Information processing system, location management method, information processing device, and program

Abstract

To provide an information processing system for stably managing a position of an object to be managed, a position management method, an information processing apparatus, and a program.SOLUTION: An information processing system includes an image capturing device arranged to capture an image of a range in which a position of a management object is managed, and a work process management system which recognizes a code image corresponding to the management object from a captured image captured by the image capturing device and manages the position of the management object. The work process management system includes: a position information generation unit which generates spatial coordinate information on the code image based on a position of the code image recognized from the captured image, a size of the code image recognized from the captured image, and a size of a reference code image; and a management unit which manages the position of the management object corresponding to the code image, with plane coordinate information on the code image corrected using the spatial coordinate information on the code image.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing system, a location management method, an information processing device, and a program. [Background technology]

[0002] A technique for managing the progress of a job consisting of multiple work steps by utilizing a barcode or the like has been known in the past. For example, a technique for capturing a code image associated with a job at each location associated with a work step, recognizing the job from the code image, and managing the work step associated with the location where the image was captured and the job recognized from the code image in association with each other has been known in the past. Also, a technique for managing the work step associated with the location where the image was captured and the job recognized from the code image in association with each other and providing a user with information regarding the progress of multiple jobs consisting of multiple work steps has been known in the past (for example, see Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0003] When recognizing the planar position of a code image at each location corresponding to a work process from an image captured by a camera, the planar position of the code image may not be recognized correctly for the following reasons. For example, a camera generates a captured image by projecting an object existing in space onto a planar image. Therefore, if there is variation in the height of the code image, code images at different planar positions that exist on a straight line in the camera's shooting direction will be determined to exist in the same planar position. Also, if there is variation in the height of the code image, code images that exist in the same planar position will be determined to exist in different planar positions.

[0004] The embodiments of the present invention have been made in consideration of the above points, and an object of the present invention is to provide an information processing system that can stably manage the positions of managed objects. [Means for solving the problem]

[0005] In order to achieve the above object, claim 1 of the present application is as follows: an information processing system including an imaging device which images an area in which a position of a managed object is to be managed, and an information processing device which recognizes a code image corresponding to the managed object from an image captured by the imaging device and manages the position of the managed object, the information processing device including: a position information generating means which generates spatial coordinate information of the code image based on a position of the code image recognized from the captured image, a size of the code image recognized from the captured image, and a size of a reference code image; and a management means which manages a position of the managed object corresponding to the code image with planar coordinate information of the code image corrected using the spatial coordinate information of the code image, and the management means displays the managed object corresponding to the code images of different sizes which were captured as if they existed at the same position in the captured image, as if they existed at different planar coordinates, based on the planar coordinate information of the code image corrected using the spatial coordinate information of the code image generated by the position information generating means; It is characterized by: Effect of the Invention

[0006] According to the embodiment of the present invention, the positions of the managed objects can be managed stably. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an example of a job management system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an image diagram of an example of a work instruction sheet used in the job management system according to the embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a hardware configuration of an example of a computer. [Figure 4] FIG. 2 is a functional configuration diagram of an example of a work process management system according to the present embodiment. [Diagram 5] 13 is a diagram illustrating an example of a case in which the position of a color code cannot be correctly recognized; FIG. [Figure 6] 11A and 11B are diagrams illustrating an example of a method for correctly recognizing the position of a color code. [Figure 7] 10 is a flowchart illustrating an example of a preparation process of the information processing system according to the present embodiment. [Figure 8] FIG. 13 is an image diagram of an example of a grid setting screen. [Figure 9] FIG. 13 is an image diagram of an example of an address setting screen. [Figure 10] FIG. 13 is an image diagram of an example of a camera setting screen. [Figure 11] 4 is a diagram illustrating an example of address setting information according to the embodiment. FIG. [Figure 12] FIG. 4 is a diagram illustrating an example of camera setting information according to the embodiment. [Figure 13] 10 is a flowchart illustrating an example of a management process of the information processing system according to the present embodiment. [Figure 14]FIG. 4 is a diagram illustrating an example of job status information according to the embodiment. [Figure 15] 10 is a flowchart illustrating an example of a process for correcting a position where a color code is recognized according to the present embodiment. [Figure 16] 11 is a diagram illustrating an example of a method for calculating the size of a color code in a reference plane from the size of a color code placed in the center of the reference plane. FIG. [Figure 17] 10 is a flowchart illustrating an example of a job status information update process in the job management system according to the embodiment. [Figure 18] 10 is a flowchart illustrating an example of a map display process of the job management system according to the embodiment. [Figure 19] FIG. 2 is an image diagram of an example of a map screen of the job management system according to the embodiment. [Figure 20] FIG. 2 is an image diagram of an example of a map screen of the job management system according to the embodiment. [Figure 21] FIG. 2 is an image diagram of an example of a map screen of the job management system according to the embodiment. [Figure 22] 13 is a diagram illustrating an example of a case in which the position of a color code cannot be correctly recognized; FIG. [Figure 23] 11A and 11B are diagrams illustrating an example of a method for correctly recognizing the position of a color code. [Figure 24] FIG. 4 is a diagram illustrating an example of camera setting information according to the embodiment. [Diagram 25] 10 is a flowchart illustrating an example of a process for correcting a position where a color code is recognized according to the present embodiment. [Figure 26] FIG. 4 is a diagram illustrating an example of camera setting information according to the embodiment. [Figure 27] 10 is a flowchart illustrating an example of a process for correcting a position where a color code is recognized according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that in this embodiment, an information processing system that realizes management of the work process of a job in a printing factory by grasping the position of a color code image, which is an example of a code image, will be described as an example, but the present invention is not limited to a printing factory. For example, this embodiment can be applied to grasping the position of various objects to be managed, such as grasping the position of products in a factory and grasping the position of deliveries at a collection point.

[0009] [First embodiment] <System configuration> Fig. 1 is a configuration diagram of an example of a job management system according to this embodiment. Fig. 2 is an image diagram of an example of a work instruction sheet used in the job management system according to this embodiment. In the job management system 1 of Fig. 1, a customer system 10, a work process management system 14, a printer 16, and one or more cameras 18 are connected to each other so as to be able to communicate data with each other via a network 20 such as the Internet or a LAN.

[0010] The customer system 10 is an example of an existing system used by a customer, and a work instruction sheet 800 for the customer system 10 in Fig. 2(A) is created, in which a job ID is displayed. The job ID is an example of identification information for identifying a job. Also, a barcode image 801 used on the customer system 10 side is displayed on the work instruction sheet 800 for the customer system 10 in Fig. 2(A).

[0011] The job ID may be displayed as a barcode image 801 or as text in the work instruction sheet 800 for the customer system 10. The customer system 10 provides the user with existing functions realized by the work instruction sheet 800 for the customer system 10.

[0012] The work process management system 14, the printer 16, and one or more cameras 18 constitute the information processing system 12 that adds new functions to the work instruction sheet 800. The work process management system 14 manages the progress of a job consisting of multiple work processes as described below, using a work instruction sheet 810 for the information processing system 12 to which a color code image 811 in Fig. 2(B) is added. Note that the information processing system 12 can identify the job ID from the color code image 811 as described below.

[0013] The printer 16 prints out a work instruction sheet 810 for the information processing system 12. The work instruction sheet 810 for the information processing system 12 may be hereinafter referred to as a work instruction sheet with a color-coded image. The camera 18 is installed so as to be able to capture images of positions corresponding to the work processes of a job in the printing factory. Note that the positions corresponding to the work processes of a job refer to the range in which the positions of the managed objects, such as printed matter, are managed, such as places through which the managed objects pass when moving between work processes and temporary storage locations where the managed objects are temporarily stored.

[0014] The camera 18 may be a PTZ camera or an IP camera. The PTZ camera is a camera whose PTZ (Pan Tilt Zoom) function can be operated via the network 20, and which can transmit captured images and videos via the network 20. The IP camera is a camera which can be operated via the network 20, and which can transmit captured images and videos via the network 20. The captured images and videos captured by the camera 18 are transmitted to the work process management system 14 via the network 20. The camera 18 is an example of an imaging device.

[0015] In the information processing system 12 that adds a new function to the work instruction sheet 800, a work instruction sheet 810 for the information processing system 12 is affixed to a printed material that is an example of an intermediate product or material of a job corresponding to the work instruction sheet 810. The work instruction sheet 810 is affixed to an object to be managed, such as a printed material that is likely to be photographed by the camera 18, for example.

[0016] The work process management system 14 manages the positions of the managed objects as described below by recognizing the color code image of the work instruction sheet from the images captured by each camera 18. The work process management system 14 also manages the progress of the work process of the job (job status) by managing the positions of the managed objects. The work process management system 14 may also manage the history of the work process of the job, and captured images and videos showing the state when the work instruction sheet 810 was captured.

[0017] 1 is an example. For example, the job management system 1 may include other systems, and the work process management system 14 may be called an information processing device or other similar name. The work process management system 14 may be realized in a single server environment, or in a multiple server environment.

[0018] <Hardware configuration> The customer system 10 and the work process control system 14 are realized, for example, by a computer 500 having the hardware configuration shown in FIG.

[0019] Fig. 3 is a hardware configuration diagram of an example of a computer. The computer 500 in Fig. 3 includes an input device 501, a display device 502, an external I / F 503, a RAM 504, a ROM 505, a CPU 506, a communication I / F 507, and a HDD 508, all of which are connected to each other via a bus B. The input device 501 and the display device 502 may be connected and used when necessary.

[0020] The input device 501 includes a keyboard, a mouse, a touch panel, etc., and is used by the user to input various operation signals. The display device 502 includes a display, etc., and displays the results of processing by the computer 500.

[0021] The communication I / F 507 is an interface that connects the computer 500 to various networks. This allows the computer 500 to perform data communication via the communication I / F 507.

[0022] The HDD 508 is an example of a non-volatile storage device that stores programs and data. The stored programs and data include an OS, which is basic software that controls the entire computer 500, and application software (hereinafter simply referred to as applications) that provides various functions on the OS. Note that the computer 500 may use a drive device (e.g., a solid-state drive: SSD) that uses a flash memory as a storage medium instead of the HDD 508.

[0023] The external I / F 503 is an interface with an external device. The external device may include a recording medium 503a. This allows the computer 500 to read and / or write data from and to the recording medium 503a via the external I / F 503. The recording medium 503a may include a flexible disk, a CD, a DVD, an SD memory card, a USB memory, and the like.

[0024] The ROM 505 is an example of a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 505 stores programs and data such as BIOS, OS settings, and network settings that are executed when the computer 500 is started. The RAM 504 is an example of a volatile semiconductor memory (storage device) that temporarily retains programs and data.

[0025] The CPU 506 is a calculation device that reads programs and data from storage devices such as the ROM 505 and HDD 508 onto the RAM 504 and executes processing to realize the overall control and functions of the computer 500. The customer system 10 and the work process control system 14 can realize various processes, which will be described later, by the hardware configuration of the computer 500, for example, as shown in Fig. 3. Note that a description of the hardware configuration of the printer 16 and the camera 18 will be omitted.

[0026] <Software configuration> Fig. 4 is a functional block diagram of an example of a work process management system according to this embodiment. Note that the functional block diagram shown in Fig. 4 appropriately omits configurations that are not necessary for the description of this embodiment. The work process management system 14 in Fig. 4 has a UI section 30, a setting section 32, a job ID detection section 34, a management section 36, a color code image generation section 38, a color-coded work instruction sheet creation section 40, a print instruction section 42, a photographed image acquisition section 44, a color code recognition section 46, a correction processing section 48, a setting information storage section 50, a color code management table storage section 52, and a job management table storage section 54. The correction processing section 48 has a position information generation section 60, a plane coordinate information correction section 62, and a reference code image size calculation section 64.

[0027] The UI unit 30 controls the display of various screens such as various setting screens that accept various settings required by the user and a map screen that visually displays the location of the managed object with a highlight. The setting unit 32 accepts setting information such as address setting information and camera setting information, which will be described later, from the user, and controls the process of storing the setting information in the setting information storage unit 50. For example, the setting unit 32 controls processes such as accepting the designation of a map image that includes the range in which the positions of the managed objects are managed, assigning address identification information to the squares of the divided map image, and associating with job process information.

[0028] The setting unit 32 divides the images captured by each camera 18, and sets address identification information capable of identifying the square of the map image captured by the divided image to the divided images in the captured image. The setting unit 32 controls the process so that the same address identification information is set to divided images in the captured image captured by different cameras 18 if the divided images capture the same square. In this way, the same address identification information is set to divided images captured by cameras 18 capturing the same square, even if they are from different cameras 18. The job ID detection unit 34 detects a job ID displayed as a barcode image 801 or text on the work instruction sheet 800 for the customer system 10 in FIG. 2(A), for example.

[0029] The management unit 36 ​​stores and manages available color code IDs in the color code management table storage unit 52. The management unit 36 ​​selects unused color code IDs from the color code management table storage unit 52. The management unit 36 ​​is a management means that associates the job ID detected by the job ID detection unit 34 with the selected color code ID and manages them in the color code management table storage unit 52.

[0030] The management unit 36 ​​also stores and manages job information corresponding to the job ID and the color code ID in the job management table storage unit 54. The job management table storage unit 54 manages progress information and history information of the work process of the job, and is used for visually highlighting the location of the managed object on a map screen described later, etc.

[0031] The color code image generating unit 38 generates, for example, a color code image 811 shown in Fig. 2(B) from the color code ID provided by the management unit 36. The color-coded work instruction creation unit 40 creates, for example, a work instruction 810 for the information processing system 12 to which the color code image 811 in Fig. 2(B) is added from the work instruction 800 for the customer system 10 in Fig. 2(A). The print instructing unit 42 instructs the printer 16 to print, for example, the work instruction 810 for the information processing system 12 to which the color code image 811 in Fig. 2(B) is added.

[0032] The captured image acquisition unit 44 acquires captured images and captured videos from the camera 18. The color code recognition unit 46 recognizes a color code image 811 that appears in the captured images and captured videos. The color code recognition unit 46 decodes a color code ID from the recognized color code image 811. The color code recognition unit 46 provides the correction processing unit 48 with, for example, camera identification information that identifies the camera 18 that captured the color code image 811, the decoded color code ID, and the captured image in which the color code image 811 is recognized.

[0033] Even if there is variation in the height of the color code image 811 appearing in the captured image, the correction processing unit 48 calculates the spatial position of the color code image 811 as described below, thereby improving the detection accuracy of the planar position of the color code image 811.

[0034] The positional information generating unit 60 is a positional information generating means that calculates the spatial position of the color code image 811 based on the size of the color code image 811 appearing in the captured image and a reference code size described below, as will be described later, and generates information (spatial coordinate information) indicating the spatial position of the color code image 811.

[0035] The plane coordinate information correction unit 62 corrects information (plane coordinate information) indicating the plane position of the color code image 811 recognized from the photographed image as described later, using the spatial coordinate information generated by the position information generation unit 60. The reference code image size calculation unit 64 calculates the size of the reference code image on a reference plane other than the center of the reference plane from the size of the reference code image that appears in the photographed image captured by the camera 18 of a reference code image placed at the center of a reference plane such as the ground, as described later.

[0036] The correction processing unit 48 provides the management unit 36 ​​with camera identification information that identifies the camera 18 that captured the color code image 811, the decoded color code ID, the captured image in which the color code image 811 is recognized, and planar coordinate information of the corrected color code image 811.

[0037] The management unit 36 ​​identifies the job ID corresponding to the decoded color code ID by referring to the color code management table storage unit 52. In addition, the management unit 36 ​​refers to the setting information storage unit 50, so as to be able to accurately identify the divided images of the camera 18 that recognized the color code image from the planar coordinate information of the corrected color code image 811, and to identify the address identification information set in the identified divided images.

[0038] The management unit 36 ​​can update the job status information (described below) managed in the job management table memory unit 54 based on the work process of the job corresponding to the divided image of the camera 18 that captured the color code image and the job ID corresponding to the decoded color code ID.

[0039] Each functional unit of the work process management system 14 and the customer system 10 may be implemented in an information processing device or server device in a LAN, or may be configured as an information processing device or web server on the Internet and provided as a cloud service. For example, the information processing device may receive a request from a user's terminal device via a network such as the Internet and return various screens, or receive input from the screen and execute settings. The information processing device may also receive an image from an installed camera, recognize a code image, and transmit a map screen to software such as a web browser of the terminal device, and update the position of the managed object, work status, etc. in real time using a two-way communication protocol.

[0040] <Recognizing the position of color code images> First, a description will be given of an example in which the planar position of the color code image 811 cannot be correctly recognized when recognizing the position of the color code image 811 from an image captured by the camera 18. Fig. 5 is a diagram for explaining an example in which the position of the color code cannot be correctly recognized.

[0041] In FIG. 5(A), color code image (1) exists. In FIG. 5(B), color code image (2) exists. Color code image (1) exists at a lower position than color code image (2). Color code images (1) and (2) are examples of color code images with varying heights. Note that, in FIG. 5(A) and FIG. 5(B), both color code images (1) and (2) are shown at the same time for ease of comparison, but they do not exist at the same time; only one or the other exists.

[0042] As shown in Figures 5(A) and (B), even if color code images (1) and (2) exist in different spatial positions, the camera 18 will capture the captured image as if they exist in the same planar position as long as they exist on a straight line in the shooting direction. Therefore, in the example of Figure 5, when an attempt is made to detect the planar positions of color code images (1) and (2) from the captured image, color code images (1) and (2) at different positions are recognized as existing in the same planar position.

[0043] Therefore, in this embodiment, the planar position of the color code image is detected with high accuracy by the method shown in Fig. 6. Fig. 6 is a diagram illustrating an example of a method for correctly recognizing the position of a color code. In the method shown in Fig. 6, the size of the color code image in a captured image when a color code image placed at the center of a reference plane such as the ground is captured by the camera 18 is set as the size of the reference code image. Also, in the method shown in Fig. 6, the size of the color code image whose planar position is to be detected is detected from the captured image by the camera 18.

[0044] In the method shown in FIG. 6, the spatial position of the color code image is calculated from the size of the color code image, the size of the reference code image, the planar position of the color code image recognized from the captured image, and the height of the camera 18, and the planar position of the color code image is corrected, as shown below.

[0045] Figure 6(A) shows the relationship between the size S1 of the reference code image, the planar position X1 of the color code image recognized from the captured image, the height H1 of the camera 18 from the reference plane, the size S2 of the color code image in the captured image, the planar position X2 of the color code image to be calculated, and the height H2 of the camera 18 based on the plane of the color code image to be calculated.

[0046] In the example of Figure 6, by utilizing the similarity relationship, the relationship between the size S1 of the reference code image and the size S2 of the color code image in the captured image, and the planar position X1 of the color code image recognized from the captured image and the planar position X2 of the calculated color code image can be expressed by the following formula (1).

[0047] In addition, the relationship between the size S1 of the reference code image and the size S2 of the color code image of the captured image, the height H1 of the camera 18 from the reference plane, and the height H2 of the camera 18 based on the plane of the color code image to be calculated can be expressed by the following equation (2).

[0048] 1 / S1:1 / S2=X1 2 :X2 2 …(1) 1 / S1:1 / S2=H1 2 :H2 2 …(2) From the above formula (1), formula (3) for calculating the plane position X2 of the color code image can be obtained. Also, from the above formula (2), formula (4) for calculating the height H2 of the camera 18 with respect to the plane of the color code image can be obtained.

[0049] X2=(S1 / S2) 1 / 2 *X1…(3) H2=(S1 / S2) 1 / 2 *H1…(4) Note that the above equations (1) to (4) are calculations in the X-axis direction. By performing similar calculations in the Y-axis direction, the spatial position (spatial coordinates) of the color code image can be calculated, and the planar position (planar coordinates) of the color code image can be corrected using the spatial coordinates.

[0050] Fig. 6(B) shows that the reference code image of size S1 and the color code image of size S2 in Fig. 6(A) are captured at the same position on the captured image of the camera 18. In this embodiment, as shown in Fig. 6(C) and Fig. 6(D), for example, the spatial coordinates of the color code image can be calculated based on the size S1 of the reference code image and the size S2 of the color code image, and corrected to the planar coordinates of the correct color code image.

[0051] For example, in the example of Figure 6 (A), if the height H1 of the camera 18 from the reference plane is 4 m, the size S2 of the color code image in the captured image is 400, the planar position X1 of the color code image recognized from the captured image is 2 m, and the size S1 of the reference code image is 100, the planar position X2 of the color code image and the height H2 of the camera 18 based on the plane of the color code image can be calculated as follows.

[0052] X2=(S1 / S2) 1 / 2 *X1 =(100 / 400) 1 / 2 *2 =1m H2=(S1 / S2) 1 / 2 *H1 =(100 / 400) 1 / 2 *4 =2m <Processing> Below, we will explain the preparatory processing required to manage the work process of a job in a printing factory by grasping the position of a color code image in the job management system 1 of this embodiment, and the management processing that manages the work process of a job in a printing factory by grasping the position of a color code image.

[0053] Preparation 7 is a flowchart showing an example of preparation processing of the information processing system according to the present embodiment. In step S10, the user uploads a map image of, for example, a printing factory that includes the range in which the positions of the managed objects are managed. The UI unit 30 of the work process management system 14 accepts an operation from the user to specify the map image. The setting unit 32 accepts the specification of the map image by the user.

[0054] In step S12, the UI unit 30 may display a grid setting screen 1000 as shown in FIG. 8, for example, and receive settings required for dividing the map image into grids from the user.

[0055] FIG. 8 is an image diagram of an example of a grid setting screen. In the grid setting screen 1000 shown in FIG. 8, the user can manually input the number of grids in the number of columns x number of rows. In addition, in the grid setting screen 1000 shown in FIG. 8, the user can input the width and depth of the floor, or the floor area, and the number of grids can be automatically set based on the shooting range of the camera 18 (for example, 2 m x 2 m). For example, for a floor with a width of 40 m and a depth of 20 m, grids of 20 columns x 10 rows are automatically assigned to the map image. The UI unit 30 may accept editing of the grids automatically assigned to the map image from the user. In response to the user's input to the grid setting screen 1000, the UI unit 30 additionally displays grids so that the divided grids can be visually recognized, for example, as shown in the map image of the grid setting screen 1000 in FIG. 8.

[0056] In step S14, the UI unit 30 displays an address setting screen 1100 as shown in FIG. 9, for example, and receives from the user, for each work process, settings of the squares of the map image that are ranges for managing the positions of the managed objects.

[0057] Fig. 9 is an image diagram of an example of an address setting screen. For example, in the address setting screen 1100 of Fig. 9, the user is prompted to select an operation process for which a corresponding square is to be set from a process list 1102, and to specify the range of the square where the operation process is to be placed with a mouse or the like. In this example map image, a user has already specified the range of squares corresponding to work processes A to G, and is currently specifying the range of squares corresponding to work process H. The setting unit 32 assigns address identification information to the squares in the map image where the work processes are located, and associates job process information for identifying the work processes.

[0058] It is assumed that a plurality of cameras 18 are installed at positions capable of photographing the squares of the map image within the range for managing the positions of the managed objects (the squares corresponding to the work processes A to L of the job).

[0059] In step S16, the UI unit 30 displays a camera setting screen 1200 as shown in Fig. 10, for example, and assigns the cameras 18 that will photograph the squares of each work process. Fig. 10 is an image diagram of an example of the camera setting screen.

[0060] For example, on the camera setting screen 1200 in Fig. 10, when a user selects one operation process from the operation selection field 1202, multiple squares in which the operation process is located are displayed in the camera location field 1206. In Fig. 10, 16 squares of 4 x 4 in which operation process C is located are displayed as an example.

[0061] The user selects from the camera list 1204 the cameras 18 to be placed in the multiple squares displayed in the camera placement field 1206, and specifies the range of the four 2×2 squares in which to place the cameras 18 with a mouse or the like. Note that the camera list 1204 may gray out or display a completed mark for cameras associated with other work processes.

[0062] For example, in the camera setting screen 1200 of FIG. 10, of the squares with address identification information "1" to "16" corresponding to the work process "Process C," "camera b" is placed in address identification information "1," "2," "5," and "6," "camera c" is placed in address identification information "2," "3," "6," and "7," "camera d" is placed in address identification information "3," "4," "7," and "8," and "camera e" is placed in address identification information "5," "6," "9," and "10."

[0063] Note that the camera setting screen 1200 in Fig. 10 includes an "automatically position cameras" button 1208, and by receiving a pressing operation of the button 1208 from the user, the cameras 18 may be automatically positioned, for example, as shown in the camera positioning field 1206 in Fig. 10. In the case of a work process having 16 4 x 4 squares as shown in the camera positioning field 1206 in Fig. 10, nine cameras 18 are positioned. If the squares of the work process are n x m, the number of cameras 18 to be positioned is (n-1) x (m-1).

[0064] A user can set address identification information of the squares captured by the divided images of the multiple cameras 18 by operating the camera setting screen 1200 in Fig. 10. The setting unit 32 can associate camera identification information, address identification information, and job process information with each divided image obtained by dividing the captured image of each of the multiple cameras 18 into, for example, four.

[0065] In step S16, the UI unit 30 accepts settings of the reference code size placed at the center of the reference plane and the height of the camera 18. Note that the timing for setting the reference code size and the height of the camera 18 is not limited to step S16.

[0066] By the processing of the flowchart of Fig. 7, the setting unit 32 can store address setting information as shown in Fig. 11 and camera setting information as shown in Fig. 12 in the setting information storage unit 50. Fig. 11 is a configuration diagram of an example of address setting information according to this embodiment. Fig. 12 is a configuration diagram of an example of camera setting information according to this embodiment.

[0067] The address setting information in Fig. 11 associates camera identification information, address identification information, an area of ​​a divided image in a captured image, and job process information. The camera identification information is an example of identification information for identifying the camera 18. The address identification information is an example of identification information for identifying a grid divided from a map image. The area of ​​a divided image in a captured image is an example of identification information for identifying (cutting out) an area of ​​a divided image in a captured image captured by the camera 18. The job process information is an example of identification information for identifying a work process.

[0068] By using the address setting information in Fig. 11, the work process management system 14 can specify the address identification information of the squares photographed by the divided images of each of the cameras 18, and the job process information that identifies the work process at the position of the squares. Note that the address setting information in Fig. 11 is set so that the multiple cameras 18 overlap in photographing squares that have the same address identification information, such as the divided image at the top right of the image photographed by camera A and the divided image at the top left of the image photographed by camera B.

[0069] The camera setting information in Fig. 12 associates camera identification information, a reference code size placed at the center of a reference plane, and a camera height. The reference code size placed at the center of the reference plane is an example of information indicating the size of a reference code image that appears in a captured image when a reference code image placed at the center of a reference plane such as the ground is captured by the camera 18 indicated by the camera identification information. The camera height is an example of information indicating the height of the camera 18 indicated by the camera identification information from the reference plane.

[0070] 《Administrative Processing》 For example, the work processes of a job in a printing factory include printing, cutting, folding, binding, inspection, and temporary storage. The camera 18 is installed so as to be able to capture an image of the range in which the object to be managed is placed in the work process of the job. For example, the object to be managed is assumed to have a work instruction sheet with a color code image attached to a position where it can be photographed by the camera 18.

[0071] In the information processing system 12 according to this embodiment, the progress of a job consisting of multiple work processes is managed by attaching a work instruction sheet with a color code image that can identify the job ID onto an object to be managed, such as a printed matter, and having the camera 18 photograph the object.

[0072] For example, a work instruction sheet with a color code image added thereto is created as follows: The management unit 36 ​​of the work process management system 14 selects a color code ID to be used from the color code management table storage unit 52, and manages the selected color code ID in association with the job ID of the job for which the work instruction sheet is to be created.

[0073] A color code image generating unit 38 generates a color code image from a color code ID associated with a job ID, for example, by using the technology described in JP 2017-199306 A or JP 2020-024658 A. A color-coded work instruction creating unit 40 uses the generated color code image to create a work instruction to which a color code image is added.

[0074] The print instruction unit 42 instructs the printer 16 to print a work instruction sheet to which a color code image has been added (a work instruction sheet with a color code). The printer 16 prints the work instruction sheet to which a color code image has been added in response to an instruction from the print instruction unit 42.

[0075] For example, in a printing factory, after a printed matter is output in a printing work process, a work instruction sheet with a color code image added thereto is affixed to the printed matter. After being affixed to the printed matter, which is an example of a managed object, the work instruction sheet with the color code image added thereto is photographed by the camera 18 in the work process of the job. In this way, in the job management system 1 according to this embodiment, the color code image of the work instruction sheet affixed to the printed matter, which is an example of a managed object, is photographed in the work process of the job.

[0076] In the job management system 1 according to this embodiment, when a work instruction sheet with a color code image is photographed by the camera 18, the job status of the job status information stored in the job management table memory unit 54 is updated by the process shown in FIG. 13.

[0077] 13 is a flowchart showing an example of management processing of the information processing system according to this embodiment. In step S30, the photographed image acquisition unit 44 of the work process management system 14 acquires a photographed image from the camera 18.

[0078] In step S32, the color code recognition unit 46 performs a recognition process for a color code image from the captured image acquired by the captured image acquisition unit 44, for example, according to the procedure described in JP 2017-199306 A or JP 2020-024658 A. In step S34, when the color code recognition unit 46 recognizes the color code image, it determines that the captured image contains a color code image, and restores the color code ID that was coded in the color code image.

[0079] If the color code recognition unit 46 determines that the captured image contains a color code image, the processes of steps S36 to S42 are executed. If the color code recognition unit 46 does not determine that the captured image contains a color code image, the processes of steps S36 to S42 are skipped. If the color code recognition unit 46 determines that the captured image contains a color code image, it provides the correction processing unit 48 with, for example, camera identification information for identifying the camera 18 that captured the color code image, the decoded color code ID, and the captured image in which the color code image is recognized.

[0080] In step S36, the correction processing unit 48 reads out the address setting information and camera setting information of the camera 18 that captured the color code image from the setting information storage unit 50. In step S38, the correction processing unit 48 calculates the spatial coordinates of the color code image by using the method shown in Fig. 6, and corrects the planar coordinates of the color code image by using the spatial coordinates.

[0081] The correction processing unit 48 provides the management unit 36 ​​with camera identification information that identifies the camera 18 that captured the color code image, the decoded color code ID, the captured image in which the color code image is recognized, and planar coordinate information of the corrected color code image.

[0082] In step S40, the management unit 36 ​​reads out the address setting information of the camera 18 that captured the color code image from the setting information storage unit 50. The management unit 36 ​​specifies the address identification information of the position (square) in the captured image where the color code image is recognized, using the planar coordinates of the corrected color code image. In step S42, the management unit 36 ​​uses the specified address identification information and job process information to perform an update process of the job status information, for example, as shown in Fig. 14.

[0083] Fig. 14 is a diagram showing an example of job status information according to the present embodiment. The job status information shown in Fig. 14 manages address identification information, job process information, and status in association with each other. The job of the work instruction sheet in which the color code image was recognized in step S32 is managed using the job status information as being in the position (square) of the address identification information and the work process of the job process information associated with the position in the captured image in which the color code image was recognized.

[0084] The correction process of the position where the color code image is recognized in step S38 is executed by, for example, the process shown in Fig. 15. Fig. 15 is a flowchart of an example showing the correction process of the position where the color code image is recognized according to this embodiment.

[0085] In step S50, as explained with reference to FIG. 6, the position information generating unit 60 of the correction processing unit 48 calculates the spatial coordinates of the recognized color code image based on the planar position of the color code image recognized from the captured image, the size of the recognized color code image, and the size of the reference code image.

[0086] In step S52, the plane coordinate information correction unit 62 uses the calculated spatial coordinates of the color code image to correct the plane coordinates of the color code image recognized from the photographed image to a correct position as shown in Fig. 6. The correction processing unit 48 repeats the processes of steps S50 to S54 until the processes for all the color code images recognized from the photographed images are completed.

[0087] In addition, in the correction process of the position where the color code image is recognized in step S38, the size of the reference code image on the reference plane is calculated from the size of the reference code image that appears in the image captured by camera 18 of the reference code image placed in the center of the reference plane, as shown in Figure 16.

[0088] Fig. 16 is a diagram illustrating an example of a method for calculating the size of a color code in a reference plane from the size of a color code placed in the center of the reference plane. Fig. 16 shows the relationship between the size S' of the reference code image in the center of the reference plane, the height H' of the camera 18 from the reference plane, and the size S1 of other reference code images in the reference plane.

[0089] The relationship shown in FIG. 16 can be expressed by the following equations (5) and (6).

[0090] H1=(X1 2 +H' 2 ) 1 / 2 …(5) 1 / S':1 / S1=H' 2 :H1 2 …(6) From the above equations (5) and (6), equation (7) can be obtained for calculating the size S1 of another reference code image in the reference plane.

[0091] S1=H' 2 / H1 2 *S' =H' 2 / (X1 2 +H' 2 )*S'…(7) As shown in FIG. 16, if the size S' of the reference code image at the center of the reference plane is known, the sizes S1 of the other reference code images in the reference plane can be calculated.

[0092] The job status information update process in step S42 is executed by, for example, the process shown in Fig. 17. Fig. 17 is a flowchart showing an example of the job status information update process of the job management system according to this embodiment.

[0093] In step S60, the management unit 36 ​​refers to the color code management table stored in the color code management table storage unit 52 and identifies a job ID corresponding to the color code ID of the recognized color code image. In step S62, the management unit 36 ​​updates the job status information in Fig. 14 so as to store the address identification information of the position (square) where the color code image is recognized, identified from the planar coordinate position of the corrected color code image, and the job process in association with the job ID identified in step S60. The management unit 36 ​​repeats the processes of steps S60 to S64 until the processing for all color code images recognized from the captured image is completed.

[0094] Furthermore, in the job management system 1 of this embodiment, the contents of the work process of the job managed by grasping the position of the color code image can be provided to the user, for example, in the procedure shown in Fig. 18. Fig. 18 is a flowchart of an example showing the map display process of the job management system according to this embodiment.

[0095] In step S100, the UI unit 30 displays a map image by accepting a display operation of the map image from the user, for example. The map image is a map image of a printing factory or the like including an area for managing the positions of the managed objects, and is a map image specified in the preparation process.

[0096] In step S102, the UI unit 30 accepts an input operation of the job ID of the managed object whose position is to be searched from the user. In step S104, the management unit 36 ​​searches the job status information as shown in Fig. 14, and identifies the address identification information and job process information of the position where the managed object corresponding to the input job ID exists.

[0097] In step S106, the management unit 36 ​​displays the location of the managed object corresponding to the job ID, for example, with a marker 1304 as shown in Fig. 19, according to the address identification information of the location where the managed object identified in step S104 exists and the job process information. Fig. 19 is an image diagram of an example of a map screen of the job management system according to this embodiment. The map screen 1300 in Fig. 19 has a job ID input field 1302.

[0098] 19, the location of a job searched for by job ID is visually displayed on a map image with a marker 1304. Note that the map screen 1300 may not only display the current location of a job searched for by job ID with the marker 1304, but may also visually display the trajectory of movement. In this way, the user can check the status of the managed object to which the work instruction sheet is attached on the map screen 1300 of FIG. 19, and can track the managed object.

[0099] When multiple color code images are recognized from the same square, for example, by hovering the mouse over or clicking on the marker 1304, job information 1306 for multiple jobs may be displayed as shown in Fig. 20. Fig. 20 is an image diagram of an example of a map screen of the job management system according to this embodiment.

[0100] Furthermore, when multiple color code images are recognized from the same square, for example, by hovering the mouse over or clicking on a marker 1308, job information 1306 for multiple jobs may be displayed as shown in Fig. 21. Fig. 21 is an image diagram of an example of a map screen of the job management system according to this embodiment. The marker 1308 in Fig. 21 has a shape indicating that multiple jobs exist in the same square. Note that the marker 1308 may be displayed in the same color as the color of the color-coded work instruction sheet, the border color, etc.

[0101] [Second embodiment] The first embodiment described above is an example of a camera arrangement method in which the camera 18 is installed on a ceiling or the like and captures an image of a range in which the positions of managed objects are managed from above. The second embodiment is an example of a camera arrangement method in which the camera 18 is installed on a wall or the like and captures an image of a range in which the positions of managed objects are managed from a side.

[0102] <Recognizing the position of color code images> First, an example will be described in which the planar position of a color code image cannot be correctly recognized when the position of the color code image is recognized from an image captured by the horizontally disposed camera 18. Fig. 22 is a diagram for explaining an example in which the position of the color code cannot be correctly recognized.

[0103] In Fig. 22(A), color code images (1) and (2) exist at different heights in the same planar position. In Fig. 22(B), color code images (1) and (3) exist at different planar positions at different heights. Note that Fig. 22(B) shows both color code images (1) and (3) at the same time for ease of comparison, but they do not exist at the same time; only one of them exists.

[0104] As shown in Fig. 22(A), even if color code images (1) and (2) exist in the same planar position, if they are not on a straight line in the shooting direction, the camera 18 will capture the captured image as if they exist in different planar positions. Therefore, in the example of Fig. 22(A), when an attempt is made to detect the planar positions of color code images (1) and (2) from the captured image, color code images (1) and (2) that are in the same position will be recognized as existing in different planar positions.

[0105] As shown in Fig. 22(B), even if color code images (1) and (3) are in different planar positions, if they are on a straight line in the shooting direction, the camera 18 will capture the images as if they were in the same planar position. Therefore, in the example of Fig. 22(B), when an attempt is made to detect the planar positions of color code images (1) and (3) from the captured image, color code images (1) and (3) in different positions are recognized as being in the same planar position.

[0106] Therefore, in this embodiment, the planar position of the color code image is detected with high accuracy by the method shown in Fig. 23. Fig. 23 is a diagram illustrating an example of a method for correctly recognizing the position of a color code. The method shown in Fig. 23 detects the planar position of the color code image in the same way as the method shown in Fig. 6 by projecting the planar position found from a captured image onto real space.

[0107] 23, the plane where the center of the angle of view of the camera 18 (X=0) intersects with the ground is shown as the reference plane. From the height H' of the camera 18 from the ground and the inclination θ of the camera 18, the distance H1 from the camera 18 to the reference plane can be expressed by the following formula (8). H1 = H' / cosθ…(8)

[0108] In FIG. 23, a reference code size S1 of the reference plane is calculated from the following formula (9) based on the plane position X1 of the reference plane of the color code image recognized from the photographed image and the reference code size S' of the reference plane. 1 / S':1 / S1=H1 2 :(H1 2 +X1 2 )…(9)

[0109] Based on the size S2 of the color code image in the captured image, the reference code size S1, and the plane position X1 of the reference plane of the color code image recognized from the captured image, the detection values ​​X2 and H2 are calculated using the method of the first embodiment. The calculated detection values ​​X2 and H2 are converted to X'H' coordinates using the following equations (10) and (11), whereby the plane position X'2 of the color code image and the height H'2 of the camera 18 can be calculated. H'2=(X2 2 +H2 2 ) 1 / 2 *cos(θ+tan -1 (X2 / H2))…(10) X'2=((X2 2 +H2 2 )-H'2 2 ) 1 / 2 …(11)

[0110] In addition, since the Y-axis direction is rotated around the Y-axis, coordinate conversion is not required. For example, in the example of Fig. 23, if the height H' of the camera 18 from the ground is 4m and the inclination θ of the camera 18 is 60°, the distance H1 from the camera 18 to the reference plane can be calculated as follows. H1=H' / cosθ =4 / 0.5 =8m

[0111] If the size S2 of the color code image in the captured image is 400, the plane position X1 of the reference plane of the color code image is 2m, and the reference code size S' at the plane position X1 of the reference plane is 100, the detection values ​​X2 and H2 can be calculated as follows. X2=(S1 / S2) 1 / 2 *X1 =(100 / 400) 1 / 2 *2 =1m H2=(S1 / S2) 1 / 2 *H1 =(100 / 400) 1 / 2 *8 =4m

[0112] By converting the calculated detection values ​​X2=1m and H2=4m into X'H' coordinates, the planar position X'2 of the color code image and the height H'2 of the camera 18 can be calculated as follows. H'2=(X2 2 +H2 2 ) 1 / 2 *cos(θ+tan -1 (X2 / H2) =(1 2 +4 2 ) 1 / 2*cos(60°+tan -1 (1 / 4) ≒1.13m X'2=((X2 2 +H2 2 )-H'2 2 ) 1 / 2 =((1 2 +4 2 ) -1.13 2 ) 1 / 2 ≒3.96m

[0113] The camera setting information used in the second embodiment is as shown in Fig. 24. Fig. 24 is a configuration diagram of an example of camera setting information according to this embodiment. The camera setting information in Fig. 24 has an item "camera tilt θ" added to the camera setting information in Fig. 12. The item "camera tilt θ" is an example of information indicating the tilt of the installed camera 18.

[0114] In the second embodiment, the correction process of the position where the color code image is recognized in step S38 is executed by, for example, the process shown in Fig. 25. Fig. 25 is a flowchart of an example showing the correction process of the position where the color code image is recognized according to this embodiment.

[0115] In step S200, the position information generating unit 60 of the correction processing unit 48 calculates the distance H1 from the camera 18 to the reference plane from the installation conditions (camera height, camera inclination) of the camera setting information, as described with reference to FIG.

[0116] In step S202, the position information generating unit 60 calculates a reference code size S1 of the reference plane based on the plane position X1 of the reference plane of the color code image recognized from the photographed image and the reference code size S' of the reference plane.

[0117] In step S204, the position information generating unit 60 calculates the detection values ​​X2 and H2 using the method of the first embodiment based on the size S2 of the color code image in the captured image, the reference code size S1, and the plane position X1 of the reference plane of the color code image recognized from the captured image.

[0118] In step S206, the position information generating unit 60 converts the detection values ​​X2 and H2 into X'H' coordinates, thereby making it possible to calculate the planar position X'2 of the color code image and the height H'2 of the camera 18.

[0119] In step S208, the plane coordinate information correction unit 62 uses the spatial coordinates of the color code image calculated in step S206 to correct the plane coordinates of the color code image recognized from the captured image to a correct position as shown in Fig. 23. The correction processing unit 48 repeats the processes of steps S200 to S210 until the processes for all color code images recognized from the captured images are completed.

[0120] According to the second embodiment, the positions of color code images that are placed at the same position but at different heights can be correctly recognized. For example, on the map screen 1300 in Fig. 21, color code images of managed objects placed at the same position can be displayed overlapping each other or shaded.

[0121] [Third embodiment] The first and second embodiments described above can be used in combination. In the third embodiment, since it is necessary to set a camera arrangement method for each camera 18, camera setting information as shown in FIG. 26 is used. FIG. 26 is a configuration diagram of an example of camera setting information according to this embodiment. The camera setting information in FIG. 26 is obtained by adding an item "camera arrangement method" to the camera setting information in FIG. 24. The item "camera arrangement method" is an example of information indicating whether the installed camera 18 is arranged upward or horizontally. The camera arrangement method may be automatically determined from the value of the camera inclination θ.

[0122] In the third embodiment, the correction process of the position where the color code image is recognized in step S38 is executed by, for example, the process shown in Fig. 27. Fig. 27 is a flowchart of an example showing the correction process of the position where the color code image is recognized according to this embodiment.

[0123] In step S300, the position information generating unit 60 of the correction processing unit 48 determines whether or not the camera 18 that captured the captured image is disposed upward, based on the installation conditions (camera placement method) of the camera setting information in FIG.

[0124] If the camera 18 is disposed on the top, the position information generating unit 60 executes the position correction process for the camera disposed on the top (the correction process of the first embodiment) in step S304. If the camera 18 is not disposed on the top, the position information generating unit 60 executes the position correction process for the camera disposed on the side (the correction process of the second embodiment) in step S306.

[0125] According to the third embodiment, the top-mounted camera 18 and the side-mounted camera 18 can be used in combination, making it possible to accommodate a variety of locations.

[0126] [Fourth embodiment] The above-mentioned first to third embodiments are technologies realized by a work instruction sheet to which a code image is attached to be affixed to a management object such as a printed matter. This technology can be applied to a transportation system represented by an AGV (automated guided vehicle), for example.

[0127] For example, in a transport system that transports an article, a work instruction sheet with a color code image added thereto is attached to the article, and the article is photographed by the camera 18, so that the work process of the article being transported by the transport device can be managed. In addition, since the position of the transport device transporting the article and the destination of the article can be identified, the transport system can also control the movement of the transport device that transports the article.

[0128] [Fifth embodiment] The above-mentioned first to fourth embodiments have been described with respect to the application of the technology to the management of the work process of jobs in a printing factory and to the technology of a conveying system, but the technology can also be applied to the management of the work process of items flowing on a belt conveyer. If the technology of the present embodiment is applied to the management of the work process of items flowing on a belt conveyer, it becomes possible to track the items flowing on the belt conveyer, and it is also possible to control the branching of the belt conveyer.

[0129] The present invention is not limited to the above-mentioned specifically disclosed embodiment, and various modifications and changes are possible without departing from the scope of the claims. The work process control system 14 is an example of an information processing device described in the claims. [Explanation of symbols]

[0130] 1. Job Management System 10 Customer System 12 Information Processing Systems 14 Work process control system 16 Printers 18 Camera 20 Network 30 UI section 32 Setting section 34 Job ID detection unit 36 Management Department 38 Color code image generator 40 Color-coded work instructions 42 Print instruction section 44 Image capture unit 46 Color code recognition unit 48 Correction processing section 50 Setting information storage unit 52 Color code management table storage unit 54 Job management table storage section 60 Location information generation section 62 Plane coordinate information correction section 64 Reference code image size calculation unit [Prior art documents] [Patent documents]

[0131] [Patent Document 1] JP 2020-024658 A

Claims

1. An information processing system having an image capturing device that captures an image of an area in which the position of a managed object is to be managed, and an information processing device that recognizes a code image corresponding to the managed object from an image captured by the image capturing device and manages the position of the managed object, The information processing device includes: a position information generating means for generating spatial coordinate information of the code image based on a position of the code image recognized from the photographed image, a size of the code image recognized from the photographed image, and a size of a reference code image; a management means for managing a position of the managed object corresponding to the code image based on planar coordinate information of the code image corrected using spatial coordinate information of the code image; having the management means displays the managed objects corresponding to the code images of different sizes photographed as if they existed at the same position in the photographed image, as if they existed at different plane coordinates, based on plane coordinate information of the code image corrected using spatial coordinate information of the code image generated by the position information generation means; An information processing system comprising:

2. An information processing system having a photographing device that photographs an area in which the position of a managed object is managed, and an information processing device that recognizes a code image corresponding to the managed object from an image photographed by the photographing device and manages the position of the managed object, The information processing device includes: a position information generating means for generating spatial coordinate information of the code image based on a position of the code image recognized from the photographed image, a size of the code image recognized from the photographed image, and a size of a reference code image; a management means for managing a position of the managed object corresponding to the code image based on planar coordinate information of the code image corrected using spatial coordinate information of the code image; having the management means displays the managed object corresponding to the code image photographed as if it exists at a different position in the photographed image as if it exists at the same plane coordinates based on the plane coordinate information of the code image corrected using the spatial coordinate information of the code image generated by the position information generation means; An information processing system comprising:

3. the position information generating means generates spatial coordinate information of the code image based on a ratio between a size of the code image recognized from the photographed image and a size of a reference code image, so that the spatial coordinates of the code image recognized from the photographed image are between a position of the code image recognized from the photographed image and the photographing device.

3. The information processing system according to claim 1 or 2,

4. a range for managing the position of the management object is associated with a job process for the management object; When the planar coordinate information of the code image corrected using the spatial coordinate information of the code image indicates a position within a range in which the position of the managed object is managed, the management means displays status information of the job process as if the managed object corresponding to the code image exists in the job process associated with the range.

4. The information processing system according to claim 1, wherein:

5. a reference code image size calculation means for calculating a size of the reference code image for each position on the reference plane where the code image is recognized from a size of the code image in the photographed image, the code image being placed at the center of the reference plane photographed by the photographing device; 5. The information processing system according to claim 1, further comprising:

6. When the management means receives a search request for the managed object from a user, the management means displays the position of the managed object on an image within a range in which the position of the managed object is managed, based on the planar coordinate information of the code image corrected using the spatial coordinate information of the code image.

3. The information processing system according to claim 1 or 2,

7. An information processing system having an image capturing device that captures an area in which an object is located, and an information processing device that recognizes a code image corresponding to the object from an image captured by the image capturing device, and grasps the position of the object, The information processing device includes: a position information generating means for generating spatial coordinate information of the code image based on code information of the code image recognized from the photographed image; a recognition means for recognizing a position of the object corresponding to the code image based on planar coordinate information of the code image corrected using spatial coordinate information of the code image; having the grasping means displays the object corresponding to the code images of different sizes photographed as if they existed at the same position in the photographed image as if they existed at different plane coordinates based on the plane coordinate information of the code image corrected using the spatial coordinate information of the code image generated by the position information generating means; An information processing system comprising:

8. An information processing system having an image capturing device that captures an area in which an object is located, and an information processing device that recognizes a code image corresponding to the object from an image captured by the image capturing device and grasps the position of the object, The information processing device includes: a position information generating means for generating spatial coordinate information of the code image based on code information of the code image recognized from the photographed image; a recognition means for recognizing a position of the object corresponding to the code image based on planar coordinate information of the code image corrected using spatial coordinate information of the code image; having the grasping means displays the object corresponding to the code image photographed as if it exists at a different position in the photographed image as if it exists at the same plane coordinates based on the plane coordinate information of the code image corrected using the spatial coordinate information of the code image generated by the position information generating means; An information processing system comprising:

9. The code information includes the position of the code image, the size of the code image recognized from the photographed image, and the size of a reference code image.

9. An information processing system according to claim 7 or 8.

10. A location management method executed by an information processing system having an image capturing device that captures an image of an area in which the location of a managed object is to be managed, and an information processing device that recognizes a code image corresponding to the managed object from an image captured by the image capturing device and manages the location of the managed object, comprising: a position information generating step of generating spatial coordinate information of the code image based on a position of the code image recognized from the photographed image, a size of the code image recognized from the photographed image, and a size of a reference code image; a management step of managing a position of the managed object corresponding to the code image using planar coordinate information of the code image corrected using spatial coordinate information of the code image; having the management step includes displaying the managed objects corresponding to the code images of different sizes photographed as if they were present at the same position in the photographed image as if they were present at different planar coordinates based on planar coordinate information of the code images corrected using spatial coordinate information of the code images generated in the position information generation step; A location management method comprising:

11. A location management method executed by an information processing system having an imaging device that captures an image of an area in which the location of a managed object is managed, and an information processing device that recognizes a code image corresponding to the managed object from an image captured by the imaging device and manages the location of the managed object, comprising: a position information generating step of generating spatial coordinate information of the code image based on a position of the code image recognized from the photographed image, a size of the code image recognized from the photographed image, and a size of a reference code image; a management step of managing a position of the managed object corresponding to the code image using planar coordinate information of the code image corrected using spatial coordinate information of the code image; having the management step includes displaying the managed object corresponding to the code image photographed as if it were present at a different position in the photographed image as if it were present at the same plane coordinates based on plane coordinate information of the code image corrected using spatial coordinate information of the code image generated in the position information generation step; A location management method comprising:

12. 1. An information processing device that recognizes a code image corresponding to a managed object from an image captured by an imaging device that captures an image of an area in which the position of the managed object is managed, and manages the position of the managed object, a position information generating means for generating spatial coordinate information of the code image based on a position of the code image recognized from the photographed image, a size of the code image recognized from the photographed image, and a size of a reference code image; a management means for managing a position of the managed object corresponding to the code image based on planar coordinate information of the code image corrected using spatial coordinate information of the code image; having the management means displays the managed objects corresponding to the code images of different sizes photographed as if they existed at the same position in the photographed image, as if they existed at different plane coordinates, based on plane coordinate information of the code image corrected using spatial coordinate information of the code image generated by the position information generation means; An information processing device comprising:

13. An information processing device that recognizes a code image corresponding to a managed object from an image captured by a photographing device that photographs an area in which the position of the managed object is managed, and manages the position of the managed object, comprising: a position information generating means for generating spatial coordinate information of the code image based on a position of the code image recognized from the photographed image, a size of the code image recognized from the photographed image, and a size of a reference code image; a management means for managing a position of the managed object corresponding to the code image based on planar coordinate information of the code image corrected using spatial coordinate information of the code image; having the management means displays the managed object corresponding to the code image photographed as if it exists at a different position in the photographed image as if it exists at the same plane coordinates based on the plane coordinate information of the code image corrected using the spatial coordinate information of the code image generated by the position information generation means; An information processing device comprising:

14. an information processing device that recognizes a code image corresponding to a managed object from an image captured by a photographing device that photographs an area in which the position of the managed object is to be managed, and manages the position of the managed object; a position information generating means for generating spatial coordinate information of the code image based on a position of the code image recognized from the photographed image, a size of the code image recognized from the photographed image, and a size of a reference code image; a management means for managing a position of the managed object corresponding to the code image based on planar coordinate information of the code image corrected using spatial coordinate information of the code image; Functioning as a the management means displays the managed objects corresponding to the code images of different sizes photographed as if they existed at the same position in the photographed image, as if they existed at different plane coordinates, based on plane coordinate information of the code image corrected using spatial coordinate information of the code image generated by the position information generation means; A program characterized by.

15. An information processing device that recognizes a code image corresponding to a managed object from an image captured by a photographing device that photographs an area in which the position of the managed object is managed, and manages the position of the managed object, a position information generating means for generating spatial coordinate information of the code image based on a position of the code image recognized from the photographed image, a size of the code image recognized from the photographed image, and a size of a reference code image; a management means for managing a position of the managed object corresponding to the code image based on planar coordinate information of the code image corrected using spatial coordinate information of the code image; Functioning as a the management means displays the managed object corresponding to the code image photographed as if it exists at a different position in the photographed image as if it exists at the same plane coordinates based on the plane coordinate information of the code image corrected using the spatial coordinate information of the code image generated by the position information generation means; A program characterized by.

Images