Information processing system, location management method, information processing device, and program

Abstract

To provide an information processing system for stably managing a position of an object to be managed, based on a code image.SOLUTION: An information processing system which includes a plurality of image capturing devices disposed to capture images of a management area and an information processing apparatus configured to recognize a code image corresponding to an object to be managed in the management area from a captured image and manage a location of the object comprises: address setting means configured to set address identification information associated with position information in the management area to a region of the captured image; and management means configured to, based on recognition of the code image in the region of the captured image, manage the position information associated with the address identification information set for the region, as a location of the object to be managed corresponding to the code image. The address setting means can commonly set one piece of the address identification information associated with the position information in the management area, for regions of different captured images captured by the plurality of image capturing devices.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an information processing system, a location management method, an information processing device, and a program. [Background technology]

[0002] A technique for managing the progress of a job consisting of multiple work steps by utilizing a barcode or the like has been known in the past. For example, a technique for capturing a code image associated with a job at each location associated with a work step, recognizing the job from the code image, and managing the work step associated with the location where the image was captured and the job recognized from the code image in association with each other has been known in the past. Also, a technique for managing the work step associated with the location where the image was captured and the job recognized from the code image in association with each other and providing a user with information regarding the progress of multiple jobs consisting of multiple work steps has been known in the past (for example, see Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0003] In order to capture code images without missing any images of the entire management area where work is carried out using multiple cameras, there are cases where the capture areas of the cameras are partially overlapped.

[0004] In this case, if a code image exists in an overlapping shooting area captured by multiple cameras, the code image will be recognized in each of the images captured by the multiple cameras, making it impossible to stably manage the position of the managed object based on the code image.

[0005] The embodiments of the present invention have been made in consideration of the above points, and an object of the present invention is to provide an information processing system that can stably manage the positions of managed objects based on code images. [Means for solving the problem]

[0006] In order to achieve the above object, claim 1 of the present application provides an information processing system having a plurality of photographing devices installed to photograph a management area, and an information processing device that recognizes a code image corresponding to a management object in the management area from an image photographed by the plurality of photographing devices and manages the location of the management object, wherein the information processing device has an address setting means for setting address identification information corresponding to location information within the management area for an area of ​​the photographed image photographed by the plurality of photographing devices, and a management means for managing the location information corresponding to the address identification information set for the area based on the recognition of the code image in the area of ​​the photographed image, as the location of the management object corresponding to the code image, and the address setting means is capable of commonly setting one address identification information corresponding to location information within the management area for areas of different photographed images photographed by the plurality of photographing devices respectively. The management means displays the position of the managed object in a grid on the image of the managed area. It is characterized by the above. Effect of the Invention

[0007] According to the embodiment of the present invention, it is possible to stably manage the positions of managed objects based on code images. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an example of a job management system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an image diagram of an example of a work instruction sheet used in the job management system according to the embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a hardware configuration of an example of a computer. [Figure 4] FIG. 2 is a functional configuration diagram of an example of a work process management system according to the present embodiment. [Diagram 5] 1 is a diagram illustrating an example of the installation position of a camera and the position of a managed object whose position can be identified by the camera. [Figure 6]11 is a diagram illustrating an example of a relationship between the installation position of a camera and address identification information. FIG. [Figure 7] 10 is a diagram illustrating an example of a relationship between a shooting area of ​​a camera and address identification information; [Figure 8] 10 is a flowchart illustrating an example of a preparation process of the information processing system according to the present embodiment. [Figure 9] FIG. 13 is an image diagram of an example of a grid setting screen. [Figure 10] FIG. 13 is an image diagram of an example of an address setting screen. [Figure 11] FIG. 13 is an image diagram of an example of a camera setting screen. [Figure 12] FIG. 4 is a diagram illustrating an example of setting information according to the embodiment. [Figure 13] 10 is a flowchart illustrating an example of a management process of the information processing system according to the present embodiment. [Figure 14] FIG. 4 is a diagram illustrating an example of job status information according to the embodiment. [Figure 15] 10 is a flowchart illustrating an example of a job status information update process in the job management system according to the embodiment. [Figure 16] 1 is a flowchart illustrating an example of a management process of a job management system according to the present embodiment. [Figure 17] 13 is a diagram showing an example of the configuration of job status information when job status information is updated with an image captured by a camera B after job status information is updated with an image captured by a camera A. FIG. [Figure 18] 10 is a flowchart illustrating an example of a map display process of the job management system according to the embodiment. [Figure 19] FIG. 2 is an image diagram of an example of a map screen of the job management system according to the embodiment. [Figure 20] FIG. 2 is an image diagram of an example of a map screen of the job management system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that, as an example of the present embodiment, an information processing system that realizes management of the work process of a job in a printing factory by grasping the position of a color code image, which is an example of a code image, will be described as an example, but the present embodiment is not limited to a printing factory. For example, the present embodiment can be applied to grasping the position of various objects to be managed, such as grasping the position of products in a factory, and grasping the position of deliveries at a collection point or warehouse.

[0010] [First embodiment] <System configuration> Fig. 1 is a configuration diagram of an example of a job management system according to this embodiment. Fig. 2 is an image diagram of an example of a work instruction sheet used in the job management system according to this embodiment. In the job management system 1 of Fig. 1, a customer system 10, a work process management system 14, a printer 16, and one or more cameras 18 are connected to each other so as to be able to communicate data with each other via a network 20 such as the Internet or a LAN.

[0011] The customer system 10 is an example of an existing system used by a customer, and a work instruction sheet 800 for the customer system 10 in Fig. 2(A) is created, in which a job ID is displayed. The job ID is an example of identification information for identifying a job. Also, a barcode image 801 used on the customer system 10 side is displayed on the work instruction sheet 800 for the customer system 10 in Fig. 2(A).

[0012] The job ID may be displayed as a barcode image 801 or as text in the work instruction sheet 800 for the customer system 10. The customer system 10 provides the user with existing functions realized by the work instruction sheet 800 for the customer system 10.

[0013] The work process management system 14, the printer 16, and one or more cameras 18 constitute the information processing system 12 that adds new functions to the work instruction sheet 800. The work process management system 14 manages the progress of a job consisting of multiple work processes as described below, using a work instruction sheet 810 for the information processing system 12 to which a color code image 811 in Fig. 2(B) is added. Note that the information processing system 12 can identify the job ID from the color code image 811 as described below.

[0014] The printer 16 prints out a work instruction 810 for the information processing system 12. The work instruction 810 for the information processing system 12 may be referred to as a work instruction with a color code image hereinafter. The camera 18 is installed so as to be able to capture images of positions corresponding to the work steps of a job in a printing factory. Note that the positions corresponding to the work steps of a job are one or more locations corresponding to the work steps described below. The positions corresponding to the work steps of a job are locations included in the range in which the positions of the managed objects are managed, such as locations through which the managed objects such as printed matter pass between work steps, or temporary storage locations such as storage areas, tables, racks, and rooms where the managed objects are temporarily stored. The range in which the positions of the managed objects are managed is a management area in which the managed objects and code images corresponding to the managed objects are located and are captured by the camera 18.

[0015] The camera 18 may be a PTZ camera or an IP camera. The PTZ camera is a camera whose PTZ (Pan Tilt Zoom) function can be operated via the network 20, and which can transmit captured images and videos via the network 20. The IP camera is a camera which can be operated via the network 20, and which can transmit captured images and videos via the network 20. The captured images and videos captured by the camera 18 are transmitted to the work process management system 14 via the network 20. The camera 18 is an example of an imaging device.

[0016] In the information processing system 12 that adds a new function to the work instruction sheet 800, a work instruction sheet 810 for the information processing system 12 is affixed to a printed material that is an example of an intermediate product or material of a job corresponding to the work instruction sheet 810. The work instruction sheet 810 is affixed to an object to be managed, such as a printed material that is likely to be photographed by the camera 18, for example.

[0017] The work process management system 14 recognizes the color code image of the work instruction from the images captured by each camera 18, thereby managing management information such as the location of the managed object and the work status, as described below. The work process management system 14 also uses the code image to grasp and manage the position of the corresponding managed object, thereby managing the progress of the work process of the production process such as a job (job status). The work process management system 14 may also manage the history of the work process of the job, and captured images and videos showing the state when the work instruction 810 was captured. The management information managed by the work process management system 14 is updated and accumulated as needed according to the images captured by the camera 18, and is stored as information that can be displayed on a user interface.

[0018] 1 is an example. For example, the job management system 1 may include other systems, and the work process management system 14 may be called an information processing device or other similar name. The work process management system 14 may be realized in a single server environment, or in a multiple server environment.

[0019] <Hardware configuration> The customer system 10 and the work process control system 14 are realized, for example, by a computer 500 having the hardware configuration shown in FIG.

[0020] Fig. 3 is a hardware configuration diagram of an example of a computer. The computer 500 in Fig. 3 includes an input device 501, a display device 502, an external I / F 503, a RAM 504, a ROM 505, a CPU 506, a communication I / F 507, and a HDD 508, all of which are connected to each other via a bus B. The input device 501 and the display device 502 may be connected and used when necessary.

[0021] The input device 501 includes a keyboard, a mouse, a touch panel, etc., and is used by the user to input various operation signals. The display device 502 includes a display, etc., and displays the results of processing by the computer 500.

[0022] The communication I / F 507 is an interface that connects the computer 500 to various networks. This allows the computer 500 to perform data communication via the communication I / F 507.

[0023] The HDD 508 is an example of a non-volatile storage device that stores programs and data. The stored programs and data include an OS, which is basic software that controls the entire computer 500, and application software (hereinafter simply referred to as applications) that provides various functions on the OS. Note that the computer 500 may use a drive device (e.g., a solid-state drive: SSD) that uses a flash memory as a storage medium instead of the HDD 508.

[0024] The external I / F 503 is an interface with an external device. The external device may include a recording medium 503a. This allows the computer 500 to read and / or write data from and to the recording medium 503a via the external I / F 503. The recording medium 503a may include a flexible disk, a CD, a DVD, an SD memory card, a USB memory, and the like.

[0025] The ROM 505 is an example of a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 505 stores programs and data such as BIOS, OS settings, and network settings that are executed when the computer 500 is started. The RAM 504 is an example of a volatile semiconductor memory (storage device) that temporarily retains programs and data.

[0026] The CPU 506 is a calculation device that reads programs and data from storage devices such as the ROM 505 and HDD 508 onto the RAM 504 and executes processing to realize the overall control and functions of the computer 500. The customer system 10 and the work process control system 14 can realize various processes, which will be described later, by the hardware configuration of the computer 500, for example, as shown in Fig. 3. Note that a description of the hardware configuration of the printer 16 and the camera 18 will be omitted.

[0027] <Software configuration> Fig. 4 is a functional configuration diagram of an example of a work process management system according to this embodiment. Note that the functional configuration diagram shown in Fig. 4 appropriately omits configurations that are not necessary for the description of this embodiment. The work process management system 14 in Fig. 4 has a UI section 30, an address setting section 32, a job ID detection section 34, a management section 36, a color code image generation section 38, a color-coded work instruction sheet creation section 40, a print instruction section 42, a photographed image acquisition section 44, a color code recognition section 46, a setting information storage section 50, a color code management table storage section 52, and a job management table storage section 54.

[0028] The UI unit 30 controls the display of various screens such as various setting screens that accept various settings required by the user and a map screen that visually highlights the location of the managed object. The address setting unit 32 accepts settings of setting information (described later) from the user and controls the process of storing the setting information in the setting information storage unit 50. For example, the address setting unit 32 controls processes such as accepting the designation of a map image including a layout and floor plan of a management area, which is the range in which the locations of the managed objects are managed, assigning address identification information to the squares of the divided map image, and associating with job process information.

[0029] The address setting unit 32 divides the captured image captured by each camera 18 into four parts, and sets, for each divided image in the captured image, address identification information capable of identifying the squares of the map image shown in the divided image. The division of the captured image may be by setting divided areas within the captured image. The address setting unit 32 sets a plurality of divided areas within the captured image, and sets address identification information for each area by specifying a square of the map image.

[0030] The divided areas are rectangular areas indicated by the coordinates of the start and end points in the captured image, for example. With this setting, the position information of the management area corresponding to the address identification information of the specified map image square is stored in association with the divided images (divided areas) in the captured image of each camera.

[0031] The position information within the management area is made up of coordinate information of a square to which address identification information is assigned in an image that represents a map of the management area, or identification information for each rectangular square of the square.

[0032] The address setting unit 32 can set the same address identification information for divided images in a photographed image taken by a different camera 18, as long as the divided images show actual locations corresponding to the same grid positions. Alternatively, the characteristics of the photographed images taken by the different cameras 18 can be compared and processing can be controlled so that the same address identification information is set for similar areas.

[0033] In this way, the same address identification information can be set in the divided images of cameras 18 capturing actual locations corresponding to the same square positions, even if the cameras are different cameras 18. The address setting unit 32 is an example of an address setting means. The job ID detection unit 34 detects a job ID displayed as a barcode image 801 or text on the work instruction sheet 800 for the customer system 10 in Fig. 2(A), for example.

[0034] The management unit 36 ​​stores and manages available color code IDs in the color code management table storage unit 52. The management unit 36 ​​selects unused color code IDs from the color code management table storage unit 52. The management unit 36 ​​associates the job ID detected by the job ID detection unit 34 with the selected color code ID and manages them in the color code management table storage unit 52. The management unit 36 ​​is an example of a management means.

[0035] The management unit 36 ​​also stores and manages job information corresponding to the job ID and the color code ID in the job management table storage unit 54. The job management table storage unit 54 manages progress information and history information of the work process of the job, and is used for visually highlighting the location of the managed object on a map screen described later, etc.

[0036] The color code image generating unit 38 generates, for example, a color code image 811 shown in Fig. 2(B) from the color code ID provided by the management unit 36. The color-coded work instruction creation unit 40 creates, for example, a work instruction 810 for the information processing system 12 to which the color code image 811 in Fig. 2(B) is added from the work instruction 800 for the customer system 10 in Fig. 2(A). The print instructing unit 42 instructs the printer 16 to print, for example, the work instruction 810 for the information processing system 12 to which the color code image 811 in Fig. 2(B) is added.

[0037] The captured image acquisition unit 44 acquires captured images and videos from the camera 18. The color code recognition unit 46 recognizes a color code image 811 that appears in the captured images and videos. The color code recognition unit 46 decodes a color code ID from the recognized color code image 811. The color code recognition unit 46 provides the management unit 36 ​​with, for example, camera identification information that identifies the camera 18 that captured the color code image 811 and the decoded color code ID.

[0038] Upon receiving the provided information, the management unit 36 ​​refers to the color code management table storage unit 52 to identify the job ID corresponding to the decoded color code ID. In addition, the management unit 36 ​​refers to the setting information storage unit 50 to identify the divided image or divided area of ​​the camera 18 that recognized the color code image, as described below, and to identify the address identification information set in the identified divided image or divided area. The management unit 36 ​​stores the position information associated with the address identification information set for each divided image or divided area based on the recognition of a code image such as a color code in the divided image or divided area in the captured image, as the location of the managed object corresponding to the code image. In addition, the management unit 36 ​​stores the work process associated with the position information as the process in which the managed object is located. In this way, management information, which is a status including the location and work status (the process in which the managed object is located) of the managed object, is managed.

[0039] Furthermore, the management unit 36 ​​can update job status information (described later) managed in the job management table storage unit 54 based on the work process of the job linked to the address identification information corresponding to the divided image (divided area) of the camera 18 that captured the color code image and the job ID corresponding to the decoded color code ID. When the management unit 36 ​​recognizes a code image in an area in which one address identification information is commonly set among areas of different photographed images captured by multiple photographing means, the management unit 36 ​​updates the management information so that the managed object corresponding to the code image is located at the same position and in the same work process.

[0040] Each functional part of the work process management system and the customer system may be configured as an information processing device or a web server on the Internet and provided as a cloud service. For example, the information processing device may receive a request from a user's terminal device via a network such as the Internet and return various screens, or receive input from the screen and execute settings. The information processing device may also receive an image from an installed camera, recognize a code image, and transmit a map screen to software such as a web browser of the terminal device, and update the position of the managed object, the work status, etc. in real time using a two-way communication protocol.

[0041] <Relationship between camera installation location and address identification information> First, the installation position of the camera 18 that captures the position corresponding to the work process of the job in the printing factory will be described. Fig. 5 is an example diagram for explaining the installation position of the camera and the position of the managed object whose position can be identified by the camera. Note that the managed objects 100, 102, 104, and 106 shown in Fig. 5 have work instructions with color code images affixed to them in a position that can be photographed by the camera 18, such as the top surface.

[0042] FIG. 5(A) shows the installation positions of cameras 18, in which a camera 18 whose shooting area is a position corresponding to address (1) and a camera 18 whose shooting area is a position corresponding to address (2) are arranged so that their shooting areas do not overlap.

[0043] The color code images (hereinafter simply referred to as color code images) given to the work instructions affixed to the managed objects 100 and 102 in Fig. 5(A) are included in the angle of view of the left camera 18, and therefore are recognized from the image captured by the left camera 18. Also, the color code image of the managed object 106 in Fig. 5(A) is included in the angle of view of the right camera 18, and therefore is recognized from the image captured by the right camera 18.

[0044] However, when the color code image of the managed object 104 in FIG. 5(A) is affixed to, for example, the top surface of the managed object 104, it goes beyond the angle of view of the right-side camera 18 in the height direction and therefore cannot be recognized in the image captured by the right-side camera 18.

[0045] FIG. 5(B) shows the installation positions of cameras 18, where a camera 18 whose shooting area is a position corresponding to address (1), a camera 18 whose shooting area is a position corresponding to address (2), and a camera 18 that shoots a position corresponding to address (3) are arranged so that their shooting areas overlap.

[0046] Fig. 5(B) shows an example in which a camera 18 is additionally arranged to include in its angle of view the managed object 104 that was not included in the angle of view of either the right camera 18 or the left camera 18 in Fig. 5(A). The color code image of the managed object 104 in Fig. 5(B) is included in the angle of view of the central camera 18, and therefore is recognized from the image captured by the central camera 18.

[0047] However, in the installation position of the cameras 18 in Fig. 5(B), a color code image of the managed object 102 in the shooting area where the angles of view of the left camera 18 and the central camera 18 overlap is recognized from both the images captured by the left camera 18 and the central camera 18. In the example of Fig. 5(B), when a color code image is recognized from the image captured by the left camera 18, the position of the managed object 102 is determined to be address (1). On the other hand, when a color code image is recognized from the image captured by the central camera 18, the position of the managed object 102 is determined to be address (2).

[0048] Therefore, for example, when multiple cameras 18 take images in turn, the address identification information indicating the position of the managed object 102 that has not moved from the image capturing area where the angles of view of the left camera 18 and the center camera 18 in Fig. 5(B) overlap is unnecessarily updated by the camera 18 that captured the image. For this reason, in the example of Fig. 5(B), the address identification information indicating the position of the managed object 102 is not stable.

[0049] Therefore, in this embodiment, the relationship between the installation position of the camera 18 and the address identification information is as shown in Fig. 6. Fig. 6 is a diagram illustrating an example of the relationship between the installation position of the camera and the address identification information.

[0050] As shown in Fig. 6, different address identification information is assigned to areas separated to the left and right and above and below the center so that the address identification information switches at a position near the center (approximately the center) of the angle of view of each camera 18. In the example of Fig. 6, addresses (1) and (2) are assigned to the angle of view of camera A, addresses (2) and (3) are assigned to the angle of view of camera B, and addresses (3) and (4) are assigned to the angle of view of camera C.

[0051] 6, even if a color code image of the managed object 102 in the photographing area where the angles of view of the cameras A and B overlap is recognized from both the images photographed by the cameras A and B, it is determined to have the same address (2). For this reason, even if the multiple cameras A to C take photographs in turn in the example of FIG. 6, the content of the address identification information indicating the position of the managed object 102 that has not moved from the photographing area where the angles of view of the cameras A and B overlap does not change from address (2), and the address identification information indicating the position of the managed object 102 is stable.

[0052] Next, an example will be described in which nine positions, 3 vertical by 3 horizontal, to which addresses (1) to (9) are assigned, are photographed by four cameras A to D. Fig. 7 is a diagram illustrating an example of the relationship between the photographing areas of the cameras and the address identification information.

[0053] 7, the images captured by the cameras A to D are divided into four areas based on the center of the angle of view. The cameras A to D are installed so that the capturing areas of the adjacent cameras 18 partially overlap.

[0054] FIG. 7(A) shows an example where camera A is set up so that its captured images capture positions assigned with addresses (1), (2), (4), and (5). Each of the four split images split from the image captured by camera A captures a position assigned with a different address. For example, the split image at the top left of the captured image captures a position assigned with address (1). The split image at the top right of the captured image captures a position assigned with address (2). The split image at the bottom left of the captured image captures a position assigned with address (4). The split image at the bottom right of the captured image captures a position assigned with address (5).

[0055] FIG. 7(B) shows an example where camera B is set up so that its captured images capture positions assigned with addresses (2), (3), (5), and (6). Each of the four split images created from the image captured by camera B captures a position assigned with a different address. For example, the split image at the top left of the captured image captures a position assigned with address (2). The split image at the top right of the captured image captures a position assigned with address (3). The split image at the bottom left of the captured image captures a position assigned with address (5). The split image at the bottom right of the captured image captures a position assigned with address (6).

[0056] FIG. 7(C) shows an example where camera C is set up so that its captured images capture positions assigned with addresses (4), (5), (7), and (8). Each of the four split images split from the image captured by camera C captures a position assigned with a different address. For example, the split image at the top left of the captured image captures a position assigned with address (4). The split image at the top right of the captured image captures a position assigned with address (5). The split image at the bottom left of the captured image captures a position assigned with address (7). The split image at the bottom right of the captured image captures a position assigned with address (8).

[0057] FIG. 7(D) shows an example where camera D is set up so that its captured images capture positions assigned with addresses (5), (6), (8), and (9). Each of the four split images split from the image captured by camera D captures a position assigned with a different address. For example, the split image at the top left of the captured image captures a position assigned with address (5). The split image at the top right of the captured image captures a position assigned with address (6). The split image at the bottom left of the captured image captures a position assigned with address (8). The split image at the bottom right of the captured image captures a position assigned with address (9).

[0058] <Processing> Below, we will explain the preparatory processing required to manage the work process of a job in a printing factory by grasping the position of a color code image in the information processing system 1 of this embodiment, and the management processing that realizes the management of the work process of a job in a printing factory by grasping the position of a color code image.

[0059] Preparation 8 is a flowchart showing an example of preparation processing of the information processing system according to the present embodiment. In step S10, the user uploads a map image of, for example, a printing factory that includes the range in which the positions of the managed objects are managed. The UI unit 30 of the work process management system 14 accepts an operation to specify the map image from the user. The address setting unit 32 accepts the specification of the map image by the user.

[0060] In step S12, the UI unit 30 may display a grid setting screen 1000 as shown in FIG. 9, for example, and receive settings required for dividing the map image into grids from the user.

[0061] FIG. 9 is an image diagram of an example of a grid setting screen. In the grid setting screen 1000 shown in FIG. 9, the user can manually input the number of grids in the number of columns x number of rows. In addition, in the grid setting screen 1000 shown in FIG. 9, the user can input the width and depth of the floor, or the floor area, and the number of grids can be automatically set based on the shooting range of the camera 18 (for example, 2 m x 2 m). For example, for a floor with a width of 40 m and a depth of 20 m, grids of 20 columns x 10 rows are automatically assigned to the map image. The UI unit 30 may accept editing of the grids automatically assigned to the map image from the user. In response to the user's input to the grid setting screen 1000, the UI unit 30 additionally displays grids so that the divided grids can be visually recognized, for example, as shown in the map image of the grid setting screen 1000 in FIG. 9.

[0062] In step S14, the UI unit 30 displays an address setting screen 1100 as shown in FIG. 10, for example, and receives from the user, for each work process, settings of the squares of the map image that are ranges for managing the positions of the managed objects.

[0063] Fig. 10 is an image diagram of an example of an address setting screen. For example, in the address setting screen 1100 of Fig. 10, the user is prompted to select an operation process for which corresponding squares are to be set from a process list 1102, and to specify the range of squares in which the operation process is to be placed with a mouse or the like. For example, in the example of the map image of Fig. 10, the specification of the range of squares corresponding to operation processes A to G has already been accepted from the user, and the specification of the range of squares corresponding to operation process H is currently being accepted.

[0064] The address setting unit 32 assigns address identification information to a square in the map image where a work process is located, and associates the square with job process information for identifying the work process.

[0065] In step S16, the user sets up multiple cameras 18 so as to photograph the squares of the map image within the range for managing the positions of the managed objects (squares corresponding to the work processes A to L of the job) according to the rules shown in Figures 5 and 6. Note that the timing for setting up the multiple cameras 18 is not limited to the timing shown in the flowchart of Figure 8, and may be other timing.

[0066] In step S18, the UI unit 30 displays a camera setting screen 1200 as shown in Fig. 11, for example, and assigns the cameras 18 that will photograph the squares of each work process. Fig. 11 is an image diagram of an example of the camera setting screen.

[0067] For example, on the camera setting screen 1200 in Fig. 11, when a user selects one operation process from the operation selection field 1202, multiple squares in which the operation process is located are displayed in the camera location field 1206. In Fig. 11, 16 squares of 4 x 4 in which operation process C is located are displayed as an example.

[0068] The user selects from the camera list 1204 the cameras 18 to be placed in the multiple squares displayed in the camera placement field 1206, and specifies the range of the four 2×2 squares in which to place the cameras 18 with a mouse or the like. Note that the camera list 1204 may gray out or display a completed mark for cameras associated with other work processes.

[0069] For example, in the camera setting screen 1200 of FIG. 11, of the squares with address identification information "1" to "16" corresponding to the work process "Process C," "camera b" is placed in address identification information "1," "2," "5," and "6," "camera c" is placed in address identification information "2," "3," "6," and "7," "camera d" is placed in address identification information "3," "4," "7," and "8," and "camera e" is placed in address identification information "5," "6," "9," and "10."

[0070] It should be noted that the camera setting screen 1200 in Fig. 11 includes an "automatically position cameras" button 1208, and by receiving a pressing operation of the button 1208 from the user, the cameras 18 may be automatically positioned, for example, as shown in the camera positioning field 1206 in Fig. 11. In the case of a work process having 16 4 x 4 squares as shown in the camera positioning field 1206 in Fig. 11, nine cameras 18 are positioned. If the squares of the work process are n x m, the number of cameras 18 to be positioned is (n-1) x (m-1).

[0071] A user can set address identification information for the squares captured by the divided images of each of the multiple cameras 18 by operating the camera setting screen 1200 in Fig. 11. The address setting unit 32 can set address identification information for the squares for each divided area in the captured image of each of the cameras 18. The address setting unit 32 can associate camera identification information, address identification information, and job process information with each divided image obtained by dividing the captured image of each of the multiple cameras 18 into four.

[0072] By the process of the flowchart in Fig. 8, the address setting unit 32 can store setting information as shown in Fig. 12 in the setting information storage unit 50. Fig. 12 is a configuration diagram of an example of setting information according to this embodiment.

[0073] The setting information in FIG. 12 associates camera identification information, address identification information, areas of divided images in a captured image, and job process information. The camera identification information is an example of identification information for identifying the camera 18. The address identification information is an example of identification information for identifying a grid divided from a map image. The areas of divided images in a captured image are an example of identification information for identifying or cutting out and distinguishing areas of divided images in a captured image captured by the camera 18. The job process information is an example of identification information for identifying a work process.

[0074] By using the setting information in Fig. 12, the work process management system 14 can specify the address identification information of the squares photographed by each divided image of the cameras 18, and the job process information that identifies the work process at the position of the squares. Note that the setting information in Fig. 12 is set so that the cameras 18 overlap in photographing actual locations corresponding to the squares in the map image to which the same address identification information is assigned, such as the divided image in the upper right corner of the image photographed by camera A and the divided image in the upper left corner of the image photographed by camera B, according to the rules shown in Figs. 6 and 7.

[0075] 《Administrative Processing》 For example, the work processes of a job in a printing factory include printing, cutting, folding, binding, inspection, and temporary storage. The camera 18 is installed so as to be able to capture an image of a management area, which is an area in which managed objects are placed during the work processes of the job. In the information processing system 1 according to this embodiment, a work instruction sheet with a color code image that can identify a job ID is attached to a managed object such as a printed matter, and the camera 18 captures the image, thereby managing the progress of a job consisting of multiple work processes.

[0076] For example, a work instruction sheet with a color code image added thereto is created as follows: The management unit 36 ​​of the work process management system 14 selects a color code ID to be used from the color code management table storage unit 52, and manages the selected color code ID in association with the job ID of the job for which the work instruction sheet is to be created.

[0077] A color code image generating unit 38 generates a color code image from a color code ID associated with a job ID, for example, by using the technology described in JP 2017-199306 A or JP 2020-024658 A. A color-coded work instruction creating unit 40 uses the generated color code image to create a work instruction to which a color code image is added.

[0078] The print instruction unit 42 instructs the printer 16 to print a work instruction sheet to which a color code image has been added (a work instruction sheet with a color code). The printer 16 prints the work instruction sheet to which a color code image has been added in response to an instruction from the print instruction unit 42.

[0079] For example, in a printing factory, after a printed matter is output in a printing work process, a work instruction sheet with a color code image added thereto is affixed to the printed matter. After being affixed to the printed matter, which is an example of a managed object, the work instruction sheet with the color code image added thereto is photographed by the camera 18 in the work process of the job. In this way, in the job management system 1 according to this embodiment, the color code image of the work instruction sheet affixed to the printed matter, which is an example of a managed object, is photographed in the work process of the job.

[0080] In the job management system 1 according to this embodiment, when a work instruction sheet with a color code image is photographed by the camera 18, the job status of the job status information stored in the job management table memory unit 54 is updated by the process shown in FIG. 13.

[0081] 13 is a flowchart showing an example of management processing of the information processing system according to this embodiment. In step S30, the photographed image acquisition unit 44 of the work process management system 14 acquires a photographed image from the camera 18.

[0082] In step S32, the color code recognition unit 46 performs a recognition process of a color code image from the captured image acquired by the captured image acquisition unit 44, for example, according to a procedure described in JP 2017-199306 A or JP 2020-024658 A. In step S34, when the color code recognition unit 46 recognizes the color code image, it determines that the captured image contains a color code image and restores the color code ID that was coded in the color code image. After that, the color code recognition unit 46 provides, for example, camera identification information that identifies the camera 18 that captured the color code image and the decoded color code ID to the management unit 36.

[0083] For example, the management unit 36, which has been provided with the camera identification information for identifying the camera 18 that captured the color code image and the decoded color code ID, performs the processes of steps S36 to S42. If the color code recognition unit 46 does not determine that the captured image contains a color code image, the management unit 36 ​​skips the processes of steps S36 to S42.

[0084] In step S36, the management unit 36 ​​reads out the setting information of the camera 18 that captured the color code image from the setting information in Fig. 12. In step S38, the management unit 36 ​​identifies the area of ​​the divided image in the captured image in which the color code image is recognized, from the area of ​​the captured image. In step S40, the management unit 36 ​​performs an update process of the job status information as shown in Fig. 14, for example, using the address identification information and job process information associated with the identified divided image.

[0085] Fig. 14 is a diagram showing an example of job status information according to the present embodiment. The job status information shown in Fig. 14 manages address identification information, job process information, and status in association with each other. The job of the work instruction sheet in which the color code image was recognized in step S32 is managed using the job status information as being in the position (square) of the address identification information and the work process of the job process information associated with the divided image in the captured image in which the color code image was recognized.

[0086] The job status information update process in step S40 is executed by, for example, the process shown in Fig. 15. Fig. 15 is a flowchart showing an example of the job status information update process of the job management system according to this embodiment.

[0087] In step S50, the management unit 36 ​​of the work process control system 14 refers to the setting information stored in the setting information storage unit 50 and identifies the address identification information and job process information associated with the divided image obtained by recognizing the color code image.

[0088] In step S52, the management unit 36 ​​refers to the color code management table stored in the color code management table storage unit 52 and identifies the job ID corresponding to the color code ID of the color code image recognized from the divided image.

[0089] In step S54, the management unit 36 ​​updates the job status information as shown in Fig. 14 so as to store the address identification information and job process identified in step S50 in association with the job ID identified in step S52. The management unit 36 ​​repeats the processes of steps S50 to S56 until the processes for all the color code images recognized from the captured image are completed.

[0090] The procedure of the management process of the job management system 1 shown in the flowchart of Fig. 13 is an example, and may be performed, for example, in a procedure as shown in Fig. 16. Fig. 16 is a flowchart of an example showing the management process of the job management system according to this embodiment. In the work process management system 14 that performs the management process shown in Fig. 16, the captured images are provided from the captured image acquisition unit 44 to the management unit 36.

[0091] In step S70, the photographed image acquiring unit 44 of the work process control system 14 acquires a photographed image from the camera 18. In step S72, the management unit 36 ​​is provided with the photographed image from the photographed image acquiring unit 44. The management unit 36 ​​reads out the setting information of the camera 18 that captured the photographed image from the setting information in FIG.

[0092] In step S74, the management unit 36 ​​uses the setting information of Fig. 12 read in step S72 to identify the areas of divided images in the captured image in which the color code image is recognized. The management unit 36 ​​requests the color code recognition unit 46 to perform a recognition process of the color code image for each of the identified divided image areas.

[0093] The color code recognition unit 46 performs a color code image recognition process for each region of the divided image requested by the management unit 36, for example, according to the procedure described in JP 2017-199306 A or JP 2020-024658 A. When the color code recognition unit 46 recognizes a color code image, it determines that the divided image contains a color code image and restores the color code ID that was coded in the color code image. Then, the color code recognition unit 46 returns the decoded color code ID to the management unit 36.

[0094] For example, when the management unit 36 ​​receives the decoded color code ID from the color code recognition unit 46, it determines in step S76 that there is an area of ​​the divided image in which the color code image is recognized, and performs the process of step S78. If it is not determined in step S76 that there is an area of ​​the divided image in which the color code image is recognized, the management unit 36 ​​skips the process of step S78.

[0095] In step S78, the management unit 36 ​​performs an update process of the job status information, for example, as shown in FIG. 14, using the address identification information and job process information associated with the divided images in the photographed image in which the color code image has been recognized.

[0096] Fig. 17 is a diagram showing an example of the configuration of job status information when the job status information is updated using an image captured by camera B after the job status information is updated using an image captured by camera A. In the example of Fig. 17, even if there is a managed object that has not moved from the imaging area where the angles of view of cameras A and B overlap, the same address identification information is identified by the update process of the job status information using the images captured by cameras A and B.

[0097] Therefore, according to the job management system 1 of this embodiment, even if there is a managed object that has not moved from the shooting area where the angles of view of cameras A and B overlap, the contents of the address identification information indicating the position of the managed object do not change due to the update process of the job status information using the images captured by cameras A and B. Therefore, the positions of the managed objects can be stably managed.

[0098] Furthermore, in the job management system 1 of this embodiment, the contents of the work process of the job managed by grasping the position of the color code image can be provided to the user, for example, in the procedure shown in Fig. 18. Fig. 18 is a flowchart of an example showing the map display process of the job management system according to this embodiment.

[0099] In step S100, the UI unit 30 displays a map image by accepting a display operation of the map image from the user, for example. The map image is a map image of a printing factory or the like including an area for managing the positions of the managed objects, and is a map image specified in the preparation process.

[0100] In step S102, the UI unit 30 accepts an input operation of the job ID of the managed object whose position is to be searched from the user. In step S104, the management unit 36 ​​searches the job status information as shown in Fig. 14, and identifies the address identification information and job process information of the position where the managed object corresponding to the input job ID exists.

[0101] In step S106, the management unit 36 ​​displays the location of the managed object corresponding to the job ID, for example, with a marker 1304 as shown in Fig. 19, according to the address identification information of the location where the managed object identified in step S104 exists and the job process information. Fig. 19 is an image diagram of an example of a map screen of the job management system according to this embodiment. The map screen 1300 in Fig. 19 has a job ID input field 1302.

[0102] 19, the location of a job searched for by job ID is visually displayed on a map image with a marker 1304. Note that the map screen 1300 may not only display the current location of a job searched for by job ID with the marker 1304, but may also visually display the trajectory of movement. In this way, the user can check the status of the managed object to which the work instruction sheet is attached on the map screen 1300 of FIG. 19, and can track the managed object.

[0103] When multiple color code images are recognized from the same square, for example, by hovering the mouse over or clicking on the marker 1304, job information 1306 for multiple jobs may be displayed as shown in Fig. 20. Fig. 20 is an image diagram of an example of a map screen of the job management system according to this embodiment.

[0104] [Second embodiment] The above-mentioned first embodiment is a technology that is realized by a work instruction sheet to which a code image is attached to a management object such as a printed matter. This technology can be applied to a transportation system represented by an AGV (automated guided vehicle), for example.

[0105] For example, in a transport system that transports an article, a work instruction sheet with a color code image added thereto is attached to the article, and the article is photographed by the camera 18, so that the work process of the article being transported by the transport device can be managed. In addition, since the position of the transport device transporting the article and the destination of the article can be identified, the transport system can also control the movement of the transport device that transports the article.

[0106] [Third embodiment] The above-mentioned first and second embodiments have been described with respect to the application of the technology to the management of the work process of jobs and the transportation system in a printing factory, but the technology can also be applied to the management of the work process of items flowing on a belt conveyor. If the technology of the present embodiment is applied to the management of the work process of items flowing on a belt conveyor, it becomes possible to track the items flowing on the belt conveyor and to control the branching of the belt conveyor.

[0107] The system of the present invention is applicable to a place where a work process for handling a managed object exists, such as a factory or a warehouse. When photographing a place where a code image corresponding to a managed object exists to manage the position and work status of the managed object, multiple cameras are used to photograph the code image without missing any. In this case, if the cameras are installed at a distance from each other without overlapping the photographing areas of each camera, for example, the further away from the center of the camera's angle of view (center of the angle of view), the lower the photographable area in the height direction, so that the code image may protrude in the height direction and there may be a place where the code image cannot be identified. Therefore, in order to prevent the code image from protruding in the height direction, the cameras are placed close to each other, for example, arranged side by side at intervals of several meters, and the photographing areas are overlapped, so that the photographable area in the height direction can be increased throughout the entire photographing location, making it easier to identify the code image.

[0108] Furthermore, if a code image exists in an overlapping photographed area photographed by multiple cameras, the code image is recognized in each of the photographed images of the multiple cameras. In this case, if a camera and each work process are associated and the progress of the work process is managed based on the camera of the photographed image that recognized the code image, for example, when each of the multiple cameras photographs at a predetermined time interval (for example, every few minutes), the code image existing in the overlapping photographed area is recognized in each of the photographed images of the multiple cameras, and the position is unnecessarily updated every few minutes, which may cause the position and work situation to become unstable. Therefore, by making it possible to commonly set one address identification information for the areas of different photographed images photographed by multiple photographing devices, it is possible to prevent the system from becoming unstable.

[0109] The present invention is not limited to the above-mentioned specifically disclosed embodiment, and various modifications and changes are possible without departing from the scope of the claims. The work process control system 14 is an example of an information processing device described in the claims. [Explanation of symbols]

[0110] 1. Job Management System 10 Customer System 12 Information Processing Systems 14 Work process control system 16 Printers 18 Camera 20 Network 30 UI section 32 Address setting section 34 Job ID detection unit 36 Management Department 38 Color code image generator 40 Color-coded work instructions 42 Print instruction section 44 Image capture unit 46 Color code recognition unit 50 Setting information storage unit 52 Color code management table storage unit 54 Job management table storage section [Prior art documents] [Patent documents]

[0111] [Patent Document 1] JP 2020-024658 A

Claims

1. An information processing system having a plurality of photographing devices installed to photograph a management area, and an information processing device that recognizes a code image corresponding to a management object in the management area from photographed images photographed by the plurality of photographing devices, and manages the location of the management object, The information processing device includes: an address setting means for setting address identification information associated with position information within the management area for an area of ​​the photographed images photographed by the plurality of photographing devices; a management means for managing the location information associated with the address identification information set for the area based on the code image being recognized in the area of ​​the photographed image as the location of the managed object corresponding to the code image; having the address setting means is capable of commonly setting one piece of the address identification information associated with position information within the management area for areas of different photographed images photographed by the plurality of photographing devices, respectively; The management means displays the location of the managed object in a grid on the image of the management area. An information processing system comprising:

2. The address setting means sets the address identification information in each divided area obtained by dividing the image captured by the plurality of image capturing devices at approximately the center of the angle of view of the image.

2. The information processing system according to claim 1,

3. The captured image is divided into divided images and the address identification information is set for each divided image, The divided images are images obtained by dividing the photographed image into four parts, and the divided images of the adjacent photographing devices include the common area and do not exceed the center of the angle of view of the other photographing device.

3. The information processing system according to claim 1 or 2,

4. The address setting means divides the image of the management area into squares, and sets the address identification information for each square as a position within a range for managing the position of the managed object.

4. The information processing system according to claim 1, wherein:

5. When the management means receives a search request for the managed object from a user, the management means displays the position of the managed object on the image of the management area in the position of the square.

5. The information processing system according to claim 4,

6. The position information within the management area is composed of coordinate information of the square to which the address identification information is assigned in the image showing a layout map of the management area.

6. The information processing system according to claim 5 .

7. The position information within the management area is associated with a work process for the management object.

6. The information processing system according to claim 1, wherein:

8. The management means includes: Based on the recognition of the code image in the area of ​​the photographed image, the address identification information corresponding to the area is used to store the position information of the code image in the management area as the location of the managed object, and the work process corresponding to the position information as the process in which the managed object is located, thereby managing management information including the location and work status of the managed object, and further When the code image is recognized in an area in which one address identification information is commonly set among areas of different photographed images photographed by the plurality of photographing devices, the management information is updated so that the managed object corresponding to the code image is located in the same position and in the same work process.

8. An information processing system according to any one of claims 1 to 7.

9. A location management method executed by an information processing system having a plurality of photographing devices installed to photograph a management area, and an information processing device that recognizes a code image corresponding to a management object in the management area from photographed images photographed by the plurality of photographing devices, and manages the location of the management object, comprising: an address setting step of setting address identification information associated with position information within the management area for areas of the photographed images photographed by the plurality of photographing devices; a management step of managing the location information associated with the address identification information set for the area based on the code image being recognized in the area of ​​the photographed image as the location of the managed object corresponding to the code image; having the address setting step can set one address identification information corresponding to position information within the management area commonly for areas of different photographed images photographed by the plurality of photographing devices, The management step includes displaying the location of the managed object in a grid on an image of the managed area. A location management method comprising:

10. An information processing device that recognizes code images corresponding to managed objects in a management area from images captured by a plurality of photographing devices installed to photograph the management area, and manages the locations of the managed objects, an address setting means for setting address identification information associated with position information within the management area for an area of ​​the photographed images photographed by the plurality of photographing devices; a management means for managing the location information associated with the address identification information set for the area based on the code image being recognized in the area of ​​the photographed image as the location of the managed object corresponding to the code image; having the address setting means is capable of commonly setting one piece of the address identification information associated with position information within the management area for areas of different photographed images photographed by the plurality of photographing devices, respectively; The management means displays the location of the managed object in a grid on the image of the management area. An information processing device comprising:

11. an information processing device which recognizes code images corresponding to managed objects in a management area from images taken by a plurality of photographing devices installed to photograph the management area, and manages the locations of the managed objects; an address setting means for setting address identification information corresponding to position information within the management area for the areas of the photographed images photographed by the plurality of photographing devices; a management means for managing the location information associated with the address identification information set for the area based on the code image being recognized in the area of ​​the photographed image, as the location of the managed object corresponding to the code image; Function as a the address setting means is capable of commonly setting one piece of the address identification information associated with position information within the management area for areas of different photographed images photographed by the plurality of photographing devices, respectively; The management means displays the location of the managed object in a grid on the image of the management area. A program characterized by.

Images