Information processing system, progress management method, program, and information processing device.

Abstract

To provide an information processing system that is easy for a user to check information about progress of a plurality of jobs each of which is composed of a plurality of work processes.SOLUTION: The problem can be solved by an information processing system that manages progress of a job composed of a plurality of work processes and provides a user with information about progress of the jobs. This system comprises: photographing means that, in each of places associated with work processes, photographs a slip on which a code image associated with a job is displayed; recognizing means that recognizes a job from the code image on the slip photographed by the imaging means; managing means that, based on the work process associated with the place where the slip was photographed and based on the job recognized by the recognition means, manages and associates information about progress of the job and photographic data obtained by photographing the slip; and provision means that provides user with information about progress of a plurality of jobs and photographic data obtained by photographing the slip.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an information processing system, a progress management method 、 program and information processing equipment and the like.

Background Art

[0002] Conventionally, the progress of a job consisting of a plurality of work processes has been managed by utilizing a barcode or the like printed on a work instruction sheet.

[0003] Also, a system for managing the working hours of workers and the flow of objects on which work is performed (entry and exit to / from the workplace) by attaching RFID tags to the workers and the objects on which work is performed (materials, work-in-progress, products, etc. in a factory) and reading the RFID tags at the entrance gate of the processing factory and the exit gate of the processing factory has been known conventionally (see, for example, Patent Document 1).

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional existing system for managing the progress of a job consisting of a plurality of work processes by utilizing a barcode or the like printed on a work instruction sheet, there has been a problem that it is difficult for a user to confirm the progress of each job.

[0005] An embodiment of the present invention has been made in view of the above points, and an object thereof is to provide an information processing system in which information regarding the progress of a plurality of jobs consisting of a plurality of work processes is easy for a user to confirm.

Means for Solving the Problems

[0006] To achieve the above objective, claim 1 of this application provides an information processing system comprising: a storage means for storing the status of a work process; an acquisition means for acquiring images of a location corresponding to the work process, captured by one or more cameras installed to capture the location; a recognition means for recognizing color code identification information contained in the acquired images of the location; an update means for updating the status of the work process based on the identification information when the identification information is recognized; and a provision means for providing the updated status of the work process to a user. Furthermore, when the update means recognizes the identification information, it updates the status of the work process using the identification information and the identification information of the camera that captured the image of the location including the color code. It is characterized by doing so. [Effects of the Invention]

[0007] According to an embodiment of the present invention, an information processing system can be provided that makes it easy for users to check information regarding the progress of multiple jobs consisting of multiple work processes. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram illustrating an example of a job management system according to this embodiment. [Figure 2] This is an illustrative diagram of an example of a work instruction sheet used in the job management system according to this embodiment. [Figure 3] This is a hardware configuration diagram of an example computer. [Figure 4] This is a functional configuration diagram of an example of a work process management system. [Figure 5] This is an illustrative diagram showing an example of the work process for a job in a printing factory. [Figure 6] This is a flowchart illustrating an example of the process for creating a work instruction sheet with color codes. [Figure 7] This is a flowchart illustrating an example of a process for generating color-coded images. [Figure 8] This is a diagram illustrating the coding rules that allow for the representation of ternary numbers. [Figure 9] This figure shows an example of information to be coded into optical symbols. [Figure 10] It is a diagram showing an example of an optical symbol when the number of colors assigned to each cell of the optical symbol is four. [Figure 11] It is a diagram showing a processing image for creating a work instruction with color codes. [Figure 12] It is a flowchart of an example of a process for reducing an image including a barcode image. [Figure 13] It is an image diagram of an example of a barcode image. [Figure 14] It is an image diagram of an example of a barcode image before reduction in the vertical direction. [Figure 15] It is an image diagram of an example of a barcode image after reduction in the vertical direction. [Figure 16] It is an image diagram of an example of a barcode image before reduction in the horizontal direction. [Figure 17] It is an image diagram of an example of a barcode image after reduction in the horizontal direction. [Figure 18] It is an explanatory diagram of an example of a change count confirmation process. [Figure 19] It is an image diagram of an example of a barcode image. [Figure 20] It is an explanatory diagram of an example of a change count confirmation process. [Figure 21] It is a flowchart of an example of a job status update process when passing through a gate. [Figure 22] It is a flowchart of an example of a job status update process when storing in a temporary storage location. [Figure 23] It is a transition diagram of an example of a UI screen displayed by a work process management system. [Figure 24] It is an explanatory diagram of an example of a grouping process on a UI screen. [Figure 25] It is an explanatory diagram of an example of a search condition specification process on a UI screen. [Figure 26] It is an image diagram of an example of a map display screen of a temporary storage location. [Figure 27] It is an explanatory diagram of an example of expressing the progress information of one or more work processes with markers. [Figure 28] It is an explanatory diagram of an example showing a state of switching between a job status list screen and a dashboard display screen. [Figure 29] It is an image diagram of an example of a dashboard display screen. [Figure 30] It is an explanatory diagram of an example of a marker. [Figure 31] It is an image diagram of an example of a flow line display screen. [Figure 32] It is a functional configuration diagram of another example of a work process management system.

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, a job management system that realizes the management of the job work process in a printing factory by a function using a work instruction sheet (slip) (function of the work instruction sheet) will be described as an example.

[0010] [First Embodiment] <System Configuration> FIG. 1 is a configuration diagram of an example of a job management system according to this embodiment. FIG. 2 is an image diagram of an example of a work instruction sheet used in the job management system according to this embodiment. The job management system 1 in FIG. 1 is connected so that data communication is possible via a network 20 such as the Internet or a LAN with a customer system 10, a work process management system 14, a printer 16, and one or more cameras 18.

[0011] The customer system 10 is an example of an existing system used by a customer, and creates a work instruction sheet 800 for the customer system 10 in FIG. 2(A) in which a job ID is displayed. The job ID is an example of identification information for identifying a job. In addition, at least one barcode image 801 used on the customer system 10 side is displayed on the work instruction sheet 800 for the customer system 10.

[0012] The job ID may be displayed as a barcode image 801 or as text in the work order 800 for the customer system 10. The customer system 10 provides the user with existing functions realized by the work order 800 for the customer system 10.

[0013] The work process management system 14, the printer 16, and one or more cameras 18 constitute an information processing system 12 that adds new functions to the work instruction sheet 800. The work process management system 14 manages the progress of a job consisting of multiple work processes using a work instruction sheet 810 for the information processing system 12, which is assigned the color code image 811 shown in Figure 2(B), as described below. The information processing system 12 can identify the job ID from the color code image 811, as described below.

[0014] The printer 16 prints a work instruction sheet 810 for the information processing system 12. The camera 18 is installed to capture images of locations in the printing plant that correspond to the work process of the job. Locations corresponding to the work process of the job include places where the printed material passes through during movement between work processes, and temporary storage areas where the printed material is temporarily stored.

[0015] Camera 18 can be a PTZ camera or an IP camera. A PTZ camera is a camera that can operate the PTZ (Pan Tilt Zoom) function via the network 20 and can transmit captured images and videos via the network 20. An IP camera is a camera that can be operated via the network 20 and can transmit captured images and videos via the network 20. Captured images and videos taken by camera 18 are transmitted to the work process management system 14 via the network 20.

[0016] In the information processing system 12, which adds new functions to the work instruction sheet 800, a work instruction sheet 810 for the information processing system 12 is affixed to a printed document, which is an example of an intermediate product or material for the job corresponding to the work instruction sheet 810. The work instruction sheet 810 is affixed, for example, to a printed document that is easily photographed by a camera 18.

[0017] The work process management system 14 manages the progress of the work process (job status) of a job based on the work process of the job corresponding to the camera 18 that took a picture of the work instruction sheet 810, and the job ID identified from the color code image 811 of the work instruction sheet 810. The work process management system 14 also manages the history of the work process of the job, as well as captured images and videos that show the circumstances when the work instruction sheet 810 was taken.

[0018] Note that the configuration of the job management system 1 shown in Figure 1 is just one example. For example, the job management system 1 may include other systems, and the work process management system 14 may have a different name. The work process management system 14 may be implemented in a single server environment or in a multi-server environment.

[0019] <Hardware Configuration> The customer system 10 and the work process management system 14 are implemented by a computer 500 with the hardware configuration shown in Figure 3, for example.

[0020] Figure 3 is a hardware configuration diagram of an example computer. The computer 500 in Figure 3 is equipped with an input device 501, a display device 502, an external interface 503, RAM 504, ROM 505, a CPU 506, a communication interface 507, and an HDD 508, all of which are interconnected via bus B. Note that the input device 501 and the display device 502 may be connected and used only when necessary.

[0021] The input device 501 includes a keyboard, mouse, touch panel, etc., and is used by the user to input various operation signals. The display device 502 includes a display, etc., and displays the processing results from the computer 500.

[0022] Communication I / F 507 is an interface that connects computer 500 to various networks. This allows computer 500 to perform data communication via communication I / F 507.

[0023] Furthermore, the HDD508 is an example of a non-volatile storage device that stores programs and data. The programs and data stored include the OS, which is the basic software that controls the entire computer 500, and application software (hereinafter simply referred to as "applications") that provides various functions on the OS. Note that the computer 500 may use a drive device that uses flash memory as a storage medium (for example, a solid-state drive: SSD) instead of the HDD508.

[0024] External I / F 503 is an interface to external devices. External devices include recording media 503a, etc. This allows computer 500 to read and / or write to recording media 503a via external I / F 503. Recording media 503a include flexible disks, CDs, DVDs, SD memory cards, USB memory, etc.

[0025] ROM505 is an example of non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. ROM505 stores programs and data such as the BIOS, OS settings, and network settings that are executed when the computer 500 starts up. RAM504 is an example of volatile semiconductor memory (storage device) that temporarily holds programs and data.

[0026] The CPU 506 is a processing unit that reads programs and data from storage devices such as the ROM 505 and HDD 508 onto the RAM 504 and executes processing, thereby realizing the overall control and functions of the computer 500. The customer system 10 and the work process management system 14 can realize the various processes described later with the hardware configuration of the computer 500 as shown in Figure 3, for example. The hardware configuration of the printer 16 and camera 18 will be omitted from this explanation.

[0027] <Software Configuration> Figure 4 is a functional configuration diagram of an example of a work process management system. Note that components unnecessary for the explanation of this embodiment have been appropriately omitted from the functional configuration in Figure 4. The work process management system 14 in Figure 4 has a configuration comprising a UI unit 30, a job ID detection unit 32, a job management unit 34, a color code image generation unit 36, a work instruction sheet creation unit with color codes 38, a print instruction unit 40, a captured image acquisition unit 42, a color code recognition unit 44, a color code management table storage unit 46, and a job management table storage unit 48.

[0028] The UI unit 30 controls the display of various screens, including a screen that accepts various settings from the user, the job status list screen, the job details history screen, the dashboard display screen, and the user flow display screen, as described later. The job ID detection unit 32 detects the job ID displayed as a barcode image 801 or text in the work instruction sheet 800 for the customer system 10, for example, as shown in Figure 2(A).

[0029] The job management unit 34 stores and manages available color code IDs in the color code management table storage unit 46. The job management unit 34 maximizes the size of the color code image 811 by limiting the number of available color code IDs and narrowing the range of values ​​required to represent the color code IDs, thereby facilitating color code recognition, as described later.

[0030] The job management unit 34 selects and reuses the oldest color code ID from the color code management table storage unit 46 if there are no unused color code IDs remaining in the color code management table storage unit 46. The job management unit 34 associates the job ID detected by the job ID detection unit 32 with the selected color code ID and manages them in the color code management table storage unit 46.

[0031] Furthermore, the job management unit 34 stores and manages job information corresponding to the job ID and color code ID in the job management table storage unit 48. The job management table storage unit 48 manages progress information and history information of the job's work process, as well as captured image files and video files showing what happened when the work instruction sheet 810 was photographed, and is used to display the job status list screen and the like, as described later.

[0032] The color code image generation unit 36 ​​generates a color code image 811, described later, from the color code ID provided by the job management unit 34. The color code work instruction creation unit 38 creates a work instruction 810 for the information processing system 12, with the color code image 811 shown in Figure 2(B), from the work instruction 800 for the customer system 10 shown in Figure 2(A). The print instruction unit 40 instructs the printer 16 to print the work instruction 810 for the information processing system 12, with the color code image 811 shown in Figure 2(B).

[0033] The image acquisition unit 42 acquires captured images and videos from the camera 18. The color code recognition unit 44 decodes the color code ID from the color code image 811 captured in the captured image or video. The color code recognition unit 44 provides the job management unit 34 with identification information that identifies, for example, the camera 18 that captured the color code image 811 or the work process of the job, and the decoded color code ID. The job management unit 34 identifies the job ID corresponding to the decoded color code ID by referring to the job management table storage unit 48.

[0034] As a result, the job management unit 34 can update the progress information (job status) of the job's work process, which is managed in the job management table storage unit 48, based on the work process of the job corresponding to the camera 18 that captured the color code image 811, and the job ID corresponding to the decoded color code ID.

[0035] <Processing> Figure 5 is an illustrative diagram showing an example of a job's work process in a printing factory. The work process in Figure 5 includes "printing," "cutting," "folding," "binding," "inspection," "temporary storage 1," and "temporary storage 2." Camera 18a is installed at the gate in front of the areas where the work processes "printing," "cutting," "folding," "binding," and "inspection" are performed. Camera 18b is installed at the areas where the work processes "temporary storage 1" and "temporary storage 2" are performed.

[0036] The work instructions 810 for the information processing system 12 are photographed by cameras 18a and 18b while moving between work processes or while stored in a temporary storage location. However, as shown in Figure 5, in a printing plant, the movement between work processes is not constant, and depending on the job, there may be work processes in which the work instructions 810 for the information processing system 12 are not photographed, and therefore not detected or recorded. In Figure 5, arrows represent the movement of jobs that perform all work processes and arrows represent the movement of jobs that skip some work processes.

[0037] In Figure 5, after the printout is produced through the "printing" process, the work instruction sheet 810 is attached to the printout. As a result, in the example in Figure 5, the work instruction sheet 810 attached to the printout is photographed by cameras 18a and 18b as the printout passes through the gate and while it is stored in the temporary storage area.

[0038] Creating work instructions with color codes. Figure 6 is a flowchart of an example of the process for creating a work instruction sheet with a color code. In step S11, the job ID detection unit 32 of the work process management system 14 detects the job ID displayed as a barcode image 801 or text on the work instruction sheet 800 for the customer system 10. The detection frame for detecting the job ID from the work instruction sheet 800 may be set in advance by the operator, or it may be set automatically using OCR or the like.

[0039] Proceeding to step S12, the job management unit 34 selects the color code ID to be used from the color code management table storage unit 46, and manages the selected color code ID in association with the job ID detected by the job ID detection unit 32.

[0040] The process proceeds to step S13, where the color code image generation unit 36 ​​generates a color code image 811 from the color code ID associated with the job ID using the procedure shown in Figure 7. The procedure shown in Figure 7 utilizes, for example, the technology described in Japanese Patent Application Publication No. 2017-199306.

[0041] Figure 7 is a flowchart of an example of the process for generating a color code image. In step S21, the color code image generation unit 36 ​​receives input of the color code ID to be color-coded. In step S22, the color code image generation unit 36 ​​decomposes the string of the color code ID into values ​​for each digit. In step S23, the color code image generation unit 36 ​​converts the decomposed values ​​of each digit into values ​​corresponding to the number of colors assigned to each cell of the optical symbol as described in Japanese Patent Application Publication No. 2017-199306. For example, if the number of colors assigned to each cell is 4, the color code image generation unit 36 ​​converts the values ​​of each digit into ternary values ​​according to the coding rules shown in Figure 8, for example.

[0042] Figure 8 illustrates the coding rules that allow for the representation of ternary numbers. Figure 8 shows an example of coding rules using four colors: R (red), G (green), B (blue), and K (black). When using four colors, representation in ternary numbers is possible.

[0043] For example, as illustrated in Figure 8(a), each clockwise transition, i.e., from R to K, K to B, B to G, and G to R, represents a value of "0". Similarly, each counterclockwise transition, i.e., from R to G, G to B, B to K, and K to R, represents a value of "1". Furthermore, each diagonal transition, i.e., bidirectional transition between R and B, and bidirectional transition between K and G, represents a value of "2".

[0044] For example, consider the case where we view the color transitions of each cell from left to right in the diagram, in a cell column where each cell is sequentially connected in the order of G, R, B, K, and G, as shown in Figure 8(b). In this case, the transition from G to R represents the value "0", the transition from R to B represents the value "2", the transition from B to K represents the value "1", and the transition from K to G represents the value "2". Therefore, the array in Figure 8(b) represents the value "3d2120", which is the decimal value "69". Note that the "3d" at the beginning of the value notation indicates that the following digit is a ternary value.

[0045] The coding rule using four color transitions is not limited to the example shown in Figure 8(a), but may also be the example shown in Figure 8(d). Figure 8(e) shows an example of a conversion table that associates color transitions with values ​​according to the coding rule shown in Figure 8(d). In Figure 8(e), for example, if the source cell's color is R and the value "2" is to be represented, the color of the destination cell adjacent to the source cell is set to B. Similarly, if the source cell's color is K and the value "1" is to be represented, the color of the destination cell is set to G.

[0046] Following step S23 in Figure 7, the process proceeds to step S24, in which the color code image generation unit 36 ​​assigns a color to each cell of the optical symbol according to the color of the source cell, the ternary value converted in step S23, and the conversion table of coding rules as shown in Figure 8(a). It is assumed that the color code image generation unit 36 ​​has already stored the arrangement of colors for each cell in the cell column of the main code of the optical symbol.

[0047] Figures 9 and 10 illustrate a more specific example of assigning four colors to each cell of an optical symbol. Figure 9 shows an example of information to be coded into an optical symbol. Figure 10 shows an example of an optical symbol when four colors are assigned to each cell of the optical symbol.

[0048] Figure 9 shows an example where the cell columns of the subcode section 301 connected to the cells "d1", "d2", "d3", and "d4" of the main code section 300 have the values ​​"3d1", "3d1", "3d0", and "3d1".

[0049] Figure 10 shows an example of an optical symbol coded according to the coding rules explained using Figure 8, based on the information illustrated in Figure 9. In this example, the cell row of the main code section 300 contains an even number of cells, with the leading cell being the K color, and K and R colors arranged alternately.

[0050] Furthermore, in the subcode section 301, for example, in the leftmost cell, the cell's color is assigned using the K color of the cell in the main code section 300 to which the cell is connected as the initial source color. In the example in Figure 10, the R color is assigned to the leftmost cell of the subcode section 301 of the optical symbol, according to the K color of the cell in the main code section 300 (the source cell) and the cell value "3d1" of the subcode section 301, by referring to the coding rule conversion table in Figure 8(a).

[0051] Following step S13 in Figure 6, the process proceeds to step S14, where the color-coded work instruction creation unit 38 uses the color-coded image 811 generated in step S13 to create a work instruction 810 for the information processing system 12.

[0052] Figure 11 shows an image illustrating the process for creating a work instruction sheet with color codes. The paper size and layout frame may be pre-set by the operator. In Figure 2(A), the work instruction sheet 800 for the customer system 10 is scaled down to create empty space. In Figure 2(B), the work instruction sheet 810 for the information processing system 12 has a color code image 811 pasted onto the empty space created by scaling down the work instruction sheet 800 for the customer system 10.

[0053] Furthermore, the work order sheet 800 for the customer system 10 shown in Figure 2(A) displays at least one barcode image 801 used by the customer system 10. Therefore, simply reducing the size of the work order sheet 800 for the customer system 10 may cause the barcode image 801 to stop functioning (become unrecognizable).

[0054] Therefore, in order to generate empty space without impairing the function of the barcode image 801, the color-coded work instruction sheet creation unit 38 checks the orientation of the barcode image 801 using the procedure shown in Figure 12 and performs scaling or reduction in a direction that does not impair the function of the barcode image 801.

[0055] Figure 12 is a flowchart of an example of a process for reducing the size of an image containing a barcode image. In step S31, the color-coded work instruction creation unit 38 extracts the position of the barcode image 801 from the work instruction 800 for the customer system 10 in Figure 2(A). Proceeding to step S32, the color-coded work instruction creation unit 38 determines whether the extracted barcode image, as shown in Figure 13, is a Bk (monochrome) image or an RGB image. Figure 13 is an illustrative image of an example of a barcode image.

[0056] If the image is not a Bk (monochrome) image, the color-coded work instruction creation unit 38 proceeds to step S33, converts the RGB image to a Lab or Luv image, extracts only the L component after conversion to create a monochrome image, and then proceeds to step S34. If the image is monochrome, the color-coded work instruction creation unit 38 skips step S33 and proceeds to step S34.

[0057] In step S34, the color-coded work instruction sheet creation unit 38 raster scans all pixels in the barcode image in Figure 13. Here, the pixel positions (x,y) are the positions according to the X and Y directions shown in Figure 13. The color-coded work instruction sheet creation unit 38 accumulates the values ​​of pixels with the same pixel position x, that is, it accumulates the pixel values ​​in the Y direction.

[0058] Next, proceeding to step S35, the color-coded work instruction sheet creation unit 38 accumulates all pixels. The color-coded work instruction sheet creation unit 38 calculates the average value by dividing the accumulated value of all pixels by the total number of pixels.

[0059] Proceeding to step S36, the color-coded work instruction sheet creation unit 38 checks the number of changes by comparing the cumulative value of pixels with the same pixel position x, as determined in step S34, with the average value determined in step S35, as shown in Figure 18. Figure 18 is an explanatory diagram of an example of the change count confirmation process. For example, the color-coded work instruction sheet creation unit 38 checks the total number of changes, which is the sum of the number of changes from a cumulative value higher than the average to a cumulative value lower than the average, and the number of changes from a cumulative value lower than the average to a cumulative value higher than the average.

[0060] The process proceeds to step S37, where the color-coded work instruction sheet creation unit 38 determines whether the number of changes confirmed in step S36 is 24 or more. If the number of changes is 24 or more, the process proceeds to step S38, where the color-coded work instruction sheet creation unit 38 performs scaling, for example, the scaling of the barcode image in Figure 14 to the barcode image in Figure 15, in the vertical direction. Figure 14 is an example image of a barcode image before vertical scaling. Figure 15 is an example image of a barcode image after vertical scaling.

[0061] For example, if the barcode image shown in Figure 13 is reduced in the X direction, the black lines become closer together, making it difficult to distinguish between black and white. On the other hand, reducing the barcode image shown in Figure 13 in the Y direction does not change the spacing between the black lines, so it does not affect the distinction between black and white.

[0062] On the other hand, if the number of changes is not 24 or more, the process proceeds to step S39, where the color-coded work instruction sheet creation unit 38 performs scaling, for example, to reduce the barcode image in Figure 16 horizontally, as shown in the barcode image in Figure 17.

[0063] The flowchart in Figure 12 determines the orientation of the barcode image based on the following idea. For example, in the case of a vertical barcode as shown in Figure 13, the black line has width, so the change from white line to black line and the change from black line to white line occur in pairs. The number of changes varies depending on the number of digits displayed in the barcode, but based on research on typical barcodes, if the number of changes is 24 or more, it can be determined to be a vertical barcode like the one in Figure 13.

[0064] Similarly, in the case of a horizontal barcode as shown in Figure 19, the color-coded work instruction sheet creation unit 38 checks the number of changes in step S36, as shown in Figure 20. Figure 19 is an illustrative image of an example of a barcode image. Figure 20 is an explanatory diagram of an example of the change count check process.

[0065] In the case of Figure 20, the total number of changes—the number of changes from an accumulated value higher than the average to an accumulated value lower than the average, and the number of changes from an accumulated value lower than the average to an accumulated value higher than the average—is extremely small. Therefore, even if information other than the barcode image is accumulated, the number of changes will not exceed 24. Consequently, if the number of changes is 24 or more, it can be determined to be a horizontal barcode as shown in Figure 19.

[0066] Alternatively, the orientation of a barcode image can also be determined using methods such as the Fourier transform or the Hough transform, which is a feature extraction method. However, modern printers perform halftone processing. In particular, electrophotographic printers may use line dithering.

[0067] When using the Hough transform, line segment components can be extracted from the image. Although the line segment components of the barcode are dominant, it is difficult to eliminate the effects of halftone processing, especially line dithering. To minimize the effects of line dithering, the position of the barcode image 801 must be precisely specified in step S31. Similarly, when using the Fourier transform, the problem is whether the effects of halftone processing around the barcode can be precisely eliminated.

[0068] On the other hand, in the method for determining the orientation of a barcode image according to this embodiment, even if a dithered image or some characters or images are included around the barcode image cut out in step S31, it will not be counted as a change and will not pose a problem.

[0069] Thus, in the work process management system 14 of this embodiment, empty space can be created by scaling the barcode image 801 in a direction that does not impair its functionality, and a remotely readable color code image 811 can be added to that empty space. Even if the work instruction sheet 800 for the customer system 10 is scaled, the functionality of the barcode image 801 is not impaired, and the functionality of the work instruction sheet 800 for the customer system 10 can be retained while adding the functionality of the color code image 811 to the work instruction sheet 810 for the information processing system 12.

[0070] Following step S14 in Figure 6, the process proceeds to step S15, instructing the printer 16 to print a work instruction sheet 810 for the information processing system 12 (work instruction sheet with color code) to which the color code image 811 created in step S14 has been added. The printer 16 prints a work instruction sheet 810 for the information processing system 12 to which the color code image 811 has been added, for example, as shown in Figure 2(B), based on the instructions from the print instruction unit 40.

[0071] 《Updating job status upon passing through a gate》 In the job management system 1 according to this embodiment, when a work instruction sheet 810 for the information processing system 12, to which a color code image 811 has been attached, is photographed by the camera 18a, the job status in the job management table storage unit 48 is updated as follows.

[0072] Figure 21 is a flowchart of an example of the job status update process when passing through a gate. When the image acquisition unit 42 of the work process management system 14 acquires captured images or videos from the camera 18a, it proceeds to step S52.

[0073] In step S52, the color code recognition unit 44 attempts to extract a color code image 811 from the captured image or video acquired by the captured image acquisition unit 42. Proceeding to step S53, the color code recognition unit 44 performs recognition processing of the color code image 811 using a procedure described, for example, in Japanese Patent Application Publication No. 2017-199306.

[0074] When the color code image 811 is recognized, the color code recognition unit 44 detects the image of each cell from the color code image 811. Proceeding to step S54, the color code recognition unit 44 recognizes the color information of each cell and the connection information of each cell detected in step S53 as symbol information. Proceeding to step S54, the color code recognition unit 44 recovers the color code ID that was encoded in the color code image 811 by decoding the symbol information according to the encoding rule shown in Figure 8(A), for example.

[0075] The process proceeds to step S55, where the color code recognition unit 44 provides the job management unit 34 with, for example, the identification information of the camera 18a that captured the color code image 811 and the color code ID restored by decoding. The job management unit 34 refers to the job management table storage unit 48 to identify the job ID corresponding to the color code ID.

[0076] Then, the job management unit 34 can update the job status managed by the job management table storage unit 48 from "Not Detected" to "Current Process" or "Completed," etc., based on, for example, the identification information of the camera 18a that took the color code image 811 and the job ID corresponding to the color code ID recovered from the color code image 811.

[0077] 《Updating job status when storing in temporary storage location》 In the job management system 1 according to this embodiment, when a work instruction sheet 810 for the information processing system 12, to which a color code image 811 has been attached, is photographed by the camera 18b, the job status in the job management table storage unit 48 is updated as follows.

[0078] Figure 22 is a flowchart of an example of the job status update process when storing a job in a temporary storage location. When the image acquisition unit 42 of the work process management system 14 acquires an image from the camera 18b, it proceeds to step S62.

[0079] In step S62, the color code recognition unit 44 attempts to extract a color code image 811 from the captured image or video acquired by the captured image acquisition unit 42. Proceeding to step S63, the color code recognition unit 44 performs recognition processing of the color code image 811 using a procedure described, for example, in Japanese Patent Application Publication No. 2017-199306.

[0080] If the color code image 811 is recognized in step S63, the color code recognition unit 44 proceeds from step S64 to S65 and decodes the symbol information recognized from the color code image 811 to restore the color code ID that was encoded in the color code image 811.

[0081] Proceeding to step S66, the color code recognition unit 44 provides the job management unit 34 with, for example, the identification information of the camera 18b that captured the color code image 811 and the color code ID restored by decoding. The job management unit 34 refers to the job management table storage unit 48 to identify the job ID corresponding to the color code ID.

[0082] Then, the job management unit 34 determines, for example, whether a predetermined time has elapsed without any change in the job status of a given job ID, based on the identification information of the camera 18b that captured the color code image 811 and the job ID corresponding to the color code ID restored from the color code image 811.

[0083] If the job status remains unchanged for a predetermined time, the job management unit 34 proceeds to step S70. In step S70, the job management unit 34 can update the job status managed by the job management table storage unit 48 from "Not Detected" to "Current Process". If the job status remains unchanged for a predetermined time, the job management unit 34 performs processing to display a stagnation alert on the job details history screen described later, and then proceeds to step S70. In step S70, the job management unit 34 can update the job status managed by the job management table storage unit 48 from "Current Process" to "Alert" or the like.

[0084] If the color code image 811 is not recognized in step S63, the color code recognition unit 44 proceeds from step S64 to S68 and performs processing to update the job ID, which is currently stored in the temporary storage location, in cases where the color code image 811 could not be recognized from the image captured by the camera 18b installed in the temporary storage location.

[0085] The color code recognition unit 44 provides, for example, identification information of the camera 18b to the job management unit 34. The job management unit 34 determines whether there are any job IDs in the temporary storage location corresponding to the provided camera 18b identification information whose job status is "current process" and for which a color code image 811 has not been recognized from the captured image for a predetermined number of consecutive times.

[0086] If, among the job IDs in the temporary storage location whose job status is "Current Process," there is a job ID for which the color code image 811 has not been recognized from the captured image for a predetermined number of consecutive times, the job management unit 34 proceeds to step S69, sets the color code ID associated with that job ID to Null, and proceeds to step S70. In step S70, the job management unit 34 can update the job status managed by the job management table storage unit 48 from "Current Process" to "Completed" or the like.

[0087] Furthermore, it is possible that the work instruction sheet 810 for the information processing system 12 is stored in a temporary storage location in a state where it cannot be photographed by the camera 18b, such as when printed materials are stacked on top of each other. Taking this into consideration, the job management unit 34 skips step S69 and proceeds to step S70 if there are no job IDs in the temporary storage location whose job status is "current process" and for which the color code image 811 has not been recognized from the captured images for a predetermined number of consecutive times. In step S70, the job management unit 34 can update the job status managed by the job management table storage unit 48 from "current process" to "stacked".

[0088] According to the flowcharts in Figures 21 and 22, tracking of the printed materials to which the work instructions 810 for the information processing system 12 have been attached becomes possible by automatically updating the job status of the printed materials to which the work instructions 810 for the information processing system 12 have been attached.

[0089] Job Inquiry The operator of the job management system 1 can use the various UI screens provided by the work process management system 14 to view progress information and history information of the work process of a job in the printing plant, as well as image files and video files showing what happened when the work instruction sheet 810 was photographed.

[0090] Figure 23 is a transition diagram of an example of a UI screen displayed by the work process management system. The UI unit 30 of the work process management system 14 displays, for example, the job status list screen 1000 on the display device 502. The job status list screen 1000 displays a list of job information including the job ID, color code ID, job name, last updated data, and progress information for one or more work processes.

[0091] The progress information for one or more work processes indicates whether the progress of the work process is "Not Detected," "Completed," "Current Process," "Alert," or "Accumulated." "Not Detected" for a work process indicates that the camera 18 corresponding to that work process has not captured a work instruction sheet 810 for the information processing system 12. "Completed" for a work process indicates that the camera 18 corresponding to that work process captured a work instruction sheet 810 for the information processing system 12 before its final update.

[0092] Furthermore, the "Current Process" status in the work process indicates that the work instruction sheet 810 for the information processing system 12 has been photographed by the camera 18 corresponding to that work process in its final update. The "Alert" status in the work process indicates that an alert has occurred, such as a dwell alert. The "Stacked" status in the work process indicates that the work instruction sheet 810 for the information processing system 12 was photographed by the camera 18 corresponding to that work process before its final update, and that the work instruction sheet 810 for the information processing system 12 has not been photographed by the camera 18 in its final update for that work process or other work processes. The "Stacked" status in the work process indicates a state in which it is expected that printed materials to which the work instruction sheet 810 is attached can no longer be photographed by the camera 18 due to stacking or other reasons.

[0093] Furthermore, the key icon 1002 on the job status list screen 1000 indicates that the color code ID associated with, for example, a long-term storage job is locked to prevent its automatic release. In this way, the work process management system 14 provides the operator with a lock function that prevents the reuse of color code IDs.

[0094] The operator can display the Job Details History screen 1010 by selecting a job from the Job Status List screen 1000. The Job Details History screen 1010 displays buttons to transition to screens showing the start date and time, completion date and time, and captured images and videos for each work process. If the work process is "Temporary Storage 1" or "Temporary Storage 2", the address representing the storage area at the temporary storage location is also displayed on the Job Details History screen 1010. Furthermore, the Job Details History screen 1010 displays a "Show Workflow" button to transition to the Workflow Display screen 1050, which will be described later.

[0095] Furthermore, when a button is pressed to transition to the screen 1020 that displays captured images and videos on the job details history screen 1010, the UI unit 30 transitions to the screen 1020 that displays captured images and videos. When the play button is pressed on the screen 1020 that displays captured images and videos, it displays the captured image file and video file that represent the situation when the work instruction sheet 810 was photographed.

[0096] When a button is pressed to transition to the map display screen 1030 for the job corresponding to work process "Temporary Storage 1" or "Temporary Storage 2", the UI unit 30 transitions to the map display screen 1030 shown in Figure 26. Figure 26 is an illustrative image of an example of the map display screen for temporary storage locations. The UI unit 30 displays, for example, a photograph of the temporary storage location as the background, and then displays the storage area where the printed materials are stored so that it can be identified with a mark "●" or the like.

[0097] Furthermore, the job status list screen 1000 in Figure 23 may be displayed with grouping as shown in Figure 24, or it may be configured to accept the specification of search conditions as shown in Figure 25.

[0098] Figure 24 is an explanatory diagram of an example of grouping processing in the UI screen. Figure 24(A) is an example of the Job Status List screen 1000 before grouping processing. Figures 24(B) and 24(C) are examples of the Job Status List screen 1000 after grouping processing, where the progress information of one or more work processes is grouped by "1st floor area" and "2nd floor area" to make it easier to understand visually.

[0099] In Figure 24(C), the grouped "1st floor area" is displayed in a collapsed state. When displaying the progress information of grouped work processes in a collapsed state, the progress information of the work processes included in the collapsed "1st floor area" group is displayed as the AND (logical AND) of the progress information of the work processes included in the group. For example, the AND of the progress information of the work processes included in the group is determined as "Alert > Current Process > Stacked > Passed > Not Detected".

[0100] Figure 25 is an explanatory diagram of an example of the process for specifying search conditions on a UI screen. Figure 25 is an example of the Job Status List Screen 1000 with a search row. The search row is displayed, for example, when the "Search" button on the Job Status List Screen 1000 is pressed. By specifying search conditions in the search row, the operator can search for jobs that match the search conditions. Note that the search row provided as in Figure 25 has the effect of reducing screen transitions and improving immediacy and visibility. Figure 25(A) is an example where the search condition is entered as a text item (partial match) in the search row. Figure 25(B) is an example where the search condition is entered as a date and time item (range specification) in the search row. Figure 25(C) is an example where the search condition is selected from a dropdown menu using a work process item (match) in the search row. The dropdown selection can also be made so that multiple selections can be made using the control key, for example. Also, when the "Clear Conditions" button is pressed, the search conditions are cleared all at once and the display returns to all items.

[0101] Furthermore, the progress information of one or more work processes on the job status list screen 1000 may be represented by markers, for example, as shown in Figure 27. Note that in Figure 27, the marker colors are represented by kanji characters. Time progresses in the order of Figures 27(a) to 27(e). For example, in Figure 27, the second work process, which was in progress "Current Process" in Figure 27(c), is in progress "Accumulated" in Figure 27(d).

[0102] The progress information for one or more work processes shown in Figure 27(d) will change to one of the states shown on the right side of Figure 27 when the work instruction sheet 810 for the information processing system 12 is photographed again by the camera 18.

[0103] Furthermore, the operator can display the dashboard display screen 1040 by pressing the "Dashboard" button from the job status list screen 1000. Figure 28 is an explanatory diagram illustrating an example of switching between the job status list screen and the dashboard display screen. When the "Dashboard" button is pressed while the job status list screen 1000 is accepting search criteria, the dashboard display screen 1040 will carry over and display the specified search criteria.

[0104] Furthermore, the operator can display the job status list screen 1000 by pressing the "Return to List" button from the dashboard display screen 1040. The job status list screen 1000 may also be configured to retain the original search criteria specified before transitioning to the dashboard display screen 1040.

[0105] Figure 29 is an illustrative image of an example of a dashboard display screen. Dashboard display screen 1040 displays information about the progress of multiple jobs using markers (an example of a display object) that represent the location of the jobs on the map and the number of jobs at each location. For example, in Figure 29, the number of jobs and the number of alerts within each process category are shown for "Process Category A", "Process Category B", and "Process Category C".

[0106] The dashboard display screen 1040 in Figure 29 may have process categories enclosed in frames and a factory map embedded as a background image. The position where a marker is displayed represents one or more work processes. The size of the marker represents the number of jobs in that work process. The color of a normal marker that does not contain a job with an alert (e.g., green) and the color of an alert marker that contains a job with an alert (e.g., red) are displayed separately.

[0107] Figure 30 is an explanatory diagram of an example of a marker. The marker in Figure 30 is an example represented by a bubble (circle mark), and for a specified work process, it is represented by a circle of five different sizes corresponding to the number of jobs. The parameters of the marker in Figure 30 include the range of the number of jobs, the center coordinates, the radius (size), and the color (normal marker / alert marker). Note that the color of the normal marker may differ for each of the five levels.

[0108] According to the dashboard display screen 1040, the number of jobs accumulated in each work process, as well as the work processes containing jobs that have triggered alerts, can be displayed in an easy-to-understand visual format.

[0109] Furthermore, the operator can display the movement path display screen 1050 by pressing the "Movement Path Display" button on the job details history screen 1010. Figure 31 is an illustrative image of an example of the movement path display screen. The movement path display screen 1050 displays the movement path by connecting the job history information displayed on the job details history screen 1010 with curves on a map. One example of a curve is a Bézier curve. A Bézier curve is an N-1 type curve obtained from N control points.

[0110] The workflow display screen 1050 in Figure 31 displays points in the work process as circles. The display of circles is done by specifying the center coordinates, radius, color, and transparency. Work processes that the job has not passed through are hidden. In addition, the workflow display screen 1050 in Figure 31 can visually represent the job's workflow by connecting the center coordinates of the work processes that the job has passed through with a Bézier curve. In the workflow display screen 1050 in Figure 31, hovering the mouse over a circle may display information such as the process name and passage time (all if there are multiple) of the work process corresponding to that circle.

[0111] As described above, the work process management system 14 according to this embodiment allows for a list of progress information on the work processes of jobs in a printing factory. Furthermore, the work process management system 14 according to this embodiment allows for easy confirmation of the history information of the work processes of jobs in a printing factory, as well as captured image files and video files showing the situation when the work instruction sheet 810 was photographed, a dashboard display screen 1040, and a workflow display screen 1050.

[0112] [Second Embodiment] In the first embodiment, the work process management system 14 created work instructions 810 for the information processing system 12. In the second embodiment, the customer system 10 creates work instructions 810 for the information processing system 12. The second embodiment is identical to the first embodiment except for a few parts. Therefore, explanations of parts that are identical to the first embodiment will be omitted as appropriate.

[0113] <System Configuration> Figure 32 is a functional configuration diagram of another example of a work process management system. The work process management system 14a in Figure 32 is configured by removing the color-coded work instruction creation unit 38 and the print instruction unit 40 from the work process management system 14 in Figure 4. On the other hand, the customer system 10a has the color-coded work instruction creation unit 60 and the print instruction unit 62 added.

[0114] In the job management system 1 according to the second embodiment, the generation of the color code image 811 is performed by the work process management system 14a in the same manner as in the first embodiment, and steps S14 and S15 in Figure 6 are performed by the color code work instruction creation unit 60 and the print instruction unit 62 of the customer system 10.

[0115] The job management system 1 according to the second embodiment can also achieve the same effects as the job management system 1 according to the first embodiment.

[0116] [Third Embodiment] The first and second embodiments described above are technologies realized by a work instruction sheet 810 for an information processing system 12, which adds a new code (color code image 811) to the work instruction sheet 800 for the customer system 10. This technology can be applied to transport systems, such as AGVs (Automated Guided Vehicles).

[0117] For example, in a transport system for transporting goods, a work instruction sheet 810 for an information processing system 12, with a color code image 811 added, is attached to the goods, and the goods in transport are photographed by a camera 18, allowing the transport device to manage the work process of the goods being transported. Furthermore, since the position of the transport device transporting the goods and the destination of the goods can be identified, the transport system can also control the movement of the transport device transporting the goods.

[0118] [Fourth Embodiment] The first and second embodiments described above illustrate the management of job processes in a printing factory, but they can also be applied to the management of processes for items moving on a conveyor belt, for example. For example, if the technology of this embodiment is applied to the management of processes for items moving on a conveyor belt, it becomes possible to track the items moving on the conveyor belt and also to control the branching of the conveyor belt.

[0119] The present invention is not limited to the embodiments specifically disclosed above, and various modifications and changes are possible without departing from the scope of the claims. Color code image 811 is an example of a code image associated with a job described in the claims. Work instruction sheet 810 for an information processing system is an example of a form.

[0120] Camera 18 is an example of a shooting means. Color code recognition unit 44 is an example of a recognition means. Job management unit 34 is an example of a management means. UI unit 30 is an example of a provision means. Captured image file and captured video file are examples of captured data. Job status list screen 1000 is an example of a list screen. Job detail history screen 1010 is an example of a detail screen. [Explanation of symbols]

[0121] 1. Job Management System 10 Customer System 12 Information Processing Systems 14. Work Process Management System 16 Printers 18, 18a, 18b cameras 20 Networks 30 UI section 32 Job ID Detection Unit 34 Job Management Department 36. Color Code Image Generation Unit 38, 60 Work Instruction Sheet Creation Department with Color Codes 40, 62 Print instruction section 42 Image acquisition unit 44 Color code recognition unit 46 Color Code Management Table Storage Unit 48 Job Management Table Storage Unit 800 Work order for customer system 801 Barcode image 810 Work instructions for information processing systems 811 Color Code Image 1000 Job Status List Screen 1010 Job Details History Screen 1020 Screen for displaying captured images and videos. 1030 Job map display screen 1040 Dashboard display screen 1050 Flow line display screen [Prior art documents] [Patent Documents]

[0122] [Patent Document 1] Japanese Patent Publication No. 2005-100298

Claims

1. An information processing system, A storage means for storing the status of the work process, An acquisition means for acquiring images of the location captured by one or more cameras installed to capture the location corresponding to the work process, A recognition means for recognizing the color code identification information contained in the acquired image of the aforementioned location, An update means that, upon recognizing the aforementioned identification information, updates the status of the work process based on the aforementioned identification information, A means for providing the user with the updated status of the aforementioned work process, It has, The update means is an information processing system that, upon recognizing the identification information, updates the status of the work process using the identification information and the identification information of a camera that captured an image of the location including the color code.

2. The location corresponding to the aforementioned work process is the place where the printed material with the color code is placed. The information processing system according to claim 1.

3. The aforementioned color code represents the identification information using multiple colored cells. The information processing system according to claim 1 or 2.

4. The aforementioned multiple colored cells represent the identification information using multiple colors printed adjacent to each other. The information processing system according to claim 3.

5. The aforementioned work process includes a storage process of storing items in a storage location, and the location corresponding to the aforementioned work process includes the storage location. The information processing system according to any one of claims 1 to 4.

6. The status of the aforementioned work process includes the status of storing an item in the storage location and the status of retrieving an item from the storage location. The information processing system according to claim 5.

7. The information processing system is capable of displaying the image data that the camera has captured, which represents the image of the color code. The information processing system according to any one of claims 1 to 4.

8. The photographic data representing the capture of the aforementioned color code can be displayed via the status display screen of the work process. The information processing system according to claim 7.

9. The aforementioned display screen displays the updated status of the aforementioned work process in a list using display objects. The information processing system according to claim 8.

10. The providing means displays an alert on the status display screen of the work process if a predetermined time has elapsed without the status of the item being stored in the storage location being updated. The information processing system according to claim 6.

11. A progress management method performed in an information processing system, A storage means for storing the status of the work process, An acquisition step of acquiring an image of the location captured by one or more cameras installed to capture the location corresponding to the work process, A recognition step of recognizing the color code identification information contained in the acquired image of the location, An update step in which, upon recognition of the aforementioned identification information, the status of the work process is updated based on the aforementioned identification information, A provision step for providing the user with the updated status of the aforementioned work process, It has, The update step is a progress management method that, when the identification information is recognized, updates the status of the work process using the identification information and the identification information of the camera that took the image of the location including the color code.

12. Information processing system A storage means for storing the status of a work process. Acquisition means for acquiring images of the location captured by one or more cameras installed to capture the location corresponding to the aforementioned work process, Recognition means for recognizing color code identification information contained in the acquired image of the location, When the aforementioned identification information is recognized, an update means updates the status of the work process based on the aforementioned identification information. A means for providing the user with the updated status of the aforementioned work process, To make it function as, The update means is a program that, upon recognizing the identification information, updates the status of the work process using the identification information and the identification information of the camera that captured an image of the location including the color code.

13. An information processing device, A storage means for storing the status of the work process, An acquisition means for acquiring images of the location captured by one or more cameras installed to capture the location corresponding to the work process, A recognition means for recognizing the color code identification information contained in the acquired image of the aforementioned location, An update means that, upon recognizing the aforementioned identification information, updates the status of the work process based on the aforementioned identification information, A means for providing the user with the updated status of the aforementioned work process, It has, The update means is an information processing device that, upon recognizing the identification information, updates the status of the work process using the identification information and the identification information of a camera that captured an image of the location including the color code.

Images