Information processing method, information processing device, and program

Abstract

Provided is an information processing method capable of appropriately generating and outputting manufacturing rule information. An information processing device performs the following (1) to (6). (1) Acquiring image information obtained by imaging a situation in which a plurality of manufacturing steps are sequentially executed on a product. (2) Creating, on the basis of the image information, graphic information including information indicating the plurality of manufacturing steps to be executed on a product and information indicating an execution order of the plurality of manufacturing steps. (3) Displaying the graphic information on a display device. (4) Acquiring, from an input device, correction information for the graphic information inputted by the user. (5) Generating, on the basis of the graphic information and the correction information, manufacturing rule information including the plurality of manufacturing steps related to the product and a rule of an execution order of the plurality of manufacturing steps. (6) Outputting the manufacturing rule information.

Description

Information processing method, information processing device, and program

[0001] The present disclosure relates to an information processing method, an information processing device, and a program.

[0002] A work data management system according to the background art is disclosed in Patent Document 1. This work data management system classifies work performed by workers at a work site into multiple chronologically-series detailed processes based on work data including video data of the work site, and displays the video data and the chronologically-series detailed processes.

[0003] Japanese Patent No. 6824838

[0004] The work data management system according to the background art does not consider at all the generation and output of manufacturing rule information used in factory simulations such as CPS (Cyber-Physical System) that optimize manufacturing operations.

[0005] The present disclosure aims to provide an information processing method, an information processing device, and a program that can appropriately generate and output manufacturing rule information including multiple manufacturing processes related to a product and rules for the execution order of the multiple manufacturing processes, thereby enabling optimization of manufacturing operations through factory simulation using the manufacturing rule information.

[0006] An information processing method according to one aspect of the present disclosure is an information processing method for generating manufacturing rule information for a product manufactured through multiple manufacturing processes, in which an information processing device performs the following steps (1) to (6): (1) acquire image information obtained by photographing a situation in which the multiple manufacturing processes are performed on the product in sequence; (2) create, based on the image information, graphical information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution order of the multiple manufacturing processes; (3) display the graphical information on a display device; (4) acquire, from an input device, correction information for the graphical information input by a user; (5) generate, based on the graphical information and the correction information, manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution order of the multiple manufacturing processes; and (6) output the manufacturing rule information.

[0007] An information processing device according to another aspect of the present disclosure is an information processing device for generating manufacturing rule information for a product manufactured through multiple manufacturing processes, and performs the following operations: acquires image information obtained by photographing the situation in which the multiple manufacturing processes are performed on the product in sequence; creates, based on the image information, graphical information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution order of the multiple manufacturing processes; displays the graphical information on a display device; acquires, from an input device, correction information for the graphical information input by a user; generates, based on the graphical information and the correction information, manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution order of the multiple manufacturing processes; and outputs the manufacturing rule information.

[0008] According to the present disclosure, it is possible to appropriately generate and output manufacturing rule information including multiple manufacturing processes related to a product and rules for the execution order of the multiple manufacturing processes, and as a result, it is possible to optimize manufacturing operations through factory simulation using the manufacturing rule information.

[0009] 1 is a simplified diagram illustrating an overall configuration of an information processing system according to an embodiment of the present disclosure. FIG. 2 is a flowchart illustrating a flow of processing executed by an information processing device according to an embodiment of the present disclosure. FIG. 3 is a simplified diagram illustrating an example of analysis of image information. FIG. 4 is a simplified diagram illustrating an example of graphic information displayed on a display device. FIG. 5 is a simplified diagram illustrating an example of manufacturing rule information generated when no correction information is input. FIG. 6 is a simplified diagram illustrating an example of graphic information displayed on a display device after a deletion correction. FIG. 7 is a simplified diagram illustrating an example of manufacturing rule information generated when deletion correction information is input. FIG. 8 is a simplified diagram illustrating an example of graphic information displayed on a display device after a branch condition addition correction. FIG. 9 is a simplified diagram illustrating an example of manufacturing rule information generated when branch condition addition correction information is input. FIG. 10 is a simplified diagram illustrating an example of graphic information displayed on a display device after a division correction. FIG. 11 is a simplified diagram illustrating an example of manufacturing rule information generated when division correction information is input. FIG. 12 is a simplified diagram illustrating an example of graphic information displayed on a display device after an integration correction. FIG. 13 is a simplified diagram illustrating an example of manufacturing rule information generated when integration correction information is input. FIG. 1 is a simplified diagram showing an example of graphical information displayed on a display device. FIG. 2 is a simplified diagram showing an example of graphical information displayed on a display device after an addition correction. FIG. 3 is a simplified diagram showing an example of graphical information displayed on a display device after an integration correction. FIG. 4 is a simplified diagram showing an example of graphical information to which proposal information has been added. FIG. 5 is a simplified diagram showing another example of graphical information displayed on a display device. FIG. 6 is a simplified diagram showing another example of graphical information displayed on a display device. FIG. 7 is a simplified diagram showing another example of graphical information displayed on a display device. FIG. 8 is a flowchart showing the flow of processing of a modified example executed by an information processing device according to an embodiment of the present disclosure.

[0010] (Knowledge forming the basis of the present disclosure) In order to optimize manufacturing operations in a manufacturing factory, it is desirable to build a factory simulator using a CPS (Cyber-Physical System) etc. Building a highly accurate factory simulator makes it possible to discover manufacturing operation methods that improve production efficiency, verify resilience to fluctuations in demand or supply, or verify the load on manufacturing equipment or manufacturing processes when additionally manufacturing a new product or changing the manufacturing rules of an existing product, without affecting the operation of the actual factory.

[0011] However, not all manufacturing plants have accumulated or managed sufficient manufacturing rule information to build a factory simulator. In particular, it is difficult to obtain detailed information about processes or operations that are not mechanized, making it difficult to build a highly accurate factory simulator.

[0012] In order to solve this problem, the present inventors have come up with the idea of ​​the present disclosure based on the knowledge that by using image information obtained by photographing the situation in which multiple manufacturing processes are performed in sequence on a product, and also taking into account correction information input by a user such as an operator, it is possible to appropriately generate manufacturing rule information including multiple manufacturing processes related to the product and rules for the order in which the multiple manufacturing processes are performed.

[0013] Next, each aspect of the present disclosure will be described.

[0014] An information processing method according to a first aspect of the present disclosure is an information processing method for generating manufacturing rule information for a product manufactured through multiple manufacturing processes, in which an information processing device performs the following steps (1) to (6): (1) acquire image information obtained by photographing a situation in which the multiple manufacturing processes are performed on the product in sequence; (2) create, based on the image information, graphic information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution order of the multiple manufacturing processes; (3) display the graphic information on a display device; (4) acquire, from an input device, correction information for the graphic information input by a user; (5) generate, based on the graphic information and the correction information, manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution order of the multiple manufacturing processes; and (6) output the manufacturing rule information.

[0015] According to the first aspect, it is possible to appropriately generate and output manufacturing rule information including a plurality of manufacturing processes related to a product and rules for the execution order of the plurality of manufacturing processes based on graphic information and modification information input by a user. As a result, it is possible to optimize manufacturing operations through a factory simulation using the manufacturing rule information.

[0016] In the information processing method according to the second aspect of the present disclosure, in the first aspect, the correction information may include information for deleting or changing a manufacturing process included in the graphic information.

[0017] According to the second aspect, it is possible to appropriately generate manufacturing rule information based on modification information including information for deleting or changing a manufacturing process.

[0018] In the information processing method according to the third aspect of the present disclosure, in the first or second aspect, the modification information may include information that adds a branching condition from a manufacturing process included in the graphic information to multiple subsequent manufacturing processes.

[0019] According to the third aspect, it is possible to appropriately generate manufacturing rule information based on correction information including information for adding a branch condition.

[0020] An information processing method according to a fourth aspect of the present disclosure may be any one of the first to third aspects, in which the products include multiple products with different attributes, and the correction information includes information for dividing or integrating graphic information according to the attributes.

[0021] According to the fourth aspect, it is possible to appropriately generate manufacturing rule information based on correction information including information for dividing or integrating graphic information.

[0022] An information processing method according to a fifth aspect of the present disclosure is any one of the first to fourth aspects, in which the manufactured products include multiple manufactured products with different attributes, and in creating the graphic information, the multiple manufactured products are classified into multiple groups based on their attributes, and the graphic information is created for each group.

[0023] According to the fifth aspect, graphic information is created for each group, eliminating the need for the user to classify the groups.

[0024] In an information processing method according to a sixth aspect of the present disclosure, in any one of the first to fifth aspects, the image information may include manufacturing equipment that performed each manufacturing process, and the graphic information may further include information indicating at least one manufacturing equipment that performs each manufacturing process.

[0025] According to the sixth aspect, since the graphic information includes information indicating the manufacturing equipment, more detailed manufacturing rule information can be generated and output.

[0026] An information processing method according to a seventh aspect of the present disclosure is based on the sixth aspect, and it is preferable that the correction information includes information for adding manufacturing equipment capable of executing the manufacturing process included in the graphic information.

[0027] According to the seventh aspect, it is possible to appropriately generate manufacturing rule information based on correction information including information for adding manufacturing equipment.

[0028] An information processing method according to an eighth aspect of the present disclosure is the sixth or seventh aspect, wherein the correction information includes information for integrating a plurality of pieces of manufacturing equipment corresponding to the manufacturing process included in the graphic information.

[0029] According to the eighth aspect, it is possible to appropriately generate manufacturing rule information based on correction information including information for integrating a plurality of manufacturing facilities.

[0030] An information processing method according to a ninth aspect of the present disclosure may be any one of the sixth to eighth aspects, in which the manufactured products include a plurality of manufactured products with different attributes, and in creating the graphic information, proposal information is created based on the attributes to propose the addition of manufacturing equipment capable of executing the manufacturing process included in the graphic information, the proposal information is added to the graphic information, and the correction information includes information indicating whether or not to adopt the proposal.

[0031] According to the ninth aspect, since proposal information proposing the addition of manufacturing equipment is added to the graphic information, the user can easily perform the work of adding manufacturing equipment.

[0032] An information processing method according to a tenth aspect of the present disclosure is any one of the first to ninth aspects, in which the graphical information includes nodes representing each manufacturing process and links connecting multiple nodes, and in creating the graphical information, the display mode of the corresponding node is set based on the processing time required for each manufacturing process.

[0033] According to the tenth aspect, the display mode of the node is set based on the time required for processing in the manufacturing process, so that the user can easily determine the time required for processing in the manufacturing process.

[0034] An information processing method according to an eleventh aspect of the present disclosure is, in the tenth aspect, preferably further comprising the step of setting the display mode of the corresponding link based on the travel time of the product between manufacturing processes when creating the graphical information.

[0035] According to the eleventh aspect, the display mode of the link is set based on the time it takes for the product to travel between the manufacturing processes, so that the user can easily determine the time it takes for the product to travel between the manufacturing processes.

[0036] In an information processing method according to a twelfth aspect of the present disclosure, in any one of the first to eleventh aspects, the graphic information further includes processing time distribution information indicating a distribution of times required for processing in the manufacturing process. Furthermore, processing time setting information indicating a set value for the times required for processing in the manufacturing process set by a user based on the processing time distribution information is acquired from the input device, and the manufacturing rule information is generated based on the graphic information, the correction information, and the processing time setting information.

[0037] According to the twelfth aspect, the user can easily set a setting value for the required processing time based on the processing time distribution information. Also, the manufacturing rule information can be appropriately generated based on the graphic information, the correction information, and the processing time setting information.

[0038] In an information processing method according to a thirteenth aspect of the present disclosure, in the twelfth aspect, the graphic information further includes travel time distribution information indicating a distribution of times required for moving products between manufacturing processes. Furthermore, travel time setting information indicating a setting value for times required for moving the products between manufacturing processes, which is set by a user based on the travel time distribution information, is acquired from an input device, and in generating the manufacturing rule information, the manufacturing rule information may be generated based on the graphic information, the correction information, the processing time setting information, and the travel time setting information.

[0039] According to the thirteenth aspect, the user can easily set a setting value for the required time for movement based on the movement time distribution information. Also, the manufacturing rule information can be appropriately generated based on the graphic information, the correction information, the processing time setting information, and the movement time setting information.

[0040] The information processing method according to a fourteenth aspect of the present disclosure, in any one of the first to thirteenth aspects, may further create difference information indicating the difference between the manufacturing rule information and a manufacturing process performed within a predetermined period in the past, and display the difference information on the display device.

[0041] According to the fourteenth aspect, the difference information is displayed on the display device, so that the user can easily check the difference information.

[0042] An information processing device according to a fifteenth aspect of the present disclosure is an information processing device for generating manufacturing rule information for a product manufactured through multiple manufacturing processes, and acquires image information obtained by photographing a situation in which the multiple manufacturing processes are performed in sequence on the product. The information processing device creates graphical information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution order of the multiple manufacturing processes based on the image information. The information processing device displays the graphical information on a display device. The information processing device acquires correction information for the graphical information input by a user from an input device. The information processing device generates manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution order of the multiple manufacturing processes based on the graphical information and the correction information. The information processing device outputs the manufacturing rule information.

[0043] According to the fifteenth aspect, it is possible to appropriately generate and output manufacturing rule information including a plurality of manufacturing processes related to a product and rules for the execution order of the plurality of manufacturing processes based on graphic information and modification information input by a user. As a result, it is possible to optimize manufacturing operations by performing a factory simulation using the manufacturing rule information.

[0044] A sixteenth aspect of the present disclosure provides a program for causing an information processing device to execute a process for generating manufacturing rule information for a product manufactured through multiple manufacturing processes, the process acquiring image information obtained by photographing a situation in which the multiple manufacturing processes are performed on the product in sequence. The process creates, based on the image information, graphical information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution order of the multiple manufacturing processes. The process displays the graphical information on a display device. The process acquires, from an input device, correction information for the graphical information input by a user. The process generates, based on the graphical information and the correction information, manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution order of the multiple manufacturing processes. The process outputs the manufacturing rule information.

[0045] According to the sixteenth aspect, it is possible to appropriately generate and output manufacturing rule information including a plurality of manufacturing processes related to a product and rules for the execution order of the plurality of manufacturing processes based on graphic information and modification information input by a user. As a result, it is possible to optimize manufacturing operations by performing a factory simulation using the manufacturing rule information.

[0046] The present disclosure can also be realized as a program that causes a computer to execute each characteristic configuration included in such a method or apparatus, or as a system operated by this program. Needless to say, such a computer program can be distributed on a computer-readable non-transitory recording medium such as a CD-ROM or via a communication network such as the Internet.

[0047] (Embodiments of the Present Disclosure) Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Elements with the same reference numerals in different drawings indicate the same or corresponding elements. Furthermore, the components, the layout positions of the components, the connection forms, the order of operations, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. The present disclosure is limited only by the claims. Therefore, among the components in the following embodiments, components that are not described in the independent claims that represent the superordinate concept of the present disclosure are not necessarily required to achieve the object of the present disclosure, but are described as constituting a more preferred embodiment.

[0048] 1 is a simplified diagram illustrating the overall configuration of an information processing system 1 according to an embodiment of the present disclosure. The information processing system 1 may be a computer terminal or the like located in a manufacturing factory where products are manufactured. The products are manufactured through multiple manufacturing processes in the manufacturing factory. The manufacturing processes include a processing process, an assembly process, an inspection process, and the like.

[0049] The information processing system 1 includes an information processing device 11 , a storage device 12 , an input device 13 , a display device 14 , a communication device 15 , and an imaging device 16 .

[0050] The information processing device 11 includes a processor such as a CPU (Central Processing Unit). The storage device 12 includes a hard disk drive (HDD), a solid state device (SSD), or other semiconductor memory. The input device 13 includes a mouse, keyboard, or touch panel. The display device 14 includes a liquid crystal display or an organic electroluminescence (EL) display. The communication device 15 includes a communication module compatible with any communication standard such as Internet Protocol (IP). The imaging device 16 includes a camera installed on the ceiling or wall of the manufacturing factory or above a work space where workers work within the manufacturing factory. Multiple cameras may be installed within the manufacturing factory. Alternatively, a camera may be installed for each piece of manufacturing equipment. The camera may be a wide-angle camera or a 360-degree camera. The imaging device 16 captures images of multiple manufacturing processes being performed on a product in sequence and outputs image information 32 of moving images obtained by the capture. The image information 32 is stored in the storage device 12. The input device 13 and the display device 14 may be integrally configured as a touch panel display.

[0051] The storage device 12 stores a program 31, image information 32, and graphic information 40. The storage device 12 includes a computer-readable non-volatile storage medium, and the program 31 is stored in the storage medium.

[0052] The information processing device 11 has an acquisition unit 21, a processing unit 22, a display control unit 23, and an output unit 24 as functions realized by the information processing device 11 executing the program 31 read from the storage device 12. Details of the processing contents of these functions will be described later.

[0053] FIG. 2 is a flowchart showing the flow of processing executed by the information processing device 11.

[0054] First, in step S01 , the acquisition unit 21 acquires the image information 32 by reading it from the storage device 12 .

[0055] Next, in step S02 , the processing unit 22 creates graphic information 40 (see FIG. 4 , which will be described later) based on the image information 32 acquired in step S01 , and stores the graphic information 40 in the storage device 12 .

[0056] 3 is a simplified diagram showing an example of analysis of image information 32. The image information 32 is a plurality of pieces of image information 32 of a moving image. １ ~32 ３ The processing unit 22 includes image information 32 １ ~32 ３ By analyzing each of the above, the image information 32 １ ~32 ３ As a first example, the processing unit 22 applies image information 32 to a machine-learned convolutional neural network. １ ~32 ３ As a second example, the processing unit 22 inputs the image information 32 into the object detection model or the skeleton estimation model, and extracts the feature amount for each frame of the image by obtaining the result of the intermediate layer. １ ~32 ３ By inputting the above, feature quantities such as object type and its coordinates, or body part type and its coordinates, for each image frame are extracted. Then, the processing unit 22 performs clustering using the extracted feature quantities. As a third example, the processing unit 22 performs clustering by calculating the similarity between pairs of temporally adjacent images and defining division times for dividing the video based on the magnitude of the similarity.

[0057] The manufacturing process includes a manufacturing process performed using manufacturing equipment and a manufacturing process performed manually by an operator. １ is classified into a plurality of manufacturing processes (processes A, B, and C) in time series, and image information 32 ２ is classified into a plurality of manufacturing processes (processes A, D, and C) in time series, and image information 32 ３is classified into a plurality of manufacturing processes (processes A, B, and C) in chronological order. Each classified process may be a manufacturing process such as a processing process, an assembly process, or an inspection process. Alternatively, each classified process may be a work procedure included in one manufacturing process. For example, if the manufacturing process is a welding process, each classified process may be a work procedure for adjusting the position of the product, a work procedure for moving a welding machine to a predetermined position, or a work procedure for performing welding.

[0058] The image information corresponding to each process may include product identification information for identifying the product, manufacturing process identification information for identifying the manufacturing process performed on the product, manufacturing equipment identification information for identifying the manufacturing equipment that performed the manufacturing process, and time information indicating the time when the manufacturing process was performed. These pieces of information may be extracted from images captured by the image capture device 16. Alternatively, these pieces of information may be extracted from manufacturing history information in which log data indicating the operation history of the manufacturing equipment is accumulated as a database. Furthermore, these pieces of information may be extracted by reading tags such as RFIDs or two-dimensional codes attached to the products or workers.

[0059] The product identification information includes attribute information indicating the attributes of the product. The attribute information includes a lot ID indicating the lot number of the product and a product number ID indicating the product number. The attribute information may also include a worker ID identifying the worker. The manufacturing process identification information includes a process ID indicating the identification number of the manufacturing process. The manufacturing equipment identification information includes an equipment ID indicating the identification number of the manufacturing equipment. The time information includes a start time indicating the time when the manufacturing process started and an end time indicating the time when the manufacturing process ended.

[0060] Next, in step S03, the display control unit 23 causes the display device 14 to display the graphic information 40 created in step S02.

[0061] FIG. 4 is a simplified diagram showing an example of graphical information 40 displayed on the display device 14. The graphical information 40 includes information indicating a plurality of manufacturing processes to be performed on a product and information indicating the execution sequence of the plurality of manufacturing processes. In the example shown in FIG. 4, the graphical information 40 represents a directed graph including nodes representing each manufacturing process and links directly or indirectly connecting the plurality of nodes. Note that the graphical information 40 is not limited to a directed graph and may be a Gantt chart or the like.

[0062] 4, the nodes include a start node NS, multiple nodes NA to ND corresponding to multiple manufacturing processes (processes A to D), and an end node NE. The links include a link LSA extending from the start node NS to the node NA, a link LAB extending from the node NA to the node NB, a link LBC extending from the node NB to the node NC, a link LAD extending from the node NA to the node ND, a link LDC extending from the node ND to the node NC, and a link LCE extending from the node NC to the end node NE. The graphic information 40 also includes an icon 38 labeled "Edit" and an icon 39 labeled "Confirm."

[0063] Next, in step S04, the processing unit 22 determines whether or not the user has input correction information from the input device 13.

[0064] If the "Edit" icon 38 is not selected and the "Confirm" icon 39 is selected instead, this indicates that no correction information has been input by the user. If no correction information has been input (step S04: NO), then in step S06, the processing unit 22 generates manufacturing rule information including a plurality of manufacturing processes related to the product and rules for the execution order of the plurality of manufacturing processes, based on the graphic information 40.

[0065] 5 is a simplified diagram showing an example of manufacturing rule information 50 generated when no correction information is input. The manufacturing rule information 50 indicates a process path of "A, B, C" or a process path of "A, D, C." The process path of "A, B, C" corresponds to the links LAB and LBC shown in FIG. 4 and indicates a rule that multiple manufacturing processes are performed in the order of process A → process B → process C. The process path of "A, D, C" corresponds to the links LAD and LDC shown in FIG. 4 and indicates a rule that multiple manufacturing processes are performed in the order of process A → process D → process C.

[0066] Next, in step S07, the output unit 24 outputs the manufacturing rule information 50 generated in step S06. The output manufacturing rule information 50 may be input to a factory simulator using a CPS (Cyber-Physical System) or the like, or may be stored in any storage device. The manufacturing rule information 50 may be written in a programming language that can be input to a factory simulator, or may be in the data format of master data registered in a factory database or the like as management information for manufacturing rules. Alternatively, the manufacturing rule information 50 may be in the format of a work procedure manual used by workers to carry out manufacturing.

[0067] On the other hand, the user can select the “Edit” icon 38 and then edit the graphical information 40 by operating the input device 13. Editing the graphical information 40 indicates that the user has input edit information. Editing the graphical information 40 includes deleting or changing a manufacturing process included in the graphical information 40. Deleting a manufacturing process includes, for example, deleting process D. Changing a manufacturing process includes, for example, replacing process D with another process. In the case of an exceptional manufacturing history resulting from an unusual operation, such as the production of a sample product, deleting process D can remove it from the manufacturing rules. The user can delete or change the node ND by, for example, moving the mouse cursor over the node ND to be edited, right-clicking, and selecting “Delete” or “Edit” from the displayed menu. The following example describes a case where the user edits the node ND by deleting process D. When the user moves the mouse cursor over the node ND to be edited, the display control unit 23 may display a representative image showing a characteristic operation in the manufacturing process corresponding to the node ND on the display device 14. The representative image is created by the processing unit 22 based on the image information 32.

[0068] If correction information has been input (step S04: YES), then in step S05, the processing unit 22 acquires the correction information from the input device 13. The processing unit 22 executes the processes of steps S02 to S04 again in a manner that reflects the correction content.

[0069] 6 is a simplified diagram showing an example of the graphic information 41 displayed on the display device 14 after the deletion correction. As a result of the correction to delete the process D, the node ND and the links LAD and LDC have been deleted from the graphic information 40 shown in FIG.

[0070] When the "OK" icon 39 is selected after the corrections are completed, the processing unit 22 then generates manufacturing rule information in step S06 based on the graphic information 40 and the correction information, which includes multiple manufacturing processes related to the product and rules for the order in which the multiple manufacturing processes are executed.

[0071] 7 is a simplified diagram showing an example of manufacturing rule information 51 generated when modification information for deletion is input. As a result of the modification to delete process D, the process path "A, D, C" is deleted from the manufacturing rule information 50 shown in FIG. 5, and the manufacturing rule information 51 includes only the process path "A, B, C."

[0072] Next, in step S07, the output unit 24 outputs the manufacturing rule information 51 generated in step S06. The output manufacturing rule information 51 may be input to a factory simulator using a CPS or the like, or may be stored in any storage device. The manufacturing rule information 51 may be written in a programming language that can be input to a factory simulator, or may be in the data format of master data registered in a factory database or the like as management information for manufacturing rules. Alternatively, the manufacturing rule information 51 may be in the format of a work procedure manual used by workers to carry out manufacturing.

[0073] According to this embodiment, when modification information is input by the user, the information processing device 11 can appropriately generate and output manufacturing rule information 51 including multiple manufacturing processes related to the product and rules for the execution order of the multiple manufacturing processes, based on the graphic information 40 and the modification information. As a result, it becomes possible to optimize manufacturing operations by performing a factory simulation using the manufacturing rule information 51.

[0074] Furthermore, the information processing device 11 can appropriately generate the manufacturing rule information 51 based on the modification information including information for deleting or changing the manufacturing process.

[0075] Various modifications of the above embodiment will be described below, and the modifications described below can be applied in any combination.

[0076] (First Modification) The modification information may include information for adding a branching condition from a certain manufacturing process included in the graphic information 40 to a plurality of subsequent manufacturing processes. In the example of the graphic information 40 shown in Fig. 4, a branch from a node NA corresponding to a process A to a node NB corresponding to a process B or a node ND corresponding to a process D is included. For example, the user can add a branching condition from the node NA to the node ND by moving the mouse cursor over the link LAD to which the branching condition is to be added and then right-clicking to display a menu, thereby selecting "Add Branching Condition."

[0077] FIG. 8 is a simplified diagram showing an example of the graphic information 42 displayed on the display device 14 after correction by adding a branching condition. A branching condition "if process A. abnormal" has been added to the link LAD. This branching condition can be selected from a column included in the log data, or even if it is not included in the log data, new information associated with the log data, such as a lot or equipment, can be added. This branching condition means that if process A is abnormal, branch from node NA to node ND.

[0078] 9 is a simplified diagram showing an example of manufacturing rule information 52 generated when correction information for adding a branch condition is input. As a result of a correction for adding a branch condition from node NA to node ND, the output unit 24 outputs manufacturing rule information 52 indicating that if an abnormality occurs in process A, the process path "A, D, C" is adopted, and otherwise the process path "A, B, C" is adopted.

[0079] According to this modification, the information processing device 11 can appropriately generate the manufacturing rule information 52 based on the correction information including information for adding a branch condition.

[0080] (Second Modification) The correction information may include information for dividing or integrating the graphic information 40 according to the attributes of the product. An example in which the attribute is a product number ID will be described below. However, the attribute may also be a lot ID, a manufacturing date and time, etc. The graphic information 40 shown in FIG. 4 includes information related to all product number IDs. The user can divide the graphic information 40 by product number ID, for example, by moving the mouse cursor over the start node NS and right-clicking in that state to display a menu and selecting "Split by product number."

[0081] 10 is a simplified diagram showing an example of graphic information 43 displayed on display device 14 after the division has been corrected. Graphic information 43 includes graphic information 43X corresponding to product number X, graphic information 43Y corresponding to product number Y, and graphic information 43Z corresponding to product number Z. In other words, graphic information 40 is divided into graphic information 43X, 43Y, and 43Z according to the product number ID.

[0082] 11 is a simplified diagram showing an example of manufacturing rule information 53 generated when division correction information is input. By dividing the graphic information 40 by product number ID, the output unit 24 outputs manufacturing rule information 53 indicating that the process route "A, B, C" is adopted for product number X, the process route "A, D, C" is adopted for product number Y, and the process route "A, B, C" is adopted for product number Z.

[0083] 10, the graphic information 43X corresponding to product number X is the same as the graphic information 43Z corresponding to product number Z. The user can integrate the graphic information 43Z with the graphic information 43X, for example, by specifying the range of the graphic information 43Z with a mouse and then dragging and dropping the graphic information 43Z onto the graphic information 43X.

[0084] 12 is a simplified diagram showing an example of the graphic information 44 displayed on the display device 14 after the integration correction. The graphic information 44 corresponds to the group G1 including the part numbers X and Z. １ and graphic information 44 corresponding to group G2 including product number Y. ２That is, the graphic information 43X and 43Z are included in the graphic information 44 according to the product number ID. １ has been integrated into.

[0085] 13 is a simplified diagram showing an example of manufacturing rule information 54 generated when integration correction information is input. １ As a result of the integration, the output unit 24 outputs manufacturing rule information 54, which means that the process route of "A, B, C" is adopted for the group G1 and the process route of "A, D, C" is adopted for the group G2.

[0086] For groups in which there is a large difference between the part number properties, etc. and the manufacturing rule information 53, 54, the processing unit 22 may display an alert on the display device 14 when displaying the graphic information 43, 44. For example, an alert may be displayed when data on part number properties, such as the types of parts to be assembled or parameters of the manufacturing process, are separately stored and parts are divided into different groups despite similar properties. Alternatively, an alert may be displayed when parts are integrated into the same group despite significantly different properties. The user may accept or delete the displayed alert.

[0087] According to this modification, the manufacturing rule information 53 can be appropriately generated based on the modification information including information for dividing the graphic information 40 according to the attributes of the product. Also, the simplified manufacturing rule information 54 can be appropriately generated based on the modification information including information for integrating multiple pieces of graphic information 43 according to the attributes of the product. This is particularly effective in a manufacturing factory with a large variety of products.

[0088] (Third Modification) In the second modification, the graphic information is divided or integrated by the user's operation. However, in this modification, the processing unit 22 classifies a plurality of products into a plurality of groups G1 and G2 based on the attribute information, and generates graphic information 44 for each of the groups G1 and G2 instead of the graphic information 40. １ , 44 ２ may be automatically created and suggested to the user.

[0089] The processing unit 22 classifies multiple product numbers into multiple groups based on the similarity of the graph structures in the graphic information 43 for each product number. As a first example, the processing unit 22 calculates, as the similarity, the number of operations required to convert a directed graph corresponding to a certain product number into a directed graph corresponding to another product number. The processing unit 22 then classifies multiple product numbers into multiple groups by performing clustering using a similarity matrix. As a second example, the processing unit 22 vectorizes the process path for each product number using a vector that indicates, with a binary value of "0" or "1," whether or not each process is included in the process path. The processing unit 22 then classifies multiple product numbers into multiple groups by performing clustering using the vectorized process path.

[0090] The user selects the graphic information 44 for each of the groups G1 and G2 proposed by the processing unit 22. １ , 44 ２ The user inputs whether or not to adopt the graphic information 44 for each of the groups G1 and G2 proposed by the processing unit 22 by operating the input device 13. １ , 44 ２ may be further modified by the division or integration operations described in the second modification.

[0091] According to this modification, the information processing device 11 processes the graphic information 44 for each of the groups G1 and G2. １ , 44 ２ This automatically creates groups, eliminating the need for users to perform group classification operations, improving user convenience. This is particularly effective in manufacturing plants with a wide variety of products.

[0092] (Fourth Modification) In the embodiment, the graphic information 40 includes nodes NA to ND representing each manufacturing process, but instead of the nodes NA to ND, it may include a node indicating at least one piece of manufacturing equipment that performs each manufacturing process.

[0093] 14 is a simplified diagram showing an example of graphic information 45 displayed on display device 14. Processing unit 22 creates graphic information 45 by identifying manufacturing equipment based on manufacturing equipment identification information included in image information corresponding to each process. Graphic information 45 includes nodes NA1 and NA2 indicating manufacturing equipment (equipment U1 and U2) that performs process A, nodes NB1 to NB3 indicating manufacturing equipment (equipment V1 to V3) that performs process B, and nodes NC1 and NC2 indicating manufacturing equipment (equipment W1 and W2) that performs process C. Graphic information 45 also includes links connecting nodes NA1 and NA2, nodes NB1 to NB3, and nodes NC1 and NC2 as needed.

[0094] In this modification, the modification information may include information for adding manufacturing equipment capable of executing the manufacturing process. After selecting the "Modify" icon 38, the user can modify the graphic information 45 by operating the input device 13. For example, the user can add a node and a link by moving the mouse cursor to the location where the user wants to add a node, right-clicking, and selecting "Add" from the displayed menu.

[0095] 15 is a simplified diagram showing an example of the graphical information 46 displayed on the display device 14 after the additional correction. A node NB4 corresponding to equipment V4 capable of executing process B has been added to the graphical information 45 shown in FIG. 14. Furthermore, a link LU2V4 connecting the node NA2 and the node NB4, and a link LV4W2 connecting the node NB4 and the node NC2 have been added to the graphical information 45 shown in FIG.

[0096] The modification information may also include information for integrating multiple pieces of manufacturing equipment included in the graphic information 46. In the example shown in Fig. 15, equipment V3 and V4 share the upstream equipment U2 and the downstream equipment W2. The user can integrate node NB4 into node NB3, for example, by specifying the range of node NB4 with a mouse and then dragging and dropping it onto node NB3.

[0097] 16 is a simplified diagram showing an example of graphic information 47 displayed on the display device 14 after the integration correction. The graphic information 47 includes a node NB34 formed by integrating the nodes NB3 and NB4 shown in FIG. 15, a link LU2V34 formed by integrating the links LU2V3 and LU2V4 shown in FIG. 15, and a link LV34W2 formed by integrating the links LV3W2 and LV4W2 shown in FIG. 15.

[0098] According to this modification, the graphic information 45 includes information indicating the manufacturing equipment, so the information processing device 11 can generate and output more detailed manufacturing rule information that includes the order of the manufacturing equipment instead of the manufacturing process.

[0099] Furthermore, according to this modification, it is possible to appropriately generate manufacturing rule information corresponding to the graphic information 46 based on the correction information including information on adding manufacturing equipment.

[0100] Furthermore, according to this modification, simplified manufacturing rule information corresponding to the graphic information 47 can be appropriately generated based on the modification information including information for integrating a plurality of pieces of manufacturing equipment. This is particularly effective in a manufacturing factory with many types of manufacturing equipment.

[0101] (Fifth Modification) In the fourth modification, node NB4 is added by user operation, but the processing unit 22 may automatically create manufacturing equipment capable of executing the manufacturing process included in the graphic information 45 based on the attribute information and manufacturing equipment identification information and propose it to the user.

[0102] The processing unit 22 creates proposal information proposing the addition of manufacturing equipment capable of executing the manufacturing process included in the graphic information 45 based on the similarity between the product numbers and the similarity between the manufacturing equipment, and adds the proposal information to the graphic information 45. For example, when equipment V3 is being used for a certain product number, the processing unit 22 determines that equipment V4, which has a similarity with equipment V3 equal to or greater than a predetermined value, is also likely to be usable for that product number, and creates proposal information proposing the addition of equipment V4.

[0103] 17 is a simplified diagram showing an example of graphical information 48 in which proposal information has been added to graphical information 45. Graphic information 48 includes proposal information shown by dashed lines in FIG. 17. The proposal information includes node NB4 corresponding to facility V4. The proposal information also includes link LU2V4 connecting node NA2 and node NB4, and link LV4W2 connecting node NB4 and node NC2. The user can input, as correction information, from input device 13 whether or not to adopt the proposal shown in the proposal information.

[0104] According to this modification, the graphic information 48 contains proposal information that suggests adding manufacturing equipment, which makes it easier for the user to add manufacturing equipment, thereby improving user convenience.

[0105] (Sixth Modification) When creating the graphic information 40, the processing unit 22 may set different display modes for corresponding nodes based on the time required for processing in each manufacturing process. Furthermore, when creating the graphic information 40, the processing unit 22 may set different display modes for corresponding links based on the travel time of the product between manufacturing processes. The travel time includes the transport time for transporting the product between manufacturing devices and the waiting time or residence time until processing by the manufacturing device is started after the transport is completed.

[0106] 18 is a simplified diagram showing an example of the graphic information 40 displayed on the display device 14. The longer the processing time required for each manufacturing process, the thicker the line width of the edge at the corresponding node is displayed. Also, the longer the transportation time of the product between manufacturing processes, the thicker the line width of the corresponding link is displayed. Note that the processing unit 22 may set the display mode of the node or link to be different depending on the occurrence frequency, etc., instead of the required time. Also, the processing unit 22 may set the display mode to be different depending on the display color, etc., instead of the line width.

[0107] According to this modified example, the display mode of the node is set based on the required time or frequency of occurrence of the process in the manufacturing process, so that the user can easily visually determine the required time or frequency of occurrence of the process in the manufacturing process.

[0108] Furthermore, since the display mode of the link is set based on the travel time or occurrence frequency of the product between the manufacturing processes, the user can easily visually determine the travel time or occurrence frequency of the product between the manufacturing processes.

[0109] (Seventh Modification) The graphic information 40 may further include processing time distribution information that indicates the distribution of the time required for processing in each manufacturing process.

[0110] 19 and 20 are simplified diagrams illustrating another example of the graphic information 40 displayed on the display device 14. When the user moves the mouse cursor over a node ND, for example, the processing unit 22 generates processing time distribution information K1 for the process D corresponding to the node ND, as shown in FIG. 19 . The display control unit 23 displays the generated processing time distribution information K1 as a pop-up on the display device 14. The horizontal axis of the processing time distribution information K1 indicates time, and the vertical axis indicates occurrence frequency. When the user inputs a product number specification, the processing unit 22 generates processing time distribution information K2, which indicates a distribution of the processing time required for each specified product number, as shown in FIG. 20 . The display control unit 23 displays the generated processing time distribution information K2 as a pop-up on the display device 14. The processing time distribution information K2 shown in FIG. 20 includes two pieces of distribution information for two product numbers. The user inputs representative values ​​for each piece of distribution information as the setting values ​​V1 and V2 for the processing time required. The representative value may be the average value, the mode value, the median value, or the top m% (m is arbitrary) values, and the position of each value may be specified on the screen by user operation, or each value may be calculated by statistical processing by the processing unit 22.

[0111] The acquisition unit 21 acquires processing time setting information indicating the setting values ​​V1 and V2 from the input device 13. The processing unit 22 generates manufacturing rule information based on the graphic information 40, the correction information, and the processing time setting information. The manufacturing rule information may include processing time distribution information K2. The processing time distribution information K2 included in the manufacturing rule information may be defined using a function formula representing a normal distribution approximated by maximum likelihood estimation or the like.

[0112] Similarly, the graphic information 40 may further include movement time distribution information that indicates the distribution of the time required for moving products between manufacturing processes.

[0113] FIG. 21 is a simplified diagram illustrating another example of the graphic information 40 displayed on the display device 14. When the user moves the mouse cursor over a link LAD, for example, the processing unit 22 generates travel time distribution information corresponding to the link LAD. When the user inputs a product number, the processing unit 22 generates travel time distribution information K3, which indicates the distribution of travel times for each specified product number, as shown in FIG. 21. The display control unit 23 displays the generated travel time distribution information K3 as a pop-up on the display device 14. The travel time distribution information K3 shown in FIG. 21 includes two pieces of distribution information for two product numbers. The user inputs representative values ​​for each piece of distribution information as set values ​​V3 and V4 for travel times. The representative value may be the average, mode, median, or the top m% (m is arbitrary) value. The position of each value may be specified on the screen by the user's operation, or each value may be calculated by statistical processing by the processing unit 22.

[0114] The acquisition unit 21 acquires travel time setting information indicating the setting values ​​V3 and V4 from the input device 13. The processing unit 22 generates manufacturing rule information based on the graphic information 40, the correction information, the processing time setting information, and the travel time setting information. The manufacturing rule information may include travel time distribution information K3. The travel time distribution information K3 included in the manufacturing rule information may be defined using a function formula representing a normal distribution approximated by maximum likelihood estimation or the like.

[0115] According to this modification, the user can easily set the set values ​​V1 and V2 related to the required processing times based on the processing time distribution information K2. Furthermore, the information processing device 11 can appropriately generate manufacturing rule information based on the graphic information 40, the correction information, and the processing time setting information.

[0116] Furthermore, according to this modification, the user can easily set the set values ​​V3 and V4 related to the travel time based on the travel time distribution information K3. Furthermore, the information processing device 11 can appropriately generate manufacturing rule information based on the graphic information 40, the correction information, the processing time setting information, and the travel time setting information.

[0117] Furthermore, according to this modified example, the manufacturing rule information includes set values ​​for the processing time required for each product number in each manufacturing process, and set values ​​for the travel time required for each product number between manufacturing processes, making it possible to build a highly accurate factory simulator that includes the processing time and travel time required, which are essential for simulation.

[0118] (Eighth Modification) The processing unit 22 may create difference information indicating the difference between the manufacturing rule information and the manufacturing process performed within a predetermined period in the past, and the display control unit 23 may display the created difference information on the display device 14.

[0119] FIG. 22 is a flowchart showing the flow of processing in the modified example executed by the information processing device 11.

[0120] In step S08, the processing unit 22 compares the manufacturing rule information created in step S06 with the log data of the manufacturing process executed within the most recent predetermined period. The processing unit 22 extracts log data that violates the manufacturing rule information and creates difference information that summarizes the extracted log data in a list format.

[0121] Next, in step S09, the display control unit 23 causes the display device 14 to display the difference information created in step S08.

[0122] According to this modification, the difference information is displayed on the display device 14, so that the user can easily check the difference information. As a result, it becomes possible to determine the cause of the violation for log data that violates the manufacturing rule information.

[0123] (Ninth Modification) The correction information may include information for changing notation such as "Process A" included in the graphic information 40 to the actual name of each process such as "Work for removing jigs and work-in-progress" or "Processing using jigs." This makes it easier for the user to intuitively understand.

[0124] (Tenth Modification) The correction information may include information for dividing or integrating steps in the graphic information 40. The graphic information 40 is constructed based on the clusters proposed by the processing unit 22.

[0125] For example, the user moves the mouse cursor over node NA, and then right-clicks to display a menu, selects "Split Process" from the menu, and selects the time of division, thereby dividing process A of the graphic information 40 into two processes. At this time, the processing unit 22 may further cluster the data of each image included in process A and present the user with a position suitable for division.

[0126] Alternatively, the user may move the mouse cursor over node NA, right-click, select "Merge Processes" from the displayed menu, and then select another process (e.g., process B) as the process to be merged, thereby merging process A and process B in the graphic information 40 to create a new process A. At this time, the processing unit 22 may present the user with the similarity between process A and the other process. This similarity is calculated, for example, based on the similarity between the central images of each process.

[0127] According to this modification, even if the proposed clustering result is inaccurate, the inaccurate clustering result can be corrected, and as a result, accurate manufacturing rule information for each process can be output.

[0128] (Eleventh Modification) The output unit 24 may have a function of outputting image data for each process unit corrected by the user in the tenth modification. Furthermore, the processing unit 22 may perform clustering on new data using a machine learning model trained using this image data as training data.

[0129] According to this modification, the accuracy of clustering can be improved by using a machine learning model that has been trained using the target image data itself.

[0130] As another modification, the display control unit 23 may display the programming code of the manufacturing rule information on the display device 14. This makes it easier for developers who are familiar with programming to intuitively understand the code.

[0131] The present disclosure is widely applicable to manufacturing management systems for building factory simulators and the like.

[0132] REFERENCE SIGNS LIST 1 Information processing system 11 Information processing device 12 Storage device 13 Input device 14 Display device 15 Communication device 16 Imaging device 21 Acquisition unit 22 Processing unit 23 Display control unit 24 Output unit 31 Program 32, 32 １ , 32 ２ , 32 ３ Image information 40-48 Graphic information 50-54 Manufacturing rule information

Claims

1. An information processing method for generating manufacturing rule information for a product that is manufactured through multiple manufacturing processes, comprising the steps of: an information processing device acquiring image information obtained by photographing the situation in which the multiple manufacturing processes are performed in sequence on the product; creating, based on the image information, graphical information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution order of the multiple manufacturing processes; displaying the graphical information on a display device; acquiring, from an input device, correction information for the graphical information input by a user; generating, based on the graphical information and the correction information, manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution order of the multiple manufacturing processes; and outputting the manufacturing rule information.

2. The information processing method according to claim 1, wherein the modification information includes information for deleting or changing a manufacturing process included in the graphic information.

3. The information processing method according to claim 1, wherein the modification information includes information for adding a branching condition from a manufacturing process included in the graphic information to a plurality of subsequent manufacturing processes.

4. The information processing method according to claim 1, wherein the manufactured products include a plurality of manufactured products with different attributes, and the correction information includes information for dividing or integrating the graphic information according to the attributes.

5. The information processing method according to claim 1, wherein the manufactured products include a plurality of manufactured products with different attributes, and in creating the graphic information, the plurality of manufactured products are classified into a plurality of groups based on the attributes, and the graphic information is created for each group.

6. The information processing method according to claim 1, wherein the image information includes manufacturing equipment that performed each manufacturing process, and the graphic information further includes information indicating at least one piece of manufacturing equipment that performed each manufacturing process.

7. The information processing method according to claim 6, wherein the correction information includes information for adding manufacturing equipment capable of executing the manufacturing process included in the graphic information.

8. The information processing method according to claim 6, wherein the correction information includes information for integrating a plurality of pieces of manufacturing equipment corresponding to the manufacturing process included in the graphic information.

9. The information processing method according to claim 6, wherein the manufactured products include a plurality of manufactured products with different attributes, and in creating the graphic information, proposal information is created based on the attributes to propose the addition of manufacturing equipment capable of carrying out the manufacturing process included in the graphic information, and the proposal information is added to the graphic information, and the correction information includes information indicating whether or not the proposal is to be adopted.

10. An information processing method as described in claim 1, wherein the graphical information includes nodes representing each manufacturing process and links connecting multiple nodes, and in creating the graphical information, the display mode of the corresponding node is set based on the processing time required for each manufacturing process.

11. The information processing method according to claim 10, further comprising the step of: in creating said graphic information, setting a display mode for a corresponding link based on the travel time of said product between manufacturing processes.

12. The information processing method of claim 1, wherein the graphic information further includes processing time distribution information indicating a distribution of the time required for processing in the manufacturing process, and further, processing time setting information indicating a setting value for the time required for processing in the manufacturing process set by a user based on the processing time distribution information is obtained from the input device, and in generating the manufacturing rule information, the manufacturing rule information is generated based on the graphic information, the correction information, and the processing time setting information.

13. The information processing method of claim 12, wherein the graphic information further includes travel time distribution information indicating a distribution of the time required for the product to travel between manufacturing processes, and further, travel time setting information indicating a setting value for the time required for the product to travel between manufacturing processes set by a user based on the travel time distribution information is acquired from the input device, and in generating the manufacturing rule information, the manufacturing rule information is generated based on the graphic information, the correction information, the processing time setting information, and the travel time setting information.

14. The information processing method according to claim 1, further comprising: creating difference information indicating the difference between the manufacturing rule information and a manufacturing process performed within a predetermined period in the past; and displaying the difference information on the display device.

15. An information processing device for generating manufacturing rule information for a product manufactured through multiple manufacturing processes, the information processing device acquiring image information obtained by photographing a situation in which multiple manufacturing processes are performed in sequence on the product, creating graphic information based on the image information, including information indicating the multiple manufacturing processes to be performed on the product and information indicating the order in which the multiple manufacturing processes are to be performed, displaying the graphic information on a display device, acquiring correction information for the graphic information input by a user from an input device, generating manufacturing rule information based on the graphic information and the correction information, including the multiple manufacturing processes for the product and rules for the order in which the multiple manufacturing processes are to be performed, and outputting the manufacturing rule information.

16. A program for causing an information processing device to execute processing to generate manufacturing rule information for a product manufactured through multiple manufacturing processes, the processing comprising: acquiring image information obtained by photographing the situation in which multiple manufacturing processes are performed on the product in sequence; creating, based on the image information, graphic information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution sequence of the multiple manufacturing processes; displaying the graphic information on a display device; acquiring correction information for the graphic information input by a user from an input device; generating, based on the graphic information and the correction information, manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution sequence of the multiple manufacturing processes; and outputting the manufacturing rule information.

Images