Information processing device and information processing method

Abstract

[Subject] An information processing device is provided that can prompt an operator with the amount of information required by the operator when the operator needs it, without reconstructing large-scale information. [Solution] An information processing device is provided that outputs information related to a first operation and a second operation performed by an operator on an object as an operation object, namely, operation information, wherein the information processing device comprises: a scene estimation unit that obtains a first image, which is an image of a scene in which the operator performs the first operation and the second operation, and the scene estimation unit estimates the scene using a first learning completion model that records the association between the first image and a scene ID that uniquely represents the scene; a data block estimation unit that obtains an image of the object of the first operation and the second operation, namely, a second image, and estimates the scene using one of a plurality of second learning completion models that stores the association between the second image and one or more data block meta-IDs as follows The data block, the one or more data blocks correspond one-to-one with a data block ID that uniquely represents the data block, and the data block is information after the operation information is divided or prompts the operation information; a migration target prompting unit, which compares the actually performed first operation and the second operation with a state transition table that pre-stores the relationship between the first operation and the second operation when performing the second operation, and outputs prompt information, which prompts the next operation; and an output unit, which outputs the data block, the data block estimation unit uses a model ID that corresponds one-to-one to a scene ID to select one of the multiple second learning completion models, and the data block meta ID uniquely represents a data block meta value as information related to the properties of the object.

Description

Technical Field

[0001] The present invention relates to an information processing device and an information processing method. Background Art

[0002] For example, in the work assistance system of patent document 1, rules that describe the judgment conditions of the work object or work status are generated based on a manual that describes the process, content, precautions or other matters of the work, the work object and work status are identified based on sensor information from the equipment worn by the worker, and the work assistance information is output based on the generated rules and the recognition results of the recognition unit.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-109844 Summary of the Invention

[0006] Problems to be solved by the invention

[0007] However, the conventional method described in Patent Document 1 only allows information stored in documents such as manuals to be retrieved on a per-document basis. For example, when searching for documents on a paragraph-by-paragraph basis, the documents must be reconstructed into structured information. Considering cost and benefit, reconstructing all documents to be searched is often impractical. Furthermore, there are problems with searching for information on a per-document basis, as a large amount of unnecessary information may be viewed, and document viewers may not be able to respond quickly.

[0008] An object of one aspect of an embodiment of the present invention is to provide an information processing device that presents an operator with information of an amount necessary for the operator when the operator needs it, without reconstructing the information on a large scale.

[0009] Means for solving problems

[0010] An information processing device that outputs information related to a first operation and a second operation performed by an operator on an object as an operation object, namely, operation information, wherein the information processing device comprises: a scene estimation unit that obtains a first image, which is an image of a situation in which the operator performs the first operation and the second operation, namely, an image of a scene, and estimates the scene using a first learning completion model that records the association between the first image and a scene ID that uniquely represents the scene; and a data block estimation unit that obtains an image of the object of the first operation and the second operation, namely, a second image, and estimates the scene using one of a plurality of second learning completion models that stores the association between the second image and one or more data block meta-IDs as follows: A data block, one or more data blocks correspond one-to-one to a data block ID that uniquely represents the data block using a meta-ID, and the data block is information after the job information is divided or prompts the job information; a migration target prompting unit, which, when performing the second job, compares the actually performed first job and the second job with a state migration table that pre-stores the relationship between the first job and the second job, and outputs prompt information, which provides a prompt related to the next job; and an output unit, which outputs the data block, a data block estimation unit uses a model ID that corresponds one-to-one to a scene ID to select one of multiple second learning completion models, and the data block meta-ID uniquely represents a data block meta-value as information related to the properties of the object.

[0011] Provided is an information processing method, which is executed by an information processing device, the information processing device outputting information related to a first operation and a second operation performed by an operator on an object as an operation object, namely, operation information, wherein the information processing method has the following steps: a first step of obtaining a first image, which is an image of a situation in which the operator performs the first operation and the second operation, namely, an image of a scene, and in the first step, using a first learning completion model that records the association between the first image and a scene ID that uniquely represents the scene to estimate the scene; a second step of obtaining an image of the object of the first operation and the second operation, namely, a second image, using a plurality of second learning completion models that store the association between the second image and one or more data block meta-IDs as follows A data block is estimated based on one of the bundle models, one or more data blocks are corresponded one-to-one with a data block ID that uniquely represents the data block using a meta ID, and the data block is information after the job information is divided or prompts the job information; in the third step, when the second job is performed, the first job and the second job actually performed are compared with the state transition table that pre-stores the relationship between the first job and the second job, and prompt information is output, which prompts the next job; and in the fourth step, the data block is output. In this information processing method, a model ID that corresponds one-to-one to a scene ID is used to select one of the multiple second learning completion models, and the data block is uniquely represented by a meta value as information related to the properties of the object using a meta ID.

[0012] Effects of the Invention

[0013] According to one aspect of the present invention, an information processing device can be realized that presents an operator with a required amount of information when the operator needs it, without reconstructing the information on a large scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a block diagram showing the configuration of the information processing device in the use phase of this embodiment.

[0015] Figure 2 This is a block diagram showing the configuration of the information processing device in the learning phase of this embodiment.

[0016] Figure 3 These are diagrams showing the first image and the second image during the removal operation and the packaging operation, respectively, according to the present embodiment.

[0017] Figure 4 1 and 2 are diagrams showing a first learning completion model and a second learning completion model according to the present embodiment.

[0018] Figure 5 This is a diagram showing information stored in the auxiliary storage device according to this embodiment.

[0019] Figure 6 This is a timing chart for explaining the scene estimation function, data block estimation function, and data block output function of this embodiment.

[0020] Figure 7 This is a sequence diagram for explaining the first learning completion model generation function and the second learning completion model generation function of the present embodiment.

[0021] Figure 8 This is a flowchart showing the processing procedure of information processing in the use phase of this embodiment.

[0022] Figure 9 This is a flowchart showing the processing procedure of the confirmation process in this embodiment. DETAILED DESCRIPTION

[0023] The following describes one embodiment of the present invention in detail using the accompanying drawings. For example, information referenced by an operator at a logistics hub, etc., will be described. The operator performs the following operations: an operation of removing an object to be operated (hereinafter, also referred to as the first operation) and a packaging operation of a new object by packaging the object in the first operation (hereinafter, also referred to as the second operation). The operators in the first and second operations may be different, or the same operator may be used for both operations.

[0024] (Present embodiment)

[0025] First, use Figure 1 The information processing apparatus 1 in the use phase will be described. Figure 1 1 is a block diagram showing the configuration of the information processing device 1 in the use phase of the present embodiment. The information processing device 1 includes a central processing unit 2 , a main storage device 3 , and an auxiliary storage device 11 .

[0026] The central processing unit 2 is, for example, a CPU (Central Processing Unit), and executes processing by calling programs stored in the main storage device 3. The main storage device 3 is, for example, a RAM (Random Access Memory), and stores programs such as the scene estimation unit 4, the data block estimation unit 5, the data block output unit 6, the confirmation unit 7, the first learning completion model generation unit 9, and the second learning completion model generation unit 10, which will be described later.

[0027] In addition, the program including the scene estimation unit 4, the data block estimation unit 5, the data block output unit 6 and the confirmation unit 7 can be referred to as the control unit 15, and the program including the first learning completion model generation unit 9 and the second learning completion model generation unit 10 can be referred to as the learning completion model generation unit 16.

[0028] The auxiliary storage device 11 is, for example, an SSD (Solid State Drive) or an HDD (Hard Disk Drive), and stores databases such as the first learning completion model DB1, the first learning model DB1', the second learning completion model DB2, and the second learning model DB2' described later, as well as tables such as the scene table TB1, the model table TB2, the content table TB3, the scene / content table TB4, the content / data block table TB5, the data block / meta table TB6, the data block table TB7, the data block meta table TB8, and the state transition table TB9.

[0029] like Figure 1 As shown, an information processing device 1 that outputs information related to a job performed by a worker, namely, job information, comprises, during the use phase, a scene estimation unit 4, a data block estimation unit 5, a data block output unit 6 that outputs data blocks (chunks) representing the job information being segmented or displayed, and a confirmation unit 7. Here, the job information may also be referred to as content, and a content ID uniquely identifies the job information. The data block output unit 6 outputs the data blocks, for example, to a user terminal 12.

[0030] The scene estimation unit 4 estimates a scene as a situation in which the operator is performing. Specifically, the scene estimation unit 4 obtains first images 20 and 40 (described later) during the unloading operation and the packaging operation, and estimates the scene using the first learning model DB 1 that stores the association between the first image 20 and a scene ID that uniquely represents the scene.

[0031] The scene estimation unit 4 uses the scene ID as a search keyword, obtains the scene name from the scene table TB1, which is a table that associates scene IDs with scene names in a one-to-one manner, and sends it to the user terminal 12. The user terminal 12 presents the scene name received from the scene estimation unit 4 to the operator.

[0032] Furthermore, the scene estimation unit 4 determines whether the image is being taken out or being packed by, for example, referring to the temporally consecutive images preceding and following the first images 20 and 40. For example, in the temporally preceding image of the first image 20, the box 21 in which the object 23, described later, is packed is captured, while the temporally following image of the first image 20 captures the object 23 being taken out of the box. Therefore, the scene estimation unit 4 can determine that the first image 20 is an image of the taking-out operation.

[0033] In addition, regarding the first image 40, in the image that is closer to the front in the time series of the first image 40, the unpackaged objects 23 and 43 are photographed, and in the image that is closer to the back in the time series of the first image 40, the box 41 after the objects 23 and 43 are packaged is photographed. Therefore, the scene estimation unit 4 can determine that the first image 20 is an image during the packaging operation.

[0034] The data block estimation unit 5 obtains an image of the object 23 of the first operation and an image of the object 51 of the second operation, i.e., the second images 30 and 50, and estimates the data block using one of a plurality of second learning completion model DBs that store the association between the second images 30 and 50 and one or more data block meta-IDs that correspond one-to-one to a data block ID that uniquely represents the data block.

[0035] The data block estimation unit 5 uses the model ID corresponding to the scene ID in a one-to-one manner to select one of the plurality of second learned model DBs 2. The data block meta-ID uniquely represents a data block meta-value, which is information on the properties of the objects 23, 43, and 51.

[0036] The data block estimation unit 5 uses the scene ID as a search key to obtain the model ID from the model table TB2, a table in which model IDs and scene IDs are associated one-to-one. Furthermore, the data block estimation unit 5 uses the data block meta ID as a search key to obtain the data block ID from the data block / meta table TB6, a table in which data block IDs and data block meta IDs are associated one-to-one or one-to-many.

[0037] The data block estimation unit 5 uses the data block ID as a search key to obtain a data block summary from the data block table TB7, and transmits the data block summary to the user terminal 12. The user terminal 12 presents the data block summary received from the data block estimation unit 5 to the operator.

[0038] Furthermore, the data block estimation unit 5 obtains a data block from the data block table TB7 using the data block ID as a search key, and transmits the data block to the user terminal 12. The user terminal 12 presents the data block received from the data block estimation unit 5 to the operator.

[0039] The data block table TB7 is a table in which data blocks, data block digests, and hash values ​​are associated with each other in a one-to-one manner for each data block ID. The hash value is used, for example, to check whether a data block has been changed.

[0040] When processing information related to the second job, the transition destination presentation unit 7 compares the first job actually being performed and the second job with a state transition table TB9 that stores the relationship between the first and second jobs, and outputs prompt information regarding the next job. The transition destination presentation unit 7 outputs the prompt information to, for example, the user terminal 12.

[0041] Next, use Figure 2 The information processing device 1 in the learning phase is described below. For example, in the learning phase, first images 20 and 40 inputted from an input device (not shown) and one or more second images 30, 35, 50, and 55 are learned as a pair. Here, learning is assumed to refer to learning with a teacher, for example.

[0042] Figure 2 1 is a block diagram showing the configuration of the information processing device 1 in the learning phase of the present embodiment. In the learning phase, the information processing device 1 includes a first learning-completed model generating unit 9 and a second learning-completed model generating unit 10 .

[0043] The first learned model generator 9 is a program that generates the first learned model DB1 by causing the first learned model DB1 ′ to learn the scene ID and the first images 20 and 40 as a pair.

[0044] The first learned model generation unit 9 acquires the scene ID for the first images 20 and 40 from the scene table TB1 and acquires the model ID corresponding to the scene ID from the model table TB2 .

[0045] The second learned model generator 10 is a program that generates the second learned model DB2 ′ by learning one or more data blocks with the meta ID and the second images 30 and 50 as a pair by designating a model ID.

[0046] The second learning completion model generation unit 10 uses the scene ID as a search keyword and obtains the content ID from the scene / content table TB4, a table that associates scene IDs with content IDs in a one-to-many relationship. Here, the scene ID, which serves as the search keyword, is associated with the first images 20 and 40, which are paired with the second images 30 and 50 to be processed.

[0047] The second learned model generation unit 10 uses the content ID as a search key to acquire content from the content table TB3 , which is a table in which content IDs and content are associated one-to-one.

[0048] The second learned model generation unit 10 uses the content ID as a search key to obtain the data block ID from the content / data block table TB5 , which is a table in which the content ID and the data block ID are associated in a one-to-one or one-to-many manner.

[0049] The second learned model generator 10 acquires a data block from the data block table TB7 using the data block ID as a search key, and acquires a meta ID for the data block from the data block / meta table TB6 using the data block ID as a search key.

[0050] The second learned model generator 10 uses the block meta ID as a search key to obtain the block meta value from the block meta table TB8. The block meta table TB8 associates the block category ID, block category name, and block meta value with each block meta ID.

[0051] The data block category ID uniquely indicates the data block category name, which is the name of the category to which the data block meta-value belongs. In addition, the second learning completion model generation unit 10 confirms that there are no problems with the acquired data block, content, and data block meta-value based on reference to the second images 30 and 50.

[0052] By determining problematic values ​​as abnormal values ​​and not using them for tutored learning, the second learning completion model generation unit 10 can generate a highly accurate learning completion model DB2, and the information processing device 1 can perform highly accurate processing during the use phase.

[0053] Next, use Figure 3 The following describes a case where the information is acquired by the user terminal 12 and processed as information by the information processing device 1. FIG. 1 shows first images 20 and 40 and second images 30 and 50 during the unloading operation and the packaging operation according to the present embodiment.

[0054] The first images 20, 40 and the second images 30, 50 during the unloading operation and the packaging operation are stored in the auxiliary storage device 11, for example, and are displayed on the user terminal 12, for example. Figure 3 , for example, an example is shown in which the first images 20 and 40 and the second images 30 and 50 included in both the second operation and the first operation are displayed simultaneously, but the first images 20 and 40 may be displayed one at a time on the user terminal 12 .

[0055] In the first image 20 during the unloading operation, for example, an object 23, a box 21 from which the object 23 is unloaded, and an order number 22 (hereinafter also referred to as the first number) are captured. In the first image 40 during the packaging operation, for example, objects 23 and 43 serving as the work targets during the unloading operation and a box 41 in which the objects 23 and 43 are packaged are captured.

[0056] In the second image 30 during the unloading operation, for example, the object 23 is captured. In the second image 50 during the packaging operation, for example, the objects 23 and 43 that are the objects of the unloading operation, the box 41 that packages the objects 23 and 43, and the order number 52 (hereinafter also referred to as the second number) are captured.

[0057] The order number 22 is, for example, a number assigned by the person in charge of ordering at a logistics intermediate location when placing an order, and the order acceptance number 52 is, for example, a number assigned by the person in charge of ordering at the logistics intermediate location when accepting an order. For example, when determining the scenario ID based on the order number 22, the data block estimation unit 5 estimates the order acceptance number 52 using the data block meta ID.

[0058] Next, use Figure 4 The first learned model DB1 and the second learned model DB2 will be described. Figure 4 The first learned model DB1 and the second learned model DB2 of this embodiment are shown.

[0059] The first learned model DB1 stores associations between a plurality of first images 20, 40 and a plurality of scene IDs generated by performing machine learning using a plurality of pairs of first images 20, 40 and scene IDs as a pair of learning data. Here, the machine learning is, for example, a convolutional neural network (CNN).

[0060] Specifically, the association between the first image 20, 40 and the scene ID can be represented by a convolutional neural network, which is composed of Figure 4 In the figure, the nodes are represented by circles, the edges are represented by arrows, and the weight coefficients set for the edges are represented. Figure 4 As shown, the input of the first images 20 and 40 to the first learned model DB 1 is, for example, individual pixels such as pixels p1 and p2.

[0061] Multiple second learning model DBs are associated with model IDs in a one-to-one relationship. Each second learning model DB stores associations between multiple second images 30, 35, 50, and 55 and one or more data block meta-IDs, generated by performing machine learning using these pairs of learning data. The machine learning used here is, for example, a convolutional neural network (CNN).

[0062] The association between the plurality of second images 30, 35, 50, 55 and the plurality of one or more data blocks with meta-IDs can be represented by a convolutional neural network, which is formed by Figure 4 In the figure, the nodes are represented by circles, the edges are represented by arrows, and the weight coefficients set for the edges are represented. Figure 4 As shown, the input of the second images 30 , 35 , 50 , and 55 to the second learned model DB2 is, for example, pixels p1 and p2 .

[0063] Next, use Figure 5 To illustrate the information stored in the auxiliary storage device 11, namely the scene table TB1, model table TB2, content table TB3, scene / content table TB4, content / data block table TB5, data block / meta table TB6, data block table TB7, data block meta table TB8 and state transition table TB9. Figure 5 This is a diagram showing information stored in the auxiliary storage device 11 according to this embodiment.

[0064] The scene ID stored in the scene table TB1 is, for example, a 3-digit hexadecimal number such as OFD. In addition, the scene name stored in the scene table TB1 is, for example, fragile goods removal, fragile goods packaging, etc.

[0065] The model ID stored in the model table TB2, etc., is represented by a two-character alphabet and a one-digit decimal number, such as MD1. The content ID stored in the content table TB3, etc., is represented by a five-digit hexadecimal number and a two-digit decimal number, such as 1B827-01. The content stored in the content table TB3, etc., is represented by a file name with an extension, such as 1B827-01.txt, which is the content ID, and stores a pointer to the actual content.

[0066] The data block ID stored in the content / data block table TB5 is represented by 5-digit and 2-digit decimal numbers such as 82700-01. The data block meta ID stored in the data block / meta table TB6 is represented by 4-digit hexadecimal numbers such as 24FD.

[0067] The data blocks stored in the data block table TB7 are represented by the file name of the content corresponding to the target data block and a 1-digit decimal number, such as 1B827-01.txt_0, and store a pointer to a part of the entity of the content corresponding to the target data block.

[0068] The data block summary stored in the data block table TB7 is, for example, a document summarizing the content of the data block, such as "Buffering material..." The hash value stored in the data block table TB7 is, for example, a 15-digit hexadecimal number such as 564544d8f0b746e.

[0069] The data block type ID stored in the data block metatable TB8 is a 3-digit decimal number such as 394. The data block type name stored in the data block metatable TB8 is, for example, the size, color, and shape of target envelopes, boxes, trays, baskets, etc.

[0070] Examples of data block meta values ​​stored in the data block meta table TB8 include A4, 60, white, blue, envelope, box, tray, basket, etc. The values ​​of the data block category ID and the data block category name may be NULL.

[0071] The state transition ID stored in the state transition table TB9 uniquely represents a combination of two scene IDs, and is a three-digit hexadecimal number such as 04C. The state transition ID may correspond to an order ticket for managing an order, for example.

[0072] The order stored in the state transition table TB9 is set to 1 or 2. For example, 1 indicates the removal operation and 2 indicates the packaging operation. In addition, in order to uniquely identify the combination of the state transition ID and the scene ID, the state transition table TB9 stores NOs that are sequentially numbered from 1.

[0073] As shown in the scene / content table TB4, the content / data block table TB5 and the data block / meta table TB6, the data structure of the job information has a hierarchical structure in which the data block meta ID is set as the first layer as the bottom layer, the data block ID is set as the second layer, the content ID is set as the third layer, and the scene ID is set as the fourth layer as the top layer.

[0074] Next, use Figure 6 To illustrate the scene estimation function, data block estimation function and data block output function. Figure 6 This is a timing chart for explaining the scene estimation function, data block estimation function, and data block output function of this embodiment.

[0075] The information processing function in the use phase is composed of a scene estimation function implemented by a scene estimation process S60 described later, a data block estimation function implemented by a data block estimation process S80 described later, and a data block output function implemented by a data block output process S100 described later.

[0076] First, the scene estimation function will be described: The scene estimation unit 4 included in the control unit 15 receives the first images 20 and 40 from the user terminal 12 (S1), and inputs the received first images 20 and 40 into the first learned model DB1 (S2).

[0077] The first learned model DB 1 selects one or more scene IDs that are strongly associated with the received first images 20 and 40 , and outputs the selected one or more scene IDs (hereinafter also referred to as a first scene ID list) to the scene estimation unit 4 ( S3 ).

[0078] When the scene estimation unit 4 obtains the first scene ID list, it directly sends it to the user terminal 12 (S4). The user terminal 12 sends the scene estimation unit 4 information on whether each scene ID included in the first scene ID list is cached (S5).

[0079] The user terminal 12 maintains a table equivalent to the scene table TB1 for previously processed information. The user terminal 12 uses the received scene IDs from the first scene ID list as search keywords to search within the table maintained by the user terminal 12. Scene IDs that yielded search results are cached, while scene IDs that yielded no search results are not cached.

[0080] The scene estimation unit 4 searches the scene table TB1 using one or more scene IDs not cached in the user terminal 12 (hereinafter also referred to as the second scene ID list) from the scene IDs included in the first scene ID list received from the user terminal 12 as search keywords (S6).

[0081] The scene estimation unit 4 acquires, from the scene table TB1 , scene names corresponding to the respective scene IDs included in the second scene ID list (hereinafter, this may also be referred to as a scene name list) as search results ( S7 ).

[0082] The scene estimation unit 4 directly transmits the acquired scene name list (S8) to the user terminal 12. In the use phase, the information processing device 1 implements a scene estimation function of estimating the scene of the first images 20 and 40 by estimating the scene name through steps S1 to S8.

[0083] Next, the data block estimation function will be described. The user terminal 12 presents the received scene name to the operator. The operator selects, for example, one scene name from the presented scene name list. The user terminal 12 transmits the scene name selected by the operator to the data block estimation unit 5 included in the control unit 15 (S9).

[0084] The data block estimation unit 5 uses the scene ID corresponding to the scene name received from the user terminal 12 as a search key ( S10 ), searches the model table TB2 , and acquires the model ID ( S11 ).

[0085] The data block estimation unit 5 receives the second images 30, 35, 50, and 55 from the user terminal 12 (S12). The data block estimation unit 5 specifies one of the plurality of second learned model DBs 2 using the model ID obtained from the model table TB2, and inputs the second images 30 and 50 into the specified second learned model DB 2 (S13).

[0086] The second learning completion model DB2 selects one or more data block meta-IDs that are strongly associated with the second image 30, 35, 50, 55, and outputs the selected one or more data block meta-IDs (hereinafter, also referred to as a data block meta-ID list) to the data block estimation unit 5 (S14).

[0087] The data block estimation unit 5 searches the data block / meta table TB6 using one or more data block meta IDs included in the data block meta ID list as search keys ( S15 ).

[0088] The data block estimation unit 5 obtains one or more data block IDs (hereinafter also referred to as a first data block ID list) from the data block / meta table TB6 as a search result (S16). The data block estimation unit 5 directly transmits the obtained first data block ID list to the user terminal 12 (S17).

[0089] The user terminal 12 sends information on whether each block ID included in the first block ID list is cached to the block estimation unit 5 (S18). The user terminal 12 holds a table having a block ID column and a block summary column in the block table TB7 for previously processed information.

[0090] The user terminal 12 uses the block IDs in the received first block ID list as search keywords to search the table held by the user terminal 12. Block IDs that yielded search results are cached, while block IDs that yielded no search results are not cached.

[0091] The data block estimation unit 5 searches the data block table TB7 using one or more data block IDs (hereinafter also referred to as the second data block ID list) from among the data block IDs included in the first data block ID list received from the user terminal 12 and not cached in the user terminal 12 as search keywords (S6).

[0092] The data block estimation unit 5 obtains the data block digests corresponding to the data block IDs included in the second data block ID list (hereinafter also referred to as the data block digest list) from the data block table TB1 as the search result (S7). The data block estimation unit 5 directly transmits the obtained data block digest list to the user terminal 12 (S21).

[0093] In the use phase, the information processing device 1 implements a block estimation function of estimating the blocks of the objects 23 , 43 , and 51 by estimating the block digests through steps S9 to S21 .

[0094] Next, the data block output function will be described. The user terminal 12 presents the operator with a list of received data block summaries. The operator selects, for example, one data block summary from the presented list. The user terminal 12 transmits the selected data block summary to the data block output unit 6 included in the control unit 15 (S22).

[0095] The data block output unit 6 uses the data block ID corresponding to the data block digest received from the user terminal 12 as a search key ( S23 ), searches the data block table TB7 , and acquires the data block ( S24 ).

[0096] The data block output unit 6 directly sends the acquired data block to the user terminal 12 (S21). The user terminal 12 presents the received data block to the user. In the use phase, the information processing device 1 implements the data block output function of the data block of the output object 23, 43, 51 through steps S22 to S25.

[0097] Next, use Figure 7 The first learning completion model generation function and the second learning completion model generation function will be described. Figure 7 This is a sequence diagram for explaining the first learning completion model generation function and the second learning completion model generation function of the present embodiment.

[0098] The information processing function in the learning phase is composed of a first learning completion model generation function implemented by the first learning completion model generation process and a second learning completion model generation function implemented by the second learning completion model generation process.

[0099] First, the first learning completion model generation function will be described. The first learning completion model generation unit 9 included in the learning completion model generation unit 16 determines the scene name to be processed, a pair of first images 20 and 40, and one or more second images 30, 35, 50, and 55, and searches the pre-generated scene table TB1 using the scene name as a search keyword (S31).

[0100] The first learned model generator 9 acquires the scene ID from the scene table TB1 as a search result ( S32 ), and causes the first learned model DB1 ′ to learn the first images 20 and 40 and the scene ID as a pair ( S33 ).

[0101] Furthermore, the first learned model generator 9 sends the acquired scenario ID to the model table TB2 and makes a model ID acquisition request (S34). The model table TB2 generates a model ID corresponding to the received scenario ID and stores the combination of the scenario ID and the model ID.

[0102] Next, the first learned model generator 9 obtains the model ID from the model table TB2 ( S35 ). In the learning phase, the information processing device 1 implements a first learned model generator function for generating the first learned model DB1 through steps S31 to S35 .

[0103] Next, the second learning completion model generation function will be described. The second learning completion model generation unit 10 included in the learning completion model generation unit 16 searches the pre-generated scenario / content table TB4 using the scenario ID received by the first learning completion model generation unit 9 in step S32 as a search key (S36).

[0104] The second learned model generation unit 10 acquires the content ID from the scenario / content table TB4 as a search result ( S37 ), and searches the pre-generated content table TB3 using the acquired content ID as a keyword ( S38 ).

[0105] The second learned model generation unit 10 acquires content from the content table TB3 as a search result ( S39 ), and searches the pre-generated content / data block table TB5 using the content ID acquired in step S37 as a search keyword ( S40 ).

[0106] The second learned model generation unit 10 acquires a data block ID from the content / data block table TB5 as a search result ( S41 ), and searches the pre-generated data block table TB7 using the acquired data block ID as a search key ( S42 ).

[0107] The second learned model generator 10 acquires a data block from the data block table TB7 as a search result ( S43 ), and searches the pre-generated data block / meta table TB6 using the data block ID acquired in step S41 as a key ( S44 ).

[0108] The second learned model generator 10 obtains one or more data block meta IDs from the data block / meta table TB6 as search results ( S45 ), and searches the pre-generated data block meta table TB8 using each obtained data block meta ID as a key ( S46 ).

[0109] The second learned model generation unit 10 obtains the data block meta value corresponding to each data block meta ID from the data block meta table TB8 as a search result ( S47 ).

[0110] The second learning completion model generator 10 refers to the first images 20 and 40 and the second images 30 , 35 , 50 , and 55 to check whether there are any problems with the content acquired in step S30 , the data blocks acquired in step S43 , and the meta values ​​for each data block acquired in step S47 .

[0111] For example, the second learning completion model generation unit 10 confirms with reference to the order number 22 captured in the first image 20, the shape of the object 23 captured in the second image 30, the objects 23 and 43 captured in the first image 40, the received order number 52 captured in the second image 50, etc.

[0112] When the result of the reference indicates that there is a problem, the processing of the group as the object is terminated. The problem is that the data blocks and the data block meta-values ​​are obviously different from the information of the objects 23, 43, 51 captured in the second images 30, 35, 50, 55.

[0113] Next, the second learning model generator 10 causes the second learning model DB2' to learn pairs of the model ID, the second images 30, 35, 50, 55, and one or more data block meta-IDs (S48). During the learning phase, the information processing device 1 implements the second learning model generation function of generating the second learning model DB2 through steps S36 to S48.

[0114] Next, use Figure 8 To illustrate the information processing in the use phase. Figure 8 This is a flowchart showing the processing procedure of information processing in the use phase of this embodiment. The information processing in the use phase is composed of scene estimation processing S60, data block estimation processing S80, data block output processing S100 and migration destination presentation processing S110.

[0115] First, the scene estimation process S60 is described. The scene estimation process S60 is composed of steps S61 to S67. When the scene estimation unit 4 receives the first images 20 and 40 from the user terminal 12 (S61), it outputs the first images 20 and 40 to the first learned model DB1 (S62).

[0116] The scene estimation unit 4 obtains the first scene ID list from the first learned model DB 1 as an output ( S63 ), directly sends the first scene ID list to the user terminal 12 , and inquires the user terminal 12 about the presence of a cache ( S64 ).

[0117] If the response from the user terminal 12 indicates that all the data are cached (S65: No), the scene estimation process S60 ends and the data block estimation process S80 begins. If the response from the user terminal 12 indicates that none of the data are cached (S65: Yes), the scene estimation unit 4 obtains the scene name list from the scene table TB1 (S66) and directly sends it to the user terminal 12 (S67), ending the scene estimation process S60.

[0118] Next, the data block estimation process S80 will be described. The data block estimation process S80 is composed of steps S81 to S88. The data block estimation unit 5 receives the scene name selected by the operator from the user terminal 12 (S81).

[0119] Upon receiving the scene name from the user terminal 12, the data block estimation unit 5 obtains the model ID from the model table TB2 (S82). Next, the data block estimation unit 5 uses the model ID to specify one of the plurality of second learned model DBs 2 and inputs the second images 30, 35, 50, and 55 received from the user terminal 12 into the specified second learned model DB 2 (S83).

[0120] The data block estimation unit 5 obtains a data block meta ID list from the second learned model DB2 as an output (S84), and obtains a first data block ID list from the data block / meta table TB6 (S85). The data block estimation unit 5 then directly transmits the first data block ID list to the user terminal 12 and inquires about the presence of a cache in the user terminal 12 (S86).

[0121] If the response from the user terminal 12 indicates that all data blocks are cached (S86: No), the data block estimation process S80 ends and the data block output process S100 begins. If the response from the user terminal 12 indicates that none of the data blocks are cached (S86: Yes), the data block estimation unit 5 obtains a data block summary list from the data block table TB7 (S87) and transmits it directly to the user terminal 12 (S88), ending the data block estimation process S80.

[0122] Next, the data block output process S100 will be described. The data block output process S100 is composed of steps S101 to S103. The data block output unit 6 receives a data block digest selected by an operator from the user terminal 12 (S101).

[0123] When receiving the data block digest from the user terminal 12, the data block output unit 6 obtains the data block from the data block table TB7 (S102), and directly transmits it to the user terminal 12 (S103), and the data block output process S100 ends.

[0124] Next, the transition target prompting process S110 will be described. The transition target prompting process S110 consists of step S111. The transition target prompting unit 7 outputs prompting information (S111). For example, the transition target prompting unit 7 prompts the operator to perform the following summary processing on the order ticket being processed in parallel: perform the summary processing on the second operation after the summary processing on the first operation. Furthermore, the operator is prompted to perform the summary processing on the operations with the same scene ID in the first operation.

[0125] This will be explained in detail using state transition TB9. Three orders, state transition IDs "04C," "05D," and "05E," are processed in parallel. While a worker is working on scenario ID "0FD" in state transition ID "04C" (No. 1), transition target prompting unit 7 prompts the worker to work on scenario ID "0FD" in state transition ID "05D" (No. 3).

[0126] When the operator is working on the scene ID "0FD" in the state transition ID "05D" of the number "3", the transition target prompting unit 7 prompts the operator to work on the scene ID "1FD" in the state transition ID "05E" of the number "5". At this point in the work, there is no unfinished work of the order "1".

[0127] When the operator is performing work in scene ID “1FD” in state transition ID “05E” of No. “5”, the transition target prompting unit 7 prompts the operator to perform work in scene ID “0FE” in state transition ID “04C” of No. “2”.

[0128] When the operator is performing work with scene ID “0FE” in state transition ID “04C” of NO “2”, the transition target prompting unit 7 prompts the operator to perform work with scene ID “0FE” in state transition ID “05E” of NO “6”.

[0129] When the operator is performing work in scene ID “0FE” in state transition ID “05E” of NO “6”, the transition target prompting unit 7 prompts the operator to perform work in scene ID “1FE” in state transition ID “05D” of NO “4”.

[0130] Next, use Figure 9 To illustrate information processing during the learning phase. Figure 9 1 is a flowchart showing the processing procedure of information processing in the learning phase of this embodiment. The information processing in the learning phase is composed of a first learning-completed model generation process S120 and a second learning-completed model generation process S140.

[0131] First, the first learning completion model generation process S120 will be described. The first learning completion model generation process S120 consists of steps S121 through S124. After determining the scene name and the pair of the first image 20 or 40 and one or more second images 30, 35, 50, or 55, the first learning completion model generation unit 9 searches the scene table TB1 using the scene name as a keyword (S121).

[0132] The first learned model generator 9 acquires the scene ID from the scene table TB1 as a search result ( S122 ), and causes the first learned model DB1 ′ to learn the scene ID and the first images 20 and 40 as a pair ( S123 ).

[0133] Next, the first learned model generator 9 sends the scenario ID acquired in step S122 to the model table TB2 , requests acquisition of the model ID, and then acquires the model ID ( S124 ).

[0134] Next, the second learning completion model generation process S140 is described. The second learning completion model generation process S140 is composed of steps S141 to S150. The second learning completion model generation unit 10 searches the scene / content table TB4 using the scene ID obtained in step S122 as a search keyword to obtain a content ID (S141).

[0135] The second learning completion model generation unit 10 searches the content table TB3 using the obtained content ID as a search key and obtains the content (S142). In addition, the second learning completion model generation unit 10 searches the content / data block table TB5 using the obtained content ID as a search key and obtains the data block ID (S143).

[0136] Furthermore, the second learning completion model generation unit 10 searches the data block table TB7 using the acquired data block ID as a key to acquire the data block (S144). Furthermore, the second learning completion model generation unit 10 searches the data block / meta table TB6 using the acquired data block ID as a search key to acquire one or more meta IDs for the data block (S145).

[0137] Furthermore, the second learned model generator 10 searches the data block meta table TB8 using the acquired one or more data block meta IDs as search keys, and acquires data block meta values ​​corresponding to the respective data block meta IDs ( S146 ).

[0138] The second learning completion model generation unit 10 refers to the first images 20, 40 and the second images 30, 35, 50, 55 to confirm whether there are any problems with the content obtained in step S142, the data blocks obtained in step S144, and the meta values ​​of each data block obtained in step S146 (S147).

[0139] If the confirmation result indicates a problem (S148: No), information processing in the learning phase for the group being processed is terminated. If the confirmation result indicates no problem (S148: Yes), the second learning-end model generator 10 causes the second learning model DB 2' to learn the model ID, one or more data block meta-IDs, and the second images 30, 35, 50, and 55 as a pair (S149), and information processing in the learning phase for the group being processed is terminated.

[0140] As described above, the information processing device 1 of this embodiment displays data blocks, which are data blocks obtained by dividing job information or data blocks that display job information, via the user terminal 12. Therefore, by appropriately setting the data blocks, the required amount of information can be displayed. Furthermore, by setting the data blocks to display information about the entire document, large-scale information reconstruction is unnecessary.

[0141] By using the model table TB2 , even when the relationship between the first learned model DB1 and the second learned model DB2 changes, this can be addressed by simply changing the model table TB2 , thereby providing a device with excellent maintainability.

[0142] Furthermore, when the model table TB2 is not used, if the relationship between the first learned model DB1 and the second learned model DB2 changes, it is necessary to generate the learned model DB2 again.

[0143] In this embodiment, the scene estimation unit 4, the data block estimation unit 5, the data block output unit 6, the confirmation unit 7, the first learning completion model generation unit 9, the second learning completion model generation unit 10 and the recommended image output unit 13 are programs, but are not limited to this and can also be logic circuits.

[0144] In addition, the scene estimation unit 4, the data block estimation unit 5, the data block output unit 6, the confirmation unit 7, the first learning completion model generation unit 9 and the second learning completion model generation unit 10, the recommended image output unit 13, the first learning completion model DB1, the first learning model DB1', the second learning completion model DB2, the second learning model DB2', the scene table TB1, the model table TB2, the content table TB3, the scene / content table TB4, the content / data block table TB5, the data block / meta table TB6, the data block table TB7, the meta table TB8 for data blocks and the state transition table TB9 may also be installed not in one device, but in a distributed manner in multiple devices connected via a network.

[0145] In addition, in the above Figure 7 and Figure 9 In the illustrated learning phase, a case where the first and second learning completion models are generated in association with each other is described. However, the present invention is not limited thereto, and the first and second learning completion models DB1 and DB2 may be generated separately.

[0146] When the first learned model DB1 and the second learned model DB2 are generated separately, for example, when the scenario is an existing scenario and only content is added, learning related to the scenario does not need to be performed.

[0147] In this embodiment, the case of using multiple second learning completion model DB2s is described, but the present invention is not limited to this. A single second learning completion model DB2 may be used. In addition, in this embodiment, the case of using the same first learning completion model DB1 for the first and second operations is described, but the present invention is not limited to this. Different first learning completion model DB1s may be used for the unloading operation and the packaging operation.

[0148] In this embodiment, a case where the relationship between the first job and the second job is stored in advance has been described. However, the relationship between the first job and the second job may also be learned through machine learning.

[0149] In this embodiment, there is no mention of obtaining operator information, i.e., user information, of the operator performing the first operation and the operator performing the second operation, but the present invention is not limited thereto. For example, it is assumed that user information can also be obtained from the user terminal 12 and used.

[0150] Description of labels

[0151] 1: Information processing device; 2: Central processing unit; 3: Main storage device; 4: Scene estimation unit; 5: Data block estimation unit; 6: Data block output unit; 7: Migration target prompt unit; 9: First learning completion model generation unit; 10: Second learning completion model generation unit; 11: Auxiliary storage device; 12: User terminal.

Claims

1. An information processing device that outputs operation information, which is information related to a first operation and a second operation performed by an operator on an object as an operation target, wherein: The information processing device has: a scene estimation unit that acquires a first image, the first image being an image of a scene in which the worker performs the first and second tasks, and estimates the scene using a first learned model that records an association between the first image and a scene ID uniquely representing the scene; a data block estimation unit that obtains a second image that is an image of an object of the first and second operations, and estimates the data block using one of a plurality of second learning-completed models that stores associations between the second image, a data block ID that uniquely indicates a data block, and one or more data block meta-IDs corresponding to the data block IDs, the data blocks being information after the operation information is segmented or indicating the operation information; a transition target prompting unit that, when the first operation or the second operation is performed, compares the first operation and the second operation actually performed with a state transition table pre-stored with two or more state transition IDs, and outputs prompt information, the prompt information providing a prompt related to the next operation, the state transition ID uniquely indicating a combination of a scene ID corresponding to the first operation and a scene ID corresponding to the second operation; as well as an output unit that outputs the data block, The data block estimation unit selects one of the plurality of second learning completed models using a model ID corresponding one-to-one to a scene ID, wherein the data block meta-ID uniquely represents a data block meta-value that is information related to the property of the object. The transition destination presentation unit, upon acquiring a scenario ID corresponding to a first job in one state transition ID, presents information related to the first job having the same scenario ID associated with another state transition ID.

2. The information processing device according to claim 1, wherein The transition destination presentation unit then presents a first task having another scenario ID associated with another state transition ID.

3. The information processing device according to claim 2, wherein: After completing all presentations related to the first job, the transition target presentation unit, upon obtaining a scene ID corresponding to the second job in one state transition ID, presents presentations related to the second job having the same scene ID associated with another state transition ID.

4. An information processing method, executed by an information processing device, the information processing device outputting operation information, which is information related to a first operation and a second operation performed by an operator on an object as an operation target, wherein: The information processing method has the following steps: A first step of acquiring a first image, the first image being an image of a scene in which the operator performs the first and second tasks, and estimating the scene using a first learned model that records an association between the first image and a scene ID uniquely representing the scene; The second step is to obtain a second image, which is an image of an object of the first operation and the second operation, and estimate the data block using one of a plurality of second learning completion models that stores associations between the second image, a data block ID uniquely representing the data block, and one or more data block meta-IDs corresponding to the data block IDs, the data blocks being information after the operation information is segmented or indicating the operation information; In a third step, when the first operation or the second operation is performed, the first operation and the second operation actually performed are compared with a state transition table pre-stored with two or more state transition IDs, and prompt information is output, the prompt information providing a prompt related to the next operation, the state transition ID uniquely indicating a combination of a scene ID corresponding to the first operation and a scene ID corresponding to the second operation; as well as Step 4: Output the data block. In the second step, one of the plurality of second learning completed models is selected using a model ID corresponding one-to-one to a scene ID, wherein the data block meta ID uniquely represents a data block meta value that is information related to the property of the object. In the third step, when the scenario ID corresponding to the first job in one state transition ID is acquired, presentation is performed on the first job having the same scenario ID associated with another state transition ID.

Images