Information processing program, information processing method, and information processing device

Abstract

To improve the efficiency of the decision-making process. [Solution] The judgment logic acquisition unit 20A of the information processing device 10 acquires judgment logic that determines whether the state represented by the processing target conforms to the input rules represented by the input rule information. The logic execution unit 20C executes a judgment process on the processing target in accordance with the judgment logic. The output unit 20E outputs output information including the judgment result from the judgment process to the output device 22.

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing program, an information processing method, and an information processing apparatus.

Background Art

[0002] It has been done to determine whether the environment such as a work site or a maintenance site is in a state conforming to rules. For example, a technique for determining the state of a determination target such as a person according to the similarity between an estimated answer and an assumed answer estimated for a question regarding a photographed image of a manufacturing site or the like has been disclosed.

[0003] However, in the prior art, it is necessary to prepare questions and assumed answers for the questions and manually adjust the determination logic for each environment of the determination target, etc., and the determination efficiency may decrease.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem to be solved by the present invention is to provide an information processing program, an information processing method, and an information processing apparatus capable of improving determination efficiency.

Means for Solving the Problems

[0006] The information processing program of this embodiment is an information processing program that causes a computer to execute the following steps: a determination logic acquisition step of acquiring determination logic to determine whether the state represented by the processing target conforms to the input rules represented by the input rule information; a logic execution step of executing a determination process on the processing target in accordance with the determination logic; and an output step of outputting output information including the determination result of the determination process to an output device. [Brief explanation of the drawing]

[0007] [Figure 1] A schematic diagram of an information processing device. [Figure 2] Block diagram of the judgment logic acquisition unit. [Figure 3] A schematic diagram of input rule information. [Figure 4] An explanatory diagram of a collection of example judgment logics. [Figure 5] An explanatory diagram of an example of a judgment logic. [Figure 6] A diagram illustrating the generation of the judgment logic. [Figure 7] A schematic diagram of the decision logic. [Figure 8A] A diagram illustrating the output of the output information. [Figure 8B] A diagram illustrating the output of the output information. [Figure 8C] A diagram illustrating the output of the output information. [Figure 9] A flowchart illustrating the flow of information processing. [Figure 10] Hardware configuration diagram. [Modes for carrying out the invention]

[0008] The information processing program, information processing method, and information processing apparatus of this embodiment will be described in detail below with reference to the attached drawings.

[0009] Figure 1 is a schematic diagram of an example of the information processing device 10 of this embodiment.

[0010] The information processing apparatus 10 includes a storage unit 12, a UI (user interface) unit 14, a communication unit 16, a photographing unit 18, and a control unit 20. The storage unit 12, the UI unit 14, the communication unit 16, the photographing unit 18, and the control unit 20 are communicably connected via a bus 19 or the like. At least one of the external information processing apparatuses connected to the information processing apparatus 10 via the storage unit 12, the UI unit 14, and the communication unit 16 functions as an output device 22 that is an output destination for outputting various types of information.

[0011] The storage unit 12 stores various types of data. The storage unit 12 may be provided outside the information processing apparatus 10. Also, at least one of the storage unit 12, the UI unit 14, the photographing unit 18, and one or more functional units included in the control unit 20 described later may be configured to be mounted on an external information processing apparatus or the like communicably connected to the information processing apparatus 10 via a network or the like.

[0012] The UI unit 14 has a display function for displaying various types of information and an input function for receiving operation inputs by the user. The display function is, for example, a display, a projection device, or the like. The input function is, for example, a pointing device such as a mouse and a touch pad, a keyboard, or the like. It may also be a touch panel that integrally configures the display function and the input function.

[0013] The UI unit 14 may have a configuration that is communicably connected to the control unit 20 by wire or wirelessly. The UI unit 14 may be provided outside the information processing apparatus 10, and the UI unit 14 and the control unit 20 may be connected via a network or the like.

[0014] The communication unit 16 is a communication interface for communicating with an external information processing apparatus or the like of the information processing apparatus 10.

[0015] The photographing unit 18 acquires photographed image data by photographing. The photographing unit 18 outputs the photographed image data obtained by photographing to the control unit 20. Hereinafter, the photographed image data may sometimes be simply referred to as an image for explanation.

[0016] The imaging unit 18 is adjusted in advance in terms of the imaging angle of view, imaging direction, and installation position so that it can image the state of the determination target in the information processing apparatus 10 of the present embodiment.

[0017] The determination target is what is determined whether it complies with the input rules in the information processing apparatus 10 of the present embodiment. The input rules are information representing predetermined rules. Details of the input rules will be described later. The determination target is, for example, various events existing in organisms such as devices, objects, and people, machines, and nature, etc., but is not limited thereto. In the present embodiment, the case where the determination target is a person will be described by way of assumption.

[0018] The state of the determination target is the state of the target for determining whether it complies with the input rules in the information processing apparatus 10 of the present embodiment. The state of the determination target corresponds to the input rules. For example, the state of the determination target is the state of a determination target such as a person. Specifically, for example, the state of the determination target is information indicating the presence or absence of wearing items such as gloves of a person during a specific operation, etc., but is not limited thereto.

[0019] The imaging unit 18, for example, sequentially performs imaging along the time series, and sequentially outputs the images obtained by the imaging to the control unit 20. The imaging unit 18 may be provided outside the information processing apparatus 10. In this case, the imaging unit 18 may be communicably connected to the control unit 20 via the communication unit 16 or the like.

[0020] The control unit 20 executes information processing in the information processing apparatus 10. The control unit 20 includes a determination logic acquisition unit 20A, an image acquisition unit 20B, a logic execution unit 20C, a generation unit 20D, and an output unit 20E.

[0021] The judgment logic acquisition unit 20A, image acquisition unit 20B, logic execution unit 20C, generation unit 20D, and output unit 20E are implemented by, for example, one or more processors. For example, each of the above units may be implemented by having a processor such as a CPU (Central Processing Unit) execute a program, i.e., by software. Each of the above units may be implemented by a processor such as a dedicated IC or circuit, i.e., by hardware. Each of the above units may be implemented by using both software and hardware. When multiple processors are used, each processor may implement one of the above units, or two or more of the above units.

[0022] The judgment logic acquisition unit 20A acquires the judgment logic.

[0023] Decision logic is the logic used to determine whether the state of a target being processed conforms to the input rules represented by the input rule information. Logic is a programming term that describes the processing content, processing procedure, and processing method of a program. For example, decision logic is designed to perform various processes by subdividing the input rules into one or more decision criteria and inputting the target being processed. Decision logic may be designed as an algorithm in various forms, such as a decision tree, logical expression, logical operation, or Python code, or as a script describing such an algorithm. In this embodiment, the use of a decision tree in the decision logic will be explained as an example. A more detailed explanation of the decision logic will follow later.

[0024] The processing target is data that can represent the state of the judgment target described above. The processing target can be any data that represents the state of the judgment target. For example, the processing target may be an image of the state of the judgment target, audio representing the state of the judgment target, a point cloud representing the state of the judgment target, a waveform or signal representing the state of the judgment target, etc. In this embodiment, one example of the processing target being an image of the judgment target captured by the imaging unit 18 will be described.

[0025] Input rule information is information that represents input rules. Input rules are rules that are entered or set by the user and are desired by the user. In other words, input rules are rules that the user wants to be followed to determine whether or not a condition represented by the processing target is being met.

[0026] The judgment logic acquisition unit 20A generates judgment logic based on input rule information. In this embodiment, the judgment logic acquisition unit 20A generates judgment logic based on input rule information and a collection of judgment logic examples.

[0027] Figure 2 is a block diagram showing an example of the configuration of the judgment logic acquisition unit 20A.

[0028] The judgment logic acquisition unit 20A includes an input rule acquisition unit 21A, a case study acquisition unit 21B, and a judgment logic generation unit 21C. The input rule acquisition unit 21A, the case study acquisition unit 21B, and the judgment logic generation unit 21C are implemented by, for example, one or more processors. For example, each of the above units may be implemented by having a processor such as a CPU execute a program, i.e., by software. Each of the above units may be implemented by a processor such as a dedicated IC or circuit, i.e., by hardware. Each of the above units may be implemented by using both software and hardware. When multiple processors are used, each processor may implement one of the above units, or it may implement two or more of the above units.

[0029] The input rule acquisition unit 21A acquires input rule information. The user inputs desired input rule information, for example, by operating the UI unit 14. The input rule acquisition unit 21A acquires input rule information by receiving the input rule information entered by the user through operation instructions of the UI unit 14, etc. Alternatively, the input rule acquisition unit 21A may acquire input rule information by reading input rule information previously stored in the storage unit 12. Alternatively, the input rule acquisition unit 21A may acquire input rule information by receiving input rule information from an external information processing device, etc., that is communicably connected to the information processing device 10 via the communication unit 16.

[0030] Figure 3 is a schematic diagram of an example of input rule information 30. In this embodiment, we will explain as an example the case in which the input rule information 30 represents the input rule "Wearing gloves is prohibited during drilling operation."

[0031] Returning to Figure 2, we continue the explanation.

[0032] The case study acquisition unit 21B acquires a collection of judgment logic case studies.

[0033] Figure 4 is an explanatory diagram illustrating an example of the generation of judgment logic examples 34.

[0034] The "Decision Logic Example Collection 34" is a logic consisting of one or more "Decision Logic Example 36" pieces of logic. "Decision Logic Example 36" is a logic piece representing an example of a decision logic.

[0035] The judgment logic example collection 34 is generated based on document data 32.

[0036] Document data 32 is data that specifies multiple types of assumed rules relating to input rules represented by input rule information 30. Document data 32 is, for example, data of manuals and documents that are applied to environments in which the object to be determined, such as a person, may exist. This environment is, for example, a specific space in a real space such as a work site, construction site, or office.

[0037] The assumed rules are one or more rules that may apply to or may be assumed for the environment, and preferably include rules related to the input rules. "Related to the input rules" means that the rules match or are similar to the input rules.

[0038] The judgment logic example 36 is designed to perform various processes by subdividing the assumed rule 36A contained in the document data 32 into one or more judgment criteria and inputting a processing target such as an image. The judgment logic example 36 may be designed as an algorithm in various forms, such as a decision tree, a logical formula, or Python code, or as a script describing said algorithm.

[0039] In this embodiment, the case in which a decision tree is used in the judgment logic example 36 will be explained as an example. Specifically, in this embodiment, the judgment logic example 36 defines one or more first processing elements 36B for the assumed rule 36A extracted from the document data 32, the execution order of the multiple first processing elements 36B if there are multiple first processing elements 36B, the processing method used for processing the first processing elements 36B, and possible first judgment results 36C.

[0040] The first processing element 36B represents the processing to be performed on the processing target, such as an image, in order to determine whether or not it complies with the assumed rule 36A. The group of one or more first processing elements 36B included in the judgment logic example 36 defines the judgment processing to be performed by the judgment logic example 36. The first processing element 36B includes judgment processing, processing processing, etc. Figure 4 shows an example where all of the first processing elements 36B included in the judgment logic example 36 are judgment processing. The first processing element 36B may include the processing method used for the processing. The processing method is, for example, object detection, VQA (Visual Question Answering), etc., but is not limited to these.

[0041] The first judgment result 36C is information representing the result of the processing by the first processing element 36B, i.e., the judgment result that can be obtained by the judgment processing represented by the judgment logic example 36. In this embodiment, the possible first judgment result 36C includes compliance, which indicates compliance with the assumed rule 36A, and non-compliance, which indicates non-compliance with the assumed rule 36A. Compliance means following the rule. Non-compliance means not following the rule.

[0042] In the example shown in Figure 4, the execution order of the first processing element 36B is represented by arrows connecting multiple first processing elements 36B. That is, when using a decision tree in the Decision Logic Example Collection 34, each of the one or more first processing elements 36B that make up the decision tree corresponds to a node, and the direction from the parent node to the child node corresponds to the execution order of the first processing elements 36B.

[0043] The case study acquisition unit 21B analyzes the document data 32 and extracts one or more assumed rules 36A contained in the document data 32. Then, for each of the extracted assumed rules 36A, the case study acquisition unit 21B generates a judgment logic example 36 to determine whether or not the assumed rule 36A is being complied with.

[0044] For example, the case study acquisition unit 21B generates a case study collection 34 by inputting the document data 32 into a Large Language Model (LLM) that has been trained using document data 32 and the format of a judgment logic case 36 represented by a decision tree as shown in Figure 4, and which has the capability of in-context learning.

[0045] In-context learning is one of the capabilities of learning models such as LLMs, and it refers to the ability to perform new tasks based on the context of the provided data. In this case, LLMs can recognize and learn patterns from regular documents and contexts, and by doing so, they can understand and output new tasks based on the in-context context of the input data.

[0046] Examples of LLMs include BERT (Bidirection Encoder Representation from Transformers) and GPT (Generative Pre-trained Transformer).

[0047] In LLMs with the capability known as in-context learning, the process to be performed is given as an instruction beforehand, followed by examples of input and output data, and finally the desired input data. This allows the LLM to output data that is in line with the context of these examples. These examples of input and output data are sometimes referred to as in-context examples.

[0048] In detail, the case study acquisition unit 21B uses document data 32 as an in-context example. Specifically, the case study acquisition unit 21B extracts multiple assumed rules 36A contained in document data 32 as in-context examples. The case study acquisition unit 21B then inputs instructions to the LLM such as "Generate a judgment logic example 36 from assumed rules 36A." The case study acquisition unit 21B then inputs the assumed rules 36A, the first processing element 36B represented by the judgment logic example 36, the execution order of the first processing element 36B, and a template representing the first judgment result 36C into the LLM as in-context examples. Finally, the case study acquisition unit 21B inputs the assumed rules 36A into the LLM to obtain a judgment logic example 36 for each assumed rule 36A. By executing these processes for each assumed rule 36A, the case study acquisition unit 21B generates a judgment logic example collection 34.

[0049] Furthermore, the judgment logic example 36 included in the judgment logic example collection 34 only needs to be generated for each assumed rule 36A included in the document data 32, and is not limited to the form shown in Figure 4. For example, the judgment logic example 36 included in the judgment logic example collection 34 may include judgment logic example examples of various patterns, such as the judgment logic example 36 shown in Figure 5, and the judgment logic 39 described later.

[0050] Returning to Figure 4, we continue the explanation.

[0051] For example, suppose document data 32 includes the assumed rule 36A "You must not use your smartphone while walking." In this case, the case data acquisition unit 21B performs the above processing to generate a judgment logic example 36, which, after the first processing element 36B representing "process the question 'Is that person walking?' using VQA," executes the first processing element 36B representing "process the question 'Is that person using their smartphone?' using VQA" if the result of the first processing element 36B is Yes. If the result of the first processing element 36B is Yes, it obtains a first judgment result 36C representing "non-compliance," and if No, it obtains a first judgment result 36C representing "compliance." The case data acquisition unit 21B also performs the same processing for other assumed rules 36A included in document data 32 to generate a judgment logic example 36 for each assumed rule 36A included in document data 32.

[0052] Furthermore, the case study acquisition unit 21B may acquire a collection of judgment logic examples 34 consisting of one or more judgment logic examples 36 that have been input by the user through operational instructions of the UI unit 14 based on the document data 32. For example, consider a case where the assumed rule 36A included in the document data 32 is "If A is done, B must not be done" or "While A is done, B is prohibited." In this case, the user may input the collection of judgment logic examples 34 according to a template that specifies information such as "1. Process A with VQA, 2. Process B with VQA, and if the judgment results for A and B are Yes, determine that it is a non-compliance."

[0053] Alternatively, the case study acquisition unit 21B may acquire the case study collection 34 by receiving the case study collection 34 generated by an external information processing device that is communicably connected to the information processing device 10 via the communication unit 16.

[0054] Alternatively, the case study acquisition unit 21B may acquire the case study collection 34 by reading the case study collection 34, which has been prepared in advance and stored in the storage unit 12, from the storage unit 12. Alternatively, the case study acquisition unit 21B may acquire the case study collection 34 by receiving the case study collection 34 generated by an external information processing device or the like.

[0055] Returning to Figure 2, we continue the explanation.

[0056] The judgment logic generation unit 21C generates judgment logic based on the input rule information 30 and the judgment logic example collection 34.

[0057] Figure 6 is an explanatory diagram of an example of the generation of the judgment logic 38.

[0058] The judgment logic generation unit 21C generates judgment logic 38 that determines the input rules represented by the input rule information 30 according to the judgment patterns represented by the judgment logic example collection 34.

[0059] For example, the judgment logic generation unit 21C uses a pre-trained LLM (Learning Lore Machine) with the capability of in-context learning, which is based on the input rule information 30 and the judgment logic example collection 34, to input the input rule information 30 and the judgment logic example collection 34 into the LLM. The judgment logic generation unit 21C then obtains judgment logic 38 as output from the LLM. In other words, the judgment logic generation unit 21C generates judgment logic 38 to obtain a judgment result from the processing target through judgment processing in accordance with the judgment pattern represented by the judgment logic example collection 34 and the context of the input rule represented by the input rule information 30.

[0060] As described above, the determination logic 38 is logic for determining whether the state of the object to be determined, represented by the processing object such as an image, conforms to the input rules represented by the input rule information 30. In this embodiment, the case in which a decision tree is used in the determination logic will be explained as an example.

[0061] For example, the judgment logic generation unit 21C inputs the instruction "Generate judgment logic 38 from input rule information 30" to the LLM. Then, the judgment logic generation unit 21C inputs the judgment logic example collection 34 as an in-context example to the LLM. Finally, the judgment logic generation unit 21C inputs the input rule information 30 to the LLM, thereby obtaining the judgment logic 38 as an output from the LLM. Through these processes, the judgment logic generation unit 21C generates judgment logic 38 that judges the input rule information 30 according to the judgment patterns represented by the judgment logic example collection 34.

[0062] In this embodiment, the determination logic 38 defines one or more second processing elements 38B, the execution order of the multiple second processing elements 38B if there are multiple second processing elements 38B, the processing method used for processing the second processing elements 38B, and possible second determination results 38C.

[0063] The second processing element 38B represents a process to be performed on a processing target such as an image in order to determine whether or not the input rules represented by the input rule information 30 are being complied with. The determination process to be performed by the determination logic 38 is defined by a group of one or more second processing elements 38B included in the determination logic 38. The second processing element 38B includes judgment processes, processing processes, etc. The second processing element 38B may include processing methods used for said processing. The processing methods are, for example, object detection, VQA, etc., but are not limited to these.

[0064] The second determination result 38C is information representing the processing result of the second processing element 38B, which is a determination process, that is, the determination result that can be obtained by the determination process. In this embodiment, the possible second determination result 38C includes compliance, which indicates that the input rules are being followed, and non-compliance, which indicates that the input rules are not being followed.

[0065] In the example shown in Figure 6, the execution order of the second processing element 38B is represented by arrows connecting multiple second processing elements 38B. That is, when a decision tree is used in the decision logic 38, each of the one or more second processing elements 38B that make up the decision tree corresponds to a node, and the direction from the parent node to the child node corresponds to the execution order of the second processing elements 38B. Figure 6 shows an example of an execution order in which the second processing element 38B2, which is a child node of the second processing element 38B1, is executed after the second processing element 38B1, which is the parent node. Second processing elements 38B1 and 38B2 are examples of second processing elements 38B.

[0066] In the case of the judgment logic 38 shown in Figure 6, the input rule information 30 representing the input rule "Wearing gloves is prohibited while operating a drilling machine" is classified into two second processing elements 38B: "Are you working on a drilling machine?" and "Are you wearing gloves?". Two judgment processes, second processing element 38B1 "Process the question 'Are you working on a drilling machine?' using VQA" and second processing element 38B2 "Process the question 'Are you wearing gloves?' using VQA," are to be executed sequentially using VQA.

[0067] Furthermore, the judgment logic 38 generated based on the input rule information 30 and the judgment logic example collection 34 is not limited to the form shown in Figure 6.

[0068] Figure 7 is a schematic diagram of an example of another judgment logic 39.

[0069] As shown in Figure 7, the judgment logic 39 includes, as a second processing element 38B, a second processing element 38Ba which is a judgment process and a second processing element 38Bb which is a processing process. Also, Figure 7 shows, as an example of an execution order, the second processing element 38B11 is the parent node, followed by the second processing element 38B11 which is the child node of the second processing element 38B11, the second processing element 38B12 which is the child node of the second processing element 38B11, the second processing element 38B13 which is the child node of the second processing element 38B13, and the second processing element 38B14 which is the child node of the second processing element 38B13. The second processing elements 38Ba, 38Bb, and 38B11 to 38B14 are examples of second processing elements 38B.

[0070] As shown in Figure 7, the second processing element 38B may further include a processing step as the second processing element 38B. That is, the judgment logic acquisition unit 20A may use judgment logic 39 instead of judgment logic 38. The processing step is, but is not limited to, a process such as "cutting out only the person's part". Also, as shown in Figure 7, the judgment result of the second processing element 38Ba, which is the judgment step, may be "whether or not there is a person", and is not limited to a Yes or No result.

[0071] In this embodiment, one example of how the determination logic acquisition unit 20A acquires the determination logic 38 is described.

[0072] Returning to Figure 2, we continue the explanation.

[0073] The decision logic generation unit 21C may generate the decision logic 38 based on the input rule information 30 and the document data 32. For example, the decision logic generation unit 21C may generate the decision logic 38 in the same manner as described above by inputting the input rule information 30 and the document data 32 into an LLM that has the capability of in-context learning, which uses the document data 32 and a template representing the decision logic 38 represented by a decision tree or the like as shown in Figure 6.

[0074] Returning to Figure 1, we continue the explanation.

[0075] In this way, the judgment logic acquisition unit 20A acquires the judgment logic 38 by generating the judgment logic 38 based on the input rule information 30.

[0076] The determination logic acquisition unit 20A may acquire the determination logic 38 by receiving the determination logic 38 generated by an external information processing device, etc., from the information processing device, etc., via the communication unit 16. Alternatively, the determination logic acquisition unit 20A may acquire the determination logic 38 by reading the determination logic 38 that has been previously stored in the storage unit 12.

[0077] The image acquisition unit 20B acquires the object to be processed. As described above, in this embodiment, an example will be described in which an image captured by the shooting unit 18 is used as the object to be processed. For this reason, in this embodiment, the image acquisition unit 20B acquires the image captured by the shooting unit 18 as the object to be processed. The image acquisition unit 20B may sequentially acquire images from the shooting unit 18 as they are sequentially captured by the shooting unit 18. Alternatively, the image acquisition unit 20B may sequentially acquire each frame constituting a moving image or recorded video captured by the shooting unit 18 as an image. Furthermore, the image acquisition unit 20B may acquire multiple images captured from multiple viewpoints at the same time. Also, the image acquisition unit 20B may acquire an image by reading an image captured by the shooting unit 18 and stored in the storage unit 12 from the storage unit 12. Alternatively, the image acquisition unit 20B may acquire an image by receiving an image via the communication unit 16 from an external information processing device that stores an image captured by the shooting unit 18.

[0078] The logic execution unit 20C executes a judgment process according to the judgment logic 38 on the image to be processed, which was acquired by the image acquisition unit 20B. Executing a judgment process according to the judgment logic 38 means executing the judgment logic 38 on the image to be processed.

[0079] In detail, the logic execution unit 20C executes the judgment process in accordance with the judgment logic 38 by sequentially executing the judgment processes represented by each of the multiple second processing elements 38B included in the judgment logic 38 on the image in the execution order defined by the judgment logic 38. The logic execution unit 20C can execute the judgment logic 38 on the processing target using a processing method that uses a trained neural network such as VQA or object detection, which is represented by the second processing element 38B. Alternatively, the logic execution unit 20C may execute the judgment logic 38 using a technique such as Visual Programming, which combines multiple modules from the input rule information 30 to perform the desired processing. Furthermore, the logic execution unit 20C may execute the judgment process represented by the judgment logic 38 by constructing a processing flow or the like in accordance with the judgment logic 38.

[0080] The logic execution unit 20C performs a judgment process on the image in accordance with the judgment logic 38, thereby obtaining a second judgment result 38C that indicates whether the state of the object to be judged included in the image complies with or does not comply with the input rule represented by the input rule information 30.

[0081] The generation unit 20D generates output information that includes the judgment result obtained by a judgment process according to the judgment logic 38 for the image to be processed.

[0082] The output information is the information to be output to the output device 22. The output information includes information representing the determination result of the determination logic 38 for the processing target, such as an image.

[0083] For example, consider a case where the logic execution unit 20C performs a judgment process on the image to be judged in accordance with the judgment logic 38 shown in Figure 6. In this case, the logic execution unit 20C performs the processing of a plurality of second processing elements 38B represented by the judgment logic 38 on the image in the execution order defined by the judgment logic 38.

[0084] The generation unit 20D generates generated information that includes a judgment result representing "compliance" or "non-compliance," which is represented by the second judgment result 38C obtained by executing a judgment process in accordance with the judgment logic 38.

[0085] Furthermore, if the determination result indicates "non-compliance" with the input rules, the generation unit 20D generates corrective information to resolve the non-compliance based on at least one of the input rule information 30 and the determination logic 38. Then, it generates output information that further includes the corrective information.

[0086] For example, let's assume that the logic execution unit 20C performs a judgment process on the image to be judged according to the judgment logic 38 shown in Figure 6. Then, let's assume that the processing of multiple second processing elements 38B represented by the judgment logic 38 is executed in the execution order defined by the judgment logic 38, and that the second judgment result 38C from the second processing element 38B2 corresponding to an intermediate or terminal node is non-compliant.

[0087] In this case, for example, the generation unit 20D generates corrective information based on the input rule information 30, indicating that compliance with the input rule represented by the input rule information 30 is encouraged. The generation unit 20D may also generate corrective information in each of the multiple decision processes represented by the second processing element 38B, to guide the target of judgment so that the final second decision result 38C represents compliance. The generation unit 20D may also use at least some of the multiple second processing elements 38B defined by the decision logic 38 to generate corrective information to guide the target of judgment so that the final second decision result 38C represents compliance.

[0088] Alternatively, the generation unit 20D may prepare a template in advance for generating corrective information and generate the corrective information using this template. In this case, the generation unit 20D can identify the second processing element 38B that was determined to be non-compliant by working backward from the execution order shown in the determination logic 38, and generate the corrective information using the template. When using a template, the generation unit 20D may also use a dedicated database generated from the document data 32.

[0089] Furthermore, the generation unit 20D pre-prepares an LLM (Learning Language Machine) that has the capability of in-context learning, using input rule information 30, judgment logic 38, a second judgment result 38C included in the judgment logic 38, and a plurality of corrective action information which are proposed corrective actions. The generation unit 20D may then input the input rule information 30, at least a part of the judgment logic 38, the judgment result obtained by executing the judgment logic 38, and an example of corrective action information into the LLM as an in-context example, and obtain corrective action information as output from the LLM.

[0090] The output unit 20E outputs the generated information to the output device 22. As described above, an external information processing device connected via the storage unit 12, the UI unit 14, and the communication unit 16 functions as the output device 22. For example, when the output unit 20E displays output information on the display of the UI unit 14 as the output device 22, a display screen including the output information is shown on the display.

[0091] Figures 8A to 8C are explanatory diagrams illustrating an example of the output of output information 52.

[0092] Figure 8A is a schematic diagram of an example of a display screen 50A. Display screen 50A is an example of a display screen 50 displayed on the display of the UI unit 14. For example, consider a case where the judgment result of the judgment process on the image to be judged, obtained by executing a judgment process according to the judgment logic 38, is information indicating compliance with the input rules. In this case, for example, the generation unit 20D generates output information 52A that includes the judgment result 54 information "Compliant." which indicates compliance. Output information 52A is an example of output information 52. The output unit 20E displays the display screen 50A including the output information 52A on the display of the UI unit 14. The output unit 20E may also output audio representing the output information 52A from the speaker included in the UI unit 14.

[0093] Figure 8B is a schematic diagram of an example of display screen 50B. Display screen 50B is an example of display screen 50 displayed on the display of the UI unit 14. For example, consider a case where the judgment result of the judgment process for the image to be judged, obtained by executing a judgment process according to the judgment logic 38, is information indicating non-compliance with the input rules. In this case, for example, the generation unit 20D generates output information 52B that includes the judgment result 54 information "Non-compliance." and the corrective information 56 for the non-compliance, "Do not wear gloves while operating the drilling machine." Output information 52B is an example of output information 52. The output unit 20E displays the display screen 50A including the output information 52B on the display of the UI unit 14. The output unit 20E may also output audio representing the output information 52B from the speaker included in the UI unit 14.

[0094] Figure 8C is a schematic diagram of an example of a display screen 50C. Display screen 50C is an example of a display screen 50 displayed on the display of the UI unit 14. For example, consider a case where the judgment result of the judgment process on the image 60 that is the target of judgment, obtained by executing a judgment process according to the judgment logic 38, is information indicating non-compliance with the input rules. In this case, for example, the generation unit 20D may generate an overlaid image as output information 52C, in which an enhanced image E that emphasizes the part of the image 60 that indicates non-compliance is superimposed on the image 60 used in the judgment process, as information of the judgment result 54 indicating non-compliance. The generation unit 20D also generates output information 52D in the same manner as above, which includes the information of the judgment result 54 indicating non-compliance, "Non-compliance," and the corrective information 56 for the non-compliance, "Do not wear gloves while operating the drilling machine." Output information 52C and output information 52D are examples of output information 52.

[0095] The output unit 20E displays a display screen 50C containing the output information 52C on the display of the UI unit 14. Display screen 50C is an example of display screen 50. Display screen 50C may also include a button image B for receiving voice output instructions. When the user operates the UI unit 14 and the button image B is operated, the output unit 20E may output voice from the speaker included in the UI unit 14 representing the output information 52D, which includes the judgment result 54 indicating non-compliance, "Non-compliance," and the corrective information 56 for the non-compliance, "Do not wear gloves while operating the drilling machine."

[0096] Figures 8A to 8C illustrate an example in which the output unit 20E outputs a string of characters or audio representing the output information 52. However, the output format of the output information 52 is not limited to strings of characters or audio. For example, the output unit 20E may output an icon, image, animation, warning sound, etc., representing the output information 52.

[0097] Next, an example of the information processing flow performed by the information processing device 10 of this embodiment will be described.

[0098] Figure 9 is a flowchart showing an example of the information processing flow performed by the information processing device 10 of this embodiment.

[0099] The input rule acquisition unit 21A of the judgment logic acquisition unit 20A acquires input rule information 30 (step S100). The input rule acquisition unit 21A acquires input rule information 30 by receiving input rule information 30 input by the user through operation instructions of the UI unit 14, for example, from the UI unit 14.

[0100] The case study acquisition unit 21B acquires the judgment logic case study collection 34 (step S102).

[0101] The judgment logic generation unit 21C generates judgment logic 38 using LLM or the like based on the input rule information 30 obtained in step S100 and the judgment logic example collection 34 obtained in step S102 (step S104).

[0102] Next, the image acquisition unit 20B acquires an image from the shooting unit 18 (step S106).

[0103] The logic execution unit 20C performs a determination process on the image acquired by the image acquisition unit 20B in accordance with the determination logic 38 generated in step S104 (step S108).

[0104] The generation unit 20D generates output information 52 according to the result of the determination process in step S108 (step S110).

[0105] The output unit 20E outputs the output information 52 generated in step S112 to the output device 22 (step S112). As a result of the processing in step S112, for example, the output information 52 shown in Figures 8A to 8B is output to the output device 22.

[0106] The control unit 20 determines whether or not to terminate the process (step S114). For example, the control unit 20 performs the determination in step S114 by determining whether or not it has received a termination signal indicating the end of the process, such as an operation instruction from the user on the UI unit 14. If the determination in step S114 is negative (step S114: No), the process proceeds to step S116. In step S116, the image acquisition unit 20B acquires a new image from the shooting unit 18 (step S116) and proceeds to step S108. In step S106, the image acquisition unit 20B only needs to acquire an image for which the determination logic 38 in step S108 has not been executed.

[0107] On the other hand, if the decision in step S114 is affirmative (step S114: Yes), this routine terminates.

[0108] As described above, the information processing device 10 of this embodiment comprises a determination logic acquisition unit 20A, a logic execution unit 20C, and an output unit 20E. The determination logic acquisition unit 20A acquires a determination logic 38 that determines whether the state represented by the processing target (image 60) conforms to the input rules represented by the input rule information 30. The logic execution unit 20C executes a determination process on the processing target in accordance with the determination logic 38. The output unit 20E outputs output information including the determination result from the determination process to the output device 22.

[0109] In this embodiment, the information processing device 10 performs a determination process on the processing target (image 60) in accordance with the acquired determination logic 38 to determine whether the state represented by the processing target conforms to the input rules represented by the input rule information 30.

[0110] Therefore, users can input input rule information 30 without having to manually adjust or design the judgment logic 38 for each environment to be judged, and receive a judgment result indicating whether the state represented by the processing target conforms to the input rule represented by the input rule information 30. Furthermore, the information processing device 10 can execute judgment processing in accordance with the judgment logic 38 on the processing target without requiring manual adjustment of the judgment logic 38 by the user.

[0111] Therefore, the information processing device 10 of this embodiment can improve the efficiency of decision-making.

[0112] Furthermore, since the information processing device 10 of this embodiment does not require manual adjustment of the judgment logic 38 by the user, in addition to the above effects, it can reduce the burden on the user.

[0113] Furthermore, the information processing device 10 of this embodiment performs a determination process on the processing target (image 60) in accordance with the determination logic 38 to determine whether the state represented by the processing target conforms to the input rules represented by the input rule information 30. Therefore, in addition to the above effects, the information processing device 10 of this embodiment can provide a versatile information processing device 10 that can be applied regardless of the environment of the processing target.

[0114] In this embodiment, the determination logic 38 defines one or more second processing elements 38B, the execution order of the second processing elements 38B, the processing method used for processing the second processing elements 38B, and possible second determination results 38C.

[0115] Therefore, the information processing device 10 of this embodiment can automatically subdivide the context contained in the input rule information 30 by executing a determination process in accordance with the determination logic 38 on the input rule information 30, and execute a process represented by the second processing element 38B for each subdivided item.

[0116] Therefore, the information processing device 10 of this embodiment can reduce the burden on the user and provide an information processing device 10 that can be applied to various environments.

[0117] In this embodiment, the object to be determined is a person, the state of the object to be determined is whether or not the person is wearing gloves or other clothing during a specific task, and the input rule represented by the input rule information 30 is "Wearing gloves is prohibited during drilling operation." was described as an example. However, as mentioned above, the object to be determined may be a device, an object, a living being such as a person, a machine, various events existing in nature, etc., and is not limited to a person. Furthermore, the state of the object to be determined is not limited to information representing whether or not the person is wearing gloves or other clothing during a specific task.

[0118] For example, the object to be judged may be a machine, and the state of the object to be judged may be the state of the surroundings of the machine. For example, the input rule represented by the input rule information 30 may be an input rule concerning the state of the surroundings of "machine A", which is an object to be judged other than a person, such as "When operating machine A, do not place flammable materials nearby".

[0119] The application of the information processing device 10 of this embodiment is not limited. The information processing device 10 of this embodiment can be applied, for example, to determining whether various input rules are being followed in various environments such as work sites and offices.

[0120] Next, an example of the hardware configuration of the information processing device 10 (information processing device 10 to information processing device 10D) of the above embodiment will be described.

[0121] Figure 10 is a hardware configuration diagram of an example of the information processing device 10 of the above embodiment.

[0122] The information processing device 10 of the above embodiment includes a control device such as a CPU (Central Processing Unit) 90D, storage devices such as a ROM (Read Only Memory) 90E, RAM (Random Access Memory) 90F, and HDD (Hard Disk Drive) 90G, an I / F unit 90B which serves as an interface to various devices, an output unit 90A which outputs various information, an input unit 90C which accepts user operations, and a bus 90H which connects each unit, and has a hardware configuration that uses a normal computer.

[0123] In the information processing device 10 of the above embodiment, each of the above components is realized on the computer by the CPU 90D reading a program from the ROM 90E onto the RAM 90F and executing it.

[0124] The program for executing each of the above processes performed by the information processing device 10 in the above embodiment may be stored in the HDD 90G. Alternatively, the program for executing each of the above processes performed by the information processing device 10 in the above embodiment may be pre-installed and provided in the ROM 90E.

[0125] Furthermore, the program for executing the above-described process performed by the information processing device 10 of the above embodiment may be stored in an installable or executable file format on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disc), or flexible disk (FD), and provided as a computer program product. Alternatively, the program for executing the above-described process performed by the information processing device 10 of the above embodiment may be stored on a computer connected to a network such as the Internet and provided by allowing download via the network. Alternatively, the program for executing the above-described process performed by the information processing device 10 of the above embodiment may be provided or distributed via a network such as the Internet.

[0126] Although this embodiment has been described above, it is presented as an example and is not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of symbols]

[0127] 10 Information Processing Devices 20A Judgment Logic Acquisition Unit 21A Input Rule Acquisition Unit 21B Case collection acquisition department 21C Judgment Logic Generation Unit 20B Image acquisition section 20C Logic Execution Unit 20D generator 20E Output Section 22 Output device

Claims

1. A determination logic acquisition step to acquire determination logic that determines whether the state represented by the processing target conforms to the input rules represented by the input rule information, A logic execution step in which a determination process is performed on the processing target in accordance with the determination logic, An output step of outputting output information including the determination result from the aforementioned determination process to an output device, An information processing program that causes a computer to execute something.

2. The aforementioned step of obtaining the judgment logic is: The process includes a determination logic generation step that generates the determination logic based on the input rule information, The information processing program according to claim 1.

3. The aforementioned decision logic generation step is: Based on the input rule information and document data specifying multiple types of assumed rules relating to the input rules, a collection of judgment logic examples consisting of one or more assumed judgment logic examples is formed. The system generates a judgment logic that determines the input rules represented by the input rule information in accordance with the judgment patterns represented by the judgment logic example collection. The information processing program according to claim 2.

4. The aforementioned example of the judgment logic is: The following are defined for each assumed rule extracted from the document data: one or more first processing elements, the execution order of the first processing elements, the processing method used for processing the first processing elements, and possible first judgment results. The information processing program according to claim 3.

5. The aforementioned decision logic generation step is: The process includes a step of acquiring examples to generate the collection of judgment logic examples based on the aforementioned document data, The information processing program according to claim 3.

6. The aforementioned determination logic is: The system comprises one or more second processing elements, the execution order of the second processing elements, a processing method used for processing the second processing elements, and possible second determination results. The information processing program according to claim 1.

7. A generation step of generating output information including the determination result obtained by the determination process in accordance with the determination logic for the processing target, The information processing program according to claim 1, which includes the following:

8. The generation step is, If the determination result indicates non-compliance with the input rule, corrective information for resolving the non-compliance is generated based on at least one of the input rule information and the determination logic, and output information further including the corrective information is generated. The information processing program according to claim 7.

9. An information processing method performed by an information processing device, A determination logic acquisition step to acquire determination logic that determines whether the state represented by the processing target conforms to the input rules represented by the input rule information, A logic execution step in which a determination process is performed on the processing target in accordance with the determination logic, An output step of outputting output information including the determination result from the aforementioned determination process to an output device, Information processing methods including

10. A determination logic acquisition unit acquires a determination logic that determines whether the state represented by the processing target conforms to the input rules represented by the input rule information, A logic execution unit that executes a determination process in accordance with the determination logic for the target of processing, An output unit that outputs output information including the determination result from the aforementioned determination process to an output device, An information processing device equipped with the following features.

Images