Information processing device, information processing method, and program

Abstract

The present invention provides an information processing device, an information processing method, and a program that enable the determination of risks according to the nature and location of the work. [Solution] The information processing device includes an acquisition means for acquiring at least one of a document and an image related to a task, and an analysis means for generating a determination criterion for at least one of an action and a state in a task, which is determined according to the environmental attributes of the task, using a trained model from at least one of the documents and images related to the task acquired by the acquisition means.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, from the viewpoint of safety management at work sites, monitoring using surveillance cameras has been carried out. When a monitor checks the video captured by a surveillance camera and discovers a dangerous action or the like, appropriate measures are taken, such as issuing a warning to the target person or the person in charge of the site, or sending security guards to the site. In recent years, monitoring using a system that automatically discriminates and detects dangerous actions and states from the video captured by a camera has also been carried out.

[0003] In Patent Document 1, a hook of a crane is disposed at least horizontally away from a movable space, and in an image generated by a camera that photographs the space at least horizontally, a suspended load and a person suspended by the hook are identified, and based on the positional relationship between the identified suspended load and the person, a technique for determining whether it is a dangerous state is proposed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] What actions and states are dangerous actions and states differ depending on the work content and location. It is difficult to define the corresponding dangerous actions and states for each work content and location and construct a system to detect dangerous actions and states. An object of the present invention is to provide an information processing apparatus, an information processing method, and a program that enable danger determination according to work content, location, and the like.

Means for Solving the Problems

[0006] The information processing device according to the present invention is characterized by comprising: an acquisition means for acquiring at least one of a document and an image relating to a task; and an analysis means for generating a determination criterion relating to at least one of an action and a state in the task, which corresponds to the environmental attributes of the task, using a trained model obtained from at least one of the document and the image relating to the task acquired by the acquisition means. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an information processing device, an information processing method, and a program that enable the determination of risks according to the content of work, location, etc. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing an example of the configuration of an information processing system. [Figure 2] This figure shows an example of the hardware configuration of an information processing device. [Figure 3] This figure shows an example of the functional configuration of a server device. [Figure 4] This diagram illustrates an example of processing performed by a server device. [Figure 5] This is a diagram illustrating the generation of judgment criteria. [Figure 6] This diagram illustrates an example of processing performed by a server device. [Figure 7] This is a diagram illustrating an example of its application. [Figure 8] This is a diagram illustrating an example of its application. [Figure 9] This is a diagram illustrating an example of its application. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings.

[0010] Figure 1 shows an example configuration of an information processing system 100 in one embodiment of the present invention. The information processing system 100 in this embodiment includes a server device 101, one or more terminal devices 102, and one or more cameras 103. The information processing system 100 may also include one or more sensors 104. The server device 101, terminal devices 102, cameras 103, and sensors 104 are connected to each other via a network NW so that they can communicate with one another. The type of network NW is not particularly limited, as long as it is capable of connecting each of the devices constituting the information processing system 100 so that they can communicate with one another.

[0011] The server device 101 generates judgment criteria based on documents and images related to the work, according to environmental attributes (work location, conditions, etc.). The server device 101 also makes judgments regarding the risks of actions and conditions during the work based on the generated judgment criteria and images captured by the camera 103. In this embodiment, images may include both still images and moving images. Furthermore, judgment criteria may be generated not only based on documents and images, but also on audio, etc.

[0012] For example, the server device 101 takes existing documents and images related to work, such as work manuals, as input and performs inference using a pre-trained model called a large-scale multimodal model (LMM) to extract prohibited actions and recommended actions from the input documents and images and generate judgment criteria. Prohibited actions are actions that are prohibited in the work, for example, defined as "do not do...", and recommended actions are actions that are recommended in the work, for example, defined as "should do...". The large-scale multimodal model is a pre-trained model built using a large amount of training data and deep learning technology. When data called prompts, which indicate instructions or commands, are input, it performs inference based on the prompts and generates and outputs data corresponding to the input prompts.

[0013] Furthermore, for example, the server device 101 determines whether the actions and conditions in the work meet the criteria based on images of the actual work situation captured by the camera 103, and makes a hazard determination according to the environmental attributes (work location, conditions, etc.). The server device 101 also outputs the determination result to the terminal device 102, etc. The server device 101 may perform inference using a large-scale multimodal model on the captured images, etc. to make the determination, or it may perform analysis of the captured images, etc. using machine learning image analysis to make the determination. The server device 101 may also determine whether the actions and conditions in the work meet the criteria based on signals from the sensor 104.

[0014] Terminal device 102 schematically represents a device that serves as an input / output interface for a user to use server device 101. For example, terminal device 102 may be used as an input / output interface for receiving various instructions from the user to server device 101 and for providing the user with various information such as judgment results output by server device 101. Terminal device 102 can consist of, for example, a personal computer (PC), a tablet terminal, a smartphone, etc.

[0015] Camera 103 is an imaging device that captures images within the workspace (workplace, work site), which is the area of ​​interest. Images captured by camera 103 are output to server device 101 via network NW. Note that camera 103 is not limited to video cameras, but may also be a tablet terminal, smartphone, etc., with a shooting function. Sensor 104 is installed in the workspace (workplace, work site) and may, for example, be a sensor that detects the intrusion of an object into the workspace or a sensor that detects the presence or absence of an object in the workspace. Signals from sensor 104 are output to server device 101 via network NW. Note that images captured by camera 103 and signals from sensor 104 may be output to terminal device 102, and then output from terminal device 102 to server device 101.

[0016] FIG. 2 is a diagram showing an example of the hardware configuration of an information processing apparatus 200 applicable as the server apparatus 101 and the terminal apparatus 102 in the information processing system 100 shown in FIG. 1. The information processing apparatus 200 includes a CPU 201, a ROM 202, a RAM 203, an auxiliary storage device 204, and a network I / F 207. Further, the information processing apparatus 200 may include at least one of an output device 205 and an input device 206. The CPU 201, the ROM 202, the RAM 203, the auxiliary storage device 204, the output device 205, the input device 206, and the network I / F 207 are communicably connected via a system bus 208.

[0017] The CPU (Central Processing Unit) 201 is a central arithmetic unit that controls various operations of the information processing apparatus 200. For example, the CPU 201 may control the operations of the entire information processing apparatus 200. The ROM (Read Only Memory) 202 stores control programs, boot programs, etc. that are executable by the CPU 201. The RAM (Random Access Memory) 203 is the main memory of the CPU 201 and is used as a work area or a temporary storage area for developing various programs.

[0018] The auxiliary storage device 204 stores various data and various programs, etc. The auxiliary storage device 204 is realized by a storage device that can store various data temporarily or persistently, such as an HDD (Hard Disk Drive) or a non-volatile memory typified by an SSD (Solid State Drive).

[0019] The output device 205 is a device that outputs various types of information and is used for presenting various types of information to the user. For example, the output device 205 is realized by a display device such as a display. The output device 205 may present information to the user by displaying various display information. As another example, the output device 205 may be realized by an acoustic output device that outputs sounds such as voices and electronic sounds. In this case, the output device 205 may present information to the user by outputting sounds such as voices and electronic sounds. Also, the device applied as the output device 205 may be appropriately changed according to the medium used to present information to the user.

[0020] The input device 206 is used for receiving various instructions from the user. For example, the input device 206 may include input devices such as a mouse, a keyboard, and a touch panel. As another example, the input device 206 may include a sound collection device such as a microphone and may collect the voice spoken by the user. In this case, various analysis processes such as acoustic analysis and natural language processing may be performed on the collected voice, and the content indicated by this voice may be recognized as an instruction from the user. Also, the device applied as the input device 206 may be appropriately changed according to the method of recognizing instructions from the user. Also, multiple types of devices may be applied as the input device 206.

[0021] The network I / F 207 is used for communication with external devices via a network. Note that the device applied as the network I / F 207 may be appropriately changed according to the type of communication path and the applied communication method.

[0022] The CPU 201 loads the program stored in the ROM 202 or auxiliary storage device 204 into the RAM 203 and executes the program, thereby realizing the functions and processes of each device described later. The program for the information processing device 200 may be provided to the information processing device 200 by a recording medium such as a CD-ROM, or it may be downloaded via a network or the like. When the program for the information processing device 200 is provided by a recording medium, the program recorded on the recording medium is installed in the auxiliary storage device 204 when the recording medium is set in a predetermined drive device.

[0023] The configuration shown in Figure 2 is merely an example and does not necessarily limit the hardware configuration of the information processing device 200 in this embodiment. For example, some components such as the output device 205 and the input device 206 may be omitted. Another example is that configurations may be added as appropriate depending on the functions to be realized by the information processing device 200.

[0024] Figure 3 shows an example of the functional configuration of the server device 101 in this embodiment. The server device 101 includes a communication unit 301, a control unit 302, an input / output control unit 303, a storage unit 304, a first acquisition unit 305, an analysis unit 306, a second acquisition unit 307, and a determination unit 308.

[0025] The communication unit 301 is a communication interface for each component of the server device 101 to send and receive information with other devices such as the terminal device 102 via a network NW. The communication unit 301 can be implemented, for example, by a network I / F 207. In the following description, unless otherwise specified, when each component of the server device 101 sends and receives information with other devices, it will be assumed that the information is sent and received via the communication unit 301.

[0026] The control unit 302 is responsible for controlling each component of the server device 101. The input / output control unit 303 performs various processes related to presenting various information to the user and receiving information input from the user (e.g., instructions). For example, the input / output control unit 303 may perform processes related to presenting the UI (User Interface) and processes related to receiving input via the UI. This enables the server device 101 to recognize instructions from the user and present the results of processing corresponding to those instructions to the user.

[0027] The memory unit 304 schematically represents a memory area for storing various data and programs. For example, the memory unit 304 may store data and programs for each component of the server device 101 to execute processing. The memory unit 304 may also store trained models used for inference regarding processing performed in the server device 101. Furthermore, the memory unit 304 may store documents and images related to the work acquired by the first acquisition unit 305, and images captured by the camera 103 acquired by the second acquisition unit 307. The memory unit 304 may also store judgment criteria generated by the analysis unit 306 and judgment results from the judgment unit 308.

[0028] The first acquisition unit 305 acquires documents and images related to work, such as work manuals, which are used to generate judgment criteria. Documents related to work may include, for example, work standards that describe work content, and safety rules to ensure safety during work. Images related to work may include images showing work content and precautions (prohibited actions), and safety education videos that record demonstrated violations. The first acquisition unit 305 also acquires information on environmental attributes such as the work location and conditions to which the judgment criteria are applied. If information on environmental attributes (work location, conditions, etc.) can be obtained from documents and images related to work, the unit may also acquire information on environmental attributes from the acquired documents and images. In addition, information such as the time of day during which work is performed may be acquired as information on environmental attributes.

[0029] The analysis unit 306 performs inference using a trained model (e.g., a large-scale multimodal model) on documents and images related to the work and generates judgment criteria according to environmental attributes (work location, state, etc.). The analysis unit 306 extracts rules, prohibited actions, recommended actions, etc. from documents and images related to the work acquired by the first acquisition unit 305 using the trained model and generates judgment criteria according to environmental attributes (work location, state, etc.). The analysis unit 306 may also use the trained model to extract rules, prohibited actions, recommended actions, etc., along with their corresponding states from the acquired documents and images related to the work, and use the states and rules, prohibited actions, and recommended actions as a set of judgment criteria.

[0030] The second acquisition unit 307 acquires images of the actual work situation, etc., which are used to determine actions and conditions during work. In this embodiment, the second acquisition unit 307 acquires images taken by a camera 103 that photographs the work space (work location, work site). The second acquisition unit 307 may also acquire signals from a sensor 104 installed in the work space (work location, work site).

[0031] The determination unit 308 determines whether the work is being performed in accordance with the criteria, using the determination criteria generated by the analysis unit 306, based on images of the actual work situation acquired by the second acquisition unit 307. The determination unit 308 analyzes the images acquired by the second acquisition unit 307 to determine whether the actions and conditions in the actual work meet the determination criteria. The determination unit 308 also controls the system to notify users such as work supervisors and workers of the violation level (warning, caution, normal, etc.) based on the determination result as needed. Alternatively, the second acquisition unit 307 may acquire multiple images of the actual work situation with different angles of view, and the determination unit 308 may determine whether the actions and conditions in the actual work meet the determination criteria based on the combination of the acquired multiple images with different angles of view. Furthermore, if the judgment criteria consist of a set of rules, prohibited actions, recommended actions, and corresponding states, the judgment unit 308 can perform a judgment on the state in the actual work using the judgment criteria, and if the state satisfies the judgment criteria, it can then perform a judgment on the action in the actual work using the judgment criteria. For example, when making a judgment on the judgment criterion, "When performing work at height in the factory, a safety harness must be worn," the judgment unit 308 can determine whether or not it is work at height based on images of the actual work situation, and if it determines that it is work at height, it can then determine whether or not the worker is wearing a safety harness. By performing the judgment on the judgment criteria in two stages in this way, it becomes possible to perform judgments under more detailed conditions.

[0032] In the example described above, the trained model used for inference (for example, a large-scale multimodal model) is configured to be stored in the server device 101. However, it may also be configured to be stored in another information processing device that can communicate with the server device 101 via a network. In this case, the server device 101 should send input to the trained model to the other information processing device that stores the trained model via the network interface 207, etc., and receive the processing results for that input.

[0033] Figure 4 is a flowchart showing an example of processing by the server device 101 in this embodiment. Figure 4 shows an example of processing that generates judgment criteria according to environmental attributes (work location, condition, etc.) based on documents and images related to the work. By executing the processing shown in Figure 4, judgment criteria corresponding to each environmental attribute (work location, condition, etc.) can be generated. Even if the work location and work content are the same or similar, business rules and work rules may differ from company to company or factory to factory. In such cases, the corresponding judgment criteria can be generated by inputting manuals, etc., specific to that company or factory.

[0034] In step S401, the first acquisition unit 305 acquires documents, images, etc., related to the work of generating judgment criteria. Documents and images related to the work include, for example, existing documents and images related to the work, such as work manuals. The documents and images to be acquired may include, for example, documents and images showing work content, documents and images showing safety rules to ensure safety, and images recorded in safety education videos.

[0035] In step S402, the first acquisition unit 305 acquires information regarding the environmental attributes to which the generated judgment criteria apply. This information regarding environmental attributes may include information such as the work location, work conditions, and work time period to which the judgment criteria apply. This information regarding environmental attributes is set, for example, by the user via a terminal device 102 or the like.

[0036] Note that the order in which steps S401 and S402 are performed is not limited to the example described above, and the order in which steps S401 and S402 are performed is not limited to any particular order. Furthermore, if information regarding environmental attributes can be obtained from documents, images, etc. related to the work acquired in step S401, information regarding environmental attributes may be obtained from documents, images, etc. acquired in step S401.

[0037] In step S403, the analysis unit 306 generates judgment criteria using a trained model (e.g., a large-scale multimodal model) based on the documents and images related to the work acquired in step S401 and the environmental attribute information acquired in step S402. The analysis unit 306 takes the acquired documents and images related to the work as input and generates judgment criteria according to the environmental attributes by giving prompts to the trained model to extract rules, prohibited actions, recommended actions, etc. from the input documents and images. For example, actions corresponding to sentences such as "You must not do...", "You must not do...", "...is prohibited", "...must do...", and "...is requested..." are extracted from the text contained in the input documents and images and used as judgment criteria. Alternatively, dangers hidden in the input image may be extracted by inference using the trained model and used as judgment criteria. For example, suppose the manual contains the text shown in Figure 5(A). By providing the trained model with a manual containing text as shown in Figure 5(A) and prompts as shown in Figure 5(B), rules as shown in Figure 5(C) are extracted as rules for the generated judgment criteria. If the amount of data such as documents and images related to the work is large, the analysis unit 306 may divide the documents and images as appropriate and generate the judgment criteria using the trained model in multiple steps.

[0038] In inference using pre-trained models such as large-scale multimodal models, there is a method called few-shot learning that improves the accuracy of responses by including examples (samples) in the prompts, which allows the model to return an output adapted to the presented examples (samples) with high accuracy. Therefore, prompts showing positive and negative training samples can be provided to the pre-trained model as examples for few-shot learning. In this case, it is possible to improve the accuracy of extracting rules, prohibited actions, recommended actions, etc. from input documents, images, etc.

[0039] In step S404, the analysis unit 306 stores the judgment criteria generated in step S403 in the storage unit 304. After executing the process in step S404, the process shown in Figure 4 is terminated.

[0040] Figure 6 is a flowchart showing an example of processing by the server device 101 in this embodiment. Figure 6 shows an example of processing that determines whether the actual work conforms to the criteria using judgment criteria according to environmental attributes.

[0041] In step S601, the second acquisition unit 307 retrieves judgment criteria from the storage unit 304 that correspond to the environmental attributes generated and stored by executing the flowchart shown in Figure 4. In step S602, the second acquisition unit 307 acquires images of the actual work situation captured by the camera 103 that photographs the work space (work location, work site). Note that the order in which steps S601 and S602 are performed is not limited to the example described above, and the order in which steps S601 and S602 are performed is not limited to any particular order.

[0042] In step S603, the determination unit 308 determines whether the actual work conforms to the criteria based on the criteria corresponding to the environmental attributes acquired in step S601 and the images of the actual work situation acquired in step S602. For example, the determination unit 308 makes a determination regarding the dangers of the actions and conditions (danger determination) by determining whether the actions and conditions in the work meet the criteria based on the criteria corresponding to the environmental attributes and the images taken by the camera 103.

[0043] In step S603, the determination unit 308 performs object detection, for example, by image analysis using machine learning techniques on the acquired image, to determine whether the actions or states in the actual work meet the criteria. Alternatively, the determination unit 308 may take the acquired image and the criteria as input and provide a trained model (e.g., a large-scale multimodal model) with prompts to extract actions or states that violate the criteria from the input image, thereby determining whether the actions or states in the actual work meet the criteria. Alternatively, for example, the second acquisition unit 307 may acquire signals from sensors 104 installed in the work space (work location, work site), and the determination unit 308 may detect intrusion actions, etc. that violate the criteria based on the signals from sensors 104. For example, an observer may monitor images captured by the camera 103 to determine whether the criteria are met, and the determination unit 308 may be notified if actions or states that violate the criteria are detected.

[0044] In step S604, the determination unit 308 determines, based on the determination made in step S603, whether or not there are any matters to be notified to users such as work supervisors and workers. The matters to be notified may include, for example, the violation level of the judgment criteria (warning, caution, normal, etc.), and the determination of whether or not there are matters to be notified should be made according to the setting of which violation level to notify the user. If the determination unit 308 determines that there are matters to be notified to users (YES), the process in step S605 is executed. On the other hand, if the determination unit 308 determines that there are no matters to be notified to users (NO), the process shown in Figure 6 is terminated.

[0045] In step S605, the determination unit 308 controls the system to notify the user of the notification items via the input / output control unit 303. At this time, the determination unit 308 may also control the system to output different notification items to the user depending on the level of violation of the judgment criteria. After executing the process in step S605, the process shown in Figure 6 is terminated.

[0046] According to this embodiment, by using a trained model based on existing documents and images related to the work to generate judgment criteria for work according to environmental attributes (work location, conditions, etc.), it becomes possible to easily generate judgment criteria for various work content and locations. Furthermore, by using these judgment criteria for various work content and locations to determine whether the criteria are met for actions and conditions in actual work, it becomes possible to detect dangerous actions and perform risk assessment according to the work content and location.

[0047] Furthermore, if it is prohibited for unqualified persons to perform the actual work, for example, the worker can be asked to present proof of qualification (such as a license) in front of camera 103 and photographed. If it cannot be confirmed that the person is qualified, it can be determined that there has been a violation of the judgment criteria.

[0048] The following describes specific application examples. As an example, we will show how to generate criteria for prohibited actions from a factory work manual (document) and then make judgments regarding actual work. The factory work manual is assumed to include not only prohibited actions but also normal work procedures.

[0049] The first acquisition unit 305 acquires the factory's work manual as a document related to the work. Next, the analysis unit 306 generates judgment criteria using a trained model (for example, a large-scale multimodal model) based on the acquired factory's work manual. At this time, the analysis unit 306 takes the factory's work manual as input and provides the trained model with prompts as shown in Figure 7(A) to extract prohibited actions from the factory's work manual. An example of the extracted prohibited actions is shown in Figure 7(B). The analysis unit 306 generates judgment criteria related to the work using the example of prohibited actions shown in Figure 7(B) as judgment criteria.

[0050] Next, when the actual work is performed, the second acquisition unit 307 acquires judgment criteria related to the work and images of the actual work situation captured by the camera 103. Then, the judgment unit 308 takes the acquired judgment criteria related to the work and images of the actual work situation as input and monitors whether prohibited actions are being performed in the actual work by giving prompts as shown in Figure 7(C) to the trained model. If a violation is detected, the judgment unit 308 controls the system to produce an output as shown in Figure 7(D) and notifies the user, such as a work supervisor. The timing for giving prompts as shown in Figure 7(C) to the trained model can be as follows: for example, if the system is continuously monitoring images of the actual work situation in real time, the prompt should be given to the trained model at the start of monitoring. Alternatively, the prompt may be given to the trained model at any necessary timing. Furthermore, for example, if post-checking (review check) is performed on recorded images of the actual work situation, the prompt should be given to the trained model at the start of the review.

[0051] As another example, we will show an example of generating criteria for prohibited actions from a safety training video (moving image) and making judgments on actual work. In the following example, we will use a safety training video that explains the correct way to wear a safety harness, and assume that this safety training video includes demonstrations of violations.

[0052] The first acquisition unit 305 acquires images recorded in the safety training video as images related to the work. Next, the analysis unit 306 generates judgment criteria using a trained model based on the acquired images from the safety training video. At this time, the analysis unit 306 takes the images from the safety training video as input and extracts prohibited actions by giving the trained model prompts as shown in Figure 8(A). An example of the extracted prohibited actions is shown in Figure 8(B). The analysis unit 306 generates judgment criteria related to the work using the example of prohibited actions shown in Figure 8(B) as judgment criteria.

[0053] Next, when the actual work is performed, the second acquisition unit 307 acquires judgment criteria related to the work and images of the actual work situation captured by the camera 103. Then, the judgment unit 308 takes the acquired judgment criteria related to the work and images of the actual work situation as input and provides a prompt to the trained model as shown in Figure 8(C) to check if there are any violations, or in other words, whether the safety harness is properly worn. If a violation is detected, the judgment unit 308 controls the system to output a message to that effect and notifies the user, such as a work supervisor.

[0054] As another example, we will show an example in which criteria for determining prohibited actions are generated from images taken of clothing and equipment worn when working at heights, and a judgment is made regarding the worker's clothing and equipment before work begins. The first acquisition unit 305 acquires images of workers wearing clothing and equipment related to working at heights as images related to the work. Next, the analysis unit 306 generates judgment criteria using a trained model based on the acquired images of workers. At this time, the analysis unit 306 takes the images of workers wearing clothing and equipment related to working at heights as input and gives prompts as shown in Figure 9(A) to the trained model to extract prohibited actions. An example of the extracted prohibited actions is shown in Figure 9(B). The analysis unit 306 generates judgment criteria related to the work using the example of prohibited actions shown in Figure 9(B) as judgment criteria.

[0055] Next, the second acquisition unit 307 acquires judgment criteria related to the work and images of the actual worker captured by the camera 103. For example, before the start of work, images are taken to confirm whether the helmet and safety harness are properly fitted while the worker is wearing them (for example, images are taken in front of the camera 103 in a predetermined working posture), and these captured images are acquired. Next, the judgment unit 308 takes the acquired judgment criteria related to the work and images of the actual worker as input and provides a prompt to the trained model as shown in Figure 9(C) to check if there are any violations, or in other words, whether the appropriate clothing and equipment for working at height are being worn. If a violation is detected, the judgment unit 308 controls the system to output a message to that effect and notifies the user, such as a work supervisor.

[0056] It should be noted that the embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features. [Explanation of symbols]

[0057] 100 Information Processing Systems 101 Server device 102 Terminal device 103 Camera 104 Sensors 201 CPU 202 ROM 203 RAM 204 Auxiliary storage 205 Output device 206 Input Device 207 Network I / F 301 Communications Department 302 Control Unit 303 Input / Output Control Unit 304 Storage section 305 First acquisition section 306 Analysis Department 307 Second acquisition section 308 Judgment section

Claims

1. means for acquiring at least one of documents and images related to the work, An information processing apparatus comprising: an analysis means that generates a determination criterion relating to at least one of the actions and states in the work, which corresponds to the environmental attributes of the work, using a trained model obtained from at least one of the documents and images relating to the work acquired by the acquisition means.

2. The information processing apparatus according to claim 1, wherein the analysis means takes at least one of the document and the image relating to the work acquired by the acquisition means as input, provides the trained model with a prompt to extract at least one of the prohibited action and the recommended action from the input, causes it to infer, and generates the judgment criteria.

3. The information processing apparatus according to claim 2, characterized in that the analysis means provides the trained model with the prompt including the training sample to generate the decision criterion.

4. The information processing device according to claim 2, characterized in that the analysis means extracts a corresponding state along with the prohibited action or the recommended action from the input, and uses the state and the prohibited action or the recommended action as a set to form the judgment criterion.

5. The information processing device according to claim 1, characterized in that the trained model is a large-scale multimodal model.

6. A determination means that determines whether at least one of the actions and conditions in the work meets the determination criteria based on images taken of the work status of the work, The information processing apparatus according to any one of claims 1 to 5, further comprising an output means for outputting the determination result obtained by the determination means.

7. The information processing apparatus according to claim 6, characterized in that the determination means analyzes images of the work status of the work to detect at least one of the actions and states in the work and determines whether or not the determination criteria are met.

8. The information processing apparatus according to claim 6, characterized in that the determination means determines whether at least one of the actions and states in the work satisfies the determination criteria using a trained model.

9. The information processing apparatus according to claim 6, characterized in that the determination means determines whether or not the determination criteria are met based on a combination of multiple images with different angles of view in which the work status of the work is captured.

10. The information processing apparatus according to claim 6, characterized in that the output means outputs differently depending on the level of violation with respect to the judgment criteria.

11. An information processing method performed by an information processing device, A process of acquiring at least one of documents and images related to the work, An information processing method characterized by comprising: an analysis step of generating a determination criterion for at least one of the actions and states in the work, which corresponds to the environmental attributes of the work, using a trained model obtained from at least one of the documents and images related to the work acquired in the acquisition step.

12. A determination step, based on images taken of the work status of the said work, determines whether at least one of the actions and conditions in the said work meets the determination criteria, The information processing method according to claim 11, further comprising an output step for outputting the determination result in the determination step.

13. On the computer, Acquisition step of obtaining at least one of documents and images related to the work, A program for performing an analysis step that generates a determination criterion for at least one of the actions and states in the work, which is determined according to the environmental attributes of the work, using a trained model obtained from at least one of the documents and images related to the work acquired in the acquisition step.

14. A determination step in which, based on images taken of the work status of the said work, it is determined whether or not at least one of the actions and conditions in the said work meets the determination criteria, The program according to claim 13, which further executes an output step that outputs the determination result in the determination step.

Images