Information processing apparatus and information processing method

Abstract

To reduce the burden of analyzing the cause when a predetermined event occurs.SOLUTION: An information processing apparatus includes a first generation unit, a second generation unit, and a third generation unit. The first generation unit generates a character string that describes the situation in which a predetermined event occurred, from a video image captured at the event occurred. The second generation unit generates a prompt for generating a report on the predetermined event using the character string generated by the first generation unit. The third generation unit generates a report using the prompt generated by the second generation unit.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] Conventionally, there is known a technology that uses an AI technology (image analysis AI) called VQA (Visual Question Answering) to estimate a specific state from video. For example, it is known to use this technology to detect industrial accidents (near misses) and estimate equipment conditions that violate safety manuals or dangerous conditions. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-071675 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, there is room for further improvement in order to reduce the burden of the work of analyzing the cause when a predetermined event occurs.

[0005] The present application has been made in view of the above, and aims to provide an information processing device and an information processing method that can reduce the burden of analyzing the cause of a predetermined event when it occurs. [Means for solving the problem]

[0006] The information processing device of the present application is characterized by having a first generation unit that generates a string of characters explaining the situation at the time of occurrence of a specified event from a video image of the event, a second generation unit that generates a prompt for generating a report about the specified event using the string of characters generated by the first generation unit, and a third generation unit that generates a report using the prompt generated by the second generation unit. [Effects of the Invention]

[0007] According to one aspect of the embodiment, it is possible to reduce the burden of analyzing the cause of a predetermined event when it occurs. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of information processing according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a prompt according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a report according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of the configuration of a terminal device according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of the configuration of an information processing device according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a VQA model storage unit according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a past accident case storage unit according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a template storage unit according to the embodiment. [Figure 10] FIG. 10 is a flowchart (part 1) illustrating an example of information processing according to the embodiment. [Figure 11] FIG. 11 is a flowchart (part 2) illustrating an example of information processing according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, a detailed description will be given of an information processing device and an information processing method according to the present application (hereinafter referred to as an "embodiment") with reference to the drawings. Note that the information processing device and the information processing method according to the present application are not limited to the embodiment. Furthermore, the same components in the following embodiments are denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0010] Conventionally, there is known a technology that uses an AI technology called VQA to estimate a specific state from video. For example, in order to meet the need to detect equipment conditions that violate safety manuals or dangerous conditions, a technology is known that estimates equipment conditions that violate safety manuals or dangerous conditions by installing cameras at specific sites such as manufacturing sites, work sites, warehouses, and factories and using them to detect industrial accidents.

[0011] VQA is a technology that integrates computer vision and natural language processing, accepting questions in natural language about objects or scenes in a video and generating answers to those questions. Specifically, when a video and a question are input, VQA generates an answer to the question based on the video. For example, when VQA receives a video of a worker wearing gloves and the question "Are you wearing gloves?", it will output "YES" based on the video.

[0012] However, conventional occupational accident detection systems using AI technologies such as VQA only report the occurrence of an occupational accident to the on-site supervisor, and the on-site supervisor is still responsible for analyzing the cause and considering measures to prevent recurrence. This application has been made in light of the above, and aims to reduce the burden on the on-site supervisor of the work of analyzing the cause of an occupational accident.

[0013] In the following embodiment, AI technology such as VQA and generative AI technology such as GPT (Generative Pre-trained Transformer) are utilized to automate a series of tasks for recording industrial accidents, analyzing their causes, and generating reports that include consideration of measures to prevent recurrence, thereby reducing the burden on on-site managers of analyzing the causes of industrial accidents. Specifically, the generative AI technology automatically generates reports on industrial accidents based on information about industrial accidents at specific work sites detected by AI technology such as VQA. Note that although GPT is used as an example of generative AI technology, other LLMs (Large Language Models) may also be used.

[0014] (Embodiment) [Configuration of information processing system] An information processing system 1 shown in Fig. 1 will be described. As shown in Fig. 1, the information processing system 1 includes a terminal device 10 and an information processing device 100. The terminal device 10 and the information processing device 100 are connected to each other via a predetermined communication network (network N) so as to be able to communicate with each other via wired or wireless communication. Fig. 1 is a diagram showing an example of the configuration of the information processing system 1 according to an embodiment.

[0015] The terminal device 10 is an information processing device used by a user such as a site manager who performs cause analysis work when a predetermined event (such as an industrial accident) occurs. The user is, for example, a site manager who is required to submit a report on an industrial accident in order to analyze the cause and consider measures to prevent recurrence when an industrial accident occurs. The terminal device 10 may be any device that can realize the processing in the embodiment. The terminal device 10 may be, for example, a smartphone, a tablet terminal, a notebook PC, a desktop PC, a mobile phone, a PDA, or other device. FIG. 2 shows a case where the terminal device 10 is a smartphone.

[0016] The terminal device 10 is, for example, a smart device such as a smartphone or smart glasses, and is a portable terminal device capable of communicating with any server device via a wireless communication network such as 4G to 5G (Generations), or LTE (Long Term Evolution). The terminal device 10 may have a screen such as a liquid crystal display with a touch panel function, and may accept various operations on displayed data such as content, such as tapping, sliding, and scrolling, performed by a user using a finger or a stylus. In FIG. 2, the terminal device 10 is used by a user U1.

[0017] The information processing device 100 is an information processing device designed to reduce the burden of cause analysis work when a predetermined event occurs. Any device may be used as long as it can implement the processes described in the embodiments. The information processing device 100 performs a series of information processing operations to generate a report that includes recording of work-related accidents, analyzing their causes, and considering measures to prevent recurrence, for example, by utilizing AI technology such as VQA and generative AI technology. For example, the information processing device 100 automatically generates a report on a work-related accident using generative AI technology based on information about a work-related accident at a specific work site detected by AI technology such as VQA. Specifically, when the information processing device 100 acquires a video of a predetermined event, it uses AI technology such as VQA to generate a string (e.g., a sentence) from the video that describes the situation at the time of the event, generates a prompt for generating a report on the predetermined event, and then uses generative AI technology to generate the report from the prompt.

[0018] Although FIG. 1 shows a case where the terminal device 10 and the information processing device 100 are separate devices, the terminal device 10 and the information processing device 100 may be integrated.

[0019] [Example of information processing] FIG. 2 is a diagram showing an example of information processing in the information processing system 1 according to the embodiment.

[0020] The information processing device 100 acquires video images (video images captured by a camera whose setting position is known) when a predetermined event occurs (step S101). For example, when the information processing device 100 detects the occurrence of a predetermined event using some kind of detection means, it acquires video images of the occurrence of the predetermined event. Here, the detection means for detecting the occurrence of the predetermined event may be any type. For example, it may be a detection means using AI technology such as VQA, or a detection means using sensors (for example, sensors attached to clothing or protective equipment such as safety belts, helmets, gloves, and safety shoes), and is not particularly limited. In the case of a detection means using AI technology such as VQA, for example, the AI ​​technology such as VQA detects industrial accidents, such as dangerous conditions that violate safety manuals, based on video images captured by a camera installed at a specific work site, and determines whether a worker is safe or unsafe. In addition, in the case of a detection means using a sensor, for example, it may be an unsafe condition detection sensor that can determine whether a worker is safe or unsafe. The following description will be given taking as an example a case where the information processing device 100 acquires a moving image G1 as a moving image when a predetermined event occurs.

[0021] When the information processing device 100 acquires the video G1, it generates a character string indicating the situation at the time of occurrence of an event (such as a character string explaining the situation) from the video G1 (step S102). At this time, the information processing device 100 generates a character string indicating the situation at the time of occurrence of a predetermined event from the video G1 by using an AI technique such as VQA.

[0022] At this time, the information processing device 100 outputs work-related accident information such as the accident situation as a character string based on the video detected as a work-related accident and metadata (such as the shooting time and location). The work-related accident information is also output based on the estimation results of AI technology such as VQA and questions about the work-related accident. For example, if AI technology such as VQA answers "YES" to the question "Has someone fallen?", it outputs "Someone has fallen: Yes."

[0023] Then, the information processing device 100 generates a prompt for generating a report on the predetermined event using the generated character string (step S103). That is, the information processing device 100 generates a prompt that explains the circumstances of the occurrence of the work-related accident from the character string of the work-related accident information identified from the camera image. Here, the character string of the work-related accident information is information indicating, for example, the time of the accident, the location of the accident, the condition of the worker, the condition of the floor, etc.

[0024] FIG. 3 is a diagram illustrating an example of a prompt according to an embodiment. The prompt illustrated in FIG. 3 includes an instruction (command) such as, "Please output the details, causes, and countermeasures of the industrial accident that are inferred from the information below." The prompt illustrated in FIG. 3 also includes video information, camera installation location information, information indicating the situation after the accident occurred, and information indicating the situation before the accident occurred. Here, the video information is information about the video detected as an industrial accident. For example, the video information is information about the camera that captured the video (such as the camera name), the time of capture, and the location of capture. In FIG. 3, information such as "Camera name: Entrance to Building 101, Time: January 10, 2024, 12:30" is included. The camera name is information for identifying the camera. For example, information such as "Camera name: Entrance to Building 101" indicates that the camera is installed at the entrance to Building 101. The time is information for identifying the time of capture. For example, information such as "Time: January 10, 2024, 12:30" indicates that the video was captured at 12:30 on January 10, 2024.

[0025] Furthermore, the camera installation location information is information about the installation location of the camera that captured the video. For example, this information is information about the break times of workers at the installation location, the work process, and past accident cases. In FIG. 3, this information includes information such as "Break time: 12:00-13:00, Process: Pressing sheet metal, Past accident case: Tripped over cardboard in the aisle and fell." Note that the break time is information for identifying the break time of workers at the installation location. For example, information such as "Break time: 12:00-13:00" indicates that the break time of workers at the installation location is 12:00-13:00. Furthermore, the process is information for identifying the work process at the installation location. For example, information such as "Process: Pressing sheet metal" indicates that the work process at the installation location is pressing sheet metal. Furthermore, past accident cases is information for identifying past accident cases at the installation location. For example, information such as "Past accident case: person tripped over cardboard in the hallway and fell" indicates that one example of a past accident at the installation location was a case where someone tripped over cardboard in the hallway and fell.

[0026] Furthermore, information indicating the situation after an accident occurs is information indicating the situation (such as information explaining the situation) that is identified from video footage after the accident occurs (after the occurrence of a predetermined event). For example, it is information such as the number of people in the video, the person who fell, the position of the person who fell, the ladder user, the press operating machine, the person running, luggage in the aisle, and lighting. In FIG. 3, information such as "Number of people in video: 1, Person who fell: Yes, Position of person who fell: aisle, Stepladder user: No, Press operating machine: No, Person running: No, Luggage in aisle: Yes, Lighting: Off" is included. Note that the number of people in the video is information for specifying the number of people included in the video. For example, information such as "Number of people in the video: 1" indicates that the number of people included in the video is one. Furthermore, the person who fell is information for specifying whether or not a person has fallen. For example, information such as "Person who fell: Yes" indicates that a person has fallen. Furthermore, the position of the person who fell is information for specifying the position of the person who fell. For example, information such as "Position of person who fell: aisle" indicates that the person who fell is located in the aisle. Furthermore, "Stepladder User" is information for specifying whether or not there is a stepladder user. For example, information such as "Stepladder User: None" indicates that there is no stepladder user. "Press Operating" is information for specifying whether or not there is a press operating. For example, information such as "Press Operating: None" indicates that there is no press operating. "Running Person" is information for specifying whether or not there is a person running. For example, information such as "Running Person: None" indicates that there is no person running. "Luggage in Passage" is information for specifying whether or not there is luggage in the passage. For example, information such as "Luggage in Passage: Present" indicates that there is luggage in the passage. "Lighting" is information for specifying whether the lighting is on or off. For example, information such as "Lighting: OFF" indicates that the lighting is off.

[0027] The information indicating the situation before the accident is information indicating the situation (such as information explaining the situation) that is revealed from the video before the accident (before the occurrence of the predetermined event). Since it is the same as the information indicating the situation after the accident, a detailed explanation will be omitted.

[0028] When comparing the information showing the situation before the accident with the information showing the situation after the accident, the number of people who fell changed from "no people who fell" (before the accident) to "people who fell" (after the accident), which suggests that some workers fell as a result of the accident. Also, the number of people running changed from "people running" (before the accident) to "people running" (after the accident), which suggests that people who were running stopped running as a result of the accident.

[0029] By using AI technology such as VQA, the information processing device 100 generates information such as "Number of people in video: 1, Person who fell: Yes, Location of person who fell: Passage, Ladder users: No, Press operating machine: No, Person running: No, Luggage in passage: Yes, Lights: Off" (information indicating the situation after the accident) and information such as "Number of people in video: 1, Person who fell: No, Location of person who fell: Unknown, Ladder users: No, Press operating machine: No, Person running: Yes, Luggage in passage: Yes, Lights: Off" (information indicating the situation before the accident), and generates the prompt shown in Figure 3. Specifically, AI technology such as VQA answers questions such as "How many people are in the video?", "Has anyone fallen?", "Where is the person who fell?", "Is anyone using a stepladder?", "Is there a press in operation?", "Is anyone running?", "Is there luggage in the aisle?", and "Are the lights on or off?" with "YES" or "NO," thereby outputting a string such as "No person has fallen," and generating the prompt shown in FIG. 3. Furthermore, the information processing device 100 generates information contained in the video information shown in FIG. 3 and camera installation location information by using metadata, etc., and generates the prompt shown in FIG. 3. Furthermore, the information processing device 100 generates the prompt shown in FIG. 3 by including in the prompt an instruction such as "Please output the details, causes, and countermeasures of the industrial accident that are inferred from the information below."

[0030] In this way, the information processing device 100 generates a prompt by identifying character strings indicating a predetermined situation before and after the predetermined event (such as "Number of people in video: 1, Person who fell: Yes, Location of person who fell: Passage, Stepladder users: No, Press operating machine: No, Person running: No, Baggage in passage: Yes, Lights: Off"). In other words, the information processing device 100 generates a prompt by identifying character strings indicating a predetermined situation from answers obtained using AI technology such as VQA before and after the predetermined event.

[0031] Then, the information processing device 100 generates a report using the generated prompt (step S104). At this time, the information processing device 100 generates a report regarding the predetermined event by using a generative AI technology such as GPT. For example, the information processing device 100 inputs the prompt shown in FIG. 3 into the generative AI technology such as GPT, thereby causing the generative AI technology such as GPT to generate a report regarding the occurrence of an accident.

[0032] FIG. 4 is a diagram showing an example of a report according to the embodiment. The report shown in FIG. 4 includes the details, causes, and countermeasures of a work-related accident. Here, the details of the work-related accident are information relating to the details of the work-related accident. In FIG. 4, information such as "It appears that a fall accident in the corridor has occurred again. As in past accident cases, the accident was caused by the presence of an object in the corridor and the lights being off." is included.

[0033] Additionally, the cause is information about the cause of the industrial accident. For example, there is information such as there being an object in the aisle or the lights being off. In Figure 4, detailed information about the cause "there was an object in the aisle" includes information such as "The luggage placed in the aisle caused the fall. In particular, a similar situation recurred despite learning from previous accident cases." Additionally, detailed information about the cause "the lights were off" includes information such as "One of the causes was that the workers were unable to secure sufficient visibility because the lights were off."

[0034] Countermeasures are information related to measures to prevent workplace accidents. Examples include clearing aisles and disposing of luggage, ensuring adequate lighting, learning from past accident cases, educating and raising worker awareness, and regular safety inspections. In Figure 4, detailed information about the countermeasure "clearing aisles and disposing of luggage" includes information such as "Clearing luggage in aisles and ensuring passage space." Detailed information about the countermeasure "ensuring lighting" includes information such as "Ensuring sufficient lighting in work areas and aisles to allow workers to move safely." Detailed information about the countermeasure "learning from past accidents" includes information such as "Thoroughly learning from past cases of falls in aisles and taking measures to prevent similar accidents." Detailed information about the countermeasure "educating and raising worker awareness" includes information such as "Educating workers to be aware of careless behavior and obstacles in aisles." Detailed information about the countermeasure "regular safety inspections" includes information such as "Conducting regular safety inspections to identify and repair potential hazards early." In addition, information regarding measures to prevent workplace accidents is included, such as, "By implementing these measures, it is expected that falls in the corridors will be prevented."

[0035] The information processing device 100 generates information on the details, causes, and countermeasures of the work-related accident by using generative AI technology such as GPT, thereby generating the report shown in Fig. 4. Specifically, the information processing device 100 generates information indicating the details of the work-related accident, such as "Luggage placed in the aisle caused the fall. A similar situation has recurred, especially despite learning from previous accident cases," information indicating the cause of the work-related accident, such as "There was an object in the aisle: Luggage placed in the aisle caused the fall. A similar situation has recurred, especially despite learning from previous accident cases," and information indicating countermeasures for the work-related accident, such as "Arrange the aisle and arrange the luggage: Organize the luggage placed in the aisle and ensure passage space.", thereby generating the report shown in Fig. 4.

[0036] In this way, the information processing device 100 generates a report that includes at least one of the content related to the predetermined event, the cause of the predetermined event, and the countermeasure for the predetermined event.

[0037] Then, the information processing device 100 provides the generated report to the user U1 (step S105). Specifically, the information processing device 100 transmits information for displaying the generated report to the terminal device 10. Upon receiving the information transmitted from the information processing device 100, the terminal device 10 displays the report that the information processing device 100 has generated using a generative AI technology such as GPT based on the received information.

[0038] In the above embodiment, the information processing device 100 may generate a prompt for generating a report by referring to past case information. For example, the information processing device 100 may generate a prompt for generating a report (e.g., a report in the same format as the report of the similar event) that is similar to a predetermined event (i.e., the target event) that is the current event by referring to past reports. That is, the information processing device 100 may generate a report in the same format as the report of the similar event by including the report of the similar event in the prompt.

[0039] In the above embodiment, the information processing device 100 may generate a prompt for generating a report by referencing a pre-prepared template (e.g., a template selected from pre-prepared templates). For example, the information processing device 100 may generate a prompt for generating a report in a format similar to that of the pre-prepared template by referencing the pre-prepared template. That is, the information processing device 100 may generate a report in a format similar to that of the template by including the template in the prompt. In this way, if there is an existing report format, a statement to that effect may be added to the instruction text to accommodate the existing report format. For example, a statement such as "Please fill out the work-related accident report in accordance with the report entry items below" may be added. If there is an existing report format, the information processing device 100 may generate a prompt by adding a statement to that effect to generate a report. Then, the information processing device 100 may automatically output an arbitrary report template including a cause analysis, measures to prevent recurrence, etc.

[0040] In the above embodiment, the information processing device 100 generates an estimated answer from a video using a trained VQA model that has been trained on combinations of questions and estimated answers prepared in advance, and determines whether the similarity value between the expected answer and the estimated answer exceeds a threshold, determining that there is no abnormality (safety) if it exceeds the threshold, and determining that there is an abnormality (unsafety) if it does not exceed the threshold, thereby distinguishing between safety and unsafety for the worker, and performing processing for generating a report according to the above embodiment if there is an abnormality. The information processing device 100 may perform processing for determining the similarity value between the expected answer and the estimated answer when detecting the occurrence of a predetermined event, or may perform processing for determining the similarity value between the expected answer and the estimated answer in order to detect the occurrence of a predetermined event.

[0041] [Configuration of terminal device] Next, the configuration of the terminal device 10 according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the configuration of the terminal device 10 according to the embodiment. As shown in Fig. 5, the terminal device 10 has a communication unit 11, an input unit 12, an output unit 13, and a control unit 14.

[0042] (Communications Department 11) The communication unit 11 is realized by, for example, a network interface card (NIC), etc. The communication unit 11 is connected to a predetermined network N by wire or wirelessly, and transmits and receives information to and from the information processing device 100, etc., via the predetermined network N.

[0043] (Input section 12) The input unit 12 accepts various operations from a user. In FIG. 5, the input unit 12 accepts various operations from a user U1. For example, the input unit 12 may accept various operations from a user via a display screen using a touch panel function. The input unit 12 may also accept various operations from buttons provided on the terminal device 10 or a keyboard or mouse connected to the terminal device 10.

[0044] (Output section 13) The output unit 13 is a display screen of a tablet terminal or the like realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device for displaying various information. The output unit 13 displays, for example, a report based on information transmitted from the information processing device 100.

[0045] (Control unit 14) The control unit 14 is, for example, a controller, and is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like executing various programs stored in a storage device inside the terminal device 10 using a RAM (Random Access Memory) as a work area. For example, these various programs include application programs installed in the terminal device 10. For example, these various programs include an application program that displays a report based on information transmitted from the information processing device 100. The control unit 14 is also realized by an integrated circuit, such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0046] As shown in FIG. 5, the control unit 14 has a receiving unit 141 and a transmitting unit 142, and realizes or executes the information processing operations described below.

[0047] (Receiving unit 141) The receiving unit 141 receives, for example, information transmitted from the information processing device 100. For example, the receiving unit 141 receives information for displaying a report regarding a predetermined event. For example, the receiving unit 141 receives information for displaying a report generated by the information processing device 100 using a generative AI technology such as GPT. For example, the receiving unit 141 receives information for displaying a report including at least one of details regarding the predetermined event, causes of the predetermined event, and countermeasures for the predetermined event.

[0048] (Transmitter 142) The transmission unit 142 transmits, for example, user operation information. For example, when the transmission unit 142 receives from the user a designation of a video to be subjected to information processing according to the embodiment, the transmission unit 142 transmits information indicating the video designated by the user. Furthermore, for example, when the transmission unit 142 receives from the user a designation of a template for generating a report, the transmission unit 142 transmits template information designated by the user.

[0049] [Configuration of information processing device] Next, the configuration of the information processing device 100 according to the embodiment will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. As shown in Fig. 6, the information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130. Note that the information processing device 100 may also have an input unit (e.g., a keyboard or a mouse) that accepts various operations from an administrator of the information processing device 100, and a display unit (e.g., a liquid crystal display) that displays various information.

[0050] (Communication unit 110) The communication unit 110 is realized by, for example, a NIC etc. The communication unit 110 is connected to a network N by wire or wirelessly, and transmits and receives information to and from the terminal device 10 etc. via the network N.

[0051] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in Fig. 6, the storage unit 120 has a VQA model storage unit 121, a past accident case storage unit 122, and a template storage unit 123.

[0052] The VQA model storage unit 121 stores information related to the VQA model. Fig. 7 shows an example of the VQA model storage unit 121 according to the embodiment. As shown in Fig. 7, the VQA model storage unit 121 has items such as "VQA model ID" and "VQA model."

[0053] "VQA model ID" indicates identification information for identifying a VQA model. "VQA model" indicates a VQA model. In the example shown in FIG. 7, conceptual information such as "VQA model #1" and "VQA model #2" is stored in "VQA model," but in reality, training data for the VQA model or the like is stored. For example, information such as combinations of questions and estimated answers is stored.

[0054] The past accident case storage unit 122 stores information about past accident cases. Here, Fig. 8 shows an example of the past accident case storage unit 122 according to the embodiment. As shown in Fig. 8, the past accident case storage unit 122 has items such as "past accident case ID" and "past accident case".

[0055] "Past Accident Case ID" indicates identification information for identifying past accident cases. "Past Accident Case" indicates information related to past accident cases. In the example shown in Figure 8, conceptual information such as "Past Accident Case #1" and "Past Accident Case #2" is stored in "Past Accident Case", but in reality, information such as reports on past accident cases is stored. For example, information such as the content, causes, and countermeasures contained in reports on past accident cases is stored.

[0056] The template storage unit 123 stores information related to prepared report templates. An example of the template storage unit 123 according to the embodiment is shown in Fig. 9. As shown in Fig. 9, the template storage unit 123 has items such as "template ID" and "template."

[0057] "Template ID" indicates identification information for identifying a template. "Template" indicates information related to the template. In the example shown in FIG. 9, conceptual information such as "Template #1" and "Template #2" is stored in "Template," but in reality, information such as the format of the template is stored. For example, information such as the format of the content, cause, and countermeasures is stored.

[0058] (control unit 130) The control unit 130 is a controller, and is realized by, for example, a CPU or an MPU executing various programs stored in a storage device inside the information processing device 100 using RAM as a work area. The control unit 130 is also realized by, for example, an integrated circuit such as an ASIC or an FPGA.

[0059] 6, the control unit 130 has an acquisition unit 131, a first generation unit 132, a second generation unit 133, a third generation unit 134, and a provision unit 135, and realizes or executes the information processing action described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 6, and may be any other configuration as long as it performs the information processing described below.

[0060] (Acquisition part 131) The acquiring unit 131 acquires various pieces of information from the storage unit 120. The acquiring unit 131 also stores the acquired various pieces of information in the storage unit 120.

[0061] The acquisition unit 131 acquires various pieces of information from an external information processing device. The acquisition unit 131 acquires various pieces of information from other information processing devices such as the terminal device 10.

[0062] The acquisition unit 131 acquires, for example, a video (such as a moving image) when a predetermined event occurs. For example, the acquisition unit 131 acquires a video captured by a camera (or other imaging device) whose setting position is known.

[0063] (first generation unit 132) For example, the first generating unit 132 generates a character string that explains the situation at the time of occurrence of a predetermined event from a video of the event. For example, the first generating unit 132 generates a character string that explains the situation at the time of occurrence of the event from a video acquired by the acquiring unit 131.

[0064] The first generating unit 132 generates a character string indicating the situation at the time of occurrence of a predetermined event from the video by using, for example, AI technology such as VQA. At this time, the first generating unit 132 outputs work accident information such as the accident situation as a character string based on, for example, the video and metadata.

[0065] The first generation unit 132 generates character strings indicating the situation related to the occurrence of a predetermined event by, for example, identifying character strings indicating the predetermined situation from the video before and after the predetermined event. For example, the first generation unit 132 generates character strings indicating the situation related to the occurrence of a predetermined event by identifying character strings indicating the predetermined situation from the VQA response before and after the predetermined event.

[0066] (Second generation unit 133) The second generating unit 133 generates, for example, a prompt for generating a report on a predetermined event. For example, the second generating unit 133 generates a prompt for generating a report on a predetermined event using the character string generated by the first generating unit 132.

[0067] The second generating unit 133 generates a prompt for generating a report on a predetermined event that includes, for example, at least one of the content on the predetermined event, the cause of the predetermined event, and a countermeasure for the predetermined event.

[0068] The second generating unit 133 generates a prompt for generating a report on a predetermined event by referring to past case information, for example. For example, the second generating unit 133 generates a prompt for generating a report in line with a report on a similar event that is similar to the predetermined event by referring to past reports.

[0069] For example, second generating unit 133 generates a prompt for generating a report on a predetermined event by referring to a template. For example, second generating unit 133 generates a prompt for generating a report in the same format as the template by referring to a template prepared in advance.

[0070] (Third generation unit 134) For example, the third generating unit 134 generates a report on a predetermined event using a prompt for generating a report on a predetermined event. For example, the third generating unit 134 generates a report on a predetermined event using the prompt generated by the second generating unit 133.

[0071] The third generation unit 134 generates a report regarding a predetermined event by using a generative AI technology such as GPT, for example. For example, the third generation unit 134 inputs a prompt for generating a report regarding the predetermined event into the generative AI technology such as GPT, thereby causing the generative AI technology such as GPT to generate a report regarding the predetermined event.

[0072] (Provider 135) The providing unit 135 provides, for example, a report regarding a predetermined event. For example, the providing unit 135 provides a report regarding a predetermined event generated by the third generating unit 134. For example, the providing unit 135 transmits information for displaying the report regarding the predetermined event. For example, the providing unit 135 transmits information for displaying the report regarding the predetermined event generated by the third generating unit 134 using a generative AI technology such as GPT. For example, the providing unit 135 transmits information for displaying the report regarding the predetermined event, including at least one of the content regarding the predetermined event, the cause of the predetermined event, and the countermeasures regarding the predetermined event. In this manner, information regarding the predetermined event that has occurred is reported to a user such as a site manager. Furthermore, the providing unit 135 performs, for example, a process for recording information regarding the predetermined event. For example, when a report regarding the predetermined event is provided, the providing unit 135 performs a process for recording information regarding the predetermined event. For example, when a report regarding the predetermined event is provided, the providing unit 135 performs a process for storing the report in the past accident case storage unit 122.

[0073] [Information processing flow] Next, the procedure of information processing by the information processing system 1 according to the embodiment will be described with reference to Fig. 10 and Fig. 11. Fig. 10 and Fig. 11 are flowcharts showing the procedure of information processing by the information processing system 1 according to the embodiment.

[0074] 10, the information processing device 100 acquires a video when a predetermined event occurs (step S201). Step S201 is a step indicating information processing when a predetermined event is detected using an AI technique such as VQA, a sensor, or the like.

[0075] The information processing device 100 generates a character string that explains the situation when the event occurred (step S202). Step S202 is a step that indicates information processing using AI technology such as VQA.

[0076] The information processing device 100 generates a prompt for generating a report on a predetermined event (step S203).

[0077] The information processing device 100 generates a report on a predetermined event (step S204). Step S204 is a step representing information processing using generative AI technology such as GPT.

[0078] The information processing device 100 provides a report regarding the predetermined event (step S205).

[0079] As shown in FIG. 11, the information processing device 100 acquires a video (step S301).

[0080] Information processing device 100 generates estimated answers using a trained model (a model that has been trained in advance on combinations of questions and expected answers) (step S302).

[0081] The information processing device 100 determines whether the similarity value between the estimated answer and the expected answer exceeds a threshold value (step S303).

[0082] When it is determined that the similarity value between the estimated answer and the expected answer exceeds the threshold value (step S303; YES), the information processing device 100 determines that there is no abnormality (step S304) and ends the information processing.

[0083] On the other hand, when the information processing device 100 determines that the similarity value between the estimated answer and the expected answer does not exceed the threshold value (step S303; NO), it determines that an abnormality exists (step S305) and executes processing for generating a report (step S306). That is, the information processing device 100 executes steps S202 to S205.

[0084] [Other system configuration examples] The above-described configuration of the information processing system 1 is merely an example, and the information processing system 1 can employ any device configuration as long as the desired processing is possible. For example, the information processing system 1 may be configured without the terminal device 10. Furthermore, the various processes performed by the information processing device 100 may be executed by any device included in the information processing system 1, not limited to the information processing device 100. For example, at least a part of the various processes performed by the information processing device 100 (e.g., the first generation unit 132, the second generation unit 133, the third generation unit 134, etc.) may be executed by the terminal device 10. In this way, the various processes performed by the information processing device 100 may be distributed and processed by multiple devices included in the information processing system 1.

[0085] Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are within the scope and spirit of the invention, as well as the invention and its equivalents as set forth in the claims. Furthermore, these embodiments and their modifications can be combined as appropriate within the scope of the processing content. [Explanation of symbols]

[0086] 1. Information Processing Systems 10 Terminal Equipment 11 Communications Department 12 Input section 13 Output section 14 Control Unit 100 Information processing device 110 Communications Department 120 Storage section 121 VQA model memory 122 Past accident case storage unit 123 Template Memory 130 Control Unit 131 Acquisition Department 132 1st generation part 133 Second generation part 134 Third generation part 135 Provision Department 141 Receiving unit 142 Transmitter N Network

Claims

1. a first generator that generates a character string that explains a situation related to the occurrence of a predetermined event from a video image when the event occurs; a second generating unit that generates a prompt for generating a report regarding the predetermined event using the character string generated by the first generating unit; a third generating unit that generates a report using the prompts generated by the second generating unit; An information processing device comprising:

2. The first generation unit The character string is generated by identifying a character string indicating a predetermined situation before and after the predetermined event.

2. The information processing apparatus according to claim 1, wherein:

3. The first generation unit The character string is generated by identifying the character string from a response to a VQA (Visual Question Answering) 3. The information processing apparatus according to claim 2, wherein:

4. The second generation unit generating the prompt to generate the report including at least one of content related to the event, causes related to the event, and countermeasures related to the event; 2. The information processing apparatus according to claim 1, wherein:

5. The second generation unit Generate the prompt to generate the report by referencing past case information.

2. The information processing apparatus according to claim 1, wherein:

6. The second generation unit Generating the prompt to generate the report by referencing a template 2. The information processing apparatus according to claim 1, wherein:

7. The predetermined event is an event related to a work-related accident.

2. The information processing apparatus according to claim 1, wherein:

8. 1. A computer-implemented information processing method, comprising: a first generation step of generating a character string that explains a situation related to the occurrence of a predetermined event from a video image when the event occurs; a second generating step of generating a prompt for generating a report on the predetermined event using the character string generated by the first generating step; a third generating step of generating a report using the prompts generated by the second generating step; An information processing method comprising:

Images