Response proposal device

Abstract

To provide a countermeasure proposal device capable of improving convenience.SOLUTION: The response proposal device includes an acquisition unit configured to acquire an image of a site at which a response-required situation occurs, a generation unit configured to generate a caption based on the image of the response-required situation, a search unit configured to search for a response manual indicating a method of responding to the response-required situation from a plurality of response manuals stored in a storage device based on the caption, and a control unit configured to cause an output device to output the searched response manual.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a response suggestion device. [Background technology]

[0002] For example, when an incident requiring a response occurs at a work site or manufacturing site, the following flow generally occurs. First, if there are any injuries, the responder on-site will administer first aid to the injured person and call an ambulance. Next, the responder will report the situation to a manager by a prescribed means and await instructions. Next, based on the reported situation, the manager will consult a response manual or solicit the opinions of experts to consider the specifics of the response and instruct the responder accordingly. Next, the responder will carry out the response instructed by the manager.

[0003] However, with the above process, when managers consider how to respond, they will need to spend time checking response manuals and searching for experts to get hints on how to respond, which could result in delayed instructions to the field.

[0004] As a conventional technology for solving such problems, for example, Patent Document 1 discloses a response support system that detects a target (person or event) from video and presents the content of the response that should be taken to a responder who responds to the detected target. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-71967 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology of Patent Document 1 requires the user to manually set in advance the definition of the situation requiring response (for example, the detection target and whether the detection target is moving, etc.), which poses a problem in terms of convenience.

[0007] An object of one aspect of the present disclosure is to provide a response suggestion device that can achieve improved convenience. [Means for solving the problem]

[0008] A response suggestion device according to one aspect of the present disclosure includes an acquisition unit that acquires an on-site image of a response-required situation that shows the scene where the response-required situation occurred, a generation unit that generates a caption based on the on-site image of the response-required situation, a search unit that searches for a response manual that indicates how to respond to the response-required situation from among multiple response manuals stored in a storage device based on the caption, and a control unit that outputs the searched response manual to an output device. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to efficiently find a response method without having to manually define a response-requiring situation in advance, thereby achieving improved convenience. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram schematically illustrating a configuration of a response suggestion system according to an embodiment of the present disclosure. [Figure 2] 1 is a flowchart illustrating an operation of a response suggestion device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] First, the configuration of a response suggestion system 1 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram schematically showing the configuration of the response suggestion system 1.

[0013] 1 is used by, for example, an organization or group that manages a site where an incident requiring a response may occur. Examples of the site include, but are not limited to, a certain area within a work facility such as a factory, a public facility (park, hospital, school, airport, stadium, etc.), or a commercial facility (department store, shopping mall, etc.), and the like, as long as the site is a certain area within a building or outdoors.

[0014] In the present embodiment, a response-requiring incident refers to a situation that requires some kind of immediate response. Examples of the occurrence of a response-requiring incident include, but are not limited to, a natural disaster, an accident, a nuisance, a violent act, a sudden illness, a lost child, wheelchair assistance, or a large number of people stranded.

[0015] As shown in FIG. 1, the action suggestion system 1 includes an action suggestion device 10, a camera 20, a storage device 30, and an output device 40.

[0016] The response proposal device 10 is, for example, a personal computer installed at a location remote from the site. The response proposal device 10 is communicably connected to each of the camera 20, the storage device 30, and the output device 40. The communication method may be a wired or wireless internet line, or any other known communication method. Each part of the response proposal device 10 will be described later.

[0017] Camera 20 is a device that is installed at or near the scene and captures images of the scene. For example, if a response-requiring incident occurs at the scene, and an emergency button electrically connected to camera 20 is pressed by a person at the scene, the camera 20, triggered by the press, transmits an image of the scene (hereinafter referred to as a captured image) to response proposal device 10. This captured image is an image showing the scene where the response-requiring incident occurred (hereinafter referred to as a response-requiring scene image).

[0018] The on-site images when a response is required may be video or still images. As will be described in detail later, the on-site images when a response is required are used to generate captions in the response suggestion device 10. Therefore, if the on-site images when a response is required are video, they will contain time-series information, and the amount of information can be increased compared to still images. As a result, the generated captions more accurately represent the on-site situation, which can improve the accuracy of response manual searches (described in detail later). Furthermore, even when multiple still images taken by multiple cameras 20 with different shooting angles are used as on-site images when a response is required, the amount of information obtained from the images can be increased, thereby achieving the same effect as described above. Of course, using multiple video images with different shooting angles as on-site images when a response is required is more preferable because it can further increase the amount of information.

[0019] The storage device 30 is a device (for example, an external hard disk) connected to the response proposal device 10. The storage device 30 stores a response manual for each type of response-requiring incident. Thus, the storage device 30 stores a plurality of response manuals.

[0020] A response manual is electronic data that shows how to respond to an incident requiring a response. The response manual may be a still image containing text, photographs, illustrations, etc., or a video. It may also include audio. For example, a response manual for a fire (an example of an incident requiring a response) may include instructions on how to respond to a fire, such as how to extinguish the fire, how to evacuate, how to contact relevant parties, and how to provide first aid.

[0021] The response manual may also include a work procedure manual that explains the detailed procedures for a specific task. For example, a fire response manual may include work procedures such as how to use a fire extinguisher and how to use an AED (automated external defibrillator). The work procedure manual may be in the form of still images or video, and may also be accompanied by audio.

[0022] In this embodiment, an example will be described in which one or more words relating to a situation requiring a response are linked to each of a plurality of response manuals stored in the storage device 30. For example, a fire response manual is linked to words relating to fires.

[0023] The output device 40 is, for example, a terminal device (for example, a personal computer display, a tablet, a smartphone, etc.) used by a manager who is located away from the site. The manager is a person in charge of managing the site. The output device 40 receives the response manual output from the response proposal device 10 and displays an image of the response manual (which may also include audio output). This allows the manager to check the contents of the response manual.

[0024] The response proposal device 10 may be a server on the Internet. The storage device 30 may be a storage device (e.g., a hard disk) built into the response proposal device 10, or may be a server on the Internet that can communicate with the response proposal device 10. The output device 40 may be a display (e.g., with a speaker) that is provided integrally with the response proposal device 10.

[0025] Next, the configuration of the treatment proposal device 10 will be described with reference to FIG.

[0026] Although not shown in the drawings, the response proposal device 10 has, as hardware, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory) storing a computer program, a RAM (Random Access Memory) serving as a working memory, etc. Each part of the response proposal device 10 described below is realized by the CPU reading out a computer program from the ROM and executing it in the RAM.

[0027] As shown in FIG. 1, the action proposal device 10 includes an acquisition unit 110, a generation unit 120, a search unit 130, and a control unit 140.

[0028] The acquisition unit 110 acquires images of the scene when a response is required from the camera 20.

[0029] As described above, when an incident requiring a response occurs at the scene, pressing the emergency button triggers transmission of the captured image from the camera 20 to the response proposal device 10, and therefore the captured image acquired by the acquisition unit 110 is always an image of the scene when a response is required. Therefore, the acquisition of the captured image by the acquisition unit 110 can be considered as the occurrence of an incident requiring a response.

[0030] The generator 120 generates a caption based on the emergency scene image. The caption is one or more words that describe the emergency scene image.

[0031] For example, a machine learning model is used to generate captions. The generation unit 120 may use a publicly available machine learning model capable of generating captions based on images, or may create a machine learning model capable of independently generating captions. Note that, since known techniques can be applied to the caption generation process using a machine learning model, detailed description thereof will be omitted.

[0032] The search unit 130 searches, based on the caption, a response manual that includes a method for responding to the currently occurring situation that requires a response from among a plurality of response manuals stored in the storage device 30.

[0033] Specifically, the search unit 130 matches the caption with one or more words associated with each corresponding manual, and acquires from the storage device 30 the corresponding manual whose caption includes the word.

[0034] The control unit 140 outputs the searched correspondence manual to the output device 40. Under the control of the control unit 140, the output device 40 outputs the correspondence manual by at least one of image display and audio output. As a result, the correspondence manual is proposed to the administrator.

[0035] Next, the operation of the response proposal device 10 will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the flow of the operation of the response proposal device 10.

[0036] The flow shown in FIG. 2 starts, for example, when a situation requiring a response occurs at a site and an image of the site requiring a response is transmitted from the camera 20 to the response proposing device 10.

[0037] First, the acquisition unit 110 acquires an on-site image when a response is required (step S1).

[0038] Next, the generating unit 120 generates a caption for the scene image of the emergency response (step S2).

[0039] Next, the search unit 130 searches for an appropriate corresponding manual based on the caption (step S3).

[0040] Next, the control unit 140 controls the output device 40 to output the corresponding manual (step S4).

[0041] By the above operation, the output device 40 outputs a response manual for dealing with the situation that requires a response.

[0042] As described above, the response suggestion device 10 of this embodiment is characterized by having an acquisition unit 110 that acquires a scene image of the response-required situation showing the scene where the response-required situation occurred; a generation unit 120 that generates a caption based on the scene image of the response-required situation; a search unit 130 that searches for a response manual that shows how to respond to the response-required situation that has occurred from among multiple response manuals stored in the storage device 30 based on the caption; and a control unit 140 that outputs the searched response manual to the output device 40.

[0043] This allows the administrator to quickly understand how to respond, shortening the time required to consider a response and enabling quick decision on response instructions. In addition, since there is no need for the user to manually perform settings in advance (see Patent Document 1), convenience can be improved.

[0044] The present disclosure is not limited to the above-described embodiment, and various modifications are possible without departing from the spirit of the present disclosure. Modifications will be described below.

[0045] [Variation 1] In the embodiment, a case has been described in which one or more words are associated with each of the correspondence manuals stored in the storage device 30, but the present invention is not limited to this.

[0046] For example, the corresponding manuals stored in the storage device 30 may not be linked to words. In this case, the search unit 130 uses one or more words contained in each corresponding manual (for example, the title sentence, file name, or words frequently appearing in the explanations of the corresponding manual) to match the captions.

[0047] According to this modification, when storing the corresponding manual in the storage device 20, there is no need to perform the work of linking words, thereby improving convenience.

[0048] [Variation 2] In order to encourage the administrator to take appropriate action in the event of a situation requiring response, the control unit 140 may cause the output device 40 to output information regarding the situation requiring response between the time when the on-site image of the situation requiring response is acquired and the time when the searched response manual is output.

[0049] For example, the control unit 140 may further include a function of determining the urgency of the response-requiring incident based on the caption, and after the caption is generated, may generate a notification message indicating that a response-requiring incident has occurred and including the type and urgency of the response-requiring incident, and cause the output device 40 to output the notification message. Alternatively, the control unit 140 may cause the output device 40 to output the notification message immediately before causing the output device 40 to output a response manual.

[0050] According to this modification, the administrator can recognize the urgency of the situation that requires a response before the response manual is output.

[0051] [Variation 3] In the embodiment, the captured images acquired by the acquisition unit 110 are always on-site images when a response is required, and therefore the acquisition of the captured images by the acquisition unit 110 itself signifies the occurrence of a situation requiring a response, but this is not limited to this.

[0052] For example, the acquisition unit 110 may have a function of detecting whether or not a situation requiring a response has occurred at the site. This will be specifically described below.

[0053] First, the camera 20 transmits images of the scene to the response suggestion device 10 constantly or periodically, even before a response-requiring incident occurs.

[0054] The acquisition unit 110 receives the captured image and detects whether a situation requiring a response has occurred at the scene based on the captured image and a machine learning model. The machine learning model here is a model trained using as training data either an image showing the scene when a situation requiring a response has occurred or an image showing the scene when no situation requiring a response has occurred.

[0055] When the acquisition unit 110 detects the occurrence of a situation requiring a response, it acquires the captured image used for the detection as a scene image for the situation requiring a response.

[0056] In the above description, the case where the acquisition unit 110 has a function of detecting the occurrence of an incident requiring a response has been described as an example, but a detection unit having the above function may be provided separately from the acquisition unit 110.

[0057] For example, the detection unit may be provided inside the response proposal device 10. In this case, the detection unit may first acquire a photographed image from the camera 20 and detect whether a response-requiring situation has occurred based on the photographed image, and the acquisition unit 110 may acquire the photographed image as a response-requiring situation image when the occurrence of the response-requiring situation is detected.

[0058] Alternatively, for example, the detection unit may be provided outside the response proposal device 10. In this case, the detection unit may be provided as a relay device between the camera 20 and the response proposal device 10. Note that in a configuration in which the detection unit is provided outside the response proposal device 10, the response-requiring incident detection process can be separated from the caption generation process and the manual search process, which is advantageous in terms of processing speed compared to a configuration in which the detection unit is provided inside the response proposal device 10.

[0059] According to this modified example, the occurrence of an incident requiring a response can be detected more quickly and accurately than with human judgment, and therefore a response manual can be proposed quickly and at an appropriate time.

[0060] [Variation 4] The solution suggestion device 10 may further include a calculation unit that, when a solution manual is searched for by the search unit 130, calculates the degree of match (degree of suitability) between the caption and either a word linked to the searched solution manual or a word included in the searched solution manual. Then, the control unit 140 may cause the output device 40 to output the degree of match together with the searched solution manual. Alternatively, the control unit 140 may cause the output device 40 to output only solution manuals whose degree of match is equal to or greater than a threshold.

[0061] According to this modification, it is possible to propose a more appropriate response manual.

[0062] [Variation 5] The control unit 140 may output the searched response manual to the output device 40, and then store the response manual in the storage device 30 in association with the on-site image or video of the situation when response is required and the caption.

[0063] According to this modified example, past incidents requiring action can be reviewed by scene based on the captions, and this can be used to prevent recurrence, as well as to adjust the accuracy of the machine learning model that generates the captions.

[0064] [Variation 6] In the embodiment, the output device 40 is described as a terminal device used by a manager, but the present invention is not limited to this. For example, the output device 40 may be a terminal device (e.g., a personal computer display, a tablet, a smartphone, etc.) used by a person on-site. This allows the person on-site to directly check the response manual when a situation requiring a response occurs. [Industrial Applicability]

[0065] The response suggestion device of the present disclosure is useful for the general technology of suggesting a response manual when a situation requiring a response occurs. [Explanation of symbols]

[0066] 1. Response proposal system 10. Response proposal device 20 Camera 30 Storage device 40 Output Devices 110 Acquisition Department 120 Generation part 130 Search Department 140 Control Unit

Claims

1. an acquisition unit that acquires a scene image in which a scene where a response-required incident has occurred is captured; a generation unit that generates a caption based on the response-required scene image; a search unit that searches for a response manual that indicates a method of responding to the response-requiring situation from among a plurality of response manuals stored in a storage device based on the caption; and a control unit that outputs the searched corresponding manual to an output device. Response suggestion device.

2. The generation unit Use a machine learning model that can generate the captions based on publicly available images, or a custom-built machine learning model that can generate the captions based on images. The response suggestion device according to claim 1 .

3. Each of the plurality of correspondence manuals stored in the storage device is associated with one or more words or includes one or more words, The search unit searching the corresponding manual based on the result of matching the caption with the one or more words; The response suggestion device according to claim 1 .

4. The control unit outputting a notification message of information relating to the response-required situation to the output device during the period from when the response-required scene image is acquired until when the searched response manual is output; The response suggestion device according to claim 1 .

5. The acquisition unit Receive images of the scene continuously or periodically from before the occurrence of the incident requiring response; Detecting whether the incident requiring a response has occurred at the scene based on the captured image and a machine learning model trained using as training data either an image showing the scene when the incident requiring a response has occurred or an image showing the scene when the incident requiring a response has not occurred; When the occurrence of the incident requiring a response is detected, the photographed image is acquired as the incident requiring a response on-site image. The response suggestion device according to claim 1 .

6. Further, a calculation unit is provided for calculating a degree of match between the caption and either a word associated with the corresponding manual or a word included in the corresponding manual when the corresponding manual is searched for by the search unit, The control unit outputting the degree of match together with the corresponding manual to the output device, or outputting only the corresponding manual whose degree of match is equal to or greater than a threshold value to the output device; The response suggestion device according to claim 1 .

7. The control unit outputting the retrieved response manual to the output device, and then storing the response manual, the response-required scene image, and the caption in the storage device in association with each other; The response suggestion device according to claim 1 .

8. The output device is The terminal device is used by a site manager who is located in a different location from the site, or a terminal device is used by a person who is located at the site. The response suggestion device according to claim 1 .

Images