Abnormality detection system and abnormality detection method

Abstract

To provide an abnormality detection system for detecting abnormalities, and an abnormality detection method, in which the abnormality detection system is capable of notifying the analysis results concerning the abnormalities in an easy-to-understand manner, and an abnormality detection method.SOLUTION: An abnormality detection system 1 is a system for detecting abnormalities in a monitoring area S and notifying the analysis results to a user. The system includes: an imaging unit 2 for picking up images of the monitoring area; an analysis server 3 that performs anomaly analysis based on the imaging results by the imaging unit; a terminal 5 that displays the analysis results acquired by the analysis server to a user; and a relay server 4 that translates the analysis results by the analysis server into natural language and displays the analysis results translated into natural language on a terminal.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to an abnormality detection system and an abnormality detection method for detecting and notifying an abnormality.

Background Art

[0002] Conventionally, technologies for predicting the degree of collision injury of an injured person at the time of a collision accident have been developed.

[0003] For example, by acquiring external sensor data including external image data before and after a traffic accident recorded by an external vehicle sensor and inputting it into an injury prediction model created by implementing deep learning, the degree of collision injury of the injured person is predicted, a determination level of the collision injury is obtained, and a collision injury prediction system that notifies the responsible operator of the determination level, the location of the collision accident, etc. by display / audio is known (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the conventional collision injury prediction system disclosed in Patent Document 1 above, when the responsible operator is notified by the collision injury prediction system, it is required to understand the determination level of the collision injury, the location of the collision accident, etc., so there is a problem that the burden on the responsible operator is large. Therefore, there is a demand for the development of a system and method for analyzing and notifying the detected abnormality to the responsible operator in an easy-to-understand manner.

[0006] Therefore, the main object of the present disclosure is to provide an anomaly detection system and an anomaly detection method that can detect anomalies and can notify the user of the analysis results in an easy-to-understand manner.

Means for Solving the Problems

[0007] The anomaly detection system of the present disclosure is an anomaly detection system that detects anomalies in a monitoring area and notifies the user of the analysis results, and includes an imaging device that images the monitoring area, an analysis server that performs anomaly analysis based on the imaging results of the imaging device, a terminal that displays the analysis results obtained by the analysis server to the user, and a relay server that naturalizes the analysis results of the analysis server and causes the terminal to display the naturalized analysis results.

[0008] The anomaly detection method of the present disclosure is an anomaly detection method that detects anomalies in a monitoring area and notifies the user of the analysis results, and is executed by an anomaly detection system including an imaging device that images the monitoring area, an analysis server that performs anomaly analysis based on the imaging results of the imaging device, a terminal that displays the analysis results obtained by the analysis server to the user, and a relay server connected to the terminal and the analysis server. The relay server naturalizes the analysis results of the analysis server and causes the terminal to display the naturalized analysis results.

Advantages of the Invention

[0009] According to the present disclosure, it is possible to provide an anomaly detection system and an anomaly detection method that can detect anomalies and can notify the detection results in an easy-to-understand manner.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

MODE FOR CARRYING OUT THE INVENTION

[0011] The first invention made to solve the above problems is an abnormality detection system that detects an abnormality in a monitoring area and notifies a user of an analysis result, including an imaging device that images the monitoring area, an analysis server that performs abnormality analysis based on the imaging result of the imaging device, a terminal that displays the analysis result obtained by the analysis server to the user, and a relay server that naturalizes the analysis result of the analysis server and causes the terminal to display the naturalized analysis result.

[0012] According to this, the detection result of the analysis server is naturalized and displayed on the terminal. Therefore, it is possible to notify the user of the detection analysis result in an easy-to-understand manner.

[0013] Further, the second invention relates to the first invention, and the relay server generates a prompt including the analysis result, and inputs the prompt to a text generation server that naturalizes the prompt based on the prompt and outputs the result as text, thereby naturalizing the analysis result of the analysis server.

[0014] According to this, the detection result can be easily naturalized.

[0015] Further, the third invention relates to the first invention, and the analysis server calculates an evaluation value indicating the degree of abnormality detected in the monitoring area based on the imaging result from the imaging device and outputs the evaluation value to the relay server. The relay server naturalizes the analysis result of the analysis server and displays the analysis result on the terminal when the evaluation value is equal to or greater than a predetermined threshold.

[0016] According to this, it is possible to easily notify the user that the degree of abnormality is increasing.

[0017] Further, the fourth invention relates to the first invention, and the analysis server stores the monitoring video of the monitoring area imaged by the imaging device. The relay server acquires a request notification for making an analysis request for an abnormality detected in the monitoring area from the terminal to the analysis server, and based on the acquired request notification, acquires the analysis result based on the monitoring video from the analysis server, and naturalizes the analysis result and displays the analysis result on the terminal.

[0018] According to this, based on the monitoring video, the analysis result can be easily notified to the user.

[0019] Further, the fifth invention relates to the fourth invention, and the relay server notifies the terminal of the analysis result, and causes the terminal to acquire a quality evaluation value input by the user for the analysis result, and stores the analysis request, the naturalized analysis result, and the quality evaluation value together as a response history.

[0020] According to this, it is possible to obtain the user's quality evaluation value for the analysis result and hold it together with the analysis request and the natural language analysis result.

[0021] Also, the sixth invention, regarding the fifth invention, is configured such that when the relay server acquires the request notification from the terminal, it causes the terminal to present candidates for the analysis request based on the response history.

[0022] According to this, it becomes possible to propose to the user candidates with a high possibility of obtaining a high-quality response.

[0023] Also, the seventh invention, regarding the sixth invention, is configured such that the relay server displays a setting screen for receiving a setting input including the user's attributes on the terminal, and selects the candidates for the analysis request to be displayed on the terminal based on the setting input received on the setting screen and the response history.

[0024] According to this, it is possible to select candidates according to the user's attributes.

[0025] Also, the eighth invention is an abnormality detection method for detecting an abnormality in a monitoring area and notifying a user of an analysis result, and is executed by an abnormality detection system including an imaging device that images the monitoring area, an analysis server that performs abnormality analysis based on the imaging result of the imaging device, a terminal that displays the analysis result acquired by the analysis server to the user, and a relay server connected to the terminal and the analysis server. The relay server naturalizes the analysis result of the analysis server and causes the terminal to display the naturalized analysis result.

[0026] According to this, the detection result of the analysis server is naturalized and displayed on the terminal. Therefore, it becomes possible to notify the user of the result of the detection analysis in an easy-to-understand manner.

[0027] The following is an example in which an abnormality detection system of the present disclosure is used to implement an abnormality detection method that naturalizes information acquired by an imaging device such as a camera and notifies a user, and that detects an abnormality in a monitoring area and notifies the user of the detection result.

[0028] FIG. 1 is an overall configuration diagram of an abnormality detection system 1 according to an embodiment of the present disclosure. The abnormality detection system 1 is a system for analyzing traffic accidents and notifying users such as police, fire departments, and hospitals of the analysis results. Furthermore, the abnormality detection system 1 can be used to identify violations and dangerous behaviors at an early stage and prevent accidents.

[0029] In the present embodiment, the abnormality detection system 1 implements an abnormality detection method of detecting an abnormality in the monitoring area S and notifying the user, and notifying the user of the detection result regarding the detected past abnormality, by executing a reporting notification process and an analysis notification process. In the reporting notification process, the abnormality detection system 1 images the monitoring area S and notifies the user when an abnormality occurs. In the analysis notification process, the abnormality detection system 1 performs an analysis based on an analysis request from the user and notifies the user.

[0030] The abnormality detection system 1 includes an imaging device 2, an analysis server 3, and a relay server 4. The imaging device 2, the analysis server 3, and the relay server 4 can be operated by an operator who operates the abnormality detection system 1. In addition, the abnormality detection system 1 includes a terminal 5 used by a user. The users of the abnormality detection system 1 may be those who work at police stations, fire departments, or hospitals. Specifically, the users of the abnormality detection system 1 may be operators who perform operations for dispatching police officers, firefighters, and doctors to handle traffic accidents at police stations, fire departments, or hospitals. In addition, the users may include an administrator who manages the abnormality detection system 1.

[0031] The imaging device 2, the analysis server 3, the relay server 4, and the terminal 5 can communicate with each other via a known communication network 6 such as the Internet or an intranet.

[0032] The relay server 4 can communicate with the text generation server 7 via a known communication network 6 such as the Internet.

[0033] Fig. 2 shows a block diagram of the imaging device 2, the analysis server 3, the relay server 4, the terminal 5, and the text generation server 7.

[0034] As shown in Fig. 2, the text generation server 7 can be configured by a computer equipped with known hardware such as a processor 7A (CPU, MPU, etc.), a memory 7B (RAM, ROM, etc.), a display, an input device, a network interface 7C, and a storage device 7D (HDD, SSD).

[0035] When the text generation server 7 (processor 7A) receives a prompt (instruction text) via the communication network 6 (when the prompt is input), it has a function of generating text in natural language according to the prompt and returning the generated text. That is, the text generation server 7 has a function as a so-called generative AI that automatically generates text when a prompt is sent. Here, the prompt corresponds to an instruction / command text that gives an instruction for text generation to a generative AI such as the text generation server 7. The text generation server 7 may be configured by, for example, an artificial intelligence chatbot such as ChatGPT (registered trademark) that sends an answer based on the sent prompt.

[0036] The text generation server 7 may be operated by the operator who operates the anomaly detection system 1, or may be operated by an operator different from the operator who operates the anomaly detection system 1. The text generation server 7 may be configured so that it cannot be directly connected to the analysis server 3 via the communication network 6. For example, the terminal 5, the analysis server 3, and the relay server 4 may be connected by an intranet respectively, and only the relay server 4 may be configured to be connectable to the text generation server 7 via the Internet.

[0037] In FIG. 1, for convenience of explanation, one terminal 5 is shown. However, the anomaly detection system 1 may include a plurality of terminals 5. Further, the anomaly detection system 1 may include a text generation server 7, or the functions of the text generation server 7 may be installed in the relay server 4. Here, it is described that the terminal 5 is connected only to the relay server 4, and the relay server 4 is connected to the analysis server 3 and the text generation server 7.

[0038] As shown in FIG. 1, the imaging device 2 images a monitoring area S (also referred to as a monitoring region) provided on the road and outputs the captured video to the analysis server 3. The imaging device 2 may be composed of one or more cameras 2A arranged near the monitoring area S as shown in FIG. 2. Here, only the imaging device 2 is provided as the sensor, but various sensors (radar, lidar, sonar, etc.) can be used in combination. The imaging device 2 may also be composed of one or more various devices and sensors that can acquire a video capable of grasping the situation of the monitoring area S. The anomaly detection system 1 may include one or more other sensors for detecting the situation of the monitoring area S in addition to (or instead of) the imaging device 2.

[0039] The imaging device 2 may be composed of a plurality of monitoring cameras provided near the monitoring area S.

[0040] The terminal 5 includes a computer that executes processes necessary to display the analysis result to the user. The terminal 5 includes an information device having a communication function such as a smartphone, a tablet terminal, or a notebook PC used by the user. As shown in FIG. 2, the terminal 5 may include known hardware such as a processor 5A (CPU, MPU, etc.), a memory 5B (RAM, ROM, etc.), a display, an input device, a network interface 5C, and a storage device 5D (HDD, SSD, etc.).

[0041] In this embodiment, the terminal 5 is composed of a smartphone.

[0042] As shown in FIG. 3, the terminal 5 has, as functional units, a communication unit 11, an operation input unit 12, a display unit 13, a storage unit 14, and a control unit 15.

[0043] The communication unit 11 has the same functions and configuration as the communication unit 11 of the terminal 5 and is constituted by a network interface 5C. The communication unit 11 performs wireless communication or wired communication with other devices (here, the relay server 4 etc.) via the communication network 6 in accordance with a known communication protocol.

[0044] The operation input unit 12 receives an input from the user. The operation input unit 12 includes a known input device such as a touch panel display 5E capable of input operations by the user.

[0045] The display unit 13 notifies the user by performing screen display. The display unit 13 includes a known display device such as a liquid crystal display. However, the touch panel display 5E used as an input device may function as a display device.

[0046] The storage unit 14 can be constituted by a memory 5B and a storage device 5D. The storage unit 14 stores various information required for performing the display on the display unit 13, various information related to the input received at the operation input unit 12, and various information necessary for the processing in the terminal 5.

[0047] The control unit 15 can be realized by the processor 5A performing processing in accordance with a program stored in the storage unit 14. The control unit 15 includes a transmission execution unit 21, an analysis request reception unit 22, a result notification unit 23, a response history transmission unit 24, an input assist unit 25, and a setting reception unit 26.

[0048] When the transmission execution unit 21 receives a transmission request from the relay server 4, it causes the display unit 13 to display a transmission screen 27 (see FIG. 4) and notifies the user that an abnormality has been detected. The transmission request transmitted from the relay server 4 may include text for notifying the user of details of the transmission content, such as "An accident has occurred at the intersection of 1-chome". The transmission request transmitted from the relay server 4 may include text for requesting the user to take measures such as emergency, fire, and police. Further, the transmission request transmitted from the relay server 4 may include text indicating the number of ambulances and fire trucks required (for example, "3 ambulances are required"). The text 27A included in the transmission request is displayed on the transmission screen 27. Note that the symbol of AI in FIG. 4 means a notification or answer output by the abnormality detection system 1.

[0049] The analysis request reception unit 22 receives an input for requesting an analysis of the abnormality history in the monitoring area S, that is, receives an analysis request. The analysis request reception unit 22 receives the analysis request by displaying an input field 28 on the transmission screen 27 (see FIGS. 4 and 5) or by causing the display unit 13 to display a chat screen 29 (see FIG. 6) including the input field 28. Note that the symbol of AI in FIGS. 5 and 6 means a notification or answer output by the abnormality detection system 1 as in FIG. 4. Also, the symbol of U in FIGS. 5 and 6 means an input by the user.

[0050] When an analysis request is input (that is, the analysis request is received), the analysis request reception unit 22 transmits a signal including the analysis request (hereinafter, analysis request notification) to the relay server 4. When the relay server 4 receives the analysis request signal, it transmits a signal including the corresponding natural language analysis result (hereinafter, analysis result notification) to the terminal 5 (result notification unit 23). The analysis result notification may include, for example, text for notifying the analysis result.

[0051] When the result notification unit 23 acquires the analysis result notification, it displays the corresponding natural language analysis result from the relay server 4 on the transmission screen 27 or the chat screen 29 in text 30. The result notification unit 23 may display the analysis result by a graph, an image, or the like in addition to the text 30.

[0052] After that, the analysis request reception unit 22 receives the input of the analysis request again. When there is an input of the analysis request, the analysis request reception unit 22 transmits the corresponding analysis request signal to the relay server 4.

[0053] When the analysis request reception unit 22 displays the analysis result on the display unit 13, the response history transmission unit 24 displays an input field 32 for the quality evaluation value for the analysis result on the display unit 13. When the quality evaluation value is input, the response history transmission unit 24 transmits the analysis request received by the analysis request reception unit 22, the natural language analysis result obtained from the relay server 4, and the corresponding quality evaluation value to the relay server 4 as the response history.

[0054] The input assist unit 25 presents an input example of the analysis request received by the analysis request reception unit 22 on the chat screen 29 to assist the user's input. When the analysis request candidate is included in the analysis result notification transmitted from the relay server 4, the input assist unit 25 may display the analysis request candidate on the display unit 13 as the input example 33.

[0055] The setting reception unit 26 receives a setting input for setting the text displayed on the display unit 13 and the input example presented by the input assist unit 25. FIG. 7 shows an example of the setting reception screen 34 that the setting reception unit 26 displays on the display unit 13 of the terminal 5. When there is an input to the setting button 27B at the upper right of the reporting screen 27 or the chat screen 29, the setting reception unit 26 may display the setting reception screen 34 on the display unit 13 of the terminal 5.

[0056] The setting reception screen 34 may include various input fields 35 for setting the text that the result notification unit 23 displays on the display unit 13.

[0057] In the example shown in FIG. 7, the setting reception screen 34 may include an input field 35A related to the degree of determination as an abnormality, an input field 35B related to items and calculation methods to be considered when calculating the degree of abnormality, an input field 35C related to the attributes of the user, and an input field 35D for the formality degree of the response text displayed by the terminal 5 in the chat. The attributes of the user may be, for example, occupations such as emergency, fire, and police. The setting reception screen 34 may further include an input field 35E for the number of input examples (question texts) proposed by the input assist unit 25, an input field 35F related to the threshold value of the quality of the input examples, and a setting completion button 36. When there is an input to the setting completion button 36, the setting reception unit 26 stores the setting input received in the input field 35 as setting information 14A in the storage unit 14 of the terminal 5.

[0058] When the setting information 14A is recorded in the storage unit 14, when the transmission execution unit 21, the analysis request reception unit 22, and the response history transmission unit 24 transmit various data to the relay server 4, it is preferable to transmit the setting information 14A together.

[0059] Next, the analysis server 3 includes a computer that executes processing necessary for performing abnormality analysis based on the video (imaging result) imaged by the imaging device 2. As shown in FIG. 2, the analysis server 3 may include known hardware such as a processor 3A (CPU, MPU, etc.), a memory 3B (RAM, ROM, etc.), a display, an input device, a network interface 3C, and a storage device 3D (HDD, SSD, etc.).

[0060] As shown in FIG. 8, the analysis server 3 includes, as functional units, a communication unit 41, a storage unit 42, and a control unit 43.

[0061] The communication unit 41 has the same functions and configuration as the communication unit 11 of the terminal 5 and may be configured by the network interface 3C. The communication unit 41 performs wireless communication or wired communication with other devices (here, the relay server 4) via the communication network 6 according to a known communication protocol.

[0062] The storage unit 42 can be constituted by a memory 3B or a storage device 3D. The storage unit 42 stores various programs necessary for performing abnormality analysis, a learned learning model 42A, video data acquired by the imaging device 2 (hereinafter also referred to as surveillance video 42B or surveillance video data), and the like.

[0063] The learning model 42A stored in the storage unit 42 includes an abnormality detection model that detects the degree of abnormality such as a car accident, a pedestrian fall, or a bicycle fall from images and videos captured by the imaging device 2. The abnormality detection model is used for the control unit 43 to output the probability that an image or video indicating a car accident, an image or video indicating a pedestrian fall, or an image or video indicating a bicycle fall is included in the images and videos acquired by the imaging device 2.

[0064] In addition, the learning model 42A stored in the storage unit 42 may include an injured person detection model that detects information on the number of injured persons, the age and gender of the injured persons, etc. based on the video captured by the imaging device 2. The learning model 42A stored in the storage unit 42 may include an accident response model that obtains the accident response method including the necessity of a stretcher and the priority level of accident response based on the acquired video.

[0065] The control unit 43 can be realized by the processor 3A performing processing according to the programs stored in the storage unit 42. The control unit 43 includes an abnormality analysis unit 46, a surveillance video update unit 47, a reporting evaluation unit 48, and a response acquisition unit 49.

[0066] The abnormality analysis unit 46 uses various learning models 42A stored in the storage unit 42 to perform an analysis process (hereinafter referred to as abnormality analysis) for detecting and analyzing abnormalities such as the occurrence of an accident with respect to the detection result (i.e., the captured video) of the imaging device 2.

[0067] The analysis performed by the abnormality analysis unit 46 may include detection of the occurrence of an accident in the monitoring area S. In addition, the analysis performed by the abnormality analysis unit 46 may include detection of information on the injured person based on the injured person detection model, a method for dealing with an accident based on the accident response model, and acquisition of the priority level of the accident response.

[0068] The monitoring video update unit 47 acquires the captured image from the imaging device 2 in real time and stores it in the storage unit 42 as the monitoring video 42B.

[0069] The reporting evaluation unit 48 uses the analysis result obtained by the abnormality analysis unit 46 to determine in real time whether an abnormality to be notified to the user has been detected. Abnormalities to be notified to the user may include, for example, traffic accidents.

[0070] In the present embodiment, the reporting evaluation unit 48 calculates the degree (hereinafter referred to as the reporting evaluation value) to notify the user that an abnormality has occurred based on the analysis result of the abnormality analysis obtained by the abnormality analysis unit 46. The reporting evaluation value is a numerical value indicating the degree of abnormality (abnormality degree), and is set to increase as the degree to be notified increases (that is, as the degree for the user to deal with increases).

[0071] The reporting evaluation value may be calculated based on the probability obtained by the abnormality detection model. For example, the reporting evaluation value may be calculated using the probability of an automobile accident occurring, the probability of a pedestrian falling, and the probability of a bicycle falling, which are obtained by the abnormality detection model. For example, the reporting evaluation value may be calculated as the maximum value of the probabilities of the three indicators of the probability of an automobile accident occurring, the probability of a pedestrian falling, and the probability of a bicycle falling, which are obtained by the abnormality detection model. The reporting evaluation value is preferably calculated based on the calculation method input in the input field 35B of the setting reception screen 34.

[0072] When the transmission evaluation unit 48 determines that the transmission evaluation value is equal to or greater than a predetermined threshold, it transmits a notification indicating that transmission should be made (hereinafter referred to as a transmission notification) to the relay server 4. At this time, the transmission evaluation unit 48 may include the transmission evaluation value and the analysis result of the abnormality analysis obtained by the abnormality analysis unit 46 (for example, the number of ambulances required, etc.) in the transmission notification and transmit it to the relay server 4.

[0073] When the transmission evaluation unit 48 determines that the transmission evaluation value is equal to or greater than a predetermined threshold, it may store in the storage unit 42 the accident occurrence time and information related to the details of the accident (hereinafter referred to as accident information) obtained by analysis. In this case, the storage unit 42 may hold an accident information database in which the accident information is recorded.

[0074] The response acquisition unit 49 acquires (collects) the analysis result corresponding to the analysis request transmitted by the relay server 4 from the abnormality analysis unit 46 and outputs it to the relay server 4.

[0075] When the response acquisition unit 49 receives from the relay server 4 an analysis request for obtaining information on the injured, it acquires information on the injured from the abnormality analysis unit 46 and transmits it to the relay server 4 as an analysis result. When the response acquisition unit 49 receives an analysis request for obtaining information on the injured, it may also transmit, together with the information on the injured, an image at the time of the accident from the surveillance video 42B and an image acquired by the imaging device 2 to the relay server 4 as an analysis result.

[0076] When the response acquisition unit 49 receives from the relay server 4 an analysis request for obtaining information related to accident response, it acquires information related to accident response from the abnormality analysis unit 46 and transmits the acquired information related to accident response to the relay server 4 as an analysis result.

[0077] The relay server 4 includes a computer that, as necessary, naturalizes the analysis results of the analysis server 3 via the text generation server 7 and executes processes necessary to display the naturalized analysis results on the terminal 5. As shown in FIG. 2, the computer constituting the relay server 4 may include known hardware such as a processor 4A (CPU, MPU, etc.), a memory 4B (RAM, ROM, etc.), a display, an input device, a network interface 4C, and a storage device 4D (HDD, SSD, etc.).

[0078] As shown in FIG. 9, the relay server 4 includes, as functional units, a communication unit 51, a storage unit 52, and a control unit 53.

[0079] The communication unit 51 is constituted by the network interface 4C. The communication unit 51 performs wireless communication or wired communication with other devices (here, the terminal 5 and the text generation server 7) via the communication network 6 according to a known communication protocol.

[0080] The storage unit 52 may be constituted by the memory 4B or the storage device 4D. The storage unit 52 stores various information necessary for the processes performed by the relay server 4. The information stored in the storage unit 52 may include a response history database 52A in which response history data transmitted from the response history transmission unit 24 of the terminal 5 is recorded.

[0081] In the response history database 52A, a naturalized analysis request received at the terminal 5, the analysis results of the analysis server 3 naturalized and displayed on the terminal 5, and the corresponding user quality evaluation value are recorded in association with each other.

[0082] Note that the user attributes for which setting inputs are received by the terminal 5 (setting reception unit 26) may be recorded in the response history database 52A in association with the analysis request, the analysis results, and the quality evaluation value.

[0083] The control unit 53 includes a reporting processing unit 61, an analysis processing unit 62, and an assist processing unit 63.

[0084] When the transmission processing unit 61 receives a transmission notification from the analysis server 3, it acquires the transmission evaluation value included in the transmission notification and the analysis result obtained by the abnormality analysis unit 46. Then, the transmission processing unit 61 generates a prompt required to naturalize the transmission evaluation value and the analysis result in the text generation server 7.

[0085] Next, the transmission processing unit 61 transmits (inputs) the generated prompt to the text generation server 7, and acquires the text in which the transmission evaluation value and the analysis result are naturalized from the text generation server 7.

[0086] After that, when the transmission processing unit 61 acquires the text in which the transmission evaluation value and the analysis result are naturalized from the text generation server 7, it transmits a transmission request including the text to the terminal 5, and causes the terminal 5 (transmission execution unit 21) to display a transmission screen 27 including the text 27A in which the transmission evaluation value and the analysis result are naturalized (see FIG. 4).

[0087] When the analysis processing unit 62 receives an analysis request notification from the terminal 5, it generates an analysis request that specifies the content of the data to be acquired from the analysis server 3 from the analysis request notification. Then, the analysis processing unit 62 transmits the analysis request to the analysis server 3, and acquires the analysis result corresponding to the analysis request from the analysis server 3 (response acquisition unit 49).

[0088] Furthermore, the analysis processing unit 62 acquires the analysis result from the analysis server 3, and generates a prompt required to naturalize the analysis result.

[0089] Next, the analysis processing unit 62 transmits (inputs) the generated prompt to the text generation server 7, and acquires the text in which the analysis result corresponding to the analysis request is naturalized from the text generation server 7.

[0090] After that, when the analysis processing unit 62 acquires the text in which the analysis result is naturalized by the text generation server 7, the analysis result notification including the text is transmitted to the terminal 5. As a result, the terminal 5 displays, on the display unit 13, a chat screen 29 (see FIG. 5) including the text 30 in which the analysis result is naturalized by the result notification unit 23.

[0091] In this way, the analysis processing unit 62 generates a prompt based on the analysis result of the analysis server 3, and inputs the prompt to the text generation server 7, thereby acquiring the text naturalized based on the prompt. By using the text generation server 7 in this way, the analysis processing unit 62 can easily naturalize the analysis result. Also, in terms of information security, even when the text generation server 7 cannot directly access the analysis server 3 or when it is desired to prohibit the text generation server 7 from directly accessing the analysis server 3, the analysis result can be easily naturalized.

[0092] Note that the analysis processing unit 62 may be configured to naturalize the analysis result of the analysis server 3, or may be configured to naturalize the prompt generated from the analysis result by itself. Thereby, the analysis processing unit 62 does not need to provide the text generation server 7, and may transmit an analysis result notification including the text acquired by naturalizing the analysis result by itself to the terminal 5.

[0093] Also, when the information related to the setting input is included in the analysis request notification transmitted from the terminal 5, the analysis processing unit 62 may transmit (input) the generated prompt to the text generation server 7 together with the information related to the setting input, and acquire the text from the text generation server 7.

[0094] Specifically, the analysis processing unit 62 requests the text generation server 7 to answer based on the user attributes included in the setting information 14A, and acquires a text indicating the analysis result along with the user attributes. Further, the analysis processing unit 62 requests the text generation server 7 to answer in consideration of the formality degree of the set answer sentence, and acquires a text indicating the analysis result that matches the set formality degree.

[0095] Based on the analysis result acquired by the analysis processing unit 62, the assist processing unit 63 refers to the answer history database 52A, and acquires one or more analysis requests to be proposed as analysis request candidates based on the information set in the setting information 14A.

[0096] The assist processing unit 63 may refer to the answer history database 52A, preferentially select (also referred to as extract) the frequently requested (highly requested) analysis requests, and acquire the selected analysis requests as analysis request candidates. Further, when the information related to the setting input is included in the analysis request notification transmitted from the terminal 5, the assist processing unit 63 may determine the number of analysis requests to be extracted based on the setting information 14A. In addition, the assist processing unit 63 may select analysis results similar to the immediately preceding analysis result from the answer history database 52A, and select a plurality of subsequent analysis requests and acquire them as analysis request candidates. Further, the assist processing unit 63 may refer to the answer history database 52A and acquire analysis request candidates based on the quality threshold of the input example 33 included in the setting information 14A and the quality evaluation value and the like.

[0097] Next, the flow until the reporting screen 27 is displayed on the terminal 5 in the anomaly detection system 1 will be described. FIG. 10 is a sequence diagram showing the flow of the reporting notification process in the anomaly detection system 1.

[0098] As shown in FIG. 10, the analysis server 3 acquires the imaging result (that is, the imaging video) from the imaging device 2 in real time (S101), and stores it in the storage unit 52 as the monitoring video 42B (S102).

[0099] Next, the analysis server 3 calculates an alarm evaluation value, which is the degree to which it should notify the user that an abnormality has occurred based on the surveillance video 42B (imaging result) of the imaging device 2. After that, the analysis server 3 determines whether the calculated alarm evaluation value is equal to or greater than a threshold value (S103). When it is determined that the alarm evaluation value is equal to or greater than the threshold value, the analysis server 3 sends an alarm notification to the relay server 4 (S104). When it is determined that the alarm evaluation value is less than the threshold value, the analysis server 3 acquires the imaging result from the imaging device 2 again, stores it as the surveillance video 42B in the storage unit 52, calculates the alarm evaluation value, and compares it with the threshold value.

[0100] When the relay server 4 receives the alarm notification from the analysis server 3, it acquires various analysis data as the analysis results obtained by the analysis server 3 and the alarm evaluation value included in the alarm notification. After that, the alarm processing unit 61 generates a prompt required to naturalize the alarm evaluation value and the analysis results in the text generation server 7 (S105).

[0101] When the generation of the prompt is completed in the relay server 4, the relay server 4 sends the prompt to the text generation server 7 (S106). When the text generation server 7 acquires the prompt from the relay server 4, it generates a text in which the alarm evaluation value and the analysis results are naturalized (S107), and sends the generated prompt to the relay server 4 (S108).

[0102] When the relay server 4 receives (acquires) the text in which the alarm evaluation value and the analysis results are naturalized from the text generation server 7, it sends an alarm request to the terminal 5 (S109). When the terminal 5 acquires the alarm request, it displays an alarm screen 27 including the naturalized alarm evaluation value and analysis results on the display unit 13 (S110).

[0103] When a traffic accident occurs in the monitoring area S, based on the reporting notification from the analysis server 3, the relay server 4 generates a prompt (see Fig. 10) that clearly explains the analysis result, such as "Accident at the intersection of the 1st block, 2 ambulances required", and sends it to the text generation server 7. The text generation server 7 generates text such as "An accident has occurred at the intersection of the 1st block. Please request 2 ambulances." (see Fig. 4) based on the sent prompt, and the terminal 5 displays the generated text.

[0104] In this way, through the reporting process performed by the relay server 4, the reporting notification by the analysis server 3 is natural languageized, and the natural languageized analysis result is displayed on the terminal 5. Therefore, it becomes possible to notify the user of the result of the anomaly analysis in an easy-to-understand manner.

[0105] Next, the flow from when an input is received in the input field 28 of the reporting screen 27 or the chat screen 29 on the terminal 5 to when the analysis result is displayed in the anomaly detection system 1 will be described. Fig. 11 is a sequence diagram showing the flow of the analysis process in the anomaly detection system 1.

[0106] When an input is made in the input field 28 of the reporting screen 27 or the chat screen 29 by the user and an analysis request is received, the terminal 5 sends an analysis request notification to the relay server 4 (S201).

[0107] When the relay server 4 receives the analysis request notification from the terminal 5, it performs text analysis etc. and obtains an analysis request (a request regarding the collection range of analysis data etc.) for querying the analysis server 3 (S202). After that, when the relay server 4 obtains the analysis request, it sends the analysis request to the analysis server 3 (S203).

[0108] When the analysis server 3 receives (obtains) the analysis request, it refers to the monitoring video 42B and performs the corresponding analysis (S204), and sends the analysis result corresponding to the analysis request to the relay server 4 (S205). At this time, when the analysis server 3 holds the accident information database, it may use the accident information database to obtain the analysis result corresponding to the analysis request.

[0109] When the relay server 4 obtains the analysis result, it generates a prompt required to naturalize the analysis result (S206). When the generation of the prompt is completed, the relay server 4 transmits (inputs) the prompt to the text generation server 7 as necessary (S207).

[0110] When the text generation server 7 receives the prompt corresponding to the analysis result from the relay server 4, it generates a text obtained by naturalizing the corresponding analysis result (S208) and transmits it to the relay server 4 (S209).

[0111] The relay server 4 generates an analysis result notification including the text transmitted from the relay server 4 (S210). When the generation of the analysis result notification is completed, the relay server 4 transmits the generated analysis result notification to the terminal 5 (S211). When the terminal 5 obtains the analysis result notification, it displays a reporting screen 27 (see FIG. 5) or a chat screen 29 (see FIG. 6) including the text 30 in which the analysis result is naturalized (S212).

[0112] When "Tell me the situation of the injured" is input in the input field 28 (the analysis request is received), the relay server 4 obtains the information of the injured from the analysis server 3 and generates a prompt to clearly explain the analysis result such as "Slight injury: 1 female in her 30s, 1 elderly male" (see FIG. 10) and transmits it to the text generation server 7. Based on the transmitted prompt, the text generation server 7 generates a text such as "It seems to be a slight injury to a female in her 30s and an elderly male", and the terminal 5 displays the generated text (see FIGS. 5 and 6). In this way, since the reporting notification by the analysis server 3 is naturalized and displayed on the terminal 5, it is possible to notify the user of the result of the abnormality analysis in an easy-to-understand manner.

[0113] In addition, when "What should I do?" is entered in the input field 28, the analysis server 3 obtains the accident handling method and the priority of accident response based on the accident handling model, and generates keywords such as "Possible after accident handling in the 3rd block". The analysis server 3 sends the generated keywords to the text generation server 7, and accordingly, the text generation server 7 generates text such as "It is considered that there is no problem with the response after the accident in the 3rd block." Then, the terminal 5 displays the text generated by the text generation server 7 (see FIGS. 5 and 6).

[0114] When "Is there a possibility of a fire?" is entered in the input field 28, the analysis server 3 analyzes the possibility of a fire. When the analysis server 3 obtains the analysis result, it uses the text generation server 7 to generate text including the probability of a fire, etc., and causes the terminal 5 to display "According to the analysis result information, the possibility of a vehicle fire is as low as 15%. However, since the degree of damage to the vehicle is large, rescue is considered necessary." etc.

[0115] In this way, through the analysis notification process performed by the relay server 4, the analysis result of the abnormality analysis by the analysis server 3 for the analysis request is natural languageized, and the natural languageized analysis result is displayed on the terminal 5. Therefore, it becomes possible to notify the user of the analysis result of the abnormality analysis in an easy-to-understand manner.

[0116] When the relay server 4 notifies the terminal 5 of the analysis result of the abnormality analysis, at the same time, it causes the terminal 5 to display the input field 32 for the quality evaluation value for the analysis result and obtain the quality evaluation value. When the quality evaluation value is input on the terminal 5, it is sent to the relay server 4 (S213). The relay server 4 records the received quality evaluation value, together with the analysis request and the analysis result, as a response history in the response history database 52A and stores it in the storage unit 52 (S214).

[0117] Also, when the relay server 4 receives an analysis request at the terminal 5, it appropriately acquires, as analysis request candidates, the analysis requests to be proposed based on the response history recorded in the response history database 52A (S215). At this time, when the relay server 4 receives a setting input including the user's attributes or the like at the terminal 5, it acquires them as analysis request candidates based on the setting input and the response history. When the acquisition of the analysis request candidates is completed, the relay server 4 transmits the acquired analysis request candidates to the terminal 5 (S216).

[0118] When the terminal 5 acquires the analysis request candidates, it displays the analysis request candidates on the display unit 13. In this way, by displaying the analysis request candidates, the terminal 5 proposes the analysis request to be input and assists the user in inputting the analysis request related to the user's analysis request (S217).

[0119] At this time, as shown in FIG. 6, the terminal 5 obtains the average value of the quality evaluation values given in the past similar analysis requests for the analysis request candidates, and displays the average quality evaluation value 32A (the number of stars, numerical values, etc.). Note that when acquiring the analysis request candidates, the relay server 4 may display the analysis request candidates and the display column 32A in accordance with the setting of the quality of the question text to be proposed (the input column 35F related to the quality threshold in FIG. 7).

[0120] In this way, by causing the terminal 5 to display, as analysis request candidates, the analysis requests to be proposed based on the response history, the possibility of obtaining a high-quality response is increased. Also, it becomes possible to propose candidates to the user. Further, since the relay server 4 acquires the analysis request candidates based on the setting input including the user's attributes, it can propose analysis request candidates corresponding to the user's attributes (for example, the user's occupation). Also, since the response history recorded in the response history database 52A includes quality evaluation values, the quality of the abnormality analysis performed by the analysis server 3 and the quality of the analysis requested from the relay server 4 to the analysis server 3 can be improved using these quality evaluation values.

[0121] In the above-described embodiment, an example in which the terminal 5 is configured by a smartphone has been described. However, the terminal 5 is not limited to a smartphone, and the terminal 5 may be any information device having a communication function such as a tablet terminal or a notebook PC. FIG. 11 shows an example of the reporting screen 27 when the terminal 5 is configured by a tablet terminal. As shown in FIG. 11, when reporting, the terminal 5 may display on the display unit 13, together with the text obtained by natural language processing of the reporting notification by the analysis server 3, the captured image 70 of the monitoring area S acquired by the imaging device 2.

[0122] As described above, the embodiments have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can also be applied to embodiments with changes, replacements, additions, omissions, etc. Further, it is also possible to combine the respective components described in the above embodiments to form a new embodiment.

[0123] So far, an example in which an analysis system uses image recognition and speech recognition technologies and can notify the recognition result by text obtained by natural language processing has been described in an abnormal detection system for performing abnormal detections such as traffic accidents. However, the application range of the analysis system is not limited to the abnormal detection system 1.

[0124] The analysis system can also be applied, for example, to a work analysis system that analyzes work efficiency at a work site such as a factory and aims to improve productivity.

[0125] In addition, the analysis system can be applied to a police work support system, a monitoring system for stations, airports, commercial facilities, public facilities, construction sites, etc., and a highway management system. The analysis system can further be applied to an education support system that analyzes the behavior of students and supports the improvement of classes, etc., a medical support system that analyzes the behavior of doctors, nurses, etc. and supports the improvement of medical operations, a nursing care support system that analyzes the behavior of elderly people and supports the improvement of nursing care operations, a skill improvement support system that analyzes the behavior of athletes and supports the improvement of skills, etc.

Industrial Applicability

[0126] The abnormality detection system and method according to the present disclosure can notify the detection result in an easy-to-understand manner, and are useful as an abnormality detection system and method that detect an abnormality in a monitoring area and notify the user of the detection result.

Description of Symbols

[0127] 1: Abnormality detection system 2: Imaging device 2A: Camera 3: Analysis server 3A: Processor 3B: Memory 3C: Network interface 3D: Storage device 4: Relay server 4A: Processor 4B: Memory 4C: Network interface 4D: Storage device 5: Terminal 5A: Processor 5B: Memory 5C: Network interface 5D: Storage device 5E: Touch panel display 6: Communication network 7: Text generation server 7A: Processor 7B: Memory 7C: Network interface 7D: Storage device 11: Communication unit 12: Operation input unit 13: Display unit 14: Storage unit 14A: Setting information 15: Control unit 21: Transmission execution unit 22: Analysis request reception unit 23: Result Notification Section 24: Reply History Sending Section 25: Input Assist Section 26: Setting Reception Section 27: Transmission Screen 27A: Text 27B: Setting Button 28: Input Field 29: Chat Screen 30: Text 32: Input Field 33: Input Example 34: Setting Reception Screen 35: Input Field 35A: Input Field 35B: Input Field 35C: Input Field 35D: Input Field 35E: Input Field 35F: Input Field 36: Setting Completion Button 41: Communication Section 42: Memory Section 42A: Learning Model 42B: Surveillance Video 43: Control Section 46: Abnormality Analysis Section 47: Surveillance Video Update Section 48: Transmission Evaluation Section 49: Reply Acquisition Section 51: Communication Section 52: Memory Section 52A: Reply History Database 53: Control Section 61: Transmission Processing Section 62: Analysis Processing Section 63: Assist Processing Section 70: Captured Image S: Surveillance Area

Claims

1. An abnormality detection system that detects an abnormality in a monitoring area and notifies a user of the analysis result of the abnormality, comprising: An imaging device that images the monitoring area; An analysis server that performs abnormality analysis based on the imaging result of the imaging device; A terminal that displays the analysis result obtained by the analysis server to the user; A relay server that naturalizes the analysis result of the analysis server and causes the terminal to display the naturalized analysis result.

2. The abnormality detection system according to claim 1, wherein the relay server generates a prompt including the analysis result, inputs the prompt to a sentence generation server that naturalizes the prompt based on the prompt and outputs the prompt as text, thereby naturalizing the analysis result of the analysis server.

3. The analysis server calculates an evaluation value indicating the degree of abnormality detected in the monitoring area based on the imaging result from the imaging device and outputs the evaluation value to the relay server. The abnormality detection system according to claim 1, wherein the relay server naturalizes the analysis result of the analysis server and causes the terminal to display the analysis result when the evaluation value is equal to or greater than a predetermined threshold.

4. The analysis server stores the monitoring video of the monitoring area imaged by the imaging device. The relay server: Obtains a request notification for making an analysis request for an abnormality detected in the monitoring area from the terminal to the analysis server; Based on the obtained request notification, obtains the analysis result based on the monitoring video from the analysis server, naturalizes the analysis result, and causes the terminal to display the analysis result.

5. The abnormality detection system according to claim 4, wherein the relay server notifies the terminal of the analysis result, causes the terminal to obtain a quality evaluation value input by the user for the analysis result, and stores the analysis request, the naturalized analysis result, and the quality evaluation value together as an answer history.

6. The abnormality detection system according to claim 5, wherein when the relay server obtains the request notification from the terminal, the relay server causes the terminal to present candidates for the analysis request based on the answer history.

7. The relay server: Displays a setting screen for receiving a setting input including the attributes of the user on the terminal. The abnormality detection system according to claim 6, wherein candidates for the analysis request to be displayed on the terminal are selected based on the setting input received on the setting screen and the answer history.

8. An abnormality detection method for detecting an abnormality in a monitoring area and notifying a user of an analysis result, comprising: an imaging device that images the monitoring area; an analysis server that performs an abnormality analysis based on an imaging result of the imaging device; a terminal that displays the analysis result obtained by the analysis server to the user; and a relay server connected to the terminal and the analysis server, and is executed by an abnormality detection system including the relay server, wherein the relay server naturalizes the analysis result of the analysis server and causes the terminal to display the naturalized analysis result.

Images