Abnormality management system

Abstract

To automatically issue an appropriate announcement on an abnormal event which a mobile body encounters to an appropriate announcement destination.SOLUTION: An abnormality management system includes one or a plurality of processors. The one or the plurality of processors are configured to execute: video acquisition processing to acquire a video to be photographed by a camera mounted on a mobile body; abnormal event recognition processing to automatically recognize an abnormal event appearing on the video and the contents of the abnormal event by using a machine learning model; announcement destination determination processing to automatically determine an announcement destination corresponding to the contents of the abnormal event; and announcement processing to automatically transmit to the announcement destination, abnormal event information which includes at least information indicating the contents of the abnormal event and position information indicating the position of the abnormal event.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an abnormality management system applied to a mobile object. [Background technology]

[0002] Patent Document 1 discloses an information processing system including an information processing device and an in-vehicle device. The in-vehicle device detects that the vehicle in which the in-vehicle device is installed is being tailgated by another vehicle. The information processing device receives an event detection notification from the in-vehicle device. If it determines that tailgating has occurred, the information processing device reports the event to at least one of a security company or the police.

[0003] Furthermore, each of Patent Documents 2 to 4 discloses a technique for dealing with abnormal events (for example, tailgating, traffic accidents) that a vehicle encounters. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-043552 [Patent Document 2] Patent Publication No. 2021-165906 [Patent Document 3] Japanese Patent Application Publication No. 2019-179313 [Patent Document 4] Japanese Patent Application Publication No. 11-345385 [Patent Document 5] Japanese Patent Application Publication No. 7-220191 Summary of the Invention [Problem to be solved by the invention]

[0005] While driving a moving object such as a vehicle, a driver may discover various abnormal events around them. However, it is difficult for the driver to report them while driving. Furthermore, abnormal events are not limited to tailgating and can have a variety of contents, so there is not just one appropriate place to report them. Even if a driver does decide to report an abnormal event, depending on the content of the abnormal event, they may not immediately know where to report it. If a report is made to an inappropriate place, it can lead to delays in reporting and unnecessary confusion. [Means for solving the problem]

[0006] The anomaly management system according to the present disclosure includes one or more processors configured to execute a video acquisition process for acquiring video captured by a camera mounted on a mobile object, an abnormal event recognition process for automatically recognizing an abnormal event and the details of the abnormal event captured in the video by using a machine learning model, a report destination determination process for automatically determining a report destination according to the details of the abnormal event, and a reporting process for automatically transmitting, to the report destination, abnormal event information including at least information indicating the details of the abnormal event and location information indicating the location of the abnormal event. [Effects of the Invention]

[0007] According to the present disclosure, an abnormal event is automatically recognized from the video captured by the camera and automatically reported, so the driver does not need to report it while driving. Furthermore, a report destination is automatically determined based on the content of the abnormal event, and the report is appropriately sent to the appropriate report destination, preventing delays in reporting and unnecessary confusion. Furthermore, abnormal event information including the content and location of the abnormal event is automatically sent to the report destination, so the driver does not need to explain each event point by point. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining an overview of an anomaly management system according to the present disclosure. [Figure 2] 1 is a block diagram showing an example of the configuration of an abnormality management system according to a first embodiment. [Figure 3] 10 is a flowchart showing an example of a processing flow related to management of an abnormal event according to the first embodiment. [Figure 4] FIG. 10 is a block diagram showing an example of the configuration of an abnormality management system according to a second embodiment. [Figure 5] 10 is a flowchart showing an example of a processing flow related to management of an abnormal event according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0010] [Outline of the abnormality management system] FIG. 1 is a diagram for explaining an overview of an anomaly management system according to the present disclosure. The anomaly management system is applied to a moving body. Examples of moving bodies include a vehicle, a robot, and an aircraft (e.g., a drone). As an example, in the following description, a case will be considered in which the moving body to which the anomaly management system is applied is a vehicle 10. When generalizing, the term "vehicle" in the following description will be read as "moving body."

[0011] The processes executed in the anomaly management system include the four processes shown in FIG. 1, namely, "video acquisition process," "anomaly event recognition process," "report destination determination process," and "report process."

[0012] A camera 11 (see FIG. 2) is mounted on the vehicle 10. The camera 11 obtains an image I showing the situation around the vehicle 10. The image acquisition process is a process for acquiring the image I captured by the camera 11.

[0013] The abnormal event recognition process is a process that uses a machine learning model to automatically recognize abnormal events and their details captured in the video I of the camera 11. Examples of abnormal events include traffic accidents, fires, crimes / incidents (e.g., car break-ins, theft, aggressive driving), illness, lost items on the road, flooded roads, and rising rivers. An abnormal event may occur at a stationary location. Furthermore, the same abnormal event may be commonly recognized by multiple vehicles 10 (e.g., vehicles 10-1, 10-2, and 10-3). Additionally, as described above, the recognition of an abnormal event by the abnormal event recognition process includes the host vehicle recognizing an abnormal event of the host vehicle (e.g., aggressive driving) and the host vehicle recognizing an external abnormal event (an abnormal event occurring around the host vehicle).

[0014] The abnormal event information is information about an abnormal event recognized by the abnormal event recognition process. The abnormal event information includes at least information indicating the content of the abnormal event and location information indicating the location of the abnormal event. Examples of the content of the abnormal event include the type, situation, and scale of the abnormal event.

[0015] More specifically, information indicating the content of the abnormal event is acquired from the recognition result of the abnormal event recognition process. The position of the abnormal event can be calculated by combining the position of the vehicle 10 and the position of the abnormal event in the image I of the camera 11. The position of the vehicle 10 may be approximately considered to be the position of the abnormal event. In other words, the position of the vehicle 10 when the abnormal event is recognized may be used as the position of the abnormal event.

[0016] The abnormal event information may also include video I of a target period that includes at least the timing when the abnormal event was recognized by the abnormal event recognition process. Furthermore, the abnormal event information may also include time information that indicates the time when the abnormal event was recognized.

[0017] The report destination determination process is a process of automatically determining the report destination 100 according to the content of the abnormal event included in the abnormal event information. In order to determine the report destination 100 in the report destination determination process, for example, a machine learning model or rule-based artificial intelligence (AI) technology may be used. Examples of the report destination 100 include organizations such as the police, fire department, road administrator, and local government. More specifically, the correspondence relationship between the abnormal event and the report destination 100 is, for example, as follows: ·Traffic accidents: fire department, police department ·Fire: Firefighting ·Crime / Incident: Police ·Illness: Firefighters ·Lost items on the road: Road administrator Flooding of roads and rising rivers: Local government

[0018] The reporting process is a process of automatically transmitting abnormal event information to the report destination 100 determined by the report destination determination process.

[0019] The abnormality management system according to the present disclosure may be mounted on a vehicle 10, as described in the first embodiment. In the first embodiment, all of the above four processes are executed by an information processing device 15 mounted on the vehicle 10. That is, all of the four processes are completed within the vehicle 10.

[0020] Alternatively, the anomaly management system according to the present disclosure may include an in-vehicle system 20 (a system mounted on a vehicle) and a management device 30, as described in a second embodiment. In the second embodiment, the above four processes are executed by cooperation between the in-vehicle system 20 and the management device 30. Specifically, the in-vehicle system 20 executes an image acquisition process and an abnormal event recognition process. The in-vehicle system 20 transmits abnormal event information to the management device 30. The management device 30 receives the abnormal event information from the in-vehicle system 20, and executes a report destination determination process and a report process. In this way, in the second embodiment, the management device 30 is interposed between a plurality of vehicles 10 and the report destination 100, and the management device 30 issues a report on behalf of the vehicles 10.

[0021] 1. First embodiment 1-1.Configuration example 2 is a block diagram showing an example of the configuration of an abnormality management system 1 according to the first embodiment. The abnormality management system 1 is mounted on a vehicle 10. The abnormality management system 1 includes, for example, one or more cameras 11 (hereinafter simply referred to as "cameras 11"), a position sensor 12, hazard lights 13, an HMI device 14, and an information processing device 15.

[0022] The camera 11 captures images of the surroundings of the vehicle 10. The position sensor 12 detects the position and orientation of the vehicle 10. The position sensor 12 includes, for example, a GNSS (Global Navigation Satellite System) receiver. The hazard lights 13 are attached to the body of the vehicle 10. The HMI device 14 is, for example, a touch panel. Additionally, the combination of the camera 11 and the information processing device 15 corresponds to an example of a drive recorder.

[0023] The information processing device 15 includes a communication device 16, one or more processors 17 (hereinafter simply referred to as "processors 17"), and one or more storage devices 18 (hereinafter simply referred to as "storage devices 18"). The communication device 16 communicates with the outside of the vehicle 10 (including the notification destination 100) via a communication network.

[0024] The processor 17 executes various processes, including processes related to abnormal event management (detection and reporting) (see FIG. 1 ). Examples of the processor 17 include a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), and a field-programmable gate array (FPGA). The processor 17 may also be referred to as a "circuitry" or a "processing circuitry." The "circuitry" refers to hardware programmed to implement a described function or hardware that executes a function. The storage device 18 stores various types of information. Examples of the storage device 18 include volatile memory, nonvolatile memory, a hard disk drive (HDD), and a solid-state drive (SSD). The processor 17 reads various types of information from and stores various types of information in the storage device 18. The functions of the information processing device 15 may be implemented by cooperation between the processor 17, which executes a computer program, and the storage device 18. The computer program is stored in the storage device 18. Alternatively, the computer program may be recorded on a computer-readable recording medium or provided via a network.

[0025] The storage device 18 also stores information such as driving environment information, abnormal event information, and report destination information. The driving environment information is information indicating the driving environment of the vehicle 10. The driving environment information includes, for example, the image I captured by the camera 11, location information indicating the position and direction of the vehicle 10 obtained by the position sensor 12, and time information obtained by a count timer provided in the processor 17 for time management. The driving environment information is used as abnormal event information. The abnormal event information is as described above. The report destination information includes information indicating the correspondence between each abnormal event and the report destination 100, and information necessary for communication with each report destination.

[0026] 1-2. Management of abnormal events 3 is a flowchart showing an example of the flow of processing related to abnormal event management according to the first embodiment. The processing of this flowchart is executed by the information processing device 15 (processor 17).

[0027] In step S100, the information processing device 15 determines whether the AI ​​automatic abnormality detection mode is ON. The AI ​​automatic abnormality detection mode is a mode in which the vehicle 10 automatically detects abnormal events using AI technology. The AI ​​automatic abnormality detection mode is switched ON / OFF by, for example, the driver of the vehicle 10 operating the HMI device 14. If the AI ​​automatic abnormality detection mode is OFF (step S100; No), the process proceeds to RETURN. On the other hand, if the AI ​​automatic abnormality detection mode is ON (step S100; Yes), the process proceeds to step S102.

[0028] In step S102, the information processing device 15 executes the above-described image acquisition process. That is, as already described, the information processing device 15 acquires the image I captured by the camera 11. The information processing device 15 stores the acquired image I of the camera 11 in the storage device 18. Thereafter, the process proceeds to step S104.

[0029] In step S104, the information processing device 15 executes the abnormal event recognition process described above. That is, as already described, the information processing device 15 uses a machine learning model to automatically recognize an abnormal event captured in the video I and the details of the abnormal event (type, situation, scale, etc.). The machine learning model is trained to output a recognition result of an abnormal event and the details of the abnormal event from the input video I. The machine learning model is stored in the storage device 18 of the vehicle 10. The information processing device 15 stores information indicating the details of the recognized abnormal event and location information indicating the location where the abnormal event was recognized in the storage device 18. The information processing device 15 may also store time information indicating the time when the abnormal event was recognized in the storage device 18. After step S104, the process proceeds to step S106.

[0030] In step S106, the information processing device 15 determines whether or not an abnormal event has been detected (recognized) by the abnormal event recognition process. As a result, if an abnormal event has not been detected (step S106; No), the process proceeds to RETURN. On the other hand, if an abnormal event has been detected (step S106; Yes), the process proceeds to step S108.

[0031] In addition, if an abnormal event is detected (step S104; Yes), depending on the content of the detected abnormal event, the information processing device 15 may turn on the hazard lights of the vehicle 10 for, for example, a predetermined time to alert vehicles behind the vehicle 10 (host vehicle). Also, if no abnormal event is detected (step S106; No), the information processing device 15 may start storing the video I in the storage device 18 in response to an operation of the HMI device 14 by the driver who has determined that an abnormal event has occurred. In the second embodiment, the start of storing the video I in response to such an operation of the HMI device 14 may be performed in the storage device 33 of the management device 30 (cloud).

[0032] In step S108, the information processing device 15 determines whether the AI ​​automatic reporting mode is ON. The AI ​​automatic reporting mode is a mode in which the vehicle 10 automatically reports abnormal event information using AI technology. The AI ​​automatic reporting mode is switched ON / OFF by, for example, the driver of the vehicle 10 operating the HMI device 14. If the AI ​​automatic reporting mode is OFF (step S108; No), the process proceeds to RETURN. On the other hand, if the AI ​​automatic reporting mode is ON (step S108; Yes), the process proceeds to step S110.

[0033] In step S110, the information processing device 15 executes the above-described report destination determination process. That is, as already described, the information processing device 15 automatically determines an appropriate report destination 100 according to the content of the abnormal event included in the abnormal event information. Additionally, in an example in which a machine learning model is used to automatically determine the report destination, the machine learning model is trained to output an appropriate report destination 100 for the input abnormal event information. The machine learning model is stored in the storage device 18 of the vehicle 10. After step S110, the process proceeds to step S112.

[0034] In step S112, the information processing device 15 executes the above-mentioned reporting process. That is, as already explained, the information processing device 15 automatically transmits the abnormal event information to the determined reporting destination 100. More specifically, the information processing device 15, for example, executes a process of verbalizing the abnormal event information and transmits information on the content and location of the abnormal event to the reporting destination 100. Furthermore, if the abnormal event information includes video I captured during the target period, the information processing device 15 may also transmit information indicating how to access the video I. Furthermore, if the abnormal event information includes time information of the abnormal event, the information processing device 15 may also transmit the time information.

[0035] In addition, the information processing device 15 may transmit other useful information together with the abnormal event information to the report destination 100. Examples of useful information here include current location information (location information at the time of reporting) of at least one of a vehicle (at least one of the subject vehicle and another vehicle) and a person involved in the abnormal event, information on the weather at the site or a weather forecast, etc.

[0036] 1-3.Effects As described above, according to the anomaly management system 1 of the first embodiment, an abnormal event is automatically recognized from the video I captured by the camera 11 and automatically reported, eliminating the need for the driver to report the event while driving. In addition, the report destination 100 is automatically determined according to the content of the abnormal event, and the report is appropriately sent to the appropriate report destination 100, preventing delays in reporting and unnecessary confusion. Furthermore, because abnormal event information including the content and location of the abnormal event is automatically sent to the report destination 100, the driver does not need to explain each event point by point. This is particularly effective when it is difficult for the driver to explain the exact location and content of the abnormal event.

[0037] Furthermore, the abnormal event information may include video I captured during a target period that includes at least the timing when the abnormal event was recognized, which makes it even easier to understand the circumstances of the abnormal event.

[0038] 2. Second embodiment 2-1.Configuration example 4 is a block diagram showing an example configuration of an abnormality management system 2 according to a second embodiment. The abnormality management system 2 includes a plurality of on-board systems 20 mounted on a plurality of vehicles 10 (10-1 to 10-N: N is an integer equal to or greater than 2), respectively, and a management device 30. The on-board systems 20 here include, for example, a camera 11, a position sensor 12, hazard lights 13, an HMI device 14, and an information processing device 15, similar to the configuration shown in FIG. 2. In the second embodiment, one or more processors 17 included in the information processing device 15 correspond to an example of "one or more first processors" according to the present disclosure, and will hereinafter be simply referred to as "first processor 17."

[0039] The management device 30 includes a communication device 31, one or more second processors 32 (hereinafter simply referred to as "second processors 32"), and one or more storage devices 33 (hereinafter simply referred to as "storage devices 33"). The communication device 31 communicates with the outside of the management device 30 (including multiple vehicles 10 and notification destinations 100) via a communication network. The management device 30 is a management server (for example, a cloud server) that manages abnormal events received from multiple vehicles 10.

[0040] In the second embodiment, the first processor 17 executes various processes including processes related to the management (detection) of an abnormal event, and the second processor 32 executes various processes including processes related to the management (reporting) of an abnormal event.

[0041] Examples of the second processor 32 include a CPU, a GPU, an ASIC, and an FPGA. The second processor 32 can also be referred to as a "circuitry" or a "processing circuitry." A "circuitry" is hardware that is programmed to realize a described function or that executes a function. The storage device 33 stores various information. Examples of the storage device 33 include a volatile memory, a non-volatile memory, an HDD, and an SSD. The second processor 32 reads various information from the storage device 33 and stores various information in the storage device 33. The functions of the management device 30 may be realized by cooperation between the second processor 32, which executes a computer program, and the storage device 33. The computer program is stored in the storage device 33. Alternatively, the computer program may be recorded on a computer-readable recording medium or provided via a network. The storage device 33 also stores information such as abnormal event information and notification destination information. Note that in the second embodiment, the storage device 18 of the vehicle 10 does not necessarily store notification destination information.

[0042] 2-2. Management of abnormal events According to the anomaly management system 1 of the first embodiment described above, the information processing device 15 of the vehicle 10 that detects an abnormal event transmits abnormal event information to the notification destination 100. As a result, abnormal event information regarding the same abnormal event may be transmitted to the notification destination 100 from multiple vehicles 10 equipped with the anomaly management system 1. In other words, there is a possibility that a large number of notifications regarding the same abnormal event may flood in. To prevent an unnecessary increase in notifications, the anomaly management system 2 of the second embodiment executes the following "selective notification processing" as the "report processing".

[0043] 5 is a flowchart showing an example of a process flow related to abnormal event management according to the second embodiment. The process of this flowchart is executed by cooperation between the information processing device 15 (first processor 17) and the management device 30 (second processor 32). This process differs from the process shown in FIG. 3 in the following points.

[0044] Specifically, the information processing device 15 (first processor 17) of the vehicle 10 executes the image acquisition process and the abnormal event recognition process (steps S102 and S104) in the same manner as in the process shown in Fig. 3. In Fig. 5, if an abnormal event is subsequently detected (step S106; Yes), the process proceeds to step S200.

[0045] In step S200, a process is executed to store the video I taken before and after the occurrence of the abnormal event in the management device (cloud) 30. Specifically, the information processing device 15 transmits the video I taken before and after the occurrence of the abnormal event, which is stored in the storage device 18, to the management device 30. The management device 30 receives the video I and stores the received video I in the storage device 33. In an example in which the video I is included in the abnormal event information transmitted to the notification destination 100, the process of step S200 corresponds to part of the process of transmitting and receiving the abnormal event information.

[0046] If the AI ​​automatic reporting mode is ON in step S108 following step S200, the process proceeds to step S202. In step S202, the information processing device 15 transmits abnormal event information regarding the abnormal event detected (recognized) in step S106 to the management device 30. More specifically, the transmitted abnormal event information includes at least information indicating the content of the abnormal event and location information. The transmitted abnormal event information may also include time information of the abnormal event. Thereafter, the process proceeds to step S204.

[0047] In step S204, the management device 30 (second processor 32) receives the abnormal event information from the vehicle 10 and stores the received abnormal event information in the storage device 33. Then, the management device 30 executes a report destination determination process based on the received abnormal event information. Thereafter, the process proceeds to step S206.

[0048] The processes of steps S206 and S208 correspond to an example of the above-mentioned "selective reporting process." In step S206, the management device 30 determines whether or not multiple pieces of abnormal event information related to the same abnormal event have been received (determination process). This determination process may be performed, for example, based on the location information and time information of the abnormal event included in the abnormal event information. More specifically, the management device 30 may determine that multiple abnormal events that fall within a predetermined distance range and a predetermined time range are the same.

[0049] If multiple pieces of abnormal event information relating to the same abnormal event have not been received (step S206; No), the management device 30 executes the same notification process as the process shown in Fig. 3 (step S112). On the other hand, if multiple pieces of abnormal event information have been received (step S206; Yes), the process proceeds to step S208.

[0050] In step S208, the management device 30 selectively transmits some of the abnormal event information relating to the same abnormal event to the notification destination 100. The selective notification process will be described in detail as follows.

[0051] 2-3. Details of selective reporting process Next, first to fifth examples of the selective reporting process will be explained in order.

[0052] 2-3-1. First example In a first example, when multiple pieces of abnormal event information relating to the same abnormal event include "first abnormal event information that arrived at the management device 30 at a first time" and "second abnormal event information that arrived at the management device 30 at a second time that is later than the first time," the management device 30 issues a report as follows. That is, the management device 30 transmits the first abnormal event information to the report destination 100, but refrains from transmitting (i.e., does not transmit) the second abnormal event information to the report destination 100. In this way, in the first example, the management device 30 is configured not to report the later piece of abnormal event information relating to the same abnormal event.

[0053] More specifically, the first abnormal event information transmitted to the reporting destination 100 may be, for example, the first abnormal event information acquired among multiple abnormal event information related to the same abnormal event. That is, the management device 30 may be configured to report only the earliest abnormal event information among the multiple abnormal event information, and not report any subsequent abnormal event information. Alternatively, the first abnormal event information may be two or more fixed pieces of abnormal event information that arrived at the management device 30 at two or more fixed first times prior to the second time. That is, the management device 30 may be configured to report two or more fixed pieces of abnormal event information, and not report any subsequent abnormal event information.

[0054] Alternatively, the management device 30 may control the transmission of multiple pieces of abnormal event information relating to the same abnormal event to the notification destination 100 so that the frequency of notifications decreases over time.

[0055] 2-3-2. Second example In a second example, when multiple pieces of abnormal event information related to the same abnormal event include "first abnormal event information that arrived at the management device 30 at a first time and includes a first video as the video I" and "second abnormal event information that arrived at the management device 30 at a second time that is later than the first time and includes a second video as the video I," the management device 30 issues a report as follows. That is, the management device 30 transmits the first abnormal event information to the report destination 100. The management device 30 also calculates the degree of difference between the first video and the second video based on the first video and the second video. If the calculated degree of difference exceeds a threshold, the management device 30 transmits the second abnormal event information to the report destination 100. On the other hand, if the calculated degree of difference is equal to or less than the threshold, the management device 30 refrains from transmitting the second abnormal event information to the report destination 100. In this way, in the second example, the management device 30 is configured to issue a follow-up report when there is a difference in the content or approach of the abnormal event information, even if the abnormal event information is related to the same abnormal event.

[0056] More specifically, in the second example, the management device 30 recognizes the first shooting direction, which is the shooting direction of the first video, based on the first abnormal event information, and recognizes the second shooting direction, which is the shooting direction of the second video, based on the second abnormal event information. The management device 30 then increases the degree of difference as the difference between the first and second shooting directions increases. Therefore, according to the second example, even if the same abnormal event occurs, if the shooting directions of the first and second videos are significantly different, the second abnormal event information is transmitted to the notification destination 100. On the other hand, even if the same abnormal event occurs, if the difference in shooting direction is small, the second abnormal event information is not transmitted.

[0057] Additionally, the shooting direction of the video I can be estimated, for example, by the following method. That is, the abnormal event information may include information indicating the position and orientation of the vehicle 10 equipped with the camera 11 that captured the video I. Then, the management device 30 may estimate the shooting direction of the video I based on the position and orientation of the vehicle 10.

[0058] 2-3-3. Third example The third example is the same as the second example in that it uses the "degree of difference" to determine whether to send the second abnormal event information to the notification destination 100. In addition, in the third example, the management device 30 calculates a first severity level, which is the severity level of the same abnormal event, based on the first abnormal event information, and calculates a second severity level, which is the severity level of the same abnormal event, based on the second abnormal event information. The management device 30 then increases the degree of difference as the increase from the first severity level to the second severity level increases. Therefore, according to the third example, even if the abnormal event is the same, if the severity level increases significantly, the second abnormal event information is sent to the notification destination 100. On the other hand, even if the abnormal event is the same, if the severity level does not increase, the second abnormal event information is not sent.

[0059] Additionally, the severity level indicates the scale, degree, or intensity of the abnormal event. For example, if the abnormal event is a building fire, the severity level increases as the fire spreads. The severity level may be calculated by the information processing device 15 of the vehicle 10, for example, based on the content of the abnormal event recognized by the abnormal event recognition process. The calculated severity level may then be included in the abnormal event information and provided to the management device 30. Alternatively, the severity level may be calculated by the management device 30 using the video I included in the abnormal event information and a machine learning model.

[0060] 2-3-4. Fourth Example In a fourth example, the management device 30 determines (confirms) whether a response team for a certain abnormal event has already arrived at the site. This determination can be made, for example, based on the result of communication with the notification destination 100 to which the abnormal event information for the abnormal event has been transmitted. If the response team has already arrived at the site, when the management device 30 receives new abnormal event information related to the same abnormal event as the abnormal event, the management device 30 refrains from transmitting the new abnormal event information to the notification destination 100 (i.e., a follow-up report).

[0061] 2-3-5. Fifth Example In a fifth example, when the management device 30 receives information from the notification destination 100 indicating that no further reporting is required for an abnormal event that has already been reported, the management device 30 is configured not to issue any further reporting for the same abnormal event.

[0062] Furthermore, when a response to the transmission of abnormal event information is received from the notification destination 100, the management device 30 may feed back the content of the response to the abnormal event recognition process in order to reduce the number of future reports. Specifically, in an example where the response includes the above-mentioned "information indicating that a follow-up report is not required for the reported abnormal event," the management device 30 may request the "feedback target vehicle 10X that has not yet transmitted abnormal event information related to the abnormal event to the management device 30" to perform the abnormal event recognition process so as not to recognize (detect) the abnormal event. Furthermore, in an example where the response includes "information indicating that the content of the report (i.e., the content of the abnormal event information transmitted to the notification destination 100) is inappropriate," the management device 30 may request the feedback target vehicle 10X to perform the abnormal event recognition process so as not to recognize an abnormal event with the content determined by the notification destination 100 to be inappropriate. Furthermore, in an example in which the response includes "information indicating that the shooting direction or severity is inappropriate for an abnormal event that has been reported multiple times," the management device 30 may request the feedback target vehicle 10X to perform abnormal event recognition processing so as not to recognize an abnormal event with the shooting direction or severity that has been determined to be inappropriate by the report destination 100. Note that the management device 30 may, for example, identify one or more vehicles 10 that are present within a predetermined distance range from the location of the abnormal event related to the response as the feedback target vehicle 10X.

[0063] 2-4.Effects The abnormality management system 2 according to the second embodiment described above also provides the same effects as those described in Sections 1-3 for the abnormality management system 1 according to the first embodiment.

[0064] Furthermore, according to the selective reporting process executed in the abnormality management system 2, when there is a plurality of abnormal event information related to the same abnormal event, only some of the plurality of abnormal event information is selectively reported, rather than all of the plurality of abnormal event information. This prevents an unnecessarily large number of reports from being sent to the report destination 100. This contributes to reducing the processing load and preventing confusion. [Explanation of symbols]

[0065] 1 Abnormality management system, 10 Vehicle, 11 Camera, 12 Position sensor, 15 Information processing device, 16, 31 Communication device, 17 Processor (first processor), 18, 33 Storage device, 20 In-vehicle system, 30 Management device, 32 Second processor, 100 Report destination

Claims

1. An image acquisition process for acquiring an image captured by a camera mounted on a moving object; An abnormal event recognition process that automatically recognizes an abnormal event captured in the video and the content of the abnormal event by using a machine learning model; a notification destination determination process that automatically determines a notification destination according to the content of the abnormal event; a reporting process of automatically transmitting abnormal event information to the reporting destination, the abnormal event information including at least information indicating the content of the abnormal event and location information indicating a location of the abnormal event; one or more processors configured to execute Anomaly management system.

2. The abnormality management system according to claim 1, The abnormal event information includes the video captured during a target period that includes at least the timing when the abnormal event was recognized. Anomaly management system.

3. The abnormality management system according to claim 1, the one or more processors: one or more first processors included in a plurality of information processing devices mounted on a plurality of moving bodies, respectively; one or more second processors included in a management device capable of communicating with the plurality of information processing devices; Including, The one or more first processors of each of the plurality of moving bodies Execute the video acquisition process and the abnormal event recognition process; The abnormal event information is transmitted to the management device. It is configured as follows: The one or more second processors receiving the abnormal event information from each of the plurality of moving objects; Execute the report destination determination process, The notification process includes a determination process for determining whether the management device has received a plurality of pieces of abnormal event information related to the same abnormal event, and a process for selectively transmitting some of the plurality of pieces of abnormal event information related to the same abnormal event to the notification destination when the management device has received the plurality of pieces of abnormal event information. It was configured as Anomaly management system.

4. The abnormality management system according to claim 3, the abnormal event information includes time information indicating a time when the abnormal event was recognized, In the determination process, the one or more second processors determine whether the management device has received the plurality of pieces of abnormal event information related to the same abnormal event, based on the location information and the time information. Anomaly management system.

5. The abnormality management system according to claim 3, the plurality of abnormal event information relating to the same abnormal event, First abnormal event information that arrives at the management device at a first time; second abnormal event information that arrives at the management device at a second time that is later than the first time; Including, The one or more second processors are configured to transmit the first abnormal event information to the notification destination while refraining from transmitting the second abnormal event information to the notification destination. Anomaly management system.

6. The abnormality management system according to claim 3, the abnormal event information includes the video captured during a target period that includes at least a timing when the abnormal event was recognized; the plurality of abnormal event information relating to the same abnormal event, First abnormal event information that arrives at the management device at a first time and includes a first video as the video; second abnormal event information that arrives at the management device at a second time that is later than the first time and includes a second video as the video; Including, The one or more second processors Sending the first abnormal event information to the notification destination; calculating a degree of difference between the first image and the second image based on the first image and the second image; If the degree of difference exceeds a threshold, the second abnormal event information is sent to the notification destination; If the degree of difference is equal to or less than the threshold, the second abnormal event information is not transmitted to the notification destination. It was configured as Anomaly management system.

Images