Control device, control method, and program

Abstract

The present disclosure provides: a control device that can achieve both improvement in accuracy of remote monitoring and adjustment of a data communication amount required for remote monitoring; a monitoring system; a control method; and non-transitory computer-readable medium. A control device (10) according to one aspect of the present embodiment comprises: a detection means (11) for, on the basis of environment information transmitted from a plurality of information acquisition devices that acquire environment information, detecting whether or not an attention-requiring event, which is an event requiring attention, has occurred in the periphery of any one of the information acquisition devices; and an instruction means (12) for, when the detection means (11) has detected the occurrence of the attention-requiring event, outputting an instruction to a first information acquisition device, which has transmitted environment information not related to the detection of the occurrence of the attention-requiring event, for changing the quality of environment information to be transmitted by the first information acquisition device.

Description

[Technical Field]

[0001] The present invention relates to a control device, a monitoring system, a control method, and a non-transitory computer-readable medium. [Background technology]

[0002] BACKGROUND ART Remote monitoring technologies have been advancing in recent years in fields such as autonomous driving of vehicles and management of factory operating conditions.

[0003] For example, Patent Document 1 describes a remote control device that receives internal video of a vehicle, predicts the risk of an accident occurring inside the vehicle based on the internal video and situational information indicating the vehicle's condition, and adjusts the quality of the internal video based on the prediction results.

[0004] Patent document 2 also describes that an external device determines whether or not the driver is experiencing a specified symptom based on the internal image of the vehicle, and if the specified symptom is occurring, the internal image is made high quality to clearly record the driver's condition.

[0005] Furthermore, Patent Document 3 describes that the pitch of a variable-pitch camera attached to an autonomous vehicle is changed as the autonomous vehicle moves to maximize the clarity and / or resolution of the camera's image. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2021 / 199349 [Patent Document 2] Japanese Patent Publication No. 2021-002283 [Patent Document 3] Special Publication No. 2019-512913 Summary of the Invention [Problem to be solved by the invention]

[0007] If the quality of the data, such as the video of the monitored object, is low, the accuracy of the remote monitoring will be low. However, if the data, such as the video, is high quality, there is a problem that the amount of data communication required for remote monitoring will be large.

[0008] An object of the present disclosure is to provide a control device, a monitoring system, a control method, and a non-transitory computer-readable medium that can simultaneously improve the accuracy of remote monitoring and adjust the amount of data communication required for remote monitoring. [Means for solving the problem]

[0009] A control device according to one aspect of this embodiment includes a detection means for detecting whether an attention event, which is an event requiring attention, has occurred in the vicinity of one of the information acquisition devices based on the environmental information transmitted from the multiple information acquisition devices that acquire environmental information, and an instruction means for outputting an instruction to a first information acquisition device that transmitted the environmental information not related to the detection of the occurrence of the attention event when the detection means detects the occurrence of the attention event, to change the quality of the environmental information transmitted by the first information acquisition device.

[0010] A monitoring system according to one aspect of this embodiment includes a plurality of information acquisition devices that acquire environmental information and transmit the acquired environmental information, and a control device that controls the transmission of the environmental information from the plurality of information acquisition devices based on the environmental information transmitted from the plurality of information acquisition devices. The control device includes a detection means that detects whether an attention event, which is an event that requires attention, has occurred in the vicinity of any of the information acquisition devices based on the environmental information transmitted from the plurality of information acquisition devices, and an instruction means that, when the detection means detects the occurrence of the attention event, outputs an instruction to the first information acquisition device that transmitted the environmental information that is not related to the detection of the occurrence of the attention event, to change the quality of the environmental information transmitted by the first information acquisition device.

[0011] A control method of one aspect of this embodiment is executed by a control device, which detects whether an attention event, an event that requires attention, has occurred in the vicinity of any of the information acquisition devices based on environmental information transmitted from the information acquisition devices that acquire environmental information, and when the occurrence of the attention event is detected, outputs an instruction to a first information acquisition device that transmitted environmental information that is not related to the detection of the occurrence of the attention event to change the quality of the environmental information transmitted by the first information acquisition device.

[0012] A non-transitory computer-readable medium according to one aspect of the present embodiment causes a computer to execute the following: based on environmental information transmitted from a plurality of information acquisition devices that acquire environmental information, detect whether an attention event, which is an event requiring attention, has occurred in the vicinity of any of the information acquisition devices; and, when the occurrence of the attention event is detected, output an instruction to a first information acquisition device that transmitted the environmental information unrelated to the detection of the occurrence of the attention event, to change the quality of the environmental information transmitted by the first information acquisition device. The program is stored It is something. [Effects of the Invention]

[0013] According to the present disclosure, it is possible to provide a control device, a monitoring system, a control method, and a non-transitory computer-readable medium that can simultaneously improve the accuracy of remote monitoring and adjust the amount of data communication required for remote monitoring. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing an example of a control device according to a first embodiment; [Figure 2] 4 is a flowchart illustrating an example of a process executed by the control device according to the first embodiment. [Figure 3] 1 is a block diagram illustrating an example of a monitoring system according to a first embodiment. [Figure 4] FIG. 10 is a block diagram illustrating an example of a monitoring system according to a second embodiment. [Figure 5]FIG. 10 is a block diagram showing an example of a field system according to a second embodiment. [Figure 6] FIG. 10 is a block diagram illustrating an example of a cloud according to a second embodiment. [Figure 7] FIG. 10 is a block diagram showing an example of a display system according to a second embodiment. [Figure 8] 10 is a flowchart illustrating an example of a process executed by the control system according to the second embodiment. [Figure 9A] This table shows an example of the quality of video when no alert has occurred (for AI) and when an alert has occurred (for monitors). [Figure 9B] This is a table showing examples of the quality of video when no alert has occurred (for AI) and the quality of video when an alert has occurred (for AI detailed analysis and for monitors). [Figure 10] This is a table showing examples of environmental information quality when no alert has occurred (for AI) and quality when an alert has occurred (for monitors). [Figure 11] This is a table showing an example in which the bit rate of the video sent by the alert determination unit is increased for the monitor and then decreased again to that for AI. [Figure 12] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an apparatus according to each embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] The present disclosure will be described below with reference to the drawings. Note that the following description and drawings have been omitted or simplified as appropriate for clarity of explanation. Furthermore, in this disclosure, unless otherwise specified, when multiple items are defined as "at least one of them," the definition may mean any one item or any multiple items (including all items).

[0016] Embodiment 1 (Embodiment 1A) In the embodiment 1A, a control device that detects the occurrence of a caution event, which is an event that requires attention, will be described.

[0017] 1 is a block diagram showing an example of a control device. The control device 10 includes a detection unit 11 and an instruction unit 12. Each unit (each means) of the control device 10 is controlled by a control unit (controller) (not shown) reading a program. Each component will be described below.

[0018] The detection unit 11 detects whether a warning event, which is an event requiring attention, has occurred in the vicinity of one of the information acquisition devices based on environmental information transmitted from the multiple information acquisition devices that acquire the environmental information. Here, an "information acquisition device" refers to a device capable of acquiring any information related to the environment surrounding the device, such as a camera that captures still or video images, a sensor capable of detecting its own position, such as a satellite positioning system (e.g., a GPS (Global Positioning System)) sensor, a microphone, a CAN (Controller Area Network) sensor mounted on a vehicle, or a vibration detection sensor. "Environmental information" refers to information acquired by the information acquisition device, such as video information, location information, sound information, vehicle status information, or vibration information. The environmental information acquired by the detection unit 11 from the multiple information acquisition devices may be the same type of information or different types of information. For example, all of the multiple information acquisition devices may transmit video information. In another example, one information acquisition device may transmit video information, while another information acquisition device may transmit sound information.

[0019] Furthermore, the multiple information acquisition devices may be installed on the same detection object (for example, a vehicle) or may be installed in different locations. The information acquisition devices may be installed at fixed locations or may be installed on a movable object such as a vehicle. Details of such variations will be described later in the second and subsequent embodiments.

[0020] The detection unit 11 determines, as an event requiring attention, for example, whether an accident has occurred or the possibility of an accident occurring in the near future is greater than or equal to a predetermined threshold. Specifically, when an information acquisition device is mounted on a vehicle and captures video of the vehicle's traveling direction, the detection unit 11 determines, based on the video, whether the vehicle has come into contact with an object (e.g., another vehicle) in the traveling direction or will come into contact with such an object within a predetermined time. The determination may be performed, for example, by an AI (Artificial Intelligence) model using machine learning, or by any analytical method such as image analysis. At this time, at least one of factors such as the object's properties (e.g., size, shape, and material), the distance between the information acquisition device and the object, the speed of the information acquisition device and the object, the positional relationship between the information acquisition device and the object, and their respective traveling directions may be detected from environmental information acquired by one or more information acquisition devices. The detected factors can be used by the AI ​​model or the like to determine whether an event requiring attention is present.

[0021] As another example, the detection unit 11 may detect rapid acceleration or deceleration of a vehicle equipped with an information acquisition device that is equal to or greater than a predetermined threshold based on position information or vehicle state information, or may detect a collision or acceleration or deceleration of a vehicle equipped with an information acquisition device based on sound information or vibration information. Based on such detection results, the detection unit 11 detects whether or not a cautionary event has occurred.

[0022] When the detection unit 11 detects the occurrence of a warning event, the instruction unit 12 outputs an instruction to the first information acquisition device, which has transmitted environmental information unrelated to the detection of the warning event, to change the quality of the environmental information transmitted by the first information acquisition device. The term "first information acquisition device" refers to a device that acquires environmental information unrelated to the detection of the warning event by the detection unit 11. For example, the first information acquisition device may be one from which the detection unit 11 cannot detect the occurrence of a warning event from the acquired environmental information. For example, if a vehicle equipped with multiple information acquisition devices is parked in front of another vehicle and the detection unit 11 detects the occurrence of a warning event based on video from an information acquisition device capturing the front view, the information acquisition device capturing the view other than the front of the vehicle may be considered the "first information acquisition device." Alternatively, if multiple information acquisition devices are fixed outdoors and the occurrence of a warning event is detected based on video from one information acquisition device, the information acquisition device that does not capture the area where the warning event is occurring may also be considered the "first information acquisition device." The "first information acquisition device" may be one or more information acquisition devices.

[0023] Here, the instruction unit 12 may cause the first information acquisition device to change the quality of the environmental information transmitted by the device to either increase or decrease it. Increasing the quality means, for example, if the environmental information is video, increasing the image quality of the video. Increasing the image quality may mean, for example, increasing at least one of the frame rate, bit rate, and resolution of the video. Increasing the image quality means the opposite when decreasing the image quality. Increasing the quality of the environmental information may also mean increasing the frequency of transmission of the environmental information if the environmental information is any of position information, sound information, vehicle status information, and vibration information. Increasing the quality of the environmental information means the opposite when decreasing the quality of the environmental information.

[0024] By changing the quality of the environmental information in this way, the amount of data of the environmental information to be transmitted changes. When the quality of the environmental information is high, the amount of data increases, while when the quality of the environmental information is low, the amount of data decreases. When the quality of the environmental information is high, the amount of data increases, which increases the cost of communication between the control device 10 and the information acquisition device, but it becomes possible to improve the accuracy of remote monitoring. On the other hand, when the quality of the environmental information is low, the accuracy of remote monitoring decreases, but the amount of data decreases, which makes it possible to reduce the cost of communication.

[0025] 2 is a flowchart showing an example of a typical process of the control device 10, and this flowchart explains the process of the control method executed by the control device 10. Details of the process executed by each unit below are as described above.

[0026] The detection unit 11 detects whether or not a cautionary event, which is an event that requires attention, has occurred around any of the information acquisition devices based on the environmental information transmitted from the multiple information acquisition devices (step S11; detection step). If no cautionary event has occurred (No in step S11), the detection unit 11 intermittently repeats the determination in step S11.

[0027] When the detection unit 11 detects that a caution event has occurred (Yes in step S11), the instruction unit 12 outputs an instruction to the first information acquisition device that transmitted environmental information not related to the detection of the occurrence of the caution event to change the quality of the environmental information to be transmitted (step S12; instruction step).

[0028] In this way, when the control device 10 detects the occurrence of a cautionary event, it can cause the first information acquisition device that transmitted environmental information unrelated to the detection of the occurrence of the cautionary event to change the quality of the environmental information to be transmitted. This makes it possible to improve the accuracy of remote monitoring while adjusting the amount of data communication required for remote monitoring.

[0029] The control device 10 may have a centralized configuration consisting of a single computer, or may have a distributed configuration in which multiple computers share and execute the processing of the detection unit 11 and the instruction unit 12. In a distributed configuration, multiple devices may be connected via a communication network such as a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet.

[0030] (Embodiment 1B) Next, in embodiment 1B, a monitoring system will be described.

[0031] 3 is a block diagram showing an example of a monitoring system. The monitoring system S1 includes a plurality of information acquisition devices 21A, 21B, etc., and a control device 22. Each information acquisition device 21 acquires environmental information about its surroundings and transmits the acquired environmental information to the control device 22. Details of the information acquisition device 21 are as described in embodiment 1A. The control device 22 also controls each information acquisition device 21. from Based on the transmitted environmental information, the transmission of environmental information from the plurality of information acquisition devices 21 is controlled.

[0032] The control device 22 corresponds to the control device 10 in embodiment 1A. In detail, the control device 22 includes a detection unit 11 that detects whether a cautionary event has occurred in the vicinity of any of the information acquisition devices 21 based on environmental information transmitted from the information acquisition devices 21, and an instruction unit 12 that outputs an instruction to the first information acquisition device 21, when the detection unit 11 detects the occurrence of a cautionary event, to change the quality of environmental information transmitted by the information acquisition device 21 (first information acquisition device) that transmitted environmental information not related to the detection of the occurrence of the cautionary event. The detailed description of each unit is also as described in embodiment 1A.

[0033] As described above, in the monitoring system, when the occurrence of a cautionary event is detected, the quality of the environmental information transmitted by the first information acquisition device that transmitted environmental information unrelated to the detection of the occurrence of the cautionary event can be changed. This makes it possible to improve the accuracy of remote monitoring while adjusting the amount of data communication required for remote monitoring.

[0034] Embodiment 2 Specific embodiments will be described below with reference to the drawings.

[0035] 4 is a block diagram showing an example of a monitoring system. The monitoring system S2 is a system for remotely monitoring vehicles and the like, and includes a field system 110, a cloud 120, and a display system 130. The field system 110 and the cloud 120, and the cloud 120 and the display system 130, are connected by a wireless line such as LTE (Long Term Evolution), 5G (Fifth Generation), or a wireless LAN (Local Area Network) or a wired line, respectively, and data is transmitted and received via the line. Details of each part will be described below.

[0036] FIG. 5 is a block diagram showing an example of the on-site system 110. The on-site system 110 includes cameras 111A-111D, a GPS sensor 112, a microphone 113, a CAN sensor 114, and infrastructure cameras 115A-115C. These devices transmit environmental information, such as images seen from the vehicle, location information, sound information, and images captured outdoors, to the control system 200 via a wireless network. Here, the cameras 111A-111D, the GPS sensor 112, the microphone 113, and the CAN sensor 114 are mounted on vehicle A and function as control devices for vehicle A. Here, an autonomously driven automobile is assumed as the vehicle, but the present invention can also be applied to any vehicle, such as a non-autonomous vehicle, a motorcycle, a bicycle, or a railcar. Furthermore, the present invention can also be applied to any moving object other than a vehicle, such as an aircraft or a ship. The cameras 111A-111D and the infrastructure cameras 115A-115C are also collectively referred to as cameras 111 and infrastructure cameras 115, respectively.

[0037] Cameras 111A, 111B, 111C, and 111D are installed outside vehicle A and capture video images of the front, rear, right side, and left side of vehicle A, respectively. Note that cameras may also be installed inside vehicle A to capture video images of the interior of the vehicle. GPS sensor 112 acquires location information of vehicle A using a GPS function. Microphone 113 acquires sound information around vehicle A. CAN sensor 114 is connected to the control system of vehicle A and acquires vehicle status information indicating the running status of vehicle A, the presence or absence of any malfunctions, etc. These devices transmit the acquired environmental information to cloud 120, and upon receiving a control signal related to the transmission of environmental information from cloud 120, change the method of transmitting the environmental information based on the control signal. This will be described in more detail below.

[0038] Furthermore, not only vehicle A but also vehicles B, C, etc. are equipped with cameras 111 to CAN sensors 114, similar to vehicle A. These devices also transmit the environmental information acquired by each device to the cloud 120, and when a control signal is received from the cloud 120, they change the method of transmitting the environmental information based on the control signal.

[0039] Infrastructure camera 115 is a fixed-position camera installed in different outdoor locations, each capable of capturing a different area. As an example, infrastructure camera 115 is installed in locations where vehicles can travel, such as roads, traffic lights, roadside devices, intersections, and railroad crossings (for example, locations where an accident is likely to occur), and is capable of capturing images of the surrounding area. Infrastructure camera 115 transmits the captured video to cloud 120, and upon receiving a control signal related to the transmission of environmental information from cloud 120, changes the method of transmitting environmental information based on the control signal.

[0040] FIG. 6 is a block diagram showing an example of the cloud 120. The cloud 120 relays environmental information from the on-site system 110 and distributes it to the display system 130. The cloud 120 also includes a control system 200 that has a function of issuing an alert indicating that an alert event (corresponding to the caution event in the first embodiment) has occurred in the vehicle. The control system 200 corresponds to the control device 10 in the first embodiment and includes an information receiving unit 201, a recognition unit 202, an alert determination unit 203, a monitoring information DB (Database) 204, and an information distribution unit 205. The control system 200 may have a centralized configuration consisting of a single computer, or a distributed configuration in which multiple computers share and execute the processing of the information receiving unit 201 to the information distribution unit 205. In the distributed configuration, multiple devices may be connected via a communication network such as a LAN, a WAN, or the Internet. Similarly, the entire cloud 120 may have a centralized configuration consisting of a single computer, or a distributed configuration consisting of multiple computers. The information receiving unit 201 to the information distributing unit 205 will be described in detail below.

[0041] The information receiving unit 201 receives environmental information transmitted from each device provided in the on-site system 110. The information receiving unit 201 outputs the received environmental information to each other unit of the control system 200.

[0042] The recognition unit 202 (recognition engine) identifies the area of ​​an object captured in the video captured by each camera 111 and the infrastructure camera 115 based on the environmental information acquired from the information receiving unit 201, and specifies (recognizes) the object in that area. For example, the recognition unit 202 can recognize whether an object captured in the video is a person, a vehicle, a utility pole, or another obstacle. The recognition unit 202 also estimates the posture of an object, such as a person, captured in the video by identifying its feature points. Here, the recognition unit 202 can perform recognition processing for vehicle A using at least one of the video captured by the camera 111 of vehicle A and the video captured by the infrastructure camera 115 externally of vehicle A.

[0043] Furthermore, the recognition unit 202 can estimate the distance between an object shown in the video and the camera 111 or infrastructure camera 115 that captured the video, based on the video. Furthermore, the recognition unit 202 can also recognize the driving lane of the road on which the vehicle equipped with the camera 111 that captured the video is traveling.

[0044] The recognition unit 202 performs the above processing and outputs the processing result (object recognition result) to the monitoring information DB 204. In this example, the recognition unit 202 uses an AI model that has been machine-learned in advance to identify objects in the video, estimate their posture, estimate their distance, and recognize the driving lane. However, other analysis methods may be used for these processes. Furthermore, the above recognition processing may use not only the video from the camera 111 and the infrastructure camera 115, but also sound information, position information, vehicle status information, and the like.

[0045] The alert determination unit 203 corresponds to the detection unit 11 and the instruction unit 12 according to the first embodiment, and upon acquiring the object recognition result and the environmental information, determines whether or not an alert event has occurred in at least one of the vehicles A, B, C, etc. based on the data. If it is determined that an alert event has occurred in any of the vehicles, the alert determination unit 203 outputs alert information about that vehicle to the monitoring information DB 204 and the information distribution unit 205. In the example shown below, the alert determination unit 203 determines whether or not an alert has occurred using an AI model that has been machine-learned in advance, but other analysis methods may also be used for the determination.

[0046] For example, assume that the recognition unit 202 recognizes, based on the image captured by the camera 111A of vehicle A, that another vehicle is parked within a predetermined distance from vehicle A ahead in the direction of travel of vehicle A. Based on this recognition result, the alert determination unit 203 inputs environmental information into an AI model for determination to determine whether vehicle A is approaching a parked vehicle. If vehicle A is approaching, the alert determination unit 203 determines that an alert event exists in which vehicle A may collide with the other vehicle. The alert flag for vehicle A is then set from "No" to "Yes," and the alert information is output to the monitoring information DB 204 and the information distribution unit 205. Similar to the recognition process, this determination can be performed using at least one of the image captured by the camera 111 of vehicle A and the image captured by the infrastructure camera 115 externally of vehicle A.

[0047] As another example, suppose that the recognition unit 202 recognizes, based on video footage from a camera installed inside vehicle A, that a passenger standing inside vehicle A, which is a bus, has lost his / her balance and fallen. Based on this recognition result, the alert determination unit 203 inputs environmental information into an AI model for determination, and when it determines that vehicle A has decelerated at a rate equal to or greater than a predetermined acceleration, determines that an alert event has occurred, that vehicle A may have suddenly stopped. Then, the alert flag for vehicle A is set from "No" to "Yes," and the alert information is output to the monitoring information DB 204 and the information distribution unit 205.

[0048] The above determination process may use not only the images from the camera 111 and the infrastructure camera 115, but also sound information, position information, vehicle state information, etc. For example, it may be determined that the vehicle A has decelerated at a rate equal to or greater than a predetermined acceleration based on information about the traveling state of the vehicle A acquired by the CAN sensor 114.

[0049] Furthermore, the alert determination unit 203 may identify an area in the camera image (a part of the image) related to the occurrence of an alert event as an area of ​​interest for monitoring, and include information about the area of ​​interest in the alert information. In the above example, when vehicle A is approaching a stopped vehicle, the alert determination unit 203 identifies the area in the image of camera 111A in which the vehicle ahead is captured as an area of ​​interest. In this case, buildings around the road on which vehicle A is traveling are not important areas for monitoring, and therefore may not be included in the area of ​​interest. Furthermore, when a passenger standing inside vehicle A falls, the alert determination unit 203 identifies the area in the image of the camera installed inside vehicle A in which the fallen passenger is captured as an area of ​​interest.

[0050] Furthermore, when an alert event occurs in any vehicle, the alert determination unit 203 can also output a control signal to any device in the on-site system 110 to control it to change the quality of the environmental information to be transmitted. This will be described in detail later.

[0051] The monitoring information DB 204 stores environmental information received by the information receiving unit 201 and alert information determined by the alert determination unit 203. The information stored in this monitoring information DB 204 is transmitted to the display system 130 by the information distribution unit 205 as necessary. The monitoring information DB 204 may also store information on the camera 111, GPS sensor 112, microphone 113, and CAN sensor 114 mounted on each vehicle, map information of the area managed by the control system 200 (for example, map information of the area where each infrastructure camera 115 is installed), and location information of the infrastructure camera 115. The monitoring information DB 204 may also store AI models used by the recognition unit 202 and the alert determination unit 203, respectively.

[0052] The information distribution unit 205 distributes the environmental information received by the information receiving unit 201 and the alert information determined by the alert determination unit 203 to the display system 130.

[0053] 7 is a block diagram showing an example of the display system 130. The display system 130 receives environmental information from the information distribution unit 205 and presents the information to the monitor. The display system 130 includes an information receiving unit 131, a presentation switching unit 132, an integrated display unit 133, and detailed information presentation units 134A, 134B, etc. The detailed information presentation units 134A and 134B are collectively referred to as the detailed information presentation unit 134. Each unit will be described in detail below.

[0054] The information receiving unit 131 receives environmental information and alert information distributed from the information distribution unit 205. Then, the information receiving unit 131 outputs the received information to the presentation switching unit 132. Of the environmental information distributed from the information distribution unit 205, at least the video information from the camera 111 is output to the integrated display unit 133. However, environmental information other than video, such as position information from the GPS sensor 112, may also be output to the integrated display unit 133.

[0055] The presentation switching unit 132 analyzes the acquired alert information, and if there is a vehicle for which the alert is set to "Yes", switches the presentation of information so that environmental information distributed from a device mounted on the vehicle is output to and presented by the detailed information presentation unit 134. In detail, the image from the camera 111 mounted on the vehicle for which the alert is set to "Yes", the location information detected by the GPS sensor 112, the sound information acquired by the microphone 113, and the vehicle status information acquired by the CAN sensor 114 are output to the detailed information presentation unit 134. Note that if there are multiple vehicles for which the alert is set to "Yes", the environmental information of different vehicles is output to different detailed information presentation units 134 so that the vehicles for which the alert is set to "Yes" correspond one-to-one to the detailed information presentation units 134.

[0056] In addition, the presentation switching unit 132 may display on the display unit of the detailed information presentation unit 134 not only the image from the camera 111 mounted on the vehicle for which the alert is set to "Yes," but also, if there is image from an infrastructure camera 115 that can capture the vehicle, the image from that infrastructure camera 115.

[0057] The integrated display unit 133 displays in real time the images acquired from the cameras 111 of the multiple vehicles, with the images separated for each vehicle. For example, one section of the integrated display unit 133 displays the images from the cameras 111A to 111D of vehicle A, and another section displays the images from the cameras 111A to 111D of vehicle B. Note that the integrated display unit 133 may also display images captured by the infrastructure camera 115. The integrated display unit 133 is configured, for example, with a large-screen display panel.

[0058] The detailed information presentation unit 134 is an information presentation unit provided for each of the multiple observers, and includes a display unit and a speaker. For a specific vehicle, the display unit of the detailed information presentation unit 134 displays in real time images from the cameras 111A to 111D mounted on the vehicle, location information detected by the GPS sensor 112, and vehicle status information detected by the CAN sensor 114. Images from an infrastructure camera 115 capable of capturing images of the specific vehicle may also be displayed on the display unit. Furthermore, sounds detected by the microphone 113 mounted on the vehicle are output from the speaker of the detailed information presentation unit 134. Observers can grasp the situation of the vehicle and its surroundings by visually checking the display on the display unit and listening to the sound from the speaker.

[0059] 8 is a flowchart showing an example of a typical process of the control system 200, and this flowchart explains the process of the control method executed by the control system 200. Details of the process executed by each unit below are as described above.

[0060] The information receiving unit 201 receives environmental information such as video and location information transmitted from each device of the on-site system 110 (step S21; receiving step). The recognition unit 202 identifies the area of ​​an object captured in the video captured by each camera 111 and infrastructure camera 115 based on the environmental information acquired from the information receiving unit 201, and recognizes the object in that area (step S22; recognition step).

[0061] The alert determination unit 203 determines whether an alert event has occurred in at least one of the vehicles based on the object recognition result and the environmental information (step S23; detection step). If the occurrence of an alert event is not detected (No in step S23), the control system 200 returns to step S21 and repeats the process.

[0062] If it is detected that an alert event has occurred in any vehicle (Yes in step S23), the alert determination unit 203 generates alert information and distributes it to the display system 130 via the information distribution unit 205. Furthermore, the alert determination unit 203 outputs a control signal to the device of the on-site system 110 to control it to change the quality of the environmental information to be transmitted (step S24; control step).

[0063] When the alert information is delivered, the detailed information presenting unit 134 presents the environmental information of the vehicle for which the alert is set to "Yes" to the monitor in accordance with the output from the presentation switching unit 132. At this time, the detailed information presenting unit 134 may also display, in text or the like, on the display unit that an alert event has occurred in vehicle A. Furthermore, the detailed information presenting unit 134 may display a highlighted area of ​​interest in the video from camera 111 based on the information about the area of ​​interest included in the alert information. As an example, it is possible to display a frame surrounding the area of ​​interest on the display unit.

[0064] By checking the information presented by the detailed information presentation unit 134 in this manner, the monitor can take some kind of action regarding the vehicle for which the alert has been issued. For example, when an autonomously driving vehicle A is approaching a parked vehicle, the monitor can visually check the video from the camera 111A capturing the image ahead of vehicle A, and control the driving of vehicle A to take action to avoid a collision. Furthermore, when the monitor confirms from the video or the like that an accident has occurred with the vehicle for which the alert has been issued, he or she can have a staff member rush to the location of the vehicle based on the vehicle's location information displayed on the display unit.

[0065] When the alert determination unit 203 of the control system 200 determines that the alert flag is "Yes" for a certain vehicle, the monitoring system S2 can execute the processes shown in the following embodiments 2A to 2F as specific processes of step S24 in Fig. 8. Note that in the following examples, the vehicle in which the alert event has occurred will be described as vehicle A. Furthermore, the processes shown in embodiments 2A to 2F can be applied in appropriate combinations.

[0066] (Embodiment 2A) The alert determination unit 203 can transmit a control signal to all cameras 111 mounted on vehicle A via the information receiving unit 201, instructing them to improve the overall quality of the video to be transmitted. All cameras 111 mounted on vehicle A include not only the cameras 111 that captured the video that served as the basis for determining that an alert event has occurred, but also cameras 111 that did not capture such video (i.e., cameras that are not related to the detection of the occurrence of an alert event).

[0067] When no alert event has occurred, the cameras 111A to 111D transmit video of a quality that allows a determination by the AI ​​model in the recognition unit 202 and the alert determination unit 203. However, this quality is too low for the human eye to see. Therefore, if this video is transmitted as is to the display system 130 and an observer checks the video on the detailed information presentation unit 134, there is a problem that the observer may not be able to make an accurate determination, or may feel uncomfortable or fatigued.

[0068] Based on these circumstances, the alert determination unit 203, triggered by the occurrence of an alert in vehicle A, refers to the monitoring information DB 204 to recognize all cameras 111 mounted on vehicle A and transmits a control signal to each camera 111 to change the quality of the video. Based on the control signal, each camera 111 transmits video of higher quality than when no alert has occurred to the cloud 120 and the display system 130. As described above, the presentation switching unit 132 causes the detailed information presentation unit 134 to present the video captured by the camera 111 of vehicle A where the alert has occurred. This allows the monitor to view high-quality video when checking the video of vehicle A on the detailed information presentation unit 134, making it easier for the monitor to make an accurate judgment and reducing discomfort and fatigue felt when viewing the video.

[0069] FIG. 9A is a table showing an example of the quality of video transmitted by the camera 111 of vehicle A when no alert has occurred (for AI) and the quality of video when an alert has occurred (for monitor). In this example, the frame rate, bit rate, and resolution are specified as parameters indicating the quality of the video. The frame rate, bit rate, and resolution for AI are 10 [fps], 0.5 [Mbps], and HD (High Definition), respectively, while for monitor use they are 30 [fps], 5 [Mbps], and Full HD, respectively, resulting in improved quality. Note that the parameters that change between the video for AI and the video for monitor use may be at least one of the frame rate, bit rate, and resolution. Furthermore, the quality parameters that change are not limited to this example.

[0070] The alert determination unit 203 may output a control signal to make the video have a quality suitable for a monitor only to the camera 111 that captured the video that was the basis for determining that an alert event occurred (video related to the detection of the occurrence of an alert event), rather than to all cameras 111 mounted on vehicle A. Alternatively, the alert determination unit 203 may output such a control signal to the camera 111 that captured the video that was the basis for determining that an alert event occurred, and to a camera 111 that captures a capture area adjacent to the capture area of ​​that camera 111.

[0071] For example, when the alert determination unit 203 determines that an alert event has occurred based on the video ahead of vehicle A captured by camera 111A, the alert determination unit 203 may output control signals to not only camera 111A but also cameras 111C and 111D that capture video to the sides of vehicle A, to change the video to a quality suitable for a monitor. As another example, when the area where the alert event has occurred is located to the left front of vehicle A (for example, when another vehicle is parked to the left front of vehicle A), the alert determination unit 203 determines that cameras 111A and 111D are capable of capturing images in the direction related to the parked vehicle. Then, control signals may be output to these cameras 111A and 111D to change the video to a quality suitable for a monitor.

[0072] However, as described above, the quality of the video transmitted can be improved not only for the camera 111 that captured the video that was the basis for determining that an alert event occurred, but also for all cameras 111 mounted on vehicle A, including cameras 111 that did not capture such video (i.e., cameras not related to the detection of the alert occurrence). This allows the monitor to monitor vehicle A with higher accuracy.

[0073] (Embodiment 2B) In the embodiment 2A, an example has been described in which the quality of the video captured by the camera 111 mounted on the vehicle A is changed when an alert event occurs in the vehicle A. However, the cameras whose video quality is changed may include not only the camera 111 of the vehicle A but also the infrastructure camera 115.

[0074] When an alert event occurs in vehicle A, the alert determination unit 203 acquires the current location information of vehicle A stored in the monitoring information DB 204. This location information is transmitted from the GPS sensor 112 of vehicle A. The alert determination unit 203 also acquires the location information of all infrastructure cameras 115 connected to the control system 200. Then, the alert determination unit 203 compares the current location information of vehicle A with the location information of each infrastructure camera 115, and determines whether or not there is a possibility that the infrastructure camera 115 will be able to capture an image of vehicle A.

[0075] For example, the alert determination unit 203 may determine that there is a possibility that the infrastructure camera 115 can capture an image of vehicle A if the distance between vehicle A and a specified infrastructure camera 115 is within a specified distance (i.e., they are geographically close), and may determine that there is no possibility that the infrastructure camera 115 can capture an image of vehicle A if the distance between the two is greater than the specified distance.

[0076] The monitoring information DB 204 may also store data indicating the image capture area or image capture direction on the map of each infrastructure camera 115. For an infrastructure camera 115 determined to be close to vehicle A, the alert determination unit 203 uses the image capture area or image capture direction data to determine that there is a possibility that the infrastructure camera 115 can capture an image of vehicle A if vehicle A is located within the image capture area or in the image capture direction of the infrastructure camera 115. If this is not the case, the alert determination unit 203 determines that there is no possibility that the infrastructure camera 115 can capture an image of vehicle A.

[0077] Furthermore, the alert determination unit 203 may further refer to map information stored in the monitoring information DB 204 when vehicle A is located within the imaging area or in the imaging direction of the infrastructure camera 115. By referring to the map information, the alert determination unit 203 uses data such as the height and size of buildings and the like around the infrastructure camera 115 to determine whether vehicle A is located in a blind spot of the building as seen from the infrastructure camera 115. When vehicle A is not located in a blind spot of the building as seen from the infrastructure camera 115, the alert determination unit 203 determines that there is a possibility that the infrastructure camera 115 can image vehicle A, and when vehicle A is located in a blind spot, the alert determination unit 203 determines that there is no possibility that the infrastructure camera 115 can image vehicle A.

[0078] The alert determination unit 203 transmits a control signal to the infrastructure camera 115 that has been determined to have the potential to capture an image of vehicle A, so as to adjust the quality of the output image to that required for a monitor, as described in embodiment 2A. As a result, the image from the infrastructure camera 115 that can capture an image of vehicle A is of high quality required for a monitor, and the monitor at the display system 130 side can view the image and perform highly accurate monitoring.

[0079] Furthermore, the alert determination unit 203 can also execute the above-mentioned control on the cameras 111 mounted on other vehicles such as vehicles B, C, etc., instead of or in addition to the infrastructure camera 115. Specifically, when an alert event occurs in vehicle A, the alert determination unit 203 acquires the current position information of vehicle A and vehicles B, C, etc., stored in the monitoring information DB 204. This position information is transmitted from the GPS sensor 112 mounted on each vehicle. Then, the current position information of vehicle A is compared with the position information of vehicles B, C, etc., for each vehicle, and it is determined whether there are other vehicles that may be able to capture an image of vehicle A.

[0080] For example, the alert determination unit 203 may determine that there is a possibility that the camera 111 of a vehicle can capture an image of vehicle A if the distance between vehicle A and a specified vehicle is within a specified distance, and may determine that there is no possibility that the camera 111 can capture an image of vehicle A if the distance between the two is greater than the specified distance.

[0081] Furthermore, the monitoring information DB 204 may store data indicating the photographing area or photographing direction of each vehicle. The alert determination unit 203 maps data on the photographing area or photographing direction of each camera 111 of a vehicle determined to be in close proximity to vehicle A on map information, and determines that if vehicle A is located within the photographing area or in the photographing direction, the camera 111 of that vehicle is likely to be able to photograph vehicle A. If this is not the case, the alert determination unit 203 determines that the camera 111 of that vehicle is not likely to be able to photograph vehicle A. Note that in this processing, for a vehicle determined to be in close proximity to vehicle A, the camera 111 whose photographing direction is in the direction in which vehicle A is located is determined to be likely to be able to photograph vehicle A, but the camera 111 other than that is not determined to be likely to be able to photograph vehicle A.

[0082] Furthermore, the alert determination unit 203 may further refer to map information stored in the monitoring information DB 204 when vehicle A is located within the shooting area or in the shooting direction of the camera 111 of another vehicle. By referring to the map information, the alert determination unit 203 uses data such as the height and size of buildings and the like around vehicles close to vehicle A to determine whether vehicle A is located in a blind spot of a building as viewed from that vehicle. When vehicle A is not located in a blind spot of a building as viewed from that vehicle, the alert determination unit 203 determines that there is a possibility that the camera 111 of that vehicle can capture vehicle A, and when vehicle A is located in a blind spot, the alert determination unit 203 determines that there is no possibility that the camera 111 of that vehicle can capture vehicle A.

[0083] The alert determination unit 203 transmits a control signal to the camera 111 of a vehicle that has been determined to have the potential to capture vehicle A, to improve the quality of the output image from that for AI to that for the monitor. This allows the monitor on the display system 130 side to view the image from another vehicle and perform highly accurate monitoring.

[0084] The alert determination unit 203 may repeatedly perform the above-described determination for the camera 111 of a vehicle other than vehicle A at predetermined time intervals. As a result, for example, if vehicle B moves from a position where it can capture vehicle A to a position where it cannot capture it, the alert determination unit 203 determines, based on the above-described determination, that the camera of vehicle B is a camera that has no chance of capturing images. Then, the quality of the image from that camera 111 can be lowered from the quality for an observer to the quality for an AI. Furthermore, if a vehicle moves vehicle A from a position where it cannot capture it to a position where it can capture it, the quality of the image from the camera 111 newly installed on that vehicle can be raised from the quality for an AI to the quality for an observer.

[0085] Furthermore, the video of monitor quality transmitted to the display system 130 via the cloud 120 by the infrastructure camera 115 or the camera 111 mounted on a vehicle other than vehicle A may be controlled by the presentation switching unit 132 so that it is displayed on the display unit of the detailed information presentation unit 134 that displays the video from the camera 111 of vehicle A. This allows the monitor to easily check both the video from the vehicle-mounted camera of vehicle A and the video showing vehicle A from outside.

[0086] (Embodiment 2C) In Embodiments 2A and 2B, the quality of the video controlled by the alert determination unit 203 is set to two levels, one for AI and one for a monitor. However, the quality of the video controlled by the alert determination unit 203 is not limited to two levels, one for AI and one for a monitor, and may be set to three or more levels.

[0087] FIG. 9B is a table showing an example of the quality of video transmitted by the camera 111 of vehicle A when no alert event has occurred (for AI) and the quality of video when an alert event has occurred (for AI detailed analysis and for monitors). In FIG. 9B, in addition to the settings shown in FIG. 9A, the quality of video when an alert event has occurred is set for AI detailed analysis in addition to that for monitors. The frame rate, bit rate, and resolution of the video for AI detailed analysis are 15 [fps], 2 [Mbps], and Full HD, respectively, and this quality falls between that for AI and that for monitors.

[0088] When the recognition unit 202 and the alert determination unit 203 perform determination using an AI model, the accuracy of the determination can be increased by improving the quality of the video to be determined. Therefore, when an alert event occurs in any vehicle, the alert determination unit 203 can output a control signal to a camera 111 that is not monitored by a monitor but is related to the alert event, to switch the video from the one for AI to the one for AI detailed analysis. A camera related to the alert event is, for example, a camera 111 that captures video close to the area where the alert event occurred.

[0089] For example, when the alert determination unit 203 determines that an alert event has occurred based on the video of the front of vehicle A captured by the camera 111A, the alert determination unit 203 improves the quality of the video from the camera 111A from AI quality to that for an observer. The alert determination unit 203 can also improve the quality of the video from the cameras 111C and 111D from AI quality to that for AI detailed analysis. If the area where the alert event has occurred is the left front of vehicle A, the alert determination unit 203 improves the quality of the video from the cameras 111A and 111D of vehicle A from AI quality to that for an observer, for the same reasons as those described in embodiment 2A. The alert determination unit 203 can also improve the quality of the video from the cameras 111B and 111C from AI quality to that for AI detailed analysis. However, as described in embodiment 2A, all of the video from the camera 111 mounted on vehicle A may be improved from AI quality to that for an observer.

[0090] In the above two examples, when it is determined that there is a possibility that vehicle A can be photographed by at least one of infrastructure camera 115 and camera 111 mounted on a vehicle other than vehicle A, the quality of the image from that camera may be improved from that for AI to that for AI detailed analysis. The determination of whether there is a possibility that vehicle A can be photographed is as described in embodiment 2B.

[0091] In addition, if the AI ​​model of the alert determination unit 203 detects the occurrence of an alert event based on video from a camera whose quality has been improved for AI detailed analysis, the alert determination unit 203 may improve the quality of the video transmitted by the camera from that for AI detailed analysis to that for a monitor.

[0092] When a monitor performs monitoring, if the monitoring video is kept at AI quality, the monitor may feel uncomfortable or fatigued when viewing the video. However, if the video transmitted from the camera is kept at high quality for monitoring even when no alert event occurs, the amount of data transmitted from the on-site system 110 will always be large, resulting in high costs for wireless communication. Furthermore, if the frame rate or resolution of the video to be processed is high, the amount of calculations performed in the processing will be proportional to the number of video pixels, resulting in high costs for the computer used to process the video. This becomes more pronounced as the number of cameras transmitting video increases.

[0093] However, as shown in Embodiments 2A to 2C, when an alert event occurs that is deemed to require monitoring by a monitor, the control system 200 can control the camera that is supposed to be capturing the situation so that the video image is of high quality. This makes it possible to improve the accuracy of monitoring by the monitor and reduce communication costs at the same time.

[0094] For example, when the occurrence of an alert event is detected by the camera 111A mounted on vehicle A, the alert determination unit 203 can control the cameras 111B to 111D mounted on vehicle A to increase the quality of the video to be transmitted, even if the occurrence of an alert event is not detected in the video from the camera 111. Specifically, the quality of the video can be increased from that for AI use to that for AI detailed analysis or for an observer. This makes it possible to achieve both accurate analysis by the AI ​​model or observer and reduction in communication costs during normal times.

[0095] Furthermore, for cameras capturing footage related to the detection of a specific vehicle alert, the transmitted footage can be of monitor quality, while for cameras capturing footage unrelated to the detection of an alert event but with the possibility of capturing the vehicle, the transmitted footage can be of a quality suitable for detailed AI analysis. This allows for high-quality footage from cameras highly related to the alert event, allowing monitors to monitor footage from cameras capturing important footage. Furthermore, cameras with the possibility of capturing important footage can be subjected to detailed AI analysis. Furthermore, by not immediately transmitting high-quality footage from cameras with the possibility of capturing important footage, unnecessary data traffic can be reduced, thereby reducing communication costs.

[0096] The quality of the video transmitted by the camera before detecting the occurrence of an alert event may be relatively low quality for AI, which can increase the effect of reducing communication costs.

[0097] Furthermore, the camera that improves the quality of the video for detailed AI analysis may be a camera mounted on a vehicle other than the vehicle in which the alert event was detected, or a camera that is fixed in position and can capture the vehicle from outside. This allows for more accurate monitoring by monitoring the vehicle in which the alert event was detected from outside.

[0098] In addition, in embodiments 2A to 2C, when the alert determination unit 203 determines that the alert flag in vehicle A has changed from "Yes" to "No," it may send a control signal to the camera 111, which has improved the quality of the video for a monitor or for detailed AI analysis, instructing it to lower the quality of the entire video to be transmitted for AI use.

[0099] Furthermore, if necessary, the alert determination unit 203 may improve the quality of video sent from a vehicle in which an alert event is not detected from that intended for AI to that intended for AI detailed analysis, or conversely, may lower the quality from that intended for AI detailed analysis to that intended for AI.

[0100] (Embodiment 2D) When the alert determination unit 203 detects the occurrence of an alert event related to a specific vehicle, it may change not only the quality of the video to be transmitted but also the quality of other environmental information.

[0101] For example, when the alert determination unit 203 detects the occurrence of an alert event for vehicle A, the alert determination unit 203 refers to the monitoring information DB 204 to acquire information from not only the camera 111 mounted on vehicle A but also the GPS sensor 112, microphone 113, and CAN sensor 114. Then, as shown in embodiment 2A, the alert determination unit 203 transmits a control signal to the camera 111 of vehicle A to change the quality of the video. Furthermore, the alert determination unit 203 can transmit a control signal to the GPS sensor 112, microphone 113, and CAN sensor 114 mounted on vehicle A to increase the frequency of transmission of environmental information transmitted by each device.

[0102] 10 is a table showing an example of the quality (for AI) of environmental information transmitted by the GPS sensor 112, microphone 113, and CAN sensor 114 of vehicle A when no alert has occurred and the quality (for observer) when an alert has occurred. In this example, the transmission frequencies of the GPS information, sound information, and CAN information, which are position information transmitted by each device, are 0.2 [Hz], 32 k [bps], and 1 (Hz), respectively, for AI, whereas for observer use they are 10 [Hz], 192 k [bps], and 100 (Hz), respectively, which indicates an improved transmission frequency.

[0103] The alert determination unit 203 sends a control signal to each device of vehicle A to increase the transmission frequency of environmental information from that for AI to that for the monitor, and each device then transmits environmental information to the cloud 120 and the display system 130 at the transmission frequency for the monitor. The detailed information presentation unit 134 displays the location information detected by the GPS sensor 112 and the vehicle status information detected by the CAN sensor 114 in real time at the transmission frequency for the monitor. In addition, the speaker of the detailed information presentation unit 134 outputs sound from the microphone 113 at the transmission frequency for the monitor. This allows the monitor to check information about vehicle A more accurately, thereby enabling highly accurate monitoring. It also makes it possible to reduce the amount of data communication required when the occurrence of an alert event is not detected.

[0104] The alert determination unit 203 may perform control to increase the transmission frequency of at least one of the position information, sound information, and CAN information, rather than all of them. The alert determination unit 203 may also increase the transmission frequency of other environmental information, not limited to the position information, sound information, and CAN information, in a similar manner.

[0105] Furthermore, if necessary, the alert determination unit 203 may improve the quality of environmental information other than video from a vehicle in which an alert event is not detected from the one intended for AI analysis to the one intended for AI detailed analysis, or conversely, may lower the quality from the one intended for AI detailed analysis to the one intended for AI analysis. The one intended for AI detailed analysis is transmitted more frequently than the one intended for AI analysis but less frequently than the one intended for monitors, and is insufficient for monitor monitoring but allows for detailed analysis by the AI ​​model. This control enables the control system 200 to perform detailed monitoring. Furthermore, if the AI ​​model of the alert determination unit 203 detects the occurrence of an alert event based on the environmental information improved for AI detailed analysis, the alert determination unit 203 may improve the quality of the environmental information transmitted from that vehicle from the one intended for AI detailed analysis to the one intended for monitors.

[0106] (Embodiment 2E) Even if the quality of the video is increased for the monitor by the processing described in embodiment 2A, etc., the alert determination unit 203 may decrease the quality of the video in areas other than the area of ​​interest in the video as time passes.

[0107] For example, the alert determination unit 203 detects the occurrence of an alert event based on video from the camera 111A of vehicle A, thereby improving the quality of the entire video transmitted by that camera 111A from that for AI to that for a monitor. Thereafter, the alert determination unit 203 refers to information about the area of ​​interest included in the alert information about the camera 111A, and transmits a control signal to the camera 111A so as to lower the quality of the video transmitted in areas other than the area of ​​interest as time passes.

[0108] FIG. 11 is a table illustrating an example in which the alert determination unit 203 increases the bit rate of the video transmitted for an observer and then decreases it again to the AI ​​bit rate. After detecting the occurrence of an alert event, the alert determination unit 203 increases the bit rate of the entire video transmitted from the camera 111A from 0.5 Mbps to 5 Mbps. Then, the alert determination unit 203 transmits a control signal to the camera 111A to change the bit rate in areas other than the area of ​​interest to 4 Mbps after 2 seconds, 3 Mbps after 4 seconds, 2 Mbps after 6 seconds, 1 Mbps after 8 seconds, and 0.5 Mbps after 10 seconds. The camera 111A transmits video at a quality corresponding to this control signal, allowing the observer to view, via the detailed information presentation unit 134, the video in areas other than the area of ​​interest, in which the video quality decreases over time.

[0109] In order to perform highly accurate monitoring, the monitor needs to check the video of the area of ​​interest at a quality suitable for the monitor. However, since there is little need to monitor areas other than the area of ​​interest, it is preferable to lower the quality of the video of these areas in order to reduce the amount of data communication. However, if the video quality of the area of ​​interest is set to the monitor's quality from the beginning and the video quality of the other areas is set to the AI ​​quality, the monitor may feel uncomfortable when viewing the video from the camera 111.

[0110] In embodiment 2E, to solve this problem, alert determination unit 203 increases the quality of the entire video from camera 111A, and then transmits a control signal to camera 111A to lower the quality of the video in areas other than the area of ​​interest in that video and transmit the video. This allows the monitor to view the entire video at a high monitor-specific quality and then lower the quality of the video in areas other than the area of ​​interest, thereby reducing the sense of discomfort felt by the monitor when monitoring the video.

[0111] Furthermore, when lowering the quality of video in areas other than the area of ​​interest, the alert determination unit 203 can be configured to monotonically decrease the video quality over time. Here, the alert determination unit 203 may not only lower the video quality discontinuously (in steps) as in the above example, but also monotonically decrease the quality by continuously lowering the video quality. Furthermore, when lowering the video quality in steps, any method of lowering the video quality in two or more stages can be applied. This can reduce the sense of discomfort felt by monitors when monitoring the video, compared to when the quality of video in areas other than the area of ​​interest is immediately lowered for AI use. In particular, the less the degree to which the video quality is reduced at one time, the less uncomfortable the monitor will feel.

[0112] In addition, the alert determination unit 203 may specify all areas other than the area of ​​interest as areas other than the area of ​​interest that will reduce the quality of the video, or may specify areas other than the area of ​​interest and a specified area surrounding the area of ​​interest (in other words, some areas other than the area of ​​interest).

[0113] Furthermore, the alert determination unit 203 may control the quality of the video of the area other than the attention area to be lowered from the frame rate, bit rate, or resolution for the monitor to that for the AI, without being limited to the above-mentioned examples.

[0114] Furthermore, as long as the alert determination unit 203 reduces the quality of the video of areas other than the attention area so that it is lower than the quality for the monitor, the ultimately reduced quality does not have to be for AI. For example, the alert determination unit 203 may reduce the quality of the video of areas other than the attention area so that it is ultimately for AI detailed analysis. This makes it possible to lower the quality of the video of areas that are less necessary in the video while enabling detailed analysis by an AI model. This makes it easier to detect the occurrence of an alert event in an area other than the attention area. When the occurrence of an alert event is detected in an area other than the attention area, the alert determination unit 203 again improves the quality of the entire video of the camera 111A to the quality for the monitor.

[0115] (Embodiment 2F) When the alert determination unit 203 detects the occurrence of an alert event in any vehicle, it may send a control signal to a camera that captured footage unrelated to the detection of the alert occurrence, so as to lower the quality of the footage transmitted from that camera.

[0116] For example, when the alert determination unit 203 detects the occurrence of an alert event using environmental information from any device mounted on vehicle A, it refers to information stored in the monitoring information DB 204 and determines vehicles other than vehicle A in which the occurrence of an alert event has not been detected. Then, if the video transmitted from the camera 111 mounted on that vehicle is for a monitor, the alert determination unit 203 lowers the quality of the entire video for AI detailed analysis or for AI. Furthermore, if the video transmitted from the camera 111 mounted on that vehicle is for AI detailed analysis, the alert determination unit 203 lowers the quality of the entire video for AI. The video quality to be lowered is at least one of the frame rate, bit rate, and resolution. Note that this example is not limiting, and control to lower the quality of the entire video may be similarly performed in at least one of the camera 111 and infrastructure camera 115 of a vehicle determined to have no possibility of capturing vehicle A in the determination shown in embodiment 2B.

[0117] In addition, for vehicles in which the occurrence of an alert event has not been detected or for which it has been determined that there is no possibility of photographing vehicle A, the alert determination unit 203 can also reduce the transmission frequency of environmental information from at least one of the GPS sensor 112, microphone 113, and CAN sensor 114 to AI if the transmission frequency is for the monitor.

[0118] In this way, when the alert determination unit 203 detects the occurrence of an alert event, it can transmit a control signal to a device that has transmitted environmental information unrelated to the detection of the occurrence of the alert event, to lower the quality of the transmitted environmental information. This makes it possible to reduce the amount of data communication for information that does not require monitoring.

[0119] In particular, the alert determination unit 203 can transmit a control signal to devices mounted on vehicles where no alert is detected, to lower the quality of the environmental information they transmit, thereby reducing the amount of data communication from vehicles that do not require monitoring.

[0120] The present disclosure is not limited to the above-described embodiments and may be modified as appropriate without departing from the spirit and scope of the present disclosure. For example, the quality of the video transmitted from the camera may be changed not only in two or three stages as described in the second embodiment, but also in four or more stages. Furthermore, in the second embodiment, a uniform numerical value is set for the camera in each setting for AI, AI detailed analysis, and surveillance. However, even in the same setting for AI, AI detailed analysis, or surveillance, the numerical value indicating the quality may be set differently for each camera depending on the characteristics of the camera (e.g., whether the camera is an in-vehicle camera 111 or an infrastructure camera 115) or the location of the camera. However, for a certain parameter indicating quality, when a numerical value indicating the quality of the video for AI and a numerical value indicating the quality of the video for AI detailed analysis are compared between arbitrary cameras, the latter will be of higher quality. Similarly, when a numerical value indicating the quality of the video for AI detailed analysis and a numerical value indicating the quality of the video for surveillance are compared between arbitrary cameras, the latter will be of higher quality.

[0121] Furthermore, the alert determination unit 203 of the control system 200 can determine not only whether an alert is issued but also the level of the alert when an alert event is detected. For example, when the alert determination unit 203 determines that there is a possibility of an accident occurring in vehicle A, if the probability of the accident occurring is less than a predetermined threshold, or if the time until the accident occurs is equal to or greater than a predetermined threshold, the alert level can be set to "low." On the other hand, conversely, if the probability of the accident occurring is equal to or greater than a predetermined threshold, or if the time until the accident occurs is less than a predetermined threshold, the alert level can be set to "high." The alert determination unit 203 can determine the level of the alert in this manner, store it as alert information in the monitoring information DB 204, and output it to the display system 130. This allows the monitor to easily check the status of the alert event.

[0122] Here, if the alert level is "high", the alert determination unit 203 keeps the quality of the entire image from the camera 111 mounted on the vehicle in which the alert was detected high for the monitor, whereas if the alert level is "low", the alert determination unit 203 may execute control to gradually lower the quality of the image in areas other than the area of ​​interest in the image, as shown in embodiment 2E.

[0123] Furthermore, the monitor may operate the detailed information presentation unit 134 to output a control signal for changing the quality of the environmental information to the cloud 120. For example, the monitor may determine that the occurrence of an alert event in vehicle A has been resolved by checking the video or the like displayed on the detailed information presentation unit 134. In this case, in response to the monitor's operation, the detailed information presentation unit 134 transmits a control signal to the cloud 120 to lower the quality of the overall video from the camera 111 of vehicle A from the video for the monitor to the video for AI or AI detailed analysis. In response to the signal, the alert determination unit 203 changes the alert flag in vehicle A from "Yes" to "No" and outputs an instruction to change the overall quality of the video transmitted by the camera 111 of vehicle A in accordance with the control signal. This reduces the quality of the video transmitted by the instructed camera 111, thereby reducing the amount of data communication that is not necessary.

[0124] The monitoring system according to the present disclosure can monitor not only vehicles but also any object such as a factory production line.

[0125] In the above-described embodiments, this disclosure has been described as a hardware configuration, but this disclosure is not limited to this. This disclosure can also be realized by causing a processor in a computer constituting such a device to execute a computer program to perform the processing (steps) of each device in the control device or monitoring system described in the above-described embodiments.

[0126] Fig. 12 is a block diagram showing an example of the hardware configuration of an information processing device (computer) that executes the processes of the above-described embodiments. Referring to Fig. 12, this information processing device 90 includes a signal processing circuit 91, a processor 92, and a memory 93. The information processing device 90 constitutes a control device or each device in a monitoring system.

[0127] The signal processing circuit 91 is a circuit for processing signals in accordance with the control of the processor 92. The signal processing circuit 91 may include a communication circuit for receiving signals from a transmitting device.

[0128] The processor 92 reads and executes software (computer programs) from the memory 93 to perform the processing of the device described in the above embodiment. The number of processors 92 is not limited to one, and multiple processors 92 may be provided. As an example of the processor 92, one of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Demand-Side Platform), and an ASIC (Application Specific Integrated Circuit) may be used, or multiple processors may be used in parallel.

[0129] The memory 93 is configured with a volatile memory, a nonvolatile memory, or a combination thereof. The memory 93 is not limited to one, and a plurality of memories may be provided. The volatile memory may be, for example, a RAM (Random Access Memory) such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory). The nonvolatile memory may be, for example, a PROM (Programmable Read Only Memory). Read ROMs such as EPROM (Erasable Programmable Read Only Memory) and EPROM (Erasable Programmable Read Only Memory) Read The memory may be a memory only memory, a flash memory, or an SSD (Solid State Drive).

[0130] The memory 93 is used to store one or more instructions. Here, the one or more instructions are stored as a group of software modules in the memory 93. The processor 92 can perform the processes described in the above embodiments by reading and executing the group of software modules from the memory 93.

[0131] The memory 93 may include memory built into the processor 92 in addition to memory provided outside the processor 92. The memory 93 may also include storage located away from the processors constituting the processor 92. In this case, the processor 92 can access the memory 93 via an I / O (Input / Output) interface.

[0132] As described above, one or more processors included in each device in the above-described embodiments execute one or more programs including instructions for causing a computer to execute the algorithms described using the drawings. This processing enables the signal processing method described in each embodiment to be realized.

[0133] The program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on various types of non-transitory computer-readable medium or tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disk (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0134] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) a detection means for detecting whether or not a cautionary event, which is an event requiring attention, has occurred in the vicinity of any of the information acquisition devices based on the environmental information transmitted from the information acquisition devices; an instruction means for outputting an instruction to a first information acquisition device that has transmitted the environmental information not related to the detection of the occurrence of the caution event, when the detection means detects the occurrence of the caution event, to change the quality of the environmental information transmitted by the first information acquisition device; A control device comprising: (Appendix 2) the plurality of information acquisition devices are devices mounted on the same vehicle, the instruction means, when the detection means detects the occurrence of the attention event, outputs the instruction to the first information acquisition device to increase the quality of the environmental information transmitted by the first information acquisition device. 10. The control device of claim 1. (Appendix 3) When the detection means detects the occurrence of the caution event for a predetermined vehicle, the instruction means outputs the instruction to the first information acquisition device that has the possibility of capturing an image of the vehicle to increase the quality of the image, which is the environmental information, transmitted by the first information acquisition device. 10. The control device of claim 1. (Appendix 4) When the detection means detects the occurrence of the caution event for a predetermined vehicle, the instruction means outputs the instruction to the first information acquisition device that may be able to photograph the vehicle to change the quality of the video, which is the environmental information transmitted by the first information acquisition device, from a first quality to a second quality higher than the first quality, and to the second information acquisition device that transmitted the video, which is the environmental information related to the detection of the occurrence of the caution event, to change the quality of the video, which is the second information acquisition device, from a third quality to a fourth quality higher than the second quality and the third quality. 10. The control device of claim 1. (Appendix 5) The first information acquisition device is a device mounted on a vehicle other than the vehicle on which the second information acquisition device is mounted, or a device that is fixed in position and can photograph the vehicle on which the second information acquisition device is mounted from the outside. 5. The control device according to claim 4. (Appendix 6) the first information acquisition device is a device that acquires at least one of sound information, state information of the vehicle, and position information of the vehicle as the environmental information, the instruction means, when the detection means detects the occurrence of the attention event, outputs the instruction to the first information acquisition device to increase the frequency of transmission of the environmental information transmitted by the first information acquisition device. 3. The control device according to claim 2. (Appendix 7) When the detection means detects the occurrence of the caution event, the instruction means outputs the instruction to the second information acquisition device that transmitted the video as the environmental information related to the detection of the occurrence of the caution event, to increase the quality of the entire video transmitted by the second information acquisition device, and then to lower the quality of the video of at least a part of the video other than the area related to the occurrence of the caution event and transmit the video. 7. The control device according to any one of claims 1 to 6. (Appendix 8) When lowering the quality of the image of the partial area, the instruction means sets the quality of the image of the partial area to decrease monotonically with the passage of time. 8. The control device according to claim 7. (Appendix 9) the instruction means, when the detection means detects the occurrence of the attention event, outputs the instruction to the first information acquisition device to lower the quality of the environmental information transmitted by the first information acquisition device. 10. The control device of claim 1. (Appendix 10) The first information acquisition device is a device mounted on a vehicle, and is mounted on a vehicle different from the second information acquisition device that transmitted the environmental information related to the detection of the occurrence of the caution event. 10. The control device according to claim 9. (Appendix 11) a plurality of information acquisition devices that acquire environmental information and transmit the acquired environmental information; a control device that controls transmission of the environmental information from the plurality of information acquisition devices based on the environmental information transmitted from the plurality of information acquisition devices, The control device a detection means for detecting whether or not a cautionary event, which is an event requiring attention, has occurred in the vicinity of any of the information acquisition devices based on the environmental information transmitted from the plurality of information acquisition devices; and an instruction means for outputting an instruction to a first information acquisition device that has transmitted the environmental information not related to the detection of the occurrence of the attention event, when the detection means detects the occurrence of the attention event, to change the quality of the environmental information transmitted by the first information acquisition device. Surveillance system. (Appendix 12) the plurality of information acquisition devices are devices mounted on the same vehicle, the instruction means, when the detection means detects the occurrence of the attention event, outputs the instruction to the first information acquisition device to increase the quality of the environmental information transmitted by the first information acquisition device. 12. The monitoring system described in Appendix 11. (Appendix 13) detecting whether or not a cautionary event, which is an event requiring attention, has occurred in the vicinity of any of the information acquisition devices based on the environmental information transmitted from the information acquisition devices; when detecting the occurrence of the caution event, outputting an instruction to a first information acquisition device that has transmitted the environmental information not related to the detection of the occurrence of the caution event, to change the quality of the environmental information transmitted by the first information acquisition device; A control method executed by a control device. (Appendix 14) detecting whether or not a cautionary event, which is an event requiring attention, has occurred in the vicinity of any of the information acquisition devices based on the environmental information transmitted from the information acquisition devices; when detecting the occurrence of the caution event, outputting an instruction to a first information acquisition device that has transmitted the environmental information not related to the detection of the occurrence of the caution event, to change the quality of the environmental information transmitted by the first information acquisition device; A non-transitory computer-readable medium that stores a program that causes a computer to execute a program.

[0135] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the disclosure. [Explanation of symbols]

[0136] 10 Control device 11 detection unit 12 indication unit S1 Surveillance System 21 Information acquisition device 22 Control device S2 Surveillance System 110 On-site system 111 Camera 112 GPS sensor 113 Microphone 114 CAN sensor 115 Infrastructure Camera 120 Cloud 130 Display System 131 information receiving unit 132 presentation switching unit 133 Integrated display section 134 Detailed information presentation section 200 Control System 201 Information receiving unit 202 Recognition unit 203 Alert determination unit 204 Monitoring information DB 205 Information Distribution Department

Claims

1. a detection means for detecting whether or not a cautionary event, which is an event requiring attention, has occurred in the vicinity of any of the information acquisition devices based on the environmental information transmitted from the information acquisition devices; an instruction means for outputting an instruction to a first information acquisition device, when the detection means detects the occurrence of the caution event for a predetermined vehicle, to increase the quality of the video, which is the environmental information transmitted by the first information acquisition device that has transmitted the environmental information not related to the detection of the occurrence of the caution event and has the possibility of capturing the vehicle, from a first quality to a second quality higher than the first quality; When the detection means detects the occurrence of the caution event for the vehicle, the instruction means outputs the instruction to the second information acquisition device that transmitted the video that is the environmental information related to the detection of the occurrence of the caution event, to change the quality of the video transmitted by the second information acquisition device from a third quality to a fourth quality that is higher than the second quality and the third quality. Control device.

2. The first information acquisition device is a device mounted on a vehicle other than the vehicle on which the second information acquisition device is mounted, or a device that is fixed in position and can photograph the vehicle on which the second information acquisition device is mounted from the outside. The control device according to claim 1 .

3. When the detection means detects the occurrence of the caution event, the instruction means outputs the instruction to the second information acquisition device that transmitted the video as the environmental information related to the detection of the occurrence of the caution event, to increase the quality of the entire video transmitted by the second information acquisition device, and then to lower the quality of the video of at least a part of the video other than the area related to the occurrence of the caution event and transmit the video. The control device according to claim 1 or 2.

4. When lowering the quality of the image of the partial area, the instruction means sets the quality of the image of the partial area to decrease monotonically with the passage of time. The control device according to claim 3 .

5. a detection means for detecting whether or not a cautionary event, which is an event requiring attention, has occurred in the vicinity of any of a plurality of information acquisition devices that are mounted on the same vehicle and acquire environmental information, based on the environmental information transmitted from the information acquisition devices; an instruction means for outputting an instruction to a first information acquisition device that has transmitted the environmental information not related to the detection of the occurrence of the caution event, to increase the quality of the environmental information transmitted by the first information acquisition device, when the detection means detects the occurrence of the caution event; A control device comprising:

6. the first information acquisition device is a device that acquires at least one of sound information, state information of the vehicle, and position information of the vehicle as the environmental information, the instruction means, when the detection means detects the occurrence of the attention event, outputs the instruction to the first information acquisition device to increase the frequency of transmission of the environmental information transmitted by the first information acquisition device. The control device according to claim 5 .

7. Detecting whether or not a cautionary event, which is an event requiring attention, has occurred in the vicinity of any of the information acquisition devices based on the environmental information transmitted from the information acquisition devices that acquire the environmental information; When detecting the occurrence of the caution event for a predetermined vehicle, output an instruction to a first information acquisition device that has transmitted the environmental information not related to the detection of the occurrence of the caution event and that may be able to capture the vehicle, to increase the quality of the video, which is the environmental information, from a first quality to a second quality higher than the first quality; when detecting the occurrence of the caution event for the vehicle, outputting the instruction to the second information acquisition device that transmitted the video that is the environmental information related to the detection of the occurrence of the caution event, to change the quality of the video transmitted by the second information acquisition device from a third quality to a fourth quality that is higher than the second quality and the third quality. A control method executed by a control device.

8. Detecting whether or not a cautionary event, which is an event requiring attention, has occurred in the vicinity of any of the information acquisition devices based on the environmental information transmitted from the information acquisition devices that acquire the environmental information; When detecting the occurrence of the caution event for a predetermined vehicle, output an instruction to a first information acquisition device that has transmitted the environmental information not related to the detection of the occurrence of the caution event and that may be able to capture the vehicle, to increase the quality of the video, which is the environmental information, from a first quality to a second quality higher than the first quality; when detecting the occurrence of the caution event for the vehicle, outputting the instruction to the second information acquisition device that transmitted the video that is the environmental information related to the detection of the occurrence of the caution event, to change the quality of the video transmitted by the second information acquisition device from a third quality to a fourth quality that is higher than the second quality and the third quality. A program that makes a computer do something.

Images