Driving assistance systems for mobile vehicles

Abstract

To obtain a driver assistance system that reduces the workload of operators who monitor and operate remote monitoring devices, prevents overloads in advance, and enables stable monitoring and operation to continue. [Solution] The mobile vehicle driving support system comprises: a remote monitoring necessity calculation unit that calculates the degree of need for remote monitoring by a remote monitoring device based on the state of the mobile vehicle; a remote monitoring feasibility determination unit that determines whether remote monitoring is possible for each of a plurality of mobile vehicles based on the degree of need for remote monitoring calculated by the remote monitoring necessity calculation unit; a driving method selection unit that selects a driving method for the mobile vehicle according to the determination result of whether remote monitoring is possible or not determined by the remote monitoring feasibility determination unit and the degree of need for remote monitoring calculated by the remote monitoring necessity calculation unit; and a mobile vehicle drive unit that drives the mobile vehicle based on the driving method selected by the driving method selection unit.

Description

Technical Field

[0001] The present disclosure relates to a driving support system for a moving body.

Background Art

[0002] In recent years, the automation of transport vehicles has been promoted. The introduction of automated driving is expected to solve various social problems such as improvement of driver shortages in the logistics field, improvement of traffic congestion, and response to the last-mile problem.

[0003] In the operation of an automated driving vehicle, a remote monitoring device that constantly monitors the driving state of the vehicle is required. When it is difficult to make a judgment with the automated driving function, an instruction is given from the remote monitoring device, and if necessary, the vehicle is operated by a remote operation unit provided in the remote monitoring device to continue the vehicle's travel.

[0004] When a remote monitoring device manages a large number of automated driving vehicles, the number of monitored vehicles increases and the monitoring load becomes excessive, and it may be impossible to handle all the vehicles that need to be remotely operated. A technique has been proposed to detect a state in which it is difficult for an operator to perform remote operations based on the operator's heart rate, blood pressure, posture, etc. when remotely operating, and transfer the operation authority to another operator (for example, Patent Document 1). [[ID=二十一]] [[ID=二十二]]

Prior Art Documents

Patent Documents

[0005] [[ID=二十九]] [[ID=三十]] [[ID=三十一]]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] [[ID=四十二]] The technology disclosed in Patent Document 1 describes a procedure for handing over operations to another operator if the operator of a remote monitoring device becomes overloaded and their health deteriorates. However, it does not address methods for reducing the excessive workload. If authority is transferred only after the operator's workload becomes excessive, a delay in response may result in a gap in monitoring. Furthermore, if an operator is monitoring and operating a specific autonomous vehicle, they may be unable to assist other autonomous vehicles. In such a case, the behavior of other autonomous vehicles may become unstable.

[0007] This disclosure is made to solve the problems described above. The purpose of this disclosure is to obtain a driver assistance system that reduces the workload of operators who monitor and operate remote monitoring devices, prevents overloads in advance, and enables stable monitoring and operation to continue. [Means for solving the problem]

[0008] The mobile vehicle driving assistance system related to this disclosure is A remote monitoring necessity calculation unit calculates the degree of need for remote monitoring by a remote monitoring device based on the status of the moving object. A remote monitoring feasibility determination unit determines whether remote monitoring is possible for each of the multiple mobile objects based on the degree of remote monitoring required calculated by the remote monitoring necessity calculation unit. A driving method selection unit selects the driving method of the mobile body according to the determination result of whether remote monitoring is possible, determined by the remote monitoring feasibility determination unit, and the degree of necessity for remote monitoring calculated by the remote monitoring necessity calculation unit, and It is equipped with a mobile body drive unit that moves the mobile body based on the travel method selected by the travel method selection unit. [Effects of the Invention]

[0009] According to the mobile vehicle driving support system described herein, the degree of remote monitoring necessity is calculated according to the state of the mobile vehicle, and a determination is made as to whether remote monitoring is possible. Furthermore, the driving method of the mobile vehicle is selected according to the determination result of whether remote monitoring is possible and the degree of remote monitoring necessity, thereby limiting the mobile vehicles to be monitored. This reduces the workload on operators who monitor and operate the remote monitoring device, and provides a driving support system that can prevent overload in advance and continue stable monitoring and operation. [Brief explanation of the drawing]

[0010] [Figure 1] This is a diagram showing the configuration of the operating device according to Embodiment 1. [Figure 2] This is a diagram showing the configuration of the driver assistance system and remote monitoring device according to Embodiment 1. [Figure 3] This is a hardware configuration diagram of the operating device and remote monitoring device according to Embodiment 1. [Figure 4] This is a diagram showing the configuration of the information acquisition unit of the operating device according to Embodiment 1. [Figure 5] This diagram shows the information flow for calculating the monitoring necessity of the driving device and selecting the driving method according to Embodiment 1. [Figure 6] This diagram illustrates the necessity of monitoring the travel area of ​​the driving device according to Embodiment 1. [Figure 7] This diagram illustrates the degree of need for monitoring the surrounding conditions of the operating device according to Embodiment 1. [Figure 8] This diagram illustrates the degree of need for monitoring the occupant status of the driving device according to Embodiment 1. [Figure 9] This diagram illustrates the degree of necessity for monitoring the moving state of the operating device according to Embodiment 1. [Figure 10] This figure shows a first example of calculating the remote monitoring necessity level for the operating device according to Embodiment 1. [Figure 11] This figure shows a second example of calculating the remote monitoring requirement for the operating device according to Embodiment 1. [Figure 12] This figure shows the monitoring screen of the remote monitoring device according to Embodiment 1. [Figure 13]It is a diagram showing the traveling method of the driving device according to Embodiment 1. [Figure 14] It is a first diagram showing the relationship between the monitoring necessity degree and the traveling method of the driving device according to Embodiment 1. [Figure 15] It is a second diagram showing the relationship between the monitoring necessity degree and the traveling method of the driving device according to Embodiment 1. [Figure 16] It is a first flowchart showing the processing of the driving device according to Embodiment 1. [Figure 17] It is a second flowchart showing the processing of the driving device according to Embodiment 1. [Figure 18] It is another configuration diagram of the driving device according to Embodiment 1. [Figure 19] It is another configuration diagram of the driving support system and the remote monitoring device according to Embodiment 1. [Figure 20] It is a configuration diagram of the driving support system according to Embodiment 2. [Figure 21] It is a configuration diagram of the roadside monitoring device according to Embodiment 2.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, the embodiments will be described in detail with reference to the drawings. The drawings are schematically shown, and for the sake of convenience of explanation, the configuration may be omitted or simplified as appropriate. Also, the mutual relationships of the sizes and positions of the configurations shown in different drawings are not necessarily accurately described and can be changed as appropriate. Further, in the following description, the same reference numerals are attached to the same components in the drawings, and their names and functions are also regarded as the same. Therefore, the detailed description thereof may be omitted to avoid duplication.

[0012] 1. Embodiment 1 <Configuration of the Driving Device> Figure 1 is a configuration diagram of the driving device 201 according to Embodiment 1. The driving device 201 is an automatic driving device mounted on a mobile body 200. The term "mobile body" is a concept that includes vehicles, automated guided vehicles, robots, etc., but here we will use a vehicle as an example for explanation. Automatic driving is also called autonomous driving and refers to driving a mobile body to a destination without operation by a driver. In Figure 1, the driving device 201 includes an information acquisition unit 210, a communication unit 205, a remote monitoring necessity calculation unit 203, a driving area monitoring necessity calculation unit 203a, a surrounding situation monitoring necessity calculation unit 203b, an occupant status monitoring necessity calculation unit 203c, a mobile body status monitoring necessity calculation unit 203d, a remote monitoring feasibility inquiry unit 204, a manual operation unit 202, an autonomous control unit 208, a driving method selection unit 206, and a mobile body drive unit 207.

[0013] In the driving device 201, the information acquisition unit 210 acquires driving area information, surrounding situation information, occupant information, and moving object information. From this information, the driving area monitoring necessity calculation unit 203a, the surrounding situation monitoring necessity calculation unit 203b, the occupant status monitoring necessity calculation unit 203c, and the moving object status monitoring necessity calculation unit 203d calculate the respective monitoring necessity. Based on these calculation results, the remote monitoring necessity calculation unit 203 calculates the remote monitoring necessity.

[0014] Then, the remote monitoring feasibility inquiry unit 204, via the communication unit 205, inquires with the remote monitoring feasibility determination unit 104 of the remote monitoring device 100 whether remote monitoring of the mobile body 200 is possible based on the remote monitoring necessity level. The communication unit 205 receives a response to the inquiry, and the driving method selection unit 206 selects the driving method of the mobile body according to the response result and various monitoring necessity levels. Based on the selected driving method, such as autonomous driving or manual driving, the mobile body drive unit 207 drives the mobile body. Autonomous driving means driving by autonomous operation, while manual driving means driving by operation by the occupant or the operator of the remote monitoring device 100.

[0015] The manual control unit 202 consists of a steering wheel, accelerator, brakes, etc., for the occupant of the mobile unit 200 to manually operate the mobile unit 200. The manual control unit 202 can provide a drive signal to the mobile unit drive unit 207 by operating it. The autonomous control unit 208 can provide a drive signal to the mobile unit drive unit 207 when the mobile unit 200 is operating autonomously.

[0016] <Configuration of the driver assistance system and remote monitoring device> Figure 2 is a configuration diagram of the driver assistance system 900 and remote monitoring device 100 according to Embodiment 1. The remote monitoring device 100 monitors multiple autonomously operating mobile vehicles. The remote monitoring device 100 is also referred to as a control center. The remote monitoring device 100 grasps the driving status of autonomously operating mobile vehicles, and for vehicles that require monitoring, the operator monitors them using the monitoring screen 109, and for vehicles that require manual operation, the operator manually operates them using the remote manual operation unit 102.

[0017] Figure 2 shows an example where the remote monitoring device 100 monitors mobile objects 200, 300, and 400. The mobile objects monitored by the remote monitoring device 100 are not limited to these, and it can monitor a wider range of mobile objects.

[0018] The remote monitoring device 100 can communicate with multiple mobile objects and share information via the remote monitoring communication unit 105. When the remote monitoring device 100 receives an inquiry from a mobile object regarding the feasibility of remote monitoring, the remote monitoring feasibility determination unit 104 compares the remote monitoring necessity of the mobile object in question with the remote monitoring necessity of other mobile objects and determines that monitoring is possible within the limits of the remote monitoring device 100's capabilities. The remote monitoring feasibility determination result is then sent back to the requesting mobile object.

[0019] The remote monitoring device 100 has a remote autonomous control unit 108. The remote autonomous control unit 108 can remotely perform autonomous operation of the mobile unit, replacing the autonomous control unit 208 installed in the mobile unit.

[0020] The remote monitoring feasibility determination unit 104 of the remote monitoring device and the driving device 201 of the mobile body 200 share information through communication and cooperate, allowing the driving method selection unit 206 to select the driving method of the mobile body, and based on the selected driving method, the mobile body drive unit 207 to drive the mobile body. In other words, a driving support system 900 for the mobile body is configured.

[0021] <Hardware configuration of the operating device and remote monitoring device> Figure 3 is a hardware configuration diagram of the driving device 201 and remote monitoring device 100 according to Embodiment 1. The hardware configuration shown in Figure 3 can also be applied to driving devices 301 and 401. Here, we will describe the case where it is applied to the driving device 201 as a representative example. In this embodiment, the driving device 201 is an electronic control device mounted on the mobile body 200 to enable autonomous operation of the mobile body 200. Each function of the driving device 201 is realized by the processing circuits provided in the driving device 201. Specifically, the driving device 201 includes, as processing circuits, a arithmetic processing unit 90 (computer) such as a CPU (Central Processing Unit), a storage device 91 that exchanges data with the arithmetic processing unit 90, an input circuit 92 that inputs external signals to the arithmetic processing unit 90, and an output circuit 93 that outputs signals from the arithmetic processing unit 90 to the outside. Each piece of hardware, such as the arithmetic processing unit 90, storage device 91, input circuit 92, and output circuit 93, is connected to each other by a wired network such as a bus or a wireless network.

[0022] The arithmetic processing unit 90 may include an ASIC (Application Specific Integrated Circuit), an IC (Integrated Circuit), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), various logic circuits, and various signal processing circuits. Furthermore, multiple arithmetic processing units 90 of the same or different types may be provided, with each unit performing a portion of the processing. The storage device 91 may include a RAM (Random Access Memory) configured to read and write data from the arithmetic processing unit 90, or a ROM (Read-only Memory) configured to read data from the arithmetic processing unit 90. The storage device 91 may also include non-volatile or volatile semiconductor memory such as flash memory, SSD (Solid State Drive), EPROM, EEPROM, magnetic disks, flexible disks, optical disks, compact disks, minidiscs, DVDs, etc. The input circuit 92 is connected to various sensors, switches, and communication lines, and includes an A / D converter, communication circuit, etc., which inputs the output signals and communication information of these sensors and switches to the arithmetic processing unit 90. The output circuit 93 includes a drive circuit, communication circuit, etc., which outputs control signals from the arithmetic processing unit 90. The interfaces of the input circuit 92 and output circuit 93 may be based on specifications such as CAN (Control Area Network) (registered trademark), Ethernet (registered trademark), USB (Universal Serial Bus) (registered trademark), DVI (Digital Visual Interface) (registered trademark), and HDMI (High-Definition Multimedia Interface) (registered trademark). In addition, communication may be performed by directly connecting the arithmetic processing unit 90 to the communication device 94, separate from the input circuit 92 and output circuit 93.

[0023] Each function of the operating device 201 is realized by the arithmetic processing unit 90 executing software (programs) stored in a storage device 91 such as ROM, and cooperating with other hardware of the operating device 201, such as the storage device 91, input circuit 92, and output circuit 93. Setting data such as thresholds and judgment values ​​used by the operating device 201 are stored in the storage device 91 such as ROM as part of the software (program). Each function of the operating device 201 may be composed of software modules, or it may be composed of a combination of software and hardware.

[0024] <Information acquisition section> Figure 4 is a diagram showing the configuration of the information acquisition unit 210 of the driving device 201 according to Embodiment 1. The information acquisition unit 210 includes a surrounding conditions information acquisition unit 211, a driving area information acquisition unit 212, map information 213, a crew information acquisition unit 214, a mobile object information acquisition unit 215, and an information integration unit 216.

[0025] <Surrounding Situation Information Acquisition Unit> The surrounding environment information acquisition unit 211 is equipped with sensors that grasp the surrounding environment of the moving body 200 and detect surrounding objects. Objects to be detected may include four-wheeled vehicles, two-wheeled vehicles, pedestrians, animals, other moving objects, fallen objects, signs, and other stationary objects. The sensors for grasping the surrounding environment include an image sensor 211a, a radio wave sensor 211b, and an optical sensor 211c.

[0026] The image sensor 211a captures images of objects, and the distance to the object can be calculated from the image data captured within a certain field of view. The size, direction of movement, speed of movement, and attributes of the object can also be obtained from the image data. The image sensor 211a can be a visible light camera, an infrared camera, or the like.

[0027] The radio wave sensor 211b can use millimeter-wave radar (MMWR) or the like, which utilizes the 24-79 GHz frequency band. The radio wave sensor 211b can detect the position of an object and also detect the object's velocity using the Doppler effect.

[0028] The optical sensor 211c can be a laser radar, LiDAR (Light Detection and Ranging), or the like. By irradiating a laser beam within a certain field of view and detecting point cloud data obtained from the reflection of the laser beam from an object, the position and shape of the object can be determined.

[0029] The surrounding environment information acquisition unit 211 receives information from sensors that grasp the surrounding environment, namely the image sensor 211a, the radio wave sensor 211b, and the optical sensor 211c. The surrounding environment information acquisition unit 211 can use sensor fusion technology to detect objects by combining this information. By combining information from multiple types of sensors, noise information can be removed, enabling highly reliable distance measurement, velocity detection, and attribute identification of objects. Furthermore, obstacles may be identified based on reinforcement learning such as deep learning.

[0030] While all of the information from these sensors may be processed by the surrounding environment information acquisition unit 211, information processing may also be performed for each sensor, processing the data acquired by various sensors and transmitting only the position, shape, velocity, and attribute information of the identified object to the surrounding environment information acquisition unit 211. In this way, the processing of sensor information can be distributed, reducing the amount of information that the surrounding environment information acquisition unit 211 processes alone.

[0031] Furthermore, the surrounding situation information acquisition unit 211 may use all of the image sensor 211a, radio wave sensor 211b, and optical sensor 211c, or it may use only some of the sensors. In addition, other sensors such as ultrasonic sensors may be used to understand the surrounding situation. Moreover, it may be combined with other sensors that provide information on traffic flow and weather conditions as sensor information. The surrounding situation information acquisition unit 211 transmits object information such as the position, speed, shape, and attributes of objects from the information received from the various sensors to the information integration unit 216.

[0032] <Travel Area Information Acquisition Unit> The driving area information acquisition unit 212 acquires the position information of the moving object 200 detected by the position detection unit 212a. The position detection unit 212a can calculate its own position using positioning information from a GNSS (Global Navigation Satellite System) that detects position, a distance sensor that detects the rotation speed of the wheels of the moving object 200, and a gyro sensor that detects the acceleration, velocity, angular acceleration, and angular velocity of the moving object 200.

[0033] The driving area information acquisition unit 212 acquires information about the area around the current location from the map information 213 based on the position information of the mobile body 200. The driving area information acquisition unit 212 can obtain information from the map information 213 such as the status of object entry and exit management in the currently driving area, the degree to which it is likely to become a complex situation, and whether it is an area where autonomous driving is difficult, and transmit this information to the information integration unit 216. Alternatively, information about the surrounding conditions of the mobile body 200 acquired by the surrounding conditions information acquisition unit 211 may be reflected in the map information 213 and used.

[0034] <Crew Information Acquisition Department> The occupant information acquisition unit 214 acquires information about the occupant's condition from the driver's seat camera 214a and the biosensor 214b. It can analyze the occupant's condition from biometric information such as the driver's behavior, posture, whether they are away from their seat, blood pressure, body temperature, and heart rate, and transmit the information about the occupant's condition to the information integration unit 216.

[0035] <Mobile Information Acquisition Unit> The mobile information acquisition unit 215 can read information regarding the status of various on-board equipment 220 and various control devices 230 connected to the driving device 201 of the mobile body 200 as mobile body information. Specifically, it acquires information on whether each on-board equipment 220 and each control device 230 of the mobile body 200 is operating normally, and whether there are any signs of abnormality occurring from the controlled variables.

[0036] <Information Integration Department> The information integration unit 216 shares information received from the surrounding situation information acquisition unit 211, the driving area information acquisition unit 212, the occupant information acquisition unit 214, and the mobile body information acquisition unit 215 within the driving device 201, and can also share it with the remote monitoring device 100 via the communication unit 205. The information integration unit 216 may also generate a route to a separately specified destination, determine the course of the mobile body 200, and instruct it to drive autonomously.

[0037] <Calculation of monitoring necessity and selection of driving method> Figure 5 is a diagram showing the information flow for calculating the monitoring requirement and selecting the driving method in the driving device 201 according to Embodiment 1. Based on the information acquired by the information acquisition unit 210, various monitoring requirements are calculated, and these are combined to calculate the driving area monitoring requirement. Then, the feasibility of remote monitoring is inquired about based on the driving area monitoring requirement, and the driving method is selected based on the response result and the driving area monitoring requirement.

[0038] <Requirement level for monitoring the driving area> Figure 6 is a diagram illustrating the driving area monitoring necessity level of the driving device 201 according to Embodiment 1. Based on the information acquired by the driving area information acquisition unit 212 of the information acquisition unit 210, the driving area monitoring necessity level calculation unit 203a calculates the driving area monitoring necessity level. As shown in Figure 6, the monitoring necessity level is calculated in six levels from 0 to 5, depending on the condition of the driving area in which the mobile body 200 travels.

[0039] In areas where the movement of objects such as vehicles and pedestrians around the mobile object 200 is controlled and it is known in advance that monitoring of the mobile object is unnecessary, the monitoring requirement is set to 0. For example, areas where automated guided vehicles (AGVs) travel in factories and warehouses are controlled, so these areas fall under the category of areas where the monitoring requirement is 0.

[0040] In areas where the movement of objects around the mobile device 200 is restricted and the mobile device 200 can travel stably, the monitoring requirement is set to 1. For example, factory grounds and airport premises are areas where the movement of objects is restricted and entry by the general public is prohibited, so they fall under the category of areas with a monitoring requirement of 1.

[0041] In areas where the movement of objects around the mobile vehicle 200 is not restricted and careful driving is required, the monitoring requirement is set to 2. For example, a normal road is an area where the movement of objects is not restricted and careful driving is required for other vehicles and pedestrians, so the monitoring requirement is set to 2.

[0042] In areas where autonomous driving may encounter complex situations that are difficult to handle, the monitoring requirement level is set to 3. For example, on roads and intersections experiencing intermittent severe traffic congestion, and on junctions and merging points where lane changes are difficult due to congestion, the monitoring requirement level is set to 3.

[0043] In areas where the surrounding environment of the mobile vehicle 200 is completely unknown, or in areas where autonomous driving is considered difficult, the monitoring requirement is set to 4. For example, in areas such as open fields and grasslands where roads are not registered on the map, the monitoring requirement is set to 4. In areas where roads are not registered, it is unclear where ditches and pitfalls are located, making autonomous driving difficult.

[0044] In areas where the movement of the mobile unit 200 is prohibited, the monitoring requirement is set to 5. For example, in areas where entry is prohibited, such as military bases, training grounds, and strictly restricted areas, the monitoring requirement is set to 5.

[0045] The criteria shown in Figure 6 for calculating the monitoring necessity level in six stages from 0 to 5, depending on the conditions of the driving area, are illustrative examples. The monitoring necessity levels may be increased or decreased as needed, and the criteria for each stage may be changed. Furthermore, the monitoring necessity level for the driving area may be changed according to the time of day.

[0046] <Level of need for monitoring surrounding conditions> Figure 7 is a diagram illustrating the need for monitoring the surrounding conditions of the driving device 201 according to Embodiment 1. Based on the information acquired by the surrounding conditions information acquisition unit 211 of the information acquisition unit 210, the surrounding conditions monitoring need calculation unit 203b calculates the need for monitoring the surrounding conditions. As shown in Figure 7, the monitoring need is calculated in six levels from 0 to 5, depending on the surrounding conditions in which the mobile body 200 is traveling.

[0047] If no moving objects are detected around the mobile body 200, there is sufficient space for movement in the path of the mobile body 200, the surrounding environment is not complex, and stable movement of the mobile body is expected, the monitoring requirement is set to 0.

[0048] If the number of objects detected around the mobile unit 200 is less than a predetermined number of objects, and the drivable space in the path of the mobile unit 200 is secured with a margin greater than or equal to a predetermined first range, the monitoring requirement is set to 1. This indicates that the unit can operate autonomously without any problems.

[0049] If the number of objects detected around the mobile body 200 is greater than or equal to a predetermined number of objects, and the drivable space in the path of the mobile body 200 is less than a predetermined first range, the monitoring requirement is set to 2. This corresponds to a situation where autonomous driving requires a considerable load to grasp the surrounding moving objects and where careful driving along the path is necessary. For example, the monitoring requirement is set to 2 when traffic is congested due to natural congestion.

[0050] If the area surrounding the mobile object 200 is less than a predetermined second range, which is narrower than the first range, and autonomous driving would be difficult, the monitoring requirement is set to 3. This corresponds to a situation where the mobile object and its surroundings are in a complex situation predetermined for the object's movement. For example, this would apply if the mobile object is caught in a traffic jam due to an accident, or if pedestrians are present in front of or behind the mobile object.

[0051] If the environment or equipment surrounding the mobile unit 200 is detected to be in an emergency state, or if a response to an emergency is required, the monitoring requirement level will be set to 4. For example, if an emergency evacuation order is issued due to a natural disaster, or if an accident occurs nearby, the monitoring requirement level will be set to 4.

[0052] If there is no space around the mobile object 200 that can be driven, the monitoring requirement level is set to 5. The monitoring requirement level is also set to 5 if the mobile object 200 loses its traversable road surface due to a landslide, loses its traversable road surface due to flooding, or is surrounded by a crowd.

[0053] The criteria shown in Figure 7 for calculating the level of monitoring necessity on a scale of 0 to 5 (six levels) depending on the surrounding conditions of the mobile unit 200 are illustrative. The number of monitoring necessity levels may be increased or decreased as needed, and the criteria for determining each level may be changed. Furthermore, the criteria values ​​used for determining the levels may be changed according to the detection accuracy and reliability of the information acquired by the information acquisition unit 210.

[0054] <Requirement level for monitoring crew conditions> Figure 8 is a diagram illustrating the occupant status monitoring necessity level of the driving device 201 according to Embodiment 1. Based on the information acquired by the occupant information acquisition unit 214 of the information acquisition unit 210, the occupant status monitoring necessity level calculation unit 203c calculates the occupant status monitoring necessity level. As shown in Figure 8, the monitoring necessity level is calculated in six levels from 0 to 5 depending on the occupant's status. In Figure 8, the cases of necessity level 1 and necessity level 2 are not defined.

[0055] If there are no issues requiring attention regarding the condition of the occupants of Mobile Unit 200, and the occupants have not requested monitoring, the monitoring requirement level is set to 0. The monitoring requirement level is set to 3 if the occupants' condition is analyzed based on biometric information such as their behavior, posture, whether they are away from their seats, blood pressure, body temperature, and heart rate, and attention is deemed necessary. For example, if a state of distraction in the occupants is detected, the monitoring requirement level will be set to 3.

[0056] If it is determined that there are no occupants on board the mobile unit 200, the monitoring requirement level is set to 4. If there are no occupants who should be on board, or if the mobile unit is operating unmanned, the monitoring requirement level is set to 4.

[0057] If there are crew members on board and it is determined that an emergency situation exists in which the crew members are unable to monitor the surroundings or manually operate the moving vehicle, the monitoring requirement level will be set to 5. Similarly, if an urgent monitoring request is received from the crew members, the monitoring requirement level will also be set to 5.

[0058] The criteria shown in Figure 8, which calculate the level of monitoring required on a scale of 0 to 5 based on the crew's condition, are illustrative examples. The number of monitoring levels may be increased or decreased, and the criteria for each level may be changed as needed.

[0059] <Requirement level for monitoring the status of mobile objects> Figure 9 is a diagram illustrating the necessity level for monitoring the mobile body status of the driving device 201 according to Embodiment 1. The mobile body information acquisition unit 215 acquires information on whether the various in-vehicle equipment 220 and various control devices 230 connected to the driving device 201 of the mobile body 200 are operating normally, and whether there are any signs of abnormality occurring from the controlled variables. Based on the acquired information, the mobile body status monitoring necessity level calculation unit 203d calculates the mobile body status monitoring necessity level. As shown in Figure 9, the monitoring necessity level is calculated in six levels from 0 to 5 depending on the status of the in-vehicle equipment 220 and control devices 230 of the mobile body. In Figure 9, the cases of necessity level 1 and necessity level 2 are not defined.

[0060] If there are no issues requiring attention regarding the status of the mobile unit 200 and it is considered to be operating normally, the monitoring requirement level is set to 0. If attention is deemed necessary based on the control values ​​of the on-board equipment 220 and control device 230 of the mobile unit, or on the abnormality detection information, the monitoring requirement level is set to 3.

[0061] If an abnormality is detected in the in-vehicle equipment 220 and control device 230 connected to the mobile unit 200, and a fail-safe is executed, and operation continues, the monitoring requirement is set to 4. Even if a failure is detected in some of the equipment connected to the mobile unit 200, if control continues in limp-home mode, the monitoring requirement is set to 4.

[0062] The monitoring requirement level is set to 5 if the mobile object is determined to be inoperable or on the verge of becoming inoperable. For example, this would apply to an electric vehicle where the main battery is experiencing a rapid decrease in capacity and is on the verge of losing power.

[0063] The criteria shown in Figure 9 for calculating the monitoring necessity level in six stages from 0 to 5, depending on the status of the equipment connected to the mobile unit 200, are illustrative. The monitoring necessity levels may be increased or decreased, and the criteria for each stage may be changed as needed.

[0064] <Calculation of the need for remote monitoring> The remote monitoring necessity calculation unit 203 shown in Figure 5 calculates the remote monitoring necessity based on the driving area monitoring necessity, surrounding environment monitoring necessity, occupant state monitoring necessity, and mobile body state monitoring necessity calculated by the driving area monitoring necessity calculation unit 203a, the surrounding environment monitoring necessity calculation unit 203b, the occupant state monitoring necessity calculation unit 203c, and the mobile body state monitoring necessity calculation unit 203d. The calculation formula can use a maximum value adoption method (taking the larger value). Alternatively, a method can be used in which each necessity value is multiplied by a predetermined weighting coefficient and then a simple average is taken.

[0065] Figure 10 shows a first example of calculating the remote monitoring requirement for the driving device 201 according to Embodiment 1. Figure 10 shows an example of calculating the remote monitoring requirement using a maximum value adoption method that adopts the maximum values ​​of the driving area monitoring requirement, surrounding condition monitoring requirement, occupant condition monitoring requirement, and mobile body condition monitoring requirement.

[0066] Figure 11 shows a second example of calculating the remote monitoring necessity of the operating device 201 according to Embodiment 1. Figure 11 shows an example in which the remote monitoring necessity is calculated by multiplying each necessity value by a predetermined weighting coefficient and then dividing the total value by 4 to obtain the average value. In this case, the weighting coefficient for the occupant status monitoring necessity is set to be large. The values ​​of the weighting coefficients in Figure 11 are examples, and other values ​​may be set.

[0067] <Determination of whether remote monitoring is possible> As shown in Figure 5, the operating device 201 calculates the remote monitoring necessity using the remote monitoring necessity calculation unit 203, and then requests a remote monitoring feasibility determination from the remote monitoring feasibility determination unit 104 of the remote monitoring device 100 via the remote monitoring feasibility inquiry unit 204. The remote monitoring necessity of the operating device 201 is transmitted via the communication unit 205 of the operating device 201 and the remote monitoring communication unit 105 of the remote monitoring device 100 to request a remote monitoring feasibility determination. After receiving the determination result from the remote monitoring feasibility determination unit 104, the operating device 201 selects the driving method of the mobile body 200 using the driving method selection unit 206 based on the determination result and various monitoring necessity levels.

[0068] The remote monitoring feasibility determination unit 104 of the remote monitoring device 100 receives remote monitoring necessity levels from multiple mobile objects and compares them with the remote monitoring necessity levels of the mobile object currently under management to determine whether remote monitoring is possible. If the number of mobile objects transmitting a predetermined remote monitoring necessity level exceeds a predetermined value, the remote monitoring feasibility determination for any further mobile objects may be determined as impossible. By making such a determination, monitoring requests that exceed the monitoring capacity of the remote monitoring device 100 can be rejected, thereby preventing overload on the remote monitoring device 100.

[0069] As described above, by limiting the number of moving objects to be monitored, the workload of the operator monitoring and operating the remote monitoring device 100 can be reduced, and overload can be prevented in advance. This makes it possible to obtain a driving support system 900 that can continue stable monitoring and operation.

[0070] In this case, if the request for remote monitoring is rejected, the mobile unit 200's driving method is selected by the driving method selection unit 206 based on the determination result of the remote monitoring feasibility determination unit 104 and the various monitoring necessity levels calculated. In this case, the mobile unit can choose autonomous driving without remote monitoring, manual driving by an occupant without remote monitoring, or driving prohibited.

[0071] <Monitoring screen> Figure 12 shows the monitoring screen 109 of the remote monitoring device 100 according to Embodiment 1. The predetermined number of mobile objects that defines the monitoring capacity of the remote monitoring device 100 may be determined based on the number of subwindows that the operator of the remote monitoring device 100 can display on the monitoring screen 109 at one time. Figure 12 shows the case where there are 9 mobile objects that can be monitored simultaneously by providing subwindows on the monitoring screen 109. Mobile objects are displayed in order from those with a high degree of remote monitoring need, and mobile objects with a high degree of remote monitoring need are more emphasized and displayed in a larger subwindow.

[0072] The remote monitoring device 100 displays the mobile object with the highest remote monitoring requirement in sub-window 109a. The next highest remote monitoring requirement is displayed in sub-window 109b. The next highest remote monitoring requirement is displayed in sub-window 109c. Then the next highest remote monitoring requirement is displayed in sub-window 109d. In this way, mobile objects can be monitored by setting sub-windows in order of the need for remote monitoring. Mobile objects with a lower need for remote monitoring are not displayed on the monitoring screen 109, and the remote monitoring feasibility determination unit 104 returns a response indicating that remote monitoring is not possible.

[0073] Furthermore, if an inquiry is received from a mobile object with a higher remote monitoring necessity level than the currently displayed mobile object, it may be determined that remote monitoring of the new mobile object is possible, and the currently displayed mobile object may be notified that remote monitoring is no longer possible. In this way, monitoring can always be continued on the monitoring screen 109 in order of highest remote monitoring necessity level.

[0074] The number and size of subwindows displayed on the monitoring screen 109 are not limited to the example shown in Figure 12, but should be determined according to the number of mobile objects requiring remote monitoring and the degree of remote monitoring required. A maximum number of monitoring objects may also be set for each degree of remote monitoring required.

[0075] Furthermore, by setting a focus coefficient k(n) for each level of remote monitoring necessity, the sum of the products of the focus coefficient k(n) and the number of moving objects at that focus level for each level of remote monitoring necessity can be expressed as the monitoring score (Score moni) using the following formula.

[0076]

number

[0077] The number of monitored moving objects shown on a single monitoring screen 109 may be set so that the monitoring score (Score moni) does not exceed a predetermined monitoring score threshold. The maximum number of monitored moving objects, or the monitoring score threshold, may be set uniformly for each area where the monitored moving objects are located. Alternatively, different values ​​may be set for each operator based on their knowledge and experience when monitoring and operating using the monitoring screen 109.

[0078] If the remote monitoring device 100 has multiple monitoring screens 109 and operators who monitor each of them and operate the mobile objects, the number of mobile objects that can be monitored is set by multiplying the maximum number of subwindows displayed on the monitoring screen 109 by the number of monitoring screens.

[0079] When an operator of the remote monitoring device 100 communicates with the occupant and remotely controls a specific mobile object, the maximum number of mobile objects the operator can monitor and the monitoring score threshold may be changed. In addition, when a mobile object with a high remote monitoring need is added to the monitoring target, the operator may be notified to alert them through a display change, voice guidance, beep, vibration, etc.

[0080] Monitoring of moving objects in the remote monitoring device 100 may be performed by an operator and the monitoring system. The determination of whether remote monitoring is possible by the remote monitoring feasibility determination unit 104 may be set based on the processing performance of the monitoring system. The processing capacity of the monitoring system may be calculated based on the processing speed of the CPU, the number of cores, and the cache memory capacity of the computer that the monitoring system has, and remote monitoring may be determined to be possible within the range that does not exceed this processing capacity.

[0081] The processing load required for a single mobile object may vary depending on the situation. The number of mobile objects that can be monitored may also fluctuate based on the processing resources currently in use. Taking these factors into consideration, it may be advisable to recalculate the number of mobile objects that can be remotely monitored each time.

[0082] <Driving method> Figure 13 shows the driving method of the driving device 201 according to Embodiment 1. The driving method selection unit 206 selects the driving method of the mobile body 200 based on the determination result and various monitoring requirements from the remote monitoring feasibility determination unit 104.

[0083] The driving mode can be selected from three options: autonomous driving, where the vehicle travels to its destination without operator intervention; manual driving, where the vehicle is operated manually by the operator; or remote manual driving, where the vehicle is operated manually by an operator of a remote monitoring device. If none of these options are selected, the vehicle will be disabled (stopped). Furthermore, by adding the presence or absence of monitoring by the operator or remote monitoring device to these driving modes, nine different driving modes can be selected, as shown in Figure 13.

[0084] In Figure 13, autonomous driving is selected as you move upwards, and manual driving is selected as you move downwards. Remote monitoring is selected as you move to the right, and crew monitoring is selected as you move to the left. The autonomous driving method listed at the top is labeled [Autonomous Driving]. In this case, the mobile unit 200 is autonomously controlled by the autonomous control unit 208 of the driving device 208 and drives autonomously. At this time, crew monitoring and monitoring by an operator using the monitoring screen 109 of the remote monitoring device 100 are not required.

[0085] The crew-supervised autonomous driving system on the second row from the top, on the left, is labeled as [Autonomous Driving + Crew Monitoring]. In this case, the mobile unit 200 is autonomously controlled by the autonomous control unit 208 of the driving device 208, and is also required to be monitored by the crew of the mobile unit 200. The crew is required to monitor the autonomous driving status of the mobile unit and be prepared to switch to manual driving at any time. In this case, monitoring by an operator using the monitoring screen 109 of the remote monitoring device 100 is not required.

[0086] The autonomous driving system with crew and remote monitoring, located in the center of the second row from the top, is labeled as [Autonomous Driving + Crew and Remote Monitoring]. In this case, the mobile unit 200 is autonomously controlled by the autonomous control unit 208 of the driving device 208, and further monitoring by the crew of the mobile unit 200 and by the operator via the monitoring screen 109 of the remote monitoring device 100 are required. The crew and operator are required to monitor the autonomous driving status of the mobile unit and be prepared to switch to manual driving at any time.

[0087] The remote monitoring autonomous driving system on the second row from the top, to the right, is labeled as [Autonomous Driving + Remote Monitoring]. In this case, the mobile unit 200 is autonomously controlled by the autonomous control unit 208 of the driving device 208, and further monitoring by an operator is required via the monitoring screen 109 of the remote monitoring device 100. The operator is required to monitor the autonomous driving status of the mobile unit and be prepared to switch to manual driving at any time. In this case, monitoring by an onboard occupant is not required.

[0088] The third row from the top, on the left, is labeled as "[Manual Crew Operation]". In this case, the mobile unit 200 is manually operated by a crew member operating the manual control unit 202. In this case, the crew member operating the unit manually will also perform monitoring, but this is obvious, so the description of crew monitoring is omitted. In this case, monitoring by an operator using the monitoring screen 109 of the remote monitoring device 100 is not required.

[0089] The remote monitoring and crew-operated driving system, second from the left in the third row from the top, is labeled as [crew-operated driving + remote monitoring]. In this case, the mobile unit 200 is manually driven by the crew operating the manual control unit 202, and further monitoring by the operator is required via the monitoring screen 109 of the remote monitoring device 100. In this case, the operator is required to monitor the state of manual driving by the crew of the mobile unit and be prepared to switch to remote manual driving at any time using the remote manual control unit 102. In this case, the crew operating the unit manually will also perform monitoring, but this is considered obvious, so the description of crew monitoring is omitted.

[0090] The third remote manual driving system from the left in the third row from the top is labeled as [Remote Manual Driving + Crew Monitoring]. In this case, the mobile unit 200 is manually driven by the operator of the remote monitoring device 100 operating the remote manual operation unit 102, and further monitoring by the crew is required. In this case, the crew is required to monitor the state of remote manual driving by the operator of the mobile unit and be prepared to switch to crew manual driving at any time using the manual operation unit 202. In this case, the operator performing the remote manual operation will also monitor the mobile unit 200, but this is taken for granted, so the description of remote monitoring is omitted.

[0091] The remote manual driving method on the far right of the third row from the top is labeled as [Remote Manual Driving]. In this case, the mobile unit 200 is manually driven by the operator of the remote monitoring device 100 operating the remote manual operation unit 102. In this case, the operator performing the remote manual operation will also monitor the mobile unit 200, but this is a given, so the description of remote monitoring is omitted. In this case, monitoring by the occupant is not required.

[0092] The bottom line reads "[No Driving]". In this case, the mobile unit 200 is prohibited from driving and remains stationary, or if it is moving, it is changed to a stationary state. Alternatively, it may be moved quickly to the nearest roadside and stopped via autonomous driving, manual driving by an occupant, or remote manual driving, and driving may be prohibited.

[0093] In the above description, autonomous driving was assumed to be performed using the autonomous control unit 208 of the driving device 208. However, autonomous driving may also be performed by the remote autonomous control unit 108 of the remote monitoring device 100, without using the autonomous control unit 208 of the driving device 208.

[0094] <Selection of driving method when remote monitoring is possible> Figure 14 is the first diagram showing the relationship between the monitoring requirement level and the driving method of the driving device 201 according to Embodiment 1. The selection of the driving method based on various monitoring requirements when the remote monitoring feasibility determination unit 104 determines that remote monitoring is possible will be explained in Figure 14.

[0095] The driving mode selection unit 206 selects a driving mode based on the driving area monitoring requirement, the surrounding environment monitoring requirement, the occupant status monitoring requirement, and the mobile body status monitoring requirement. The relationship between these various monitoring requirements and the remote monitoring requirement is shown in Figure 11, where the relationship is obtained from the average value of the weighted multiplication result when each monitoring requirement reaches the highest value that satisfies the requirements. The necessity of occupant monitoring and the remote monitoring requirement are appropriately set for each driving mode.

[0096] Row No. 1 indicates that [Autonomous Driving] is selected when all monitoring requirements—driving area monitoring requirement, surrounding environment monitoring requirement, occupant status monitoring requirement, and mobile body status monitoring requirement—are 0. In this case, there is no monitoring request for the occupant, and the remote monitoring requirement is 0. In this case, since the remote monitoring requirement is 0, there is no need to actually query the remote monitoring feasibility determination unit 104 for remote monitoring feasibility. Even in this case, if the load on the remote monitoring device 100 is small, it is not necessary to interfere with remote monitoring.

[0097] In row No. 2, except in the case of row No. 1, if the requirement for monitoring the driving area is 1 or less, the requirement for monitoring surrounding conditions is 1 or less, the requirement for monitoring the occupant's condition is 0, and the requirement for monitoring the moving body's condition is 0, then [autonomous driving + occupant monitoring] is selected. In this case, there is a request for monitoring the occupant, and the requirement for remote monitoring is 1. Although the requirement for remote monitoring is 1, remote monitoring is not actually required. Even in this case, if the load on the remote monitoring device 100 is small, it is not necessary to interfere with remote monitoring.

[0098] In row No. 3, except in the case of the row above, if the requirement for monitoring the driving area is 2 or less, the requirement for monitoring the surrounding conditions is 2 or less, the requirement for monitoring the occupant's condition is 0, and the requirement for monitoring the moving body's condition is 0, then [Occupant manual driving] is selected. In this case, there is a requirement for monitoring the occupant, and the requirement for remote monitoring is 1. Although the requirement for remote monitoring is 1, remote monitoring is not actually required. Even in this case, if the load on the remote monitoring device 100 is small, it is not necessary to interfere with remote monitoring.

[0099] In row No. 4, excluding the cases in the higher row, if the requirement for monitoring the driving area is 3 or less, the requirement for monitoring surrounding conditions is 3 or less, the requirement for monitoring the occupant's condition is 0, and the requirement for monitoring the moving body's condition is 0, then [autonomous driving + occupant / remote monitoring] is selected. In this case, there is a requirement for monitoring the occupant, and the requirement for remote monitoring is 3.

[0100] In row No. 5, excluding the case of the row above, if the requirement for monitoring the driving area is 1 or less, the requirement for monitoring surrounding conditions is 1 or less, the requirement for monitoring the occupant's condition is 3 or less, and the requirement for monitoring the condition of the moving object is 3 or less, then [manual occupant driving + remote monitoring] is selected. In this case, there is a requirement for monitoring the occupant, and the requirement for remote monitoring is 2.

[0101] In row No. 6, excluding the case of the row above, if the requirement for monitoring the driving area is 1 or less, the requirement for monitoring surrounding conditions is 1 or less, the requirement for monitoring the occupant's condition is 4, and the requirement for monitoring the mobile body's condition is 3 or less, then [autonomous driving + remote monitoring] is selected. In this case, there is no requirement for monitoring the occupant, and the requirement for remote monitoring is 4.

[0102] In row No. 7, excluding the case of the row above, if the requirement for monitoring the driving area is 2 or less, the requirement for monitoring surrounding conditions is 2 or less, the requirement for monitoring the occupant's condition is 3 or less, and the requirement for monitoring the condition of the moving object is 3 or less, then [remote manual driving + occupant monitoring] is selected. In this case, there is a requirement for monitoring the occupant, and the requirement for remote monitoring is 4.

[0103] In row No. 8, except in the case of the row above, if the requirement for monitoring the driving area is 2 or less, the requirement for monitoring the surrounding conditions is 2 or less, the requirement for monitoring the occupant's condition is 4, and the requirement for monitoring the moving body's condition is 3 or less, then [remote manual driving] is selected. In this case, there is no requirement for monitoring the occupant, and the requirement for remote monitoring is 4.

[0104] In row No. 9, excluding the case of the row above, if the requirement for monitoring the driving area is 3 or less, the requirement for monitoring surrounding conditions is 3 or less, the requirement for monitoring the occupant's condition is 3 or less, and the requirement for monitoring the condition of the moving object is 3 or less, then [remote manual driving + occupant monitoring] is selected. In this case, there is a requirement for monitoring the occupant, and the requirement for remote monitoring is 5.

[0105] In row No. 10, except in the case of the row above, if the requirement for monitoring the driving area is 3 or less, the requirement for monitoring surrounding conditions is 3 or less, the requirement for monitoring the occupant's condition is 4, and the requirement for monitoring the moving body's condition is 3 or less, then [remote manual driving] is selected. In this case, there is no requirement for monitoring the occupant, and the remote monitoring requirement is 5.

[0106] In row No. 11, excluding the case of the row above, [No Driving] is selected. In this case, the remote monitoring requirement is 5.

[0107] <Selection of driving method when remote monitoring is not possible> Figure 15 is a second diagram showing the relationship between the monitoring requirement level and the driving method of the driving device 201 according to Embodiment 1. Figure 15 explains the selection of driving methods based on various monitoring requirements when the remote monitoring feasibility determination unit 104 determines that remote monitoring is not possible.

[0108] The driving method selection unit 206 selects a driving method based on the driving area monitoring requirement, the surrounding environment monitoring requirement, the occupant status monitoring requirement, and the mobile body status monitoring requirement. The relationship between these various monitoring requirements and the remote monitoring requirement is shown in Figure 11, where the relationship is obtained from the average value of the weighted multiplication result when each monitoring requirement reaches the highest value that satisfies the requirements.

[0109] The difference between Figure 15 and Figure 14 is that it does not allow the selection of a driving method that includes remote monitoring. If remote monitoring is not available, then of course, remote manual driving cannot be performed either. The load on the remote monitoring device 100 is high, making remote monitoring by the operator using the monitoring screen 109 impossible, so naturally, remote manual operation by the operator using the remote manual operation unit 102 cannot be performed either.

[0110] Therefore, remote monitoring is omitted from the driving modes No. 4, No. 5, and No. 6 in Figure 15. Also, in Figure 14, No. 7 was [remote manual driving + occupant monitoring], but in Figure 15, No. 7 has been changed to [autonomous driving + occupant monitoring]. Due to the high load on the remote monitoring device 100, the parts where remote monitoring is impossible will be backed up by the functions of the autonomous control unit 208 of the driving device 201. Furthermore, by setting the driving modes No. 8 to No. 10 to [driving prohibited], the reliability of the driving assistance system is maintained and appropriate responses are ensured.

[0111] If the mobile unit 200 is changed from a state where it can be remotely monitored to a state where it cannot be remotely monitored by the remote monitoring feasibility determination unit 104, and the remote monitoring necessity level is 4 or higher, then [No Driving] may be selected. The current driving method may be notified or displayed to the occupants. If a monitoring request is made to the occupants, then a notification may be given to the occupants instructing them to monitor the area ahead.

[0112] <Operation of the operating device> Figure 16 is a first flowchart showing the processing of the operating device 201 according to Embodiment 1. Figure 17 is a second flowchart showing the processing of the operating device 201. Figure 17 shows the processing that follows Figure 16. The processing shown in Figure 16 is executed by the arithmetic processing unit of the operating device 201. This processing may be executed at predetermined intervals (for example, every 1 ms). It may also be executed in response to events such as receiving data by the communication unit 205 or acquiring new information by the information acquisition unit 210, rather than at predetermined intervals.

[0113] The process shown in Figure 16 is initiated, and in step S101, the travel area information acquisition unit 212 acquires information about the area around the current position. In step S102, the surrounding situation information acquisition unit 211 grasps the surrounding situation of the moving body 200 and detects surrounding objects.

[0114] In step S103, the crew information acquisition unit 214 acquires information regarding the status of the crew. In step S104, the mobile body information acquisition unit 215 acquires information regarding the status of the on-board equipment 220 and control device 230 connected to the driving device 201 of the mobile body 200.

[0115] In step S105, the driving area monitoring necessity calculation unit 203a calculates the driving area monitoring necessity based on the driving area information. In step S106, the surrounding condition monitoring necessity calculation unit 203b calculates the surrounding condition monitoring necessity based on the surrounding condition information.

[0116] In step S107, the crew status monitoring necessity calculation unit 203c calculates the crew status monitoring necessity based on the crew status information. In step S108, the mobile body status monitoring necessity calculation unit 203d calculates the mobile body status monitoring necessity based on the mobile body status information.

[0117] In step S109, the remote monitoring necessity calculation unit 203 calculates the remote monitoring necessity based on the driving area monitoring necessity, surrounding condition monitoring necessity, occupant condition monitoring necessity, and mobile body condition monitoring necessity. In step S110, it is confirmed whether remote monitoring is necessary. Whether remote monitoring is necessary may also be confirmed by whether the calculated remote monitoring necessity is equal to or greater than a predetermined remote monitoring necessity threshold (for example, 2 or more).

[0118] In step S111, determine whether remote monitoring is necessary. If remote monitoring is necessary (determination is YES), proceed to step S112. If remote monitoring is not necessary (determination is NO), proceed to step S114.

[0119] In step S112, the remote monitoring feasibility inquiry unit 204 requests a remote monitoring feasibility determination unit 104 of the remote monitoring device 100 to determine whether remote monitoring is possible, along with the remote monitoring necessity data. In step S113, the determination result is obtained from the remote monitoring feasibility determination unit 104.

[0120] In step S114, the driving method is selected by the driving method selection unit 206 based on the remote monitoring feasibility determination result, the need for driving area monitoring, the need for surrounding condition monitoring, the need for occupant condition monitoring, and the need for mobile body condition monitoring. In step S115, a drive instruction is transmitted to the mobile body drive unit 207 based on the driving method to drive the mobile body. The process then ends.

[0121] <Another configuration> Figure 18 is another configuration diagram of the driving device 201 according to Embodiment 1. Figure 19 is another configuration diagram of the driving support system 900 and remote monitoring device 100 according to Embodiment 1. In the configuration diagrams shown in Figures 1, 2, and 5, the remote monitoring feasibility determination unit 104 is provided in the remote monitoring device 100. The driving device 201 is provided with a remote monitoring necessity calculation unit 203, a driving area monitoring necessity calculation unit 203a, a surrounding conditions monitoring necessity calculation unit 203b, an occupant condition monitoring necessity calculation unit 203c, and a mobile body condition monitoring necessity calculation unit 203d.

[0122] In Figures 1, 2, and 5, the remote monitoring feasibility inquiry unit 204 queries the remote monitoring feasibility determination unit 104 of the remote monitoring device 100 for a determination of whether remote monitoring is possible. The configuration according to Embodiment 1 is not limited to this. As shown in Figure 18, the driving device 201 may be provided with only an information acquisition unit 210, and the information acquired by the driving area information acquisition unit 212, the surrounding situation information acquisition unit 211, the occupant information acquisition unit 214, and the mobile body information acquisition unit 215 may be transmitted to the remote monitoring device 100 via the communication unit 205.

[0123] As shown in Figure 19, the remote monitoring device 100 may be equipped with a remote monitoring necessity calculation unit 103, a driving area monitoring necessity calculation unit 103a, a surrounding conditions monitoring necessity calculation unit 103b, an occupant status monitoring necessity calculation unit 103c, a mobile body status monitoring necessity calculation unit 103d, and a driving method selection unit 106. In this way, the remote monitoring device 100 can calculate various monitoring necessity levels and then calculate the driving area monitoring necessity level by combining these.

[0124] This allows the remote monitoring device 100 to perform the process of determining whether remote monitoring is possible based on various monitoring requirements, and then selecting a driving method based on the determination result and the various monitoring requirements. The remote monitoring device 100 then transmits the selected driving method to the driving unit, which only needs to transmit a drive signal to the mobile body drive unit based on the received driving method. This limits the processing content of the driving unit 201 and reduces the processing load on the driving unit 201. In this way, it becomes unnecessary to equip the driving unit 201 with a high-speed and high-capacity information processing device, contributing to cost reduction and weight reduction of the driving unit 201.

[0125] The operating device 201 and remote monitoring device 100 in Figures 1 and 2, and the operating device 201 and remote monitoring device 100 in Figures 18 and 19, are similar in that the remote monitoring feasibility determination unit 104 is located in the remote monitoring device 100 and the mobile body drive unit 207 is located in the mobile body 200. Other functions may be located in the operating device 201, the remote monitoring device 100, or anywhere else. If the operating device 201 and the remote monitoring device 100 are connected by high-speed communication, the functions can be moved to any location. Therefore, information processing resources can be appropriately allocated, and the information processing equipment provided in the operating device 201 can be simplified, contributing to cost reduction and weight reduction of the operating device 201.

[0126] 2. Embodiment 2 Figure 20 is a configuration diagram of the driver assistance system 900 according to Embodiment 2. Figure 21 is a configuration diagram of the roadside monitoring device 810 according to Embodiment 2.

[0127] Embodiment 2 differs only in that a roadside monitoring device 810 and a roadside monitoring device 820 are added to the configuration of the driver assistance system 900. The roadside monitoring device is also called an RSU (Road Side Unit). The roadside monitoring device 810, which is installed around the road, has an image sensor 811a, a radio wave sensor 811b, and an optical sensor 811c that monitor surrounding objects. The roadside monitoring devices 810 and 820 may be installed at and around intersections. The roadside monitoring devices 810 and 820 may be arranged around intersections so that blind spots are minimized.

[0128] By using the information transmitted from the surrounding situation information acquisition unit 811 of the roadside monitoring device 810 via the information integration unit 816 and the roadside monitoring communication unit 805, the driving devices 201, 301, 401 and the remote monitoring device 100 can acquire multifaceted information, thereby enabling the construction of a more reliable driving support system. By utilizing the information from the roadside monitoring device 810, the state of the moving object can be detected with high accuracy, allowing the driving method of the moving object to be appropriately selected according to the determination result of whether remote monitoring is possible and the degree of remote monitoring necessity. Therefore, the moving objects to be monitored by the remote monitoring device 100 can be limited. This reduces the workload on the operator who monitors and operates the remote monitoring device, prevents overload in advance, and enables a driving support system that can continue stable monitoring and operation.

[0129] While this disclosure describes various exemplary embodiments and examples, the various features, aspects, and functions described in one or more embodiments are not limited to the application of a particular embodiment, but are applicable individually or in various combinations to the embodiments. Accordingly, countless variations not illustrated are envisioned within the scope of the art disclosed herein. These include, for example, modifying, adding or omitting at least one component, or even extracting at least one component and combining it with components of other embodiments.

[0130] The various aspects of this disclosure are summarized below as an appendix.

[0131] (Note 1) A remote monitoring necessity calculation unit calculates the degree of need for remote monitoring by a remote monitoring device based on the status of the moving object. A remote monitoring feasibility determination unit determines whether remote monitoring is possible for each of the multiple mobile bodies based on the degree of need for remote monitoring calculated by the remote monitoring necessity calculation unit. A driving method selection unit selects the driving method of the mobile body according to the determination result of whether remote monitoring is possible or not determined by the remote monitoring feasibility determination unit and the degree of necessity of remote monitoring calculated by the remote monitoring necessity calculation unit, and A driving support system for a mobile body, comprising a mobile body drive unit that drives the mobile body based on the driving method selected by the driving method selection unit. (Note 2) The remote monitoring feasibility determination unit determines whether the mobile body can be remotely monitored based on the maximum number of mobile bodies that can be remotely monitored by the remote monitoring device, in the mobile body driving support system described in Appendix 1. (Note 3) The remote monitoring feasibility determination unit determines whether the mobile body can be remotely monitored based on the maximum number of mobile bodies that can be displayed on the screen of the remote monitoring device, in the mobile body driving support system according to Appendix 1 or 2. (Note 4) The remote monitoring feasibility determination unit determines whether the mobile body can be remotely monitored based on the maximum processing capacity of the remote monitoring device, according to any one of the appendices 1 to 3. (Note 5) A driving area monitoring necessity calculation unit calculates the driving area monitoring necessity according to the difficulty of driving in the area in which the aforementioned moving object travels. A surrounding environment monitoring necessity calculation unit calculates the need for surrounding environment monitoring according to the conditions around the moving object. A crew status monitoring necessity calculation unit calculates the crew status monitoring necessity according to the state of the crew of the mobile body, and The system includes a mobile body condition monitoring necessity calculation unit that calculates the degree of necessity for mobile body condition monitoring according to the state of the machine part of the mobile body, A driving support system for a mobile body according to any one of the appendices 1 to 4, wherein the remote monitoring necessity calculation unit calculates the remote monitoring necessity based on the driving area monitoring necessity calculated by the driving area monitoring necessity calculation unit, the surrounding area monitoring necessity calculated by the surrounding area monitoring necessity calculation unit, the occupant condition monitoring necessity calculated by the occupant condition monitoring necessity calculation unit, and the mobile body condition monitoring necessity calculated by the mobile body condition monitoring necessity calculation unit. (Note 6) The driving assistance system for a mobile body as described in Appendix 5, wherein the remote monitoring necessity calculation unit calculates a calculation criterion for calculating the remote monitoring necessity based on the driving area monitoring necessity calculated by the driving area monitoring necessity calculation unit, the surrounding area monitoring necessity calculated by the surrounding area monitoring necessity calculation unit, the occupant state monitoring necessity calculated by the occupant state monitoring necessity calculation unit, and the mobile body state monitoring necessity calculated by the mobile body state monitoring necessity calculation unit, by inferring and setting the calculation criterion based on a plurality of predetermined combination examples of the driving area monitoring necessity, the surrounding area monitoring necessity, the occupant state monitoring necessity, the mobile body state monitoring necessity, and the remote monitoring necessity. (Note 7) A driving support system for a mobile body according to Appendix 5 or 6, wherein the driving method selection unit selects the driving method of the mobile body based on the determination result of whether remote monitoring is possible or not determined by the remote monitoring feasibility determination unit, the driving area monitoring necessity level calculated by the driving area monitoring necessity level calculation unit, the surrounding area monitoring necessity level calculated by the surrounding area monitoring necessity level calculation unit, the occupant state monitoring necessity level calculated by the occupant state monitoring necessity level calculation unit, and the mobile body state monitoring necessity level calculated by the mobile body state monitoring necessity level calculation unit. (Note 8) A driving support system for a mobile body according to any one of the appendices 5 to 7, wherein the driving method selection unit sets the driving method of the mobile body to a remote driving method in which the mobile body is operated and driven by the remote monitoring device when the occupant status monitoring necessity calculated by the occupant status monitoring necessity calculation unit is higher than a predetermined occupant status monitoring necessity threshold, and the remote monitoring necessity calculated by the remote monitoring necessity calculation unit is higher than a predetermined remote operation necessity threshold. (Note 9) The remote monitoring feasibility determination unit determines that remote monitoring is possible when the degree of need for remote monitoring calculated by the remote monitoring necessity calculation unit is higher than a predetermined remote monitoring necessity threshold, in the mobile vehicle driving support system according to any one of the appendices 1 to 8. (Note 10) The remote monitoring feasibility determination unit determines that remote monitoring is possible for multiple mobile bodies in order of priority based on the degree of need for remote monitoring calculated by the remote monitoring necessity calculation unit, according to any one of the mobile body driving support systems described in Appendix 1 to 9. (Note 11) The remote monitoring device is a driving support system for a mobile body as described in any one of the appendices 1 to 10, which displays on the screen the remote monitoring necessity level for a plurality of mobile bodies, calculated by the remote monitoring necessity level calculation unit, in descending order of necessity level. (Note 12) The remote monitoring device is a mobile vehicle driving support system according to any one of the appendices 1 to 11, which performs remote monitoring processing for multiple mobile vehicles in order of the degree of need for remote monitoring calculated by the remote monitoring need calculation unit. (Note 13) The remote monitoring device is a mobile vehicle driving support system according to any one of the appendices 1 to 12, which notifies the operator when an additional mobile vehicle to be remotely monitored is added. (Note 14) The driving system for a mobile body described in any one of the following items, selected by the driving method selection unit: an autonomous driving method in which the mobile body drives autonomously without requiring the monitoring of the occupant or the remote monitoring; an occupant-supervised autonomous driving method in which the mobile body drives autonomously with the monitoring of the occupant; a remote monitoring autonomous driving method in which the mobile body drives autonomously with the monitoring of the remote monitoring; an occupant-remote monitoring autonomous driving method in which the mobile body drives autonomously with the monitoring of the occupant and the remote monitoring; an occupant-manual driving method in which the mobile body is operated by the occupant without the remote monitoring; a remote manual driving method in which the mobile body is operated by the remote monitoring device without the monitoring of the occupant; a remote monitoring occupant-manual driving method in which the mobile body is operated by the occupant with the monitoring of the remote monitoring device; and a driving stop method in which the mobile body stops driving. (Note 15) The remote monitoring feasibility determination unit is located in the remote monitoring device. The aforementioned mobile body drive unit is a mobile body driving support system according to any one of the appendices 1 to 14, which is arranged on the mobile body. [Explanation of Symbols]

[0132] 100 Remote monitoring device, 102 Remote manual operation unit, 104 Remote monitoring feasibility determination unit, 200, 300, 400 Mobile unit, 201 Driving device, 203 Remote monitoring necessity calculation unit, 203a Driving area monitoring necessity calculation unit, 203b Surroundings monitoring necessity calculation unit, 203c Occupant status monitoring necessity calculation unit, 203d Mobile unit status monitoring necessity calculation unit, 206 Driving method selection unit, 207 Mobile unit drive unit, 900 Driving support system

Claims

1. A remote monitoring necessity calculation unit calculates the degree of need for remote monitoring by a remote monitoring device based on the status of the moving object. A remote monitoring feasibility determination unit determines whether remote monitoring is possible for each of the multiple mobile bodies based on the degree of need for remote monitoring calculated by the remote monitoring necessity calculation unit. A driving method selection unit selects the driving method of the mobile body according to the determination result of whether remote monitoring is possible or not determined by the remote monitoring feasibility determination unit and the degree of necessity of remote monitoring calculated by the remote monitoring necessity calculation unit, and A driving support system for a mobile body, comprising a mobile body drive unit that drives the mobile body based on the driving method selected by the driving method selection unit.

2. The remote monitoring feasibility determination unit determines whether the mobile body can be remotely monitored based on the maximum number of mobile bodies that can be remotely monitored by the remote monitoring device, in the mobile body driving support system according to claim 1.

3. The remote monitoring feasibility determination unit determines whether the moving body can be remotely monitored based on the maximum number of moving bodies that can be displayed on the screen of the remote monitoring device, according to the driving support system for a moving body according to claim 1.

4. The remote monitoring feasibility determination unit determines whether the mobile body can be remotely monitored based on the maximum processing capacity of the remote monitoring device that can perform the monitoring process, according to the mobile body driving support system according to claim 1.

5. A driving area monitoring necessity calculation unit calculates the driving area monitoring necessity according to the difficulty of driving in the area in which the aforementioned moving object travels. A surrounding environment monitoring necessity calculation unit calculates the need for surrounding environment monitoring according to the conditions around the moving object. A crew status monitoring necessity calculation unit calculates the crew status monitoring necessity according to the state of the crew of the mobile body, and The system includes a mobile body condition monitoring necessity calculation unit that calculates the degree of necessity for mobile body condition monitoring according to the state of the machine part of the mobile body, The driving support system for a mobile body according to claim 1, wherein the remote monitoring necessity calculation unit calculates the remote monitoring necessity based on the driving area monitoring necessity calculated by the driving area monitoring necessity calculation unit, the surrounding area monitoring necessity calculated by the surrounding area monitoring necessity calculation unit, the occupant state monitoring necessity calculated by the occupant state monitoring necessity calculation unit, and the mobile body state monitoring necessity calculated by the mobile body state monitoring necessity calculation unit.

6. The driving assistance system for a mobile body according to claim 5, wherein the remote monitoring necessity calculation unit calculates a calculation criterion for calculating the remote monitoring necessity based on the driving area monitoring necessity calculated by the driving area monitoring necessity calculation unit, the surrounding area monitoring necessity calculated by the surrounding area monitoring necessity calculation unit, the occupant state monitoring necessity calculated by the occupant state monitoring necessity calculation unit, and the mobile body state monitoring necessity calculated by the mobile body state monitoring necessity calculation unit, by inferring and setting the calculation criterion based on a plurality of predetermined combination examples of the driving area monitoring necessity, the surrounding area monitoring necessity, the occupant state monitoring necessity, the mobile body state monitoring necessity, and the remote monitoring necessity.

7. The driving assistance system for a mobile body according to claim 5, wherein the driving method selection unit selects the driving method of the mobile body based on the determination result of whether remote monitoring is possible or not determined by the remote monitoring feasibility determination unit, the driving area monitoring necessity level calculated by the driving area monitoring necessity level calculation unit, the surrounding area monitoring necessity level calculated by the surrounding area monitoring necessity level calculation unit, the occupant state monitoring necessity level calculated by the occupant state monitoring necessity level calculation unit, and the mobile body state monitoring necessity level calculated by the mobile body state monitoring necessity level calculation unit.

8. The driving support system for a mobile body according to claim 5, wherein the driving method selection unit sets the driving method of the mobile body to a remote manual driving method in which the remote monitoring device operates and drives the mobile body when the occupant status monitoring necessity calculated by the occupant status monitoring necessity calculation unit is higher than a predetermined occupant status monitoring necessity threshold, and the remote monitoring necessity calculated by the remote monitoring necessity calculation unit is higher than a predetermined remote operation necessity threshold.

9. The remote monitoring feasibility determination unit determines that remote monitoring is possible when the degree of need for remote monitoring calculated by the remote monitoring necessity calculation unit is higher than a predetermined remote monitoring necessity threshold, in the mobile vehicle driving support system according to claim 1.

10. The remote monitoring feasibility determination unit determines that remote monitoring is possible for a plurality of mobile bodies in order of priority based on the degree of need for remote monitoring calculated by the remote monitoring necessity calculation unit, according to the mobile body driving support system according to claim 1.

11. The remote monitoring device displays on the screen, in descending order of the remote monitoring necessity level calculated by the remote monitoring necessity level calculation unit, the driving support system for a mobile body according to claim 1.

12. The remote monitoring device performs remote monitoring processing on a plurality of mobile bodies in order of the degree of need for remote monitoring calculated by the remote monitoring need calculation unit, according to claim 1, which is the driving support system for a mobile body.

13. The remote monitoring device notifies the operator when an additional mobile object to be remotely monitored is added, as described in claim 1 of the mobile object driving support system.

14. The driving support system for a mobile body according to claim 1, wherein the driving mode selection unit selects one of the following: an autonomous driving mode in which the mobile body drives autonomously without requiring the monitoring of the occupant or the remote monitoring; an occupant-supervised autonomous driving mode in which the mobile body drives autonomously with the monitoring of the occupant; a remote monitoring autonomous driving mode in which the mobile body drives autonomously with the remote monitoring; an occupant- and remote monitoring autonomous driving mode in which the mobile body drives autonomously with the monitoring of the occupant and the remote monitoring; an occupant-manual driving mode in which the mobile body is operated by the occupant without the remote monitoring; a remote manual driving mode in which the mobile body is operated by the remote monitoring device without the monitoring of the occupant; a remote monitoring occupant-manual driving mode in which the mobile body is operated by the occupant with the remote monitoring; an occupant-supervised remote manual driving mode in which the mobile body is operated by the remote monitoring device with the monitoring of the occupant; and a driving stop mode in which the mobile body stops driving.

15. The remote monitoring feasibility determination unit is located in the remote monitoring device. The mobile body drive unit is arranged on the mobile body, and the mobile body driving support system is according to any one of claims 1 to 14.

Images