Remote control device, remote control system, and remote control method
The remote control device adjusts vehicle operations to prevent overlap and ensure accurate tracking by creating control values that maintain non-overlapping positional relationships, addressing the challenge of multiple vehicles in the sensor's field of view.
Patent Information
- Application Number
- JP2023086687
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-05-26
AI Technical Summary
Existing vehicle tracking systems using cameras or lidars installed remotely face challenges in accurately tracking the location of vehicles when multiple vehicles overlap, leading to potential inaccuracies in positional tracking.
A remote control device and system that utilizes sensors to acquire vehicle information, detects overlapping vehicles, and creates control values to adjust the traveling operations of vehicles to prevent overlap, ensuring accurate positional tracking by establishing a non-overlapping positional relationship.
The system effectively reduces the likelihood of positional overlap and enhances the accuracy of vehicle tracking by adjusting vehicle operations to maintain clear detection paths, even when multiple vehicles are within the sensor's field of view.
Smart Images

Figure 2024169868000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a remote control device, a remote control system, and a remote control method. [Background technology]
[0002] Conventionally, vehicles that run automatically by remote control are known (Patent Document 1). In this technology, the running of the vehicle is tracked by a monitoring system such as a camera or a lidar installed in a location different from the vehicle. In this way, if the vehicle malfunctions, the operation of the vehicle can be remotely controlled before any disruption to the running of the vehicle occurs. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Special Publication No. 2017-538619 Summary of the Invention [Problem to be solved by the invention]
[0004] When tracking the position of a vehicle using an image captured by a camera installed in a different location from the vehicle, if two or more vehicles overlap on the captured image, the position of each vehicle may not be tracked accurately. Similarly, when tracking the position of a vehicle using LIDAR information captured by a LIDAR installed in a different location from the vehicle, if two or more vehicles overlap within the detection range of the LIDAR, the position of each vehicle may not be tracked accurately. [Means for solving the problem]
[0005] The present disclosure can be realized in the following forms.
[0006] (1) According to a first aspect of the present disclosure, there is provided a remote control device. A remote control device that remotely controls the driving operation of at least some of a plurality of vehicles includes an acquisition unit that acquires sensor information representing at least some of the plurality of vehicles acquired by a sensor installed at a location different from the plurality of vehicles, a detection unit that detects the vehicles included in the sensor information, a control value creation unit that uses the sensor information to create a control value that specifies the driving operation of at least some of the vehicles among the plurality of vehicles, and a transmission unit that transmits the control value to a control target vehicle among the plurality of vehicles that is to be controlled for its driving operation, wherein the control value creation unit creates an evacuation control value as the control value that specifies the driving operation of at least one of the two or more vehicles in at least one of a first case in which two or more of the plurality of vehicles are overlapped in the sensor information, and a second case in which the two or more vehicles are expected to be overlapped when the sensor information is acquired before the sensor information is acquired, so that a detection target vehicle that is to be detected by the sensor information among the two or more vehicles is in a positional relationship that can be detected by the detection unit without overlapping with other vehicles among the plurality of vehicles. According to this embodiment, in at least either the first case or the second case, it is possible to create an evacuation control value for establishing a positional relationship in which the detection target vehicle among two or more vehicles can be detected without overlapping with other vehicles. Then, it is possible to transmit the created evacuation control value to the control target vehicle. As a result, when tracking the position of the vehicle using sensor information representing the vehicle acquired by a sensor installed at a location different from the vehicle, it is possible to reduce the possibility that two or more vehicles are present overlapping on the sensor information. Therefore, it is possible to more accurately track the position of the vehicle using the sensor information. (2) In the above embodiment, the first case is a case where a first vehicle and a second vehicle among the multiple vehicles overlap in the sensor information, and the second case is a case where the first vehicle and the second vehicle are expected to overlap when the sensor information is acquired before the sensor information is acquired, and the remote control device may further include an actual distance calculation unit that calculates an actual distance between the first vehicle and the second vehicle, and the evacuation control value may be the control value that specifies the traveling operation of at least one of the first vehicle and the second vehicle so that the actual distance becomes a target distance that is longer than the actual distance at a time when at least one of the first case and the second case occurs. According to this embodiment, the actual distance between the first vehicle and the second vehicle that overlap or are expected to overlap in the sensor information can be calculated. Then, the evacuation control value that specifies the traveling operation of at least one of the first vehicle and the second vehicle can be created so that the actual distance between the first vehicle and the second vehicle becomes a target distance that is longer than the actual distance at a time when at least one of the first case and the second case occurs. This makes it possible to more reliably reduce the possibility that the first vehicle and the second vehicle will overlap on the sensor information when tracking the positions of the first vehicle and the second vehicle using the sensor information. (3) In the above embodiment, the target distance may be set according to the external shape of the first vehicle and the external shape of the second vehicle, and the larger the external shape of the first vehicle and the external shape of the second vehicle, the longer the target distance, and the smaller the external shape of the first vehicle and the external shape of the second vehicle, the shorter the target distance. According to this embodiment, the target distance can be set according to the external shape of the first vehicle and the external shape of the second vehicle. In this way, the target distance can be set according to the external shape of the vehicle that is correlated with the area occupied by the vehicle in the sensor information. This makes it possible to suppress a decrease in the calculation accuracy of the vehicle position according to the type of vehicle. In addition, it is possible to suppress a situation where the vehicle position cannot be tracked according to the type of vehicle. (4) In the above aspect, the second vehicle may be a vehicle traveling behind the first vehicle in the same forward direction as the first vehicle, and the avoidance control value may include at least one of an acceleration control value that increases the acceleration of the first vehicle in the forward direction, an acceleration control value that decreases the acceleration of the second vehicle in the forward direction, a direction control value that causes the second vehicle to move backward in a direction opposite to the forward direction, and a stop control value that causes the traveling operation of the second vehicle to stop. According to this aspect, when the second vehicle travels behind the first vehicle in the same forward direction as the first vehicle, it is possible to create an avoidance control value that includes an acceleration control value that increases the acceleration of the first vehicle in the forward direction. As a result, it is possible to change the actual distance between the first vehicle and the second vehicle to the target distance by controlling the acceleration of the first vehicle. Also, according to this aspect, it is possible to create an avoidance control value that includes an acceleration control value that decreases the acceleration of the second vehicle in the forward direction. As a result, it is possible to change the actual distance between the first vehicle and the second vehicle to the target distance by controlling the acceleration of the second vehicle. Also, according to this embodiment, it is possible to create an evacuation control value including a direction control value for causing the second vehicle to move backward in a direction opposite to the forward direction. As a result, the actual distance between the first vehicle and the second vehicle can be changed to the target distance by causing the second vehicle to move backward in a direction opposite to the forward direction. Also, according to this embodiment, it is possible to create an evacuation control value including a stop control value for stopping the traveling operation of the second vehicle. As a result, it is possible to change the actual distance between the first vehicle and the second vehicle to the target distance by causing the second vehicle to stop. (5) In the above embodiment, the avoidance control value may further include a steering angle control value that defines a steering angle of the control target vehicle. According to this embodiment, it is possible to create an avoidance control value that includes a steering angle control value that defines a steering angle of the vehicle in addition to at least one of an acceleration control value, a direction control value, and a stop control value. This makes it possible to correct a position deviation from a predetermined target driving route in the process of changing the actual distance between the first vehicle and the second vehicle to the target distance. (6) In the above aspect, the avoidance control value may include a steering angle control value that specifies a steering angle of the control target vehicle in order to move either the first vehicle or the second vehicle in the width direction of the road. According to this aspect, it is possible to create an avoidance control value that includes a steering angle control value for moving either the first vehicle or the second vehicle in the width direction of the road. This makes it possible to change the actual distance between the first vehicle and the second vehicle to a target distance by moving either the first vehicle or the second vehicle in the width direction of the road. (7) In the above embodiment, the avoidance control value may further include an acceleration control value that specifies the acceleration of the control target vehicle. According to this embodiment, it is possible to create an avoidance control value that includes an acceleration control value in addition to a steering angle control value for moving either the first vehicle or the second vehicle in the width direction of the road. This makes it possible to adjust the moving speed when moving either the first vehicle or the second vehicle in the width direction of the road. (8) In the above embodiment, the second vehicle is the vehicle traveling behind the first vehicle in the same forward direction as the first vehicle, and the avoidance control value is the control value for moving either the first vehicle or the second vehicle in the width direction of the road to cause the second vehicle to overtake the first vehicle, and may include an acceleration control value that specifies the acceleration of the control target vehicle and a steering angle control value that specifies the steering angle of the control target vehicle. According to this embodiment, when the second vehicle travels behind the first vehicle in the same forward direction as the first vehicle, the following avoidance control value can be created. In this case, it is possible to create an avoidance control value that includes an acceleration control value and a steering angle control value for moving either the first vehicle or the second vehicle in the width direction of the road to cause the second vehicle to overtake the first vehicle. Thereby, the actual distance between the first vehicle and the second vehicle can be changed to a target distance by causing the second vehicle to overtake the first vehicle. (9) In the above embodiment, the avoidance control value may include a trajectory control value that defines a driving trajectory of the controlled vehicle by arranging target driving positions of the controlled vehicle for each predetermined time in chronological order. According to this embodiment, it is possible to create an avoidance control value that includes a trajectory control value that defines a driving trajectory of the controlled vehicle by arranging target driving positions of the controlled vehicle for each predetermined time in chronological order. This makes it possible to define the driving trajectory of the controlled vehicle without defining detailed driving operations such as acceleration and steering angle. (10) In the above aspect, a determination unit may be provided that determines whether the second case applies depending on the type of each of the two or more vehicles that are expected to be included in the sensor information when the sensor information is acquired. According to this aspect, it is possible to determine whether the second case applies depending on the type of each of the two or more vehicles that are expected to be included in the sensor information when the sensor information is acquired. This makes it possible to more reliably determine whether the second case applies. (11) In the above aspect, a determination unit may be provided that determines whether the second case applies based on the exterior shapes of the two or more vehicles that differ depending on the type of the vehicle and are expected to be included in the sensor information when the sensor information is acquired. According to this aspect, it is possible to determine whether the second case applies based on the exterior shapes of the two or more vehicles that differ depending on the type of the vehicle and are expected to be included in the sensor information when the sensor information is acquired. This makes it possible to more reliably determine whether the second case applies. (12) According to a second aspect of the present disclosure, a remote control system is provided. The remote control system includes a plurality of vehicles, a sensor that is installed at a location different from the plurality of vehicles and acquires sensor information representing at least some of the plurality of vehicles, and the remote control device according to the above aspect. According to this aspect, in at least either the first or second case, it is possible to create an evacuation control value for establishing a positional relationship in which a detection target vehicle among two or more vehicles can be detected without overlapping with other vehicles. Then, it is possible to transmit the created evacuation control value to the control target vehicle. As a result, when tracking the position of a vehicle using sensor information representing the vehicle acquired by a sensor installed at a location different from the vehicle, it is possible to reduce the possibility that two or more vehicles are overlapped on the sensor information. Therefore, it is possible to more accurately track the position of the vehicle using the sensor information. (13) According to a third aspect of the present disclosure, there is provided a remote control method. A remote control method for remotely controlling the driving operation of at least some of a plurality of vehicles includes an acquisition step of acquiring sensor information representing at least some of the plurality of vehicles, the sensor information being acquired by a sensor installed at a location different from the plurality of vehicles; a detection step of detecting the vehicles included in the sensor information; a control value creation step of creating a control value that specifies the driving operation of at least some of the vehicles of the plurality of vehicles using the sensor information; and a transmission step of transmitting the control value to a control target vehicle of the plurality of vehicles whose driving operation is to be controlled, wherein the control value creation step creates an evacuation control value as the control value that specifies the driving operation of at least one of the two or more vehicles in at least one of a first case in which two or more of the plurality of vehicles are overlapped in the sensor information and a second case in which the two or more vehicles are expected to be overlapped when the sensor information is acquired before the sensor information is acquired, so that a detection target vehicle that is to be detected by the sensor information among the two or more vehicles is in a positional relationship that can be detected in the detection step without overlapping with other vehicles among the plurality of vehicles. According to this embodiment, in at least either the first case or the second case, it is possible to create an evacuation control value for establishing a positional relationship in which the detection target vehicle among two or more vehicles can be detected without overlapping with other vehicles. Then, it is possible to transmit the created evacuation control value to the control target vehicle. As a result, when tracking the position of the vehicle using sensor information representing the vehicle acquired by a sensor installed at a location different from the vehicle, it is possible to reduce the possibility that two or more vehicles are present overlapping on the sensor information. Therefore, it is possible to more accurately track the position of the vehicle using the sensor information. The present disclosure can be realized in various forms other than the above remote control device, remote control system, and remote control method, for example, in the form of a manufacturing method for a remote control device, a control method for a remote control device and a remote control system, a computer program for realizing the control method, a non-transitory recording medium on which the computer program is recorded, etc. [Brief description of the drawings]
[0007] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a remote control system. [Diagram 2] FIG. 2 is a diagram showing a schematic configuration of a remote control device. [Diagram 3] FIG. 2 is a diagram showing details of a CPU mounted on the remote control device. [Figure 4] FIG. 11 is a diagram showing an example of a method for setting a target distance. [Diagram 5] 6 is a flowchart showing a method for calculating the position of a detection target vehicle using a captured image. [Figure 6] 6A to 6C are schematic diagrams showing examples of various images when the position calculation method shown in FIG. 5 is executed. [Figure 7] FIG. 11 is a diagram for explaining details of a coordinate point calculation step. [Figure 8] FIG. 4 is a diagram for explaining a method of acquiring a base coordinate point. [Figure 9] FIG. 1 is a diagram for explaining details of a position conversion process. [Figure 10] FIG. 2 for explaining the details of the position conversion process. [Figure 11] 4 is a first flowchart showing a remote control method in the first embodiment. [Figure 12] 5 is a second flowchart showing the remote control method in the first embodiment. [Figure 13] 6 is a third flowchart showing the remote control method in the first embodiment. [Figure 14] FIG. 4 is a diagram for explaining a method for determining vehicle classification. [Figure 15] 10 is a first flowchart showing a remote control method according to the second embodiment. [Figure 16] 10 is a second flowchart showing the remote control method in the second embodiment. [Figure 17] 10 is a third flowchart showing the remote control method in the second embodiment. [Figure 18] 13 is a first flowchart showing a remote control method according to the third embodiment. [Figure 19] 13 is a second flowchart showing the remote control method in the third embodiment. [Figure 20] 13 is a third flowchart showing a remote control method according to the third embodiment. [Figure 21] 13 is a first flowchart showing a remote control method according to the fourth embodiment. [Figure 22] 20 is a second flowchart showing the remote control method in the fourth embodiment. [Figure 23] FIG. 13 is a diagram showing details of a CPU mounted on a remote control device in a fifth embodiment. [Figure 24] 11 is a diagram showing a method for calculating the position of a detected vehicle using external lidar information. [Diagram 25] 13 is a first flowchart showing a remote control method according to the sixth embodiment. [Figure 26] 23 is a second flowchart showing the remote control method in the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] A. First embodiment: A-1. Remote control system configuration: 1 is a diagram showing a schematic configuration of a remote control system 1. The remote control system 1 includes one or more vehicles 10, a remote control device 5 that remotely controls the traveling operations of at least some of the vehicles 10, and one or more types of sensors (hereinafter, external sensors 9) installed in locations different from the multiple vehicles 10.
[0009] The external sensor 9 acquires information (hereinafter, sensor information) representing at least some of the vehicles 10 among the multiple vehicles 10. Specifically, the external sensor 9 acquires bird's-eye view information indicating the state of the vehicle 10 and the surrounding area of the vehicle 10. In this embodiment, the external sensor 9 is an imaging device 90. The imaging device 90 acquires an imaged image by capturing an image of the detection range RG including the vehicle 10 from outside the vehicle 10. The imaging device 90 transmits the captured image to the remote control device 5 together with camera identification information for identifying the multiple imaging devices 90 and the acquisition time of the captured image. The captured image transmitted to the remote control device 5 is used to calculate the position of the vehicle 10. The captured image is a two-dimensional image in which pixels arranged on the XcYc plane of the camera coordinate system are collected. The camera coordinate system is a coordinate system having the focal point of the imaging device 90 as the origin and coordinate axes indicated by the Xc axis and the Yc axis perpendicular to the Xc axis. The captured image includes at least two-dimensional data of the vehicle 10, which is a target vehicle whose position is to be calculated. The captured image is preferably a color image, but may be a gray image. The imaging device 90 is, for example, a camera having an imaging element such as a CCD image sensor or a CMOS image sensor, and an optical system.
[0010] In this embodiment, the imaging device 90 obtains an image of the lane 6 and the vehicle 10 traveling on the lane 6 from above. The installation positions and the number of the imaging devices 90 are determined in consideration of the detection range RG (angle of view) of the imaging device 90, etc., in order to image the entire lane 6 by one or more imaging devices 90. Specifically, each imaging device 90 is installed so that the first detection range RG1, which is the detection range RG of the adjacent first imaging device 901, and the second detection range RG2, which is the detection range RG of the second imaging device 902, overlap. Furthermore, each imaging device 90 is installed at a position where it can image a positioning point 10e that is set in advance for a specific part of the vehicle 10 traveling on the lane 6. In this embodiment, the positioning point 10e is the rear end of the left side of the vehicle 10 (hereinafter, the left rear end). Note that the positioning point 10e may be other than the left rear end of the vehicle 10. Further, the imaging device 90 is not limited to acquiring information from above the vehicle 10, but may acquire information from the front, rear, sides, etc. of the vehicle 10.
[0011] The vehicle 10 has a manned driving mode and a remote unmanned driving mode. In the manned driving mode, the driving conditions of the vehicle 10 are generated by the driver in the vehicle 10 operating input devices such as a steering wheel and an accelerator provided on the vehicle 10. As a result, the vehicle 10 runs according to the generated driving conditions. The driving conditions are conditions that define the driving operation of the vehicle 10. The driving conditions include, for example, the driving route, position, driving speed, acceleration, and steering angle of the wheels of the vehicle 10. In the remote unmanned driving mode, the vehicle 10 receives the driving conditions of the vehicle 10 from outside the vehicle 10 and runs based on the received driving conditions. The remote unmanned driving mode has a remote manual driving mode and a remote automatic driving mode. In the remote manual driving mode, the driving conditions of the vehicle 10 are generated by the operator operating an operator input device provided in a place different from the vehicle 10. As a result, the vehicle 10 receives the driving conditions generated by the operator input device and runs according to the received driving conditions. In the remote automatic driving mode, the remote control device 5 provided at a location different from the vehicle 10 creates control values that define the driving operation of the vehicle 10 and transmits them to the vehicle 10. As a result, the vehicle 10 receives the control values and performs automatic driving according to the received control values.
[0012] The vehicle 10 travels in a remote automatic driving mode, for example, in a factory where the vehicle 10 is produced by executing a plurality of production processes. The factory is not limited to being located in one building, or at one site or address, but may be located across multiple buildings, multiple sites, multiple addresses, etc. In this case, the vehicle 10 may travel on public roads, not limited to private roads. The vehicle 10 may also travel outside the factory in the remote automatic driving mode.
[0013] The vehicle 10 may be, for example, an electric vehicle, a hybrid vehicle, a fuel cell vehicle, a gasoline vehicle, or a diesel vehicle. The vehicle 10 may be a private vehicle such as a passenger car, or a commercial vehicle such as a truck, a bus, or a construction vehicle. The vehicle 10 may be at least one of a finished product, a semi-finished product, and a work-in-progress product.
[0014] The vehicle 10 includes a driving device 110 that accelerates the vehicle 10, a steering device 120 that changes the traveling direction of the vehicle 10, and a braking device 130 that decelerates the vehicle 10. The vehicle 10 further includes a vehicle communication unit 140 for communicating with external devices using wireless communication or the like, and a vehicle control device 150. The external devices are devices other than the vehicle 10, such as the remote control device 5 and the imaging device 90, and other vehicles 10. The vehicle communication unit 140 is, for example, a wireless communication device. The vehicle communication unit 140 communicates with external devices connected to the network N, for example, via an access point in a factory. The vehicle control device 150 includes a CPU, a storage unit, and an input / output interface. The CPU, the storage unit, and the input / output interface are connected to each other via an internal bus or an interface circuit. The input / output interface communicates with internal devices such as the driving device 110 mounted on the vehicle 10. The input / output interface is connected to the vehicle communication unit 140 so as to be able to communicate with the vehicle. The configuration of the vehicle 10 is not limited to the above. In addition, at least a part of the functions of the vehicle control device 150 may be realized as one function of the remote control device 5 or the external sensor 9.
[0015] FIG. 2 is a diagram showing a schematic configuration of the remote control device 5. The remote control device 5 uses sensor information to control the traveling operation of the vehicle 10. In this embodiment, the remote control device 5 uses a captured image, which is sensor information acquired by an imaging device 90 as an external sensor 9, to calculate the position of the detection target vehicle 10a included in the captured image. The detection target vehicle 10a is the vehicle 10 to be detected by the sensor information. The detection target vehicle 10a may be set in advance, or may be set appropriately based on at least one of data generated in the process of calculating the position of the vehicle 10 and the calculation result of the position of the vehicle 10. In this embodiment, the detection target vehicle 10a is the vehicle 10 at least a part of which is present within a predetermined recognition target area. The recognition target area is, for example, an area in which a grid line 61 is drawn as shown in FIG. 6 described later. The remote control device 5 adopts the position of the predetermined positioning point 10e of the detection target vehicle 10a as the position of the detection target vehicle 10a. Then, the remote control device 5 creates a control value that specifies the driving operation of the control-target vehicle 10b using the calculated driving conditions such as the position of the detection-target vehicle 10a. The control-target vehicle 10b is a vehicle 10 included in the sensor information and is a vehicle 10 whose driving operation is to be controlled. The control-target vehicle 10b and the detection-target vehicle 10a may be the same or different. Then, the remote control device 5 transmits the created control value to the control-target vehicle 10b. As a result, the remote control device 5 controls the driving operation of the control-target vehicle 10b included in the captured image.
[0016] The remote control device 5 includes a remote communication unit 51, a storage unit 53, and a CPU 52. The remote communication unit 51, the storage unit 53, and the CPU 52 are connected to each other via an internal bus or an interface circuit. The remote communication unit 51 communicatively connects the vehicle control device 150 and the imaging device 90 to the remote control device 5. The remote communication unit 51 is, for example, a wireless communication device.
[0017] The storage unit 53 of the remote control device 5 stores various information including various programs for controlling the operation of the remote control device 5, a position detection model Md, a distortion correction parameter Pa1, a perspective transformation parameter Pa2, a camera database D1, and a vehicle model database D2. The storage unit 53 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).
[0018] The position detection model Md is a trained machine learning model used to identify the position of the detection target vehicle 10a included in the captured image. In this embodiment, the position detection model Md is a machine learning model that has been trained in advance to mask the vehicle 10 in the input image by inputting either the captured image or various images obtained by performing various processes on the captured image. As an algorithm of the position detection model Md, for example, a deep neural network (hereinafter, DNN) having a structure of a convolutional neural network (hereinafter, CNN) that realizes semantic segmentation and instance segmentation is used. As an example of a DNN used as the algorithm of the position detection model Md, there is a DNN that performs instance segmentation such as YOLACT++. Note that the configuration of the position detection model Md is not limited to the above. The position detection model Md may be a trained machine learning model that uses an algorithm other than a neural network, for example.
[0019] The distortion correction parameter Pa1 is a parameter used when correcting distortion of a captured image. The perspective transformation parameter Pa2 is a parameter used when performing perspective transformation on a first mask image. The first mask image is an image in which a mask area is added to the vehicle 10 in an image input to the position detection model Md by masking the vehicle 10 included in the image. The correction parameter Pa1 and the perspective transformation parameter Pa2 will be described in detail later.
[0020] The camera database D1 is a database that indicates, for each imaging device 90, imaging parameters calculated based on the installation position of the imaging device 90 in a global coordinate system. In this embodiment, the imaging parameter is the height H of the imaging device 90 from the road surface 60 (FIG. 9 described later), that is, the height H with respect to the road surface 60 on which the detection target vehicle 10a is located. The camera database D1 is, for example, a database that associates camera identification information with imaging parameters for the imaging device 90 identified by the camera identification information. Note that the configuration of the camera database D1 is not limited to the above. In the camera database D1, the detection range RG, installation position, installation angle, etc. of the imaging device 90 may be associated with the camera identification information.
[0021] The vehicle model database D2 is a database including shape information. The shape information is information indicating the exterior shape of the vehicle 10 for each type of the vehicle 10. The shape information is, for example, information linking vehicle model identification information for identifying multiple types of the vehicle 10 with the exterior shape of the vehicle 10 of the type specified by the vehicle model identification information. The exterior shape of the vehicle 10 differs for each type of the vehicle 10 depending on, for example, the body shape of the vehicle 10 and the vehicle rank (also called "vehicle body") determined by the overall length, width, and height of the vehicle 10. Note that the configuration of the vehicle model database D2 is not limited to the above. At least a part of the various information stored in the storage unit 53 may be stored in a storage unit such as the vehicle control device 150 or the imaging device 90.
[0022] 3 is a diagram showing details of the CPU 52 mounted on the remote control device 5. The CPU 52 deploys various programs stored in the storage unit 53, thereby functioning as an acquisition unit 521, a position calculation unit 522, a position conversion unit 523, an actual distance calculation unit 524, a determination unit 525, a control value creation unit 526, and a transmission unit 527.
[0023] The acquisition unit 521 acquires various information. For example, the acquisition unit 521 acquires sensor information acquired by the external sensor 9. In this embodiment, the acquisition unit 521 acquires a captured image acquired by the imaging device 90 serving as the external sensor 9 as the sensor information. In addition, the acquisition unit 521 acquires imaging parameters for the imaging device 90 that is the source of the captured image to be analyzed, for example, by referring to the camera database D1 stored in the storage unit 53.
[0024] The position calculation unit 522 calculates image coordinate points using the captured image. The image coordinate points are coordinate points indicating the position of the detection target vehicle 10a in an image coordinate system described later. The position calculation unit 522 includes a distortion correction unit 522a, a rotation processing unit 522b, a crop processing unit 522c, a detection unit 522d, a deletion unit 522e, a perspective transformation unit 522f, and a coordinate point calculation unit 522g. The distortion correction unit 522a generates a corrected image by correcting the distortion of the captured image. The rotation processing unit 522b generates a rotated image by rotating the corrected image so that the direction of a vector indicating the movement direction of the detection target vehicle 10a (hereinafter, a movement vector) faces a predetermined direction. The crop processing unit 522c deletes from the rotated image, among the regions of the rotated image, regions (hereinafter, unnecessary regions) other than the vehicle region corresponding to the detection target vehicle 10a and a region including a predetermined peripheral region of the detection target vehicle 10a (hereinafter, necessary regions). As a result, the crop processing unit 522c generates a processed image in which a necessary area is cut out from the rotated image. In this embodiment, when the detection target vehicle 10a moves a distance exceeding a predetermined threshold, the crop processing unit 522c deletes from the rotated image a moved area corresponding to the distance moved by the detection target vehicle 10a, which is an unnecessary area. As a result, the crop processing unit 522c generates a processed image in which an unmoved area including the detection target vehicle 10a, which is a necessary area, is cut out from the rotated image. The detection unit 522d detects vehicles included in the processed image using the position detection model Md, and generates a first mask image. When multiple vehicles 10 are included in the first mask image, the deletion unit 522e deletes mask areas of vehicles 10 other than the detection target vehicle 10a (hereinafter, non-target vehicles 10f) from the first mask image. The non-target vehicle 10f is, for example, a vehicle 10 that is included in the captured image or is expected to be included in the captured image, and in which the positioning point 10e cannot be detected because another vehicle 10 overlaps the captured image so as to cover the positioning point 10e. The non-target vehicle 10f may also be, for example, a vehicle 10 that exists outside the recognition target area. The perspective transformation unit 522f generates a second mask image by performing perspective transformation on the first mask image. The coordinate point calculation unit 522g corrects the first coordinate point using the second coordinate point to calculate the image coordinate point.The first coordinate point is a coordinate point in the image coordinate system of a specified vertex of a first circumscribing rectangle set in a mask area in a first mask image. The second coordinate point is a coordinate point in the image coordinate system of a vertex that indicates the same position as the first coordinate point, among the vertices of a second circumscribing rectangle set in a mask area in a second mask image. Note that the configuration of the position calculation unit 522 is not limited to the above.
[0025] The position conversion unit 523 converts the image coordinate points into vehicle coordinate points by using the imaging parameters acquired by the acquisition unit 521 and the image coordinate points calculated by the position calculation unit 522. The vehicle coordinate points are coordinate points that indicate the position of the detection target vehicle 10a in the global coordinate system.
[0026] The actual distance calculation unit 524 calculates the actual distance (hereinafter, actual distance L1) between the first vehicle 101 and the second vehicle 102 as the vehicle 10 included in the sensor information. The actual distance calculation unit 524 calculates the actual distance L1 between the first vehicle 101 and the second vehicle 102, for example, by using information acquired by an on-board sensor mounted on the first vehicle 101 and the second vehicle 102. The on-board sensor is, for example, an on-board camera, an on-board radar, or an on-board lidar. In the example shown in FIG. 1, the second vehicle 102 runs behind the first vehicle 101 in the same lane as the first vehicle 101 and in the same traveling direction, but this is not limited to the above. The second vehicle 102 may be a vehicle 10 running ahead of the first vehicle 101, or may be a vehicle 10 running in a lane different from the first vehicle 101. In addition, the calculation method of the actual distance L1 between the first vehicle 101 and the second vehicle 102 is not limited to the above. The actual distance calculation unit 524 may calculate the actual distance L1 between the first vehicle 101 and the second vehicle 102 by, for example, analyzing an image such as the second mask image. The actual distance calculation unit 524 may also calculate the actual distance L1 between the first vehicle 101 and the second vehicle 102 by, for example, using the vehicle coordinate points of the first vehicle 101 and the second vehicle, their traveling speeds, the elapsed time from the time when the captured images used to calculate the vehicle coordinate points were acquired, and the like.
[0027] The determination unit 525 determines whether or not the situation corresponds to at least one of the first case and the second case. In the first case, two or more vehicles 10 among the multiple vehicles are present overlapping in the sensor information. The determination unit 525, for example, analyzes the sensor information acquired by the acquisition unit 521 to determine whether or not the situation corresponds to the first case. When the sensor information is a captured image as in this embodiment, the determination unit 525 determines whether or not two or more vehicles 10 are present overlapping by using, for example, an overlap detection model. The overlap detection model is a machine learning model that has been trained in advance so that it can determine whether or not two or more vehicles 10 are present overlapping in the input image by inputting either a captured image or various images obtained by performing various processes on the captured image. As an algorithm for the overlap detection model, for example, a DNN having a CNN structure that realizes instance segmentation is used. The overlap detection model may be realized as one function of the position detection model Md. The determination unit 525 may determine whether or not the situation corresponds to the first case by a method other than the above method. The determination unit 525 may determine whether or not the first case applies by using, for example, at least one of data generated in the process of calculating the position of the vehicle 10 and the calculation result of the position of the vehicle 10.
[0028] The second case is a case where two or more vehicles 10 are expected to overlap when the sensor information is acquired before the sensor information is acquired. The determination unit 525 determines whether or not the second case applies depending on the types of two or more vehicles 10 expected to be included in the sensor information when the sensor information is acquired, for example. Specifically, the determination unit 525 first determines whether or not two or more vehicles 10 are expected to be included in the sensor information when the sensor information is acquired, based on driving conditions such as the driving speed of each vehicle 10 and the position at a predetermined time. Then, when the determination unit 525 predicts that two or more vehicles 10 will be included in the sensor information, it specifies vehicle type identification information indicating the types of each of the two or more vehicles 10 expected to be included in the sensor information. At this time, the determination unit 525 specifies the types of each of the two or more vehicles 10 expected to be included in the sensor information, for example, using schedule information. The schedule information is information indicating which type of vehicle 10 is scheduled to travel at which time and at which position on a predetermined target travel route. Then, the determination unit 525 acquires shape information of the vehicle 10 of the type specified by the vehicle type identification information. At this time, the projected area of each vehicle 10 from the imaging direction of the imaging device 90, that is, the area occupied by each vehicle 10 on the captured image Im1, is correlated with the external shape of the vehicle 10. For example, when the sensor information is acquired, the higher the vehicle rank, the higher the possibility that two or more vehicles 10 exist overlapping in the sensor information, and the lower the vehicle rank, the lower the possibility that two or more vehicles 10 exist overlapping in the sensor information. Therefore, the determination unit 525 uses the difference in the external shape according to the type of the vehicle 10 to determine whether or not the second case applies. Specifically, the determination unit 525 determines that the second case applies when, for example, any of the total values of the overall length, vehicle width, and vehicle height of each vehicle 10 is equal to or greater than a predetermined vehicle rank threshold value. On the other hand, when any of the total values of the overall length, width, and height of each vehicle 10 is less than the vehicle rank threshold value, the determination unit 525 determines that the second case does not apply.
[0029] The determination unit 525 may determine whether the vehicle 10 corresponds to the second case by a method different from the above method. For example, the determination unit 525 may determine that the vehicle 10 corresponds to the second case when an integrated value, which is the sum of the total values of the overall length, vehicle width, and vehicle height of each vehicle 10, is equal to or greater than a predetermined integrated threshold value. In this case, the determination unit 525 determines that the vehicle 10 does not correspond to the second case when the integrated value is less than the integrated threshold value. In this manner, it is possible to determine whether the vehicle 10 corresponds to the second case. The determination unit 525 may also determine whether the vehicle 10 corresponds to the second case by using information about the imaging device 90, such as the detection range RG, installation position, and installation angle of the imaging device 90, in addition to the above information. In this manner, it is possible to more accurately determine whether the vehicle 10 corresponds to the second case.
[0030] The determination unit 525 may also specify the types of the two or more vehicles 10 that are expected to be included in the sensor information by a method different from the above method. For example, the determination unit 525 may specify the types of the two or more vehicles 10 that are expected to be included in the sensor information by using sensor information already acquired by an external sensor 9 located behind the target sensor in the traveling direction of the vehicle 10. The target sensor is an external sensor 9 that is scheduled to acquire sensor information that is the subject of the determination as to whether or not the sensor information corresponds to the second case. In this way, the type of the vehicle 10 can be specified by using past sensor information including two or more vehicles 10 that are expected to be included in the sensor information that is the subject of the determination as to whether or not the sensor information corresponds to the second case. Note that the determination unit 525 is not an essential function in the CPU 52. The control value creation unit 526 may, for example, recognize whether or not the sensor information corresponds to at least one of the first case and the second case by receiving an input from a user via an input operation unit.
[0031] The control value creation unit 526 uses the sensor information to create a control value that specifies the traveling behavior of at least some of the vehicles 10 among the multiple vehicles 10. In at least one of the first and second cases, the control value creation unit 526 creates a retraction control value in order to set the detection target vehicle 10a among the two or more vehicles 10 in a positional relationship that allows the detection unit 522d to detect the detection target vehicle 10a without overlapping with other vehicles 10 among the multiple vehicles 10 other than the detection target vehicle 10a. The retraction control value is a control value that specifies the traveling behavior of at least one of the two or more vehicles 10 in order to set the detection target vehicle 10a in a positional relationship that allows the detection unit 522d to detect the detection target vehicle 10a without overlapping with other vehicles 10. In other words, the retraction control value is a control value that causes the external sensor 9 to acquire sensor information in a state in which the detection target vehicle 10a is not overlapping with other vehicles 10. In this embodiment, the control value generating section 526 recognizes, based on the result of the determination by the determining section 525, whether or not at least one of the first case and the second case applies.
[0032] The avoidance control value in the first case is a control value that prescribes the traveling operation of at least one of the first vehicle 101 and the second vehicle 102 so that the actual distance L1 between the first vehicle 101 and the second vehicle 102, which are expected to be overlapping in the sensor information, becomes a target distance L2 that is longer than the actual distance L1 at the time when it is determined that the first case has occurred. The avoidance control value in the second case is a control value that prescribes the traveling operation of at least one of the first vehicle 101 and the second vehicle 102 so that the actual distance L1 between the first vehicle 101 and the second vehicle 102, which are expected to be overlapping in the sensor information, becomes a target distance L2 that is longer than the actual distance L1 at the time when it is determined that the second case has occurred when the sensor information is acquired.
[0033] The target distance L2 is set, for example, according to the actual distance L1 between the first vehicle 101 and the second vehicle 102 that are overlapping or expected to overlap in the sensor information, and the external shape of the first vehicle 101 and the external shape of the second vehicle 102. The larger the external shape of the first vehicle 101 and the external shape of the second vehicle 102 are, the longer the target distance L2 is. On the other hand, the smaller the external shape of the first vehicle 101 and the external shape of the second vehicle 102 are, the shorter the target distance L2 is. In the following, a case will be described in which the control value creation unit 526 sets the target distance L2, but the target distance L2 may be set by a unit other than the control value creation unit 526. The control value creation unit 526 may recognize the target distance L2 set by the user, for example, by receiving an input from the user via an input operation unit.
[0034] Fig. 4 is a diagram showing an example of a method for setting the target distance L2. In Fig. 4, the first vehicle 101 and the second vehicle 102 are vehicles 10 that are present and overlap in the sensor information, or that are expected to be present and overlap when the sensor information is acquired. The control value creation unit 526 sets the target distance L2 using the actual distance L1 between the first vehicle 101 and the second vehicle 102, the camera database D1, and the vehicle type database D2.
[0035] The control value creation unit 526 acquires coordinate values indicating the position of the imaging device 90 in the global coordinate system as information indicating the installation position of the imaging device 90 from which the captured image Im1 is acquired. Furthermore, in the first case, the control value creation unit 526 acquires the height H of the imaging device 90 from which the captured image Im1 is acquired from the road surface 60 by referring to the camera database D1. The control value creation unit 526 acquires the height H of the imaging device 90 from which the captured image Im1 is acquired from the road surface 60 by, for example, acquiring from the camera database D1 imaging parameters associated with the camera identification information transmitted together with the captured image Im1. In the second case, the control value creation unit 526 acquires the height H of the imaging device 90 from the road surface 60 for the imaging device 90 from which sensor information is expected to be acquired in which the first vehicle 101 and the second vehicle 102 are expected to overlap. Furthermore, the control value creation unit 526 acquires the height hg from the road surface 60 of the rear end portion (hereinafter, upper rear end portion 10g) on the ceiling side of the first vehicle 101 traveling ahead of the second vehicle 102 in the traveling direction.
[0036] Next, the control value creation unit 526 calculates the position of the upper rear end portion 10g of the first vehicle 101, for example, by using the relative positional relationship between the upper rear end portion 10g of the first vehicle 101 and the positioning point 10e, and the position of the positioning point 10e of the first vehicle 101. At this time, the control value creation unit 526 uses, for example, the position of the first vehicle 101 calculated by a position calculation method shown in Figures 5 to 10 described later, as the position of the positioning point 10e of the first vehicle 101. The control value creation unit 526 may use the estimated position of the first vehicle 101 estimated using the traveling speed and traveling time of the first vehicle 101 as the position of the positioning point 10e of the first vehicle 101.
[0037] Next, the control value creation unit 526 calculates the required distance L3 using the similarity relationship between the height H of the imaging device 90 from the road surface 60 and the height hg of the upper rear end 10g of the first vehicle 101 from the road surface 60. The required distance L3 is the minimum inter-vehicle distance between the first vehicle 101 and the second vehicle 102 required to set the detection target vehicle 10a in a positional relationship that allows it to be detected by the detection unit 522d without overlapping with other vehicles 10. When calculating the required distance L3 using the similarity relationship between the height H of the imaging device 90 from the road surface 60 and the height hg of the upper rear end 10g of the first vehicle 101 from the road surface 60, the control value creation unit 526 executes, for example, the following process. The control value creation unit 526 calculates the distance Lt between the first intersection Q1, which is the intersection of the first perpendicular line J1 and the road surface 60, and the second intersection Q2, which is the intersection of the second perpendicular line J2 and the road surface 60, along the traveling direction of the first vehicle 101. The first perpendicular line J1 is a perpendicular line from the front end 10i of the vehicle 10 to the road surface 60. The second perpendicular line J2 is a perpendicular line from the imaging device 90 to the road surface 60. Furthermore, the control value creation unit 526 acquires the total length Lf of the vehicle 10 close to the distance between the first intersection Q1 and the third intersection Q3. The third intersection Q3 is an intersection of the third perpendicular line J3 and the road surface 60. The third perpendicular line J3 is a perpendicular line from the upper rear end 10g of the vehicle 10 to the road surface 60. At this time, as shown in the following equation (2), the similarity ratio between the height hg of the upper rear end portion 10g of the vehicle 10 from the road surface 60 and the height H of the imaging device 90 from the road surface 60 is equal to the similarity ratio between the required distance L3 and the total distance Lp obtained by adding the required distance L3 to the separation distance Lt between the first intersection point Q1 and the second intersection point Q2 and the total length Lf of the vehicle 10. hg:H=L3:Lp=L3:(Lt+Lf+L3) Equation (1) Therefore, the control value creation unit 526 calculates the necessary distance L3 by substituting each calculated or acquired value into the above formula (1).
[0038] Next, the control value creation unit 526 acquires the actual distance L1 between the first vehicle 101 and the second vehicle 102. Then, when the actual distance L1 between the first vehicle 101 and the second vehicle 102 is equal to or less than the required distance L3, the control value creation unit 526 creates the evacuation control value by setting the calculated required distance L3 as the target distance L2, for example. On the other hand, when the actual distance L1 between the first vehicle 101 and the second vehicle 102 is greater than the required distance L3, the control value creation unit 526 sets the target distance L2 to, for example, an arbitrary distance equal to or greater than the actual distance L1. In this way, it is possible to set an appropriate target distance L2 according to the traveling situation of the vehicle 10. Note that, when the actual distance L1 between the first vehicle 101 and the second vehicle 102 is greater than the required distance L3, the control value creation unit 526 may set the target distance L2 to an arbitrary distance greater than the actual distance L1. In this way, it is possible to further reduce the possibility that the first vehicle 101 and the second vehicle 102 overlap in the sensor information.
[0039] The method of setting the target distance L2 is not limited to the above. The control value creation unit 526 may set the target distance L2 using information about the imaging device 90 from which the captured image Im1 was obtained, such as the detection range RG, installation position, and installation angle of the imaging device 90. In this way, a more appropriate target distance L2 can be set.
[0040] The avoidance control value includes, for example, a steering angle control value that specifies the steering angle of the control target vehicle 10b. In this way, by moving either one of the first vehicle 101 and the second vehicle 102 in the width direction of the road 6, the actual distance L1 between the first vehicle 101 and the second vehicle 102 can be changed to the target distance L2. Moving the vehicle 10 in the width direction of the road 6 means, for example, moving the vehicle 10 to at least one of a shoulder or a side road provided on the side of the road 6, or moving the vehicle 10 to another lane adjacent to the lane in which the vehicle 10 is traveling. This makes it possible to prevent other vehicles 10 other than the detection target vehicle 10a from interfering with the detection of the detection target vehicle 10a. In addition, when the first vehicle 101 and the second vehicle 102 are traveling along the target traveling route, the positional deviation from the target traveling route can be corrected in the process of changing the actual distance L1 between the first vehicle 101 and the second vehicle 102 to the target distance L2.
[0041] When the second vehicle 102 travels behind the first vehicle 101 in the same forward direction as the first vehicle 101, the avoidance control value may further include the following control value. In this case, the control value may include an acceleration control value that specifies the acceleration of the control target vehicle 10b. In this way, the actual distance L1 between the first vehicle 101 and the second vehicle 102 can be changed to the target distance L2 by the second vehicle 102 overtaking the first vehicle 101. As a result, for example, when it is desired to set the first vehicle 101 traveling on the front side as the detection target vehicle 10a, it is possible to prevent the second vehicle 102 from being an obstacle when detecting the first vehicle 101 by making the second vehicle 102 traveling on the rear side overtake the first vehicle 101. In addition, the moving speed when either one of the first vehicle 101 or the second vehicle 102 moves in the width direction of the road 6 can be adjusted.
[0042] Furthermore, the avoidance control value may include, for example, the following control values instead of the above control values. In this case, the avoidance control value may include, for example, at least one of an acceleration control value that increases the acceleration in the forward direction of the first vehicle 101, an acceleration control value that decreases the acceleration in the forward direction of the second vehicle 102, a direction control value that moves the second vehicle 102 backward in a direction opposite to the forward direction, and a stop control value that stops the traveling operation of the second vehicle 102. In this way, the distance in the direction along the forward direction between the first vehicle 101 and the second vehicle 102 can be made larger than the point in time when at least one of the first case and the second case is reached. This makes it possible to change the actual distance L1 between the first vehicle 101 and the second vehicle 102 to the target distance L2. Therefore, it is possible to prevent vehicles 10 other than the detection target vehicle 10a from interfering with the detection of the detection target vehicle 10a.
[0043] The types of control values included in the evacuation control value are not limited to those mentioned above. The evacuation control value may be a control value including, for example, either a destination control value or a trajectory control value, instead of the acceleration control value, the steering angle control value, the direction control value, and the stop control value. The destination control value is a control value that specifies the position of the control target vehicle 10b at the time when the control by the evacuation control value is terminated. Thus, the destination control value has a target arrival time and a target arrival location of the control target vehicle 10b. By creating an evacuation control value including the destination control value, it is possible to move the control target vehicle 10b to the target arrival location at a desired time without specifying detailed driving operations such as acceleration and steering angle. The trajectory control value is a control value that specifies a driving trajectory by arranging the target driving positions of the control target vehicle 10b for each predetermined time in chronological order. By creating an evacuation control value including the trajectory control value, it is possible to specify the driving trajectory of the control target vehicle 10b without specifying detailed driving operations such as acceleration and steering angle.
[0044] The control value creation unit 526 may create a control value other than the evacuation control value. The control value creation unit 526 may create, for example, a reference control value and a modified control value. The reference control value is a control value for driving the vehicle 10 along a predetermined target driving route. The modified control value is a control value for modifying the relative position of the vehicle 10 with respect to the target driving route. In addition, when the vehicle control device 150 receives either the reference control value or the modified control value and controls the operation of the vehicle 10 according to the received control value, the control value creation unit 526 may create the evacuation control value as a control value replacing the reference control value and the modified control value. In this way, when the vehicle 10 is driving based on a control value other than the evacuation control value at the time when at least one of the first case and the second case occurs, the operation of the vehicle 10 during driving can be changed to a driving operation according to the evacuation control value.
[0045] 3 transmits various information. For example, the transmission unit 527 transmits the control value created by the control value creation unit 526 to the control target vehicle 10b. Note that the configuration of the CPU 52 is not limited to the above. At least a part of the functions of the CPU 52 may be realized as one function of the vehicle control device 150 or the imaging device 90.
[0046] A-2. Method for calculating the position of the detected vehicle: FIG. 5 is a flow chart showing an example of a method for calculating the position of the detection target vehicle 10a using one captured image acquired by one imaging device 90. The method shown in FIG. 5 is repeatedly executed at predetermined time intervals after the detection target vehicle 10a starts traveling in the remote automatic driving mode, for example. First, a method for calculating the position of the detection target vehicle 10a in a case not corresponding to either the first case or the second case will be described with reference to each of FIG. 5 to FIG. 10 described later. In the position calculation method shown in FIG. 5 to FIG. 10 described later, the second vehicle 102 is the detection target vehicle 10a. The first vehicle 101 is the non-target vehicle 10f. In the example shown in FIG. 6, the first vehicle 101 is determined to be the non-target vehicle 10f because it is outside the recognition target area.
[0047] FIG. 6 is a schematic diagram showing examples of various images when the position calculation method shown in FIG. 5 is executed. In FIG. 6, step numbers corresponding to the steps in FIG. 5 are assigned. In the position calculation method shown in FIG. 5 to FIG. 10, a case will be described in which the first vehicle 101 and the second vehicle 102 move (run) along the same traveling direction on a road surface 60 on which mesh-like lattice lines 61 are drawn along the Xg axis parallel to the traveling direction of the first vehicle 101 and the second vehicle 102 and the Yg axis perpendicular to the Xg axis. The Xg axis and the Yg axis are coordinate axes in a global coordinate system. In other embodiments, the lattice lines 61 may be omitted.
[0048] As shown in Fig. 5, in the position calculation method, first, an image acquisition step (step S1) is executed. As shown in Fig. 6, the image acquisition step is a step of acquiring a captured image Im1 including the detection target vehicle 10a. In the image acquisition step, the acquisition unit 521 acquires the captured image Im1 acquired by the imaging device 90.
[0049] 5, after the image acquiring step, a position calculating step is executed. The position calculating step is a step of calculating image coordinate points of the detection target vehicle 10a by using the captured image Im1 acquired in the image acquiring step.
[0050] In the position calculation process, first, a distortion correction process (step S2) is executed. As shown in FIG. 6, the distortion correction process is a process of correcting the distortion of the captured image Im1. In the distortion correction process, the distortion correction unit 522a corrects the distortion of the captured image Im1 to generate a corrected image Im2. Specifically, the distortion correction unit 522a corrects the distortion of the captured image Im1, for example, by using a distortion correction parameter Pa1 stored in advance in the storage unit 53. The distortion correction parameter Pa1 is, for example, a parameter related to position information of the grid lines 61 obtained by calibration. Note that the distortion correction method is not limited to the above. The distortion correction parameter Pa1 may be any parameter other than those described above.
[0051] As shown in FIG. 5, the rotation process (step S3) is executed after the distortion correction process. As shown in FIG. 6, the rotation process is a process of rotating the corrected image Im2 so that the direction of the movement vector of the detection target vehicle 10a included in the corrected image Im2 faces a predetermined direction. In the rotation process, the rotation processing unit 522b rotates the corrected image Im2 so that the direction of the movement vector for the detection target vehicle 10a included in the corrected image Im2 faces a predetermined direction. In this way, the rotation processing unit 522b generates a rotated image Im3. Specifically, the rotation processing unit 522b rotates the corrected image Im2 with the center of gravity of the detection target vehicle 10a in the corrected image Im2 as the rotation center so that the direction of the movement vector of the detection target vehicle 10a faces upward on the screen of the display device that displays the corrected image Im2. The movement of a feature point (e.g., center of gravity) of the detection target vehicle 10a can be expressed as the direction of the movement vector by, for example, an optical flow method. The amount and direction of the movement vector of the detection target vehicle 10a are estimated based on, for example, the change in position of a feature point between image frames that is appropriately set on the correction image Im2. Note that the rotation processing method is not limited to the above.
[0052] As shown in FIG. 5, the cropping process (step S4) is executed after the rotation process. As shown in FIG. 6, the cropping process is a process of generating a processed image Im4 including a necessary area from the rotated image Im3. In this embodiment, in the cropping process, when the detection target vehicle 10a moves a distance exceeding a predetermined threshold, the cropping processing unit 522c deletes the moved area A2 corresponding to the distance moved by the detection target vehicle 10a from the rotated image Im3 as an unnecessary area. As a result, the cropping processing unit 522c generates a processed image Im4 by cutting out the unmoved area A1 including the detection target vehicle 10a as a necessary area from the rotated image Im3. At this time, the cropping processing unit 522c estimates the moved area A2 by recognizing the distance moved by the detection target vehicle 10a from the estimated movement vector amount of the detection target vehicle 10a, for example. Note that, in the position calculation process, either the rotation process or the cropping process may be executed first. Also, the cropping method is not limited to the above.
[0053] As shown in FIG. 5, the detection step (step S5) is executed after the crop processing step. As shown in FIG. 6, the detection step is a step of detecting the outer shape (contour) of the vehicle 10 included in the processed image Im4 using the position detection model Md. In the detection step, the detection unit 522d inputs the processed image Im4 to the position detection model Md. As a result, the detection unit 522d detects the first vehicle 101 included in the processed image Im4 and the second vehicle 102 as the detection target vehicle 10a. Then, the detection unit 522d generates a first mask image Im5 in which the first vehicle 101 in the processed image Im4 is masked to add a first mask region M1, and the second vehicle 102 in the processed image Im4 is masked to add a second mask region M2. Note that the method of detecting the outer shape of the vehicle 10 is not limited to the above.
[0054] As shown in FIG. 5, the detection step is followed by a deletion step (step S6). As shown in FIG. 6, the deletion step is a step of deleting a mask region M1 of a non-target vehicle 10f, which is a vehicle 10 other than the detection target vehicle 10a, from a first mask image Im5. In the example shown in FIG. 6, in the deletion step, the deletion unit 522e deletes the first mask region M1 from the first mask image Im5. As a result, the deletion unit 522e generates a first mask image Im6 including only the mask region M2 for the detection target vehicle 10a. In the following, the first mask image Im6 after deleting the mask region M1 for the vehicle 10 other than the detection target vehicle 10a, that is, the first mask image Im6 including only the mask region M2 for the detection target vehicle 10a, is referred to as the "first mask image Im6". Note that the deletion method is not limited to the above. Furthermore, the deletion step is omitted when only one vehicle 10 is included in the captured image Im1, and when calculating the positions of two or more vehicles 10 included in the captured image Im1 in a case that does not fall under either the first or second case. Hereinafter, the first mask image Im6 when the deletion step is executed refers to the first mask image Im6 generated in the deletion step, i.e., the first mask image Im6 after deleting the mask regions M1 for vehicles 10 other than the detection target vehicle 10a. The first mask image Im5 when the deletion step is omitted refers to the first mask image Im5 generated in the detection step.
[0055] As shown in FIG. 5, after the deletion step, a perspective transformation step (step S7) is executed. As shown in FIG. 6, the perspective transformation step is a step of performing perspective transformation on the first mask image Im6. In the perspective transformation step, the perspective transformation unit 522f performs perspective transformation on the first mask image Im6 to generate a second mask image Im7. Specifically, the perspective transformation unit 522f performs perspective transformation on the first mask image Im6 into a bird's-eye image viewed from a viewpoint above the detection target vehicle 10a (for example, directly above the detection target vehicle 10a) that is substantially perpendicular to the road surface 60, for example, using a perspective transformation parameter Pa2 previously determined and stored in the storage unit 53. The perspective transformation parameter Pa2 is, for example, a parameter related to position information and internal parameters of the imaging device 90 obtained by calibration. As a result, the perspective transformation unit 522f generates a second mask image Im7 expressed in an image coordinate system from the first mask image Im6 expressed in a camera coordinate system. The image coordinate system is a coordinate system having a point on the image plane projected by the perspective transformation as the origin and coordinate axes indicated by the Xi axis and the Yi axis perpendicular to the Xi axis. Note that the perspective transformation method is not limited to the above. The perspective transformation parameter Pa2 may be any parameter other than those described above.
[0056] As shown in Fig. 5, after the perspective transformation step, a coordinate point calculation step (step S8) is executed. Fig. 7 is a diagram for explaining the details of the coordinate point calculation step. The coordinate point calculation step is a step for calculating an image coordinate point P3 for the detection target vehicle 10a.
[0057] In the coordinate point calculation step, the coordinate point calculation unit 522g acquires a base coordinate point P0 from a first circumscribing rectangle R1 set in a mask area M2 in a first mask image Im6, which is an image before perspective transformation. FIG. 8 is a diagram for explaining a method for acquiring the base coordinate point P0. In order to acquire the base coordinate point P0, the coordinate point calculation unit 522g sets a base circumscribing rectangle R0 for the mask area M2 in the first mask image Im6. Next, the coordinate point calculation unit 522g rotates the first mask image Im6 by a required rotation amount around the center of gravity C of the mask area M2 as the rotation center so that the direction of the movement vector V of the detection target vehicle 10a corresponding to the mask area M2 in the first mask image Im6 faces a predetermined direction. The predetermined direction is, for example, the upward direction on the screen of the display device. Next, the coordinate point calculation unit 522g sets a first circumscribing rectangle R1 for the mask area M2 of the rotated first mask image Im6 so that the long side is parallel to the direction of the movement vector V. Next, the coordinate point calculation unit 522g rotates the first mask image Im6 to which the first circumscribing rectangle R1 has been added by the above-mentioned rotation amount with the center of gravity C of the mask area M2 as the rotation center. As a result, the coordinate point calculation unit 522g acquires, as a base coordinate point P0, the coordinate point of one of the four vertices of the first circumscribing rectangle R1, which has the coordinates closest to the positioning point 10e of the detection target vehicle 10a. Then, as shown in FIG. 7, the coordinate point calculation unit 522g performs perspective transformation on the first mask image Im6 after the reverse rotation, that is, the first mask image Im6 after the base coordinate point P0 is acquired. As a result, the coordinate point calculation unit 522g acquires the first coordinate point P1 as the coordinate point corresponding to the base coordinate point P0 in the first circumscribed rectangle R1 transformed by the perspective transformation.
[0058] Furthermore, the coordinate point calculation unit 522g sets a second circumscribing rectangle R2 for the mask region M2 in the second mask image Im7 obtained by performing perspective transformation on the first mask image Im6. Then, the coordinate point calculation unit 522g acquires, as the second coordinate point P2, a vertex indicating the same position as the first coordinate point P1 among the vertices of the second circumscribing rectangle R2. Specifically, the coordinate point calculation unit 522g acquires, as the second coordinate point P2, one of the four vertices of the second circumscribing rectangle R2, a vertex having coordinates that are closest to the positioning point 10e of the detection target vehicle 10a. That is, the first coordinate point P1 and the second coordinate point P2 are coordinate points indicating the same position, and therefore have a correlation with each other.
[0059] Furthermore, the coordinate point calculation unit 522g performs correction to replace the coordinates (Xi1, Yi1) of the first coordinate point P1 with the coordinates (Xi2, Yi2) of the second coordinate point P2 depending on the magnitude relationship between the coordinate value of the first coordinate point P1 and the coordinate value of the second coordinate point P2. When the coordinate value Xi1 in the Xi direction of the first coordinate point P1 is greater than the coordinate value Xi2 in the Xi direction of the second coordinate point P2 (Xi1>Xi2), the coordinate point calculation unit 522g replaces the coordinate value Xi1 in the Xi direction of the first coordinate point P1 with the coordinate value Xi2 in the Xi direction of the second coordinate point P2. When the coordinate value Yi1 of the first coordinate point P1 in the Yi direction is greater than the coordinate value Yi2 of the second coordinate point P2 in the Yi direction (Yi1>Yi2), the coordinate point calculation unit 522g replaces the coordinate value Yi1 of the first coordinate point P1 in the Yi direction with the coordinate value Yi2 of the second coordinate point P2 in the Yi direction. In the example shown in FIG. 7, the coordinate value Xi1 of the first coordinate point P1 in the Xi direction is greater than the coordinate value Xi2 of the second coordinate point P2 in the Xi direction. Also, the coordinate value Yi of the first coordinate point P1 in the Yi direction is smaller than the coordinate value Yi2 of the second coordinate point P2 in the Yi direction. Therefore, the image coordinate point P3 has coordinates (Xi2, Yi1). In this way, the coordinate point calculation unit 522g calculates the image coordinate point P3 indicating the position of the detection target vehicle 10a in the image coordinate system by correcting the first coordinate point P1 using the second coordinate point P2. The method of calculating the image coordinate point P3 is not limited to the above.
[0060] As shown in FIG. 5, a position conversion step (step S9) is executed after the position calculation step. The position conversion step is a step of converting the image coordinate point P3 into a vehicle coordinate point to calculate a vehicle coordinate point indicating the position of the detection target vehicle 10a in the global coordinate system. In the position conversion step, the position conversion unit 523 converts the image coordinate point P3 into a vehicle coordinate point using the imaging parameters acquired by the acquisition unit 521 and predetermined vehicle parameters. The vehicle parameters are parameters related to the distance from the road surface 60 to the positioning point 10e of the detection target vehicle 10a. In this embodiment, the vehicle parameters are heights h (FIG. 9 described later) of the positioning points 10e of the detection target vehicle 10a from the road surface 60. The vehicle parameters may be parameters related to the distance to the positioning point 10e of a vehicle 10 of a predetermined type. In other words, the vehicle parameters may be variables according to the type of the vehicle 10, or may be predetermined constants.
[0061] The position conversion unit 523 converts the image coordinate point P3 into a vehicle coordinate point using equations (2) to (4) described below, which have the vehicle coordinate point as a target variable and include the image coordinate point P3, the imaging parameters, and the vehicle parameters as explanatory variables. In this case, the position conversion unit 523 substitutes the coordinate value of the image coordinate point P3 calculated by the position calculation unit 522 into the relational expressions expressed by equations (2) to (4). The position conversion unit 523 substitutes the imaging parameters acquired by the acquisition unit 521, i.e., the values of the imaging parameters corresponding to the imaging device 90 that acquired the captured image Im1, and the values of the vehicle parameters, into the relational expressions expressed by equations (2) to (4).
[0062] FIG. 9 is a first diagram for explaining the details of the position conversion process. FIG. 9 illustrates the state of the detection target vehicle 10a when viewed from the left side. FIG. 10 is a second diagram for explaining the details of the position conversion process. FIG. 10 illustrates the state of the detection target vehicle 10a when viewed from the roof side. The global coordinate system in FIG. 9 and FIG. 10 is a coordinate system having a fixed coordinate point Pf indicating an arbitrary reference position of the road surface 60 as the origin and coordinate axes indicated by the Xg axis and the Yg axis perpendicular to the Xg axis. The imaging coordinate point Pc is the position of the imaging device 90 that acquired the captured image Im1 used to calculate the image coordinate point P3, and is a coordinate point indicating the position of the imaging device 90 in the global coordinate system. The fixed coordinate point Pf and the imaging coordinate point Pc are stored in advance in the storage unit 53.
[0063] As shown in Fig. 9, the observation distance on the XgYg plane between the position of the imaging device 90 and the position (image coordinate point P3) of the detection target vehicle 10a is set to Do. The observation error is set to ΔD. The height [m] of the imaging device 90 from the road surface 60 as an imaging parameter is set to H. The height [m] of the positioning point 10e of the detection target vehicle 10a from the road surface 60 as a vehicle parameter is set to h. In this case, the observation error ΔD can be expressed by the following equation (2). ΔD=h / H×Do Equation (2) That is, the larger the observation distance Do, the larger the observation error ΔD.
[0064] Next, if the actual distance between the position of the imaging device 90 and the position of the positioning point 10e of the detection target vehicle 10a (hereinafter referred to as the first distance) is D, the first distance D can be expressed by the following formula (3). D=Do×(1-h / H) Equation (3) That is, the first distance D is determined by the observation distance Do, the height H of the imaging device 90 as an imaging parameter, and the height h of the positioning point 10e of the detection target vehicle 10a as a vehicle parameter.
[0065] Then, as shown in FIG. 10, if the estimated distance between the reference position and the position of the vehicle 10a to be detected is Dp, and the actual distance between the reference position and the vehicle 10a to be detected (hereinafter, the second distance) is Dt, the second distance Dt can be expressed by the following equation (4). Dt = Dp × (1-h / H) Equation (4)
[0066] Here, the estimated distance Dp can be calculated using an actual distance (hereinafter, the third distance Dc) calculated from the fixed coordinate point Pf and the image coordinate point Pc, the image coordinate point P3, and the fixed coordinate point Pf. Therefore, the position conversion unit 523 can calculate the vehicle coordinate point Pv using the second distance Dt calculated by correcting the estimated distance Dp using the above formula (4) and the fixed coordinate point Pf. At this time, the calculated vehicle coordinate point Pv is a coordinate point indicating the position of the detection target vehicle 10a in the global coordinate system, and therefore corresponds to the position of the detection target vehicle 10a in the real space.
[0067] A-3.Remote control method: FIG. 11 is a first flowchart showing the remote control method in the first embodiment. FIG. 12 is a second flowchart showing the remote control method in the first embodiment. FIG. 13 is a third flowchart showing the remote control method in the first embodiment. The remote control method shown in FIG. 11 to FIG. 13 is repeatedly executed at predetermined time intervals after the vehicle 10 starts traveling in the remote automatic driving mode, for example. In FIG. 11 to FIG. 13, the same steps as those shown in FIG. 5 are denoted by the same reference numerals and description thereof is omitted.
[0068] 11, the determination unit 525 determines whether the second case applies (step S0). If the determination unit 525 determines that the second case does not apply (step S0: No), the remote control device 5 executes an acquisition step, a distortion correction step, a rotation step, a crop processing step, and a detection step in this order.
[0069] In the process of calculating the position of the vehicle 10, the determination unit 525 uses the first mask image Im5 as data generated by the detection unit 522d to determine whether or not two or more vehicles 10 are included in the captured image Im1 (step S10). When the determination unit 525 determines that only one vehicle 10 is included in the captured image Im1 (step S10: No), the remote control device 5 sets the vehicle 10 included in the captured image Im1 as the detection target vehicle 10a. Then, as shown in FIG. 5, the remote control device 5 executes a perspective transformation process, a coordinate point calculation process, and a position transformation process in this order to calculate a vehicle coordinate point Pv indicating the position of the detection target vehicle 10a. The calculated vehicle coordinate point Pv is used to create a control value such as a corrected control value, for example.
[0070] As shown in FIG. 11, when the determination unit 525 determines that two or more vehicles 10 are included in the captured image Im1 (step S10: Yes), the determination unit 525 determines whether or not the first case applies (step S20).
[0071] When the determination unit 525 determines that the first case does not apply (step S20: No), the determination unit 525 determines that each vehicle 10 included in the captured image Im1 is a detection target vehicle 10a. Then, as shown in FIG. 12, the remote control device 5 executes a deletion process, a perspective transformation process, a coordinate point calculation process, and a position conversion process for each detection target vehicle 10a in this order. FIG. 12 illustrates a processing flow when the captured image Im1 includes a first vehicle 101 and a second vehicle 102. As a result, the remote control device 5 calculates a vehicle coordinate point Pv indicating the positions of two or more vehicles 10 included in the captured image Im1 for each detection target vehicle 10a. The calculated vehicle coordinate point Pv is used to create a control value such as a correction control value, for example.
[0072] On the other hand, as shown in FIG. 11, if it is determined that the first case applies (step S20: Yes), the judgment unit 525 determines whether to classify each of the two or more vehicles 10 included in the captured image Im1 as a detection target vehicle 10a or a non-target vehicle 10f, as shown in FIG. 13 (step S31).
[0073] FIG. 14 is a diagram for explaining an example of a determination method for determining whether to classify a vehicle into a detection target vehicle 10a or a non-target vehicle 10f. A determination method in a case where both a first vehicle 101 and a second vehicle 102 are present within a recognition target area will be described with reference to FIG. 14. In FIG. 14, the grid lines 61 are omitted. Also, FIG. 14 illustrates a case where the second vehicle 102 overlaps the positioning point 10e of the first vehicle 101 in the captured image Im1 so as to cover it. That is, in the captured image Im1, the first vehicle 101 is located on the back side of the second vehicle 102. In this case, the determination unit 525 determines whether to classify each of the first vehicle 101 and the second vehicle 102 included in the captured image Im1 into a detection target vehicle 10a or a non-target vehicle 10f depending on whether the positioning point 10e can be detected in the captured image Im1. Specifically, the determination unit 525 determines that the second vehicle 102 that can detect the positioning point 10e is the detection target vehicle 10a. On the other hand, the determination unit 525 determines that the first vehicle 101 that cannot detect the positioning point 10e because another vehicle 10 overlaps and covers the positioning point 10e in the captured image Im1 is the non-target vehicle 10f.
[0074] 13, the remote control device 5 creates a stop control value as the evacuation control value for the non-target vehicle 10f without executing the deletion process, the perspective transformation process, the coordinate point calculation process, and the position conversion process (step S75). The transmission unit 527 transmits the evacuation control value created by the control value creation unit 526 to the control target vehicle 10b (step S80).
[0075] Meanwhile, the remote control device 5 executes a deletion process, a perspective transformation process, a coordinate point calculation process, and a position conversion process for each detection target vehicle 10a in this order. As a result, the remote control device 5 calculates a vehicle coordinate point Pv indicating the position of the detection target vehicle 10a among two or more vehicles 10 included in the captured image Im1 for each detection target vehicle 10a. Then, the determination unit 525 selects a vehicle 10 to be a control target vehicle 10b from among the detection target vehicles 10a (step S60). The actual distance calculation unit 524 calculates an actual distance L1 between the detection target vehicle 10a selected as the control target vehicle 10b by the determination unit 525 and a vehicle 10 traveling without any other vehicle 10 between the detection target vehicle 10a and the detection target vehicle 10a (step S40). The control value creation unit 526 sets a target distance L2 (step S50). Then, the control value creation unit 526 creates an evacuation control value for the detection target vehicle 10a selected as the control target vehicle 10b by the determination unit 525 (step S71). The transmission unit 527 transmits the evacuation control value created by the control value creation unit 526 to the control target vehicle 10b (step S80). Note that in the first case, when there is only one vehicle 10 determined to be the detection target vehicle 10a, step S60 is omitted.
[0076] As shown in FIG. 11, when the determination unit 525 determines that the second case applies (step S0: Yes), the determination unit 525 determines that each of the two or more vehicles 10 expected to be included in the captured image Im1 is a detection target vehicle 10a. Then, as shown in FIG. 13, the determination unit 525 selects a vehicle 10 to be a control target vehicle 10b from the detection target vehicles 10a (step S60). Then, the control value creation unit 526 creates an evacuation control value for the detection target vehicle 10a selected as the control target vehicle 10b by the determination unit 525 (step S71). The transmission unit 527 transmits the evacuation control value created by the control value creation unit 526 to the control target vehicle 10b (step S80).
[0077] According to the first embodiment, in at least either the first case or the second case, it is possible to create an evacuation control value for making the detection target vehicle 10a of the two or more vehicles 10 have a positional relationship in which the detection target vehicle 10a can be detected by the detection unit 522d without overlapping with the other vehicles 10. Then, it is possible to transmit the created evacuation control value to the control target vehicle 10b. As a result, when tracking the position of the vehicle 10 using the captured image Im1 acquired by the imaging device 90 installed in a location different from the vehicle 10, it is possible to reduce the possibility that two or more vehicles 10 are present overlapping on the captured image Im1. Therefore, it is possible to more accurately track the position of the vehicle 10 using the captured image Im1.
[0078] Moreover, according to the first embodiment, before the captured image Im1 is acquired as the sensor information, it is possible to predict whether or not two or more vehicles 10 will be present overlapping when the captured image Im1 is acquired. Then, when it is predicted that two or more vehicles 10 will be present overlapping when the captured image Im1 is acquired, it is possible to create an evacuation control value and transmit it to the control target vehicle 10b. In this way, it is possible to control the traveling operation of the vehicle 10 before a state in which two or more vehicles 10 are present overlapping on the captured image Im1 is reached. This makes it possible to further reduce the possibility that two or more vehicles 10 are present overlapping on the captured image Im1 when tracking the position of the vehicle 10 using the captured image Im1. Therefore, it is possible to suppress the situation in which the position of the vehicle 10 cannot be tracked.
[0079] Furthermore, according to the first embodiment, it is possible to calculate the actual distance L1 between the first vehicle 101 and the second vehicle 102 that are overlapping in the captured image Im1 or that are expected to overlap when the captured image Im1 is acquired. Then, it is possible to create an evacuation control value that specifies the traveling operation of at least one of the first vehicle 101 and the second vehicle 102 so that the actual distance L1 between the first vehicle 101 and the second vehicle 102 becomes a target distance L2 that is longer than the actual distance L1 at the time when it is determined that at least one of the first case and the second case has occurred. In this way, it is possible to more reliably reduce the possibility that the first vehicle 101 and the second vehicle 102 overlap on the captured image Im1 when tracking the positions of the first vehicle 101 and the second vehicle 102 using the captured image Im1.
[0080] Furthermore, according to the first embodiment, the target distance L2 can be set according to the external shape of the first vehicle 101 and the external shape of the second vehicle 102. In this way, the target distance L2 can be set according to the external shape of the vehicle 10 that is correlated with the area that the vehicle 10 occupies on the captured image Im1. This makes it possible to suppress a decrease in the calculation accuracy and the calculation possibility of the position of the vehicle 10 depending on the type of the vehicle 10.
[0081] Furthermore, according to the first embodiment, it is possible to generate a stop control value for stopping a vehicle 10 whose position cannot be calculated due to two or more vehicles 10 overlapping in the captured image Im1. This allows the vehicle 10 whose position cannot be calculated to be stopped safely.
[0082] Furthermore, according to the first embodiment, when the captured image Im1 is acquired, it is possible to determine whether or not the captured image Im1 corresponds to the second case depending on the types of the two or more vehicles 10 that are expected to be included in the captured image Im1. This makes it possible to more reliably determine whether or not the captured image Im1 corresponds to the second case.
[0083] Furthermore, according to the first embodiment, whether or not the second case applies can be determined according to the external shape of each of two or more vehicles 10 that are expected to be included in the captured image Im1 when the captured image Im1 is acquired, which differs depending on the type of vehicle 10. In this way, whether or not the second case applies can be determined according to the external shape of the vehicle 10 that is correlated with the area occupied by the vehicle 10 on the captured image Im1. This makes it possible to more reliably determine whether or not the second case applies.
[0084] Furthermore, according to the first embodiment described above, it is possible to determine whether each of two or more vehicles 10 included in the captured image Im1 should be classified as a detection target vehicle 10a or a non-target vehicle 10f, depending on the positional relationship of the two or more vehicles 10 on the captured image Im1.
[0085] According to the first embodiment, a vehicle 10 whose positioning point 10e cannot be detected because another vehicle 10 overlaps the positioning point 10e so as to cover it can be set as a non-target vehicle 10f. Then, the deletion step, perspective transformation step, coordinate point calculation step, and position conversion step for calculating the position of the non-target vehicle 10f can be omitted. This reduces the processing load in the remote control method.
[0086] B. Second embodiment: FIG. 15 is a first flowchart showing the remote control method in the second embodiment. FIG. 16 is a second flowchart showing the remote control method in the second embodiment. FIG. 16 illustrates a processing flow when the captured image Im1 includes the first vehicle 101 and the second vehicle 102. FIG. 17 is a third flowchart showing the remote control method in the second embodiment. The remote control method shown in FIG. 15 to FIG. 17 is repeatedly executed at predetermined time intervals after the vehicle 10 starts traveling in the remote automatic driving mode, for example.
[0087] In this embodiment, the determination unit 525 determines whether to classify each of the two or more vehicles 10 included in the captured image Im1 as a detection target vehicle 10a or a non-target vehicle 10f depending on whether the image coordinate point P3 has been calculated. Therefore, part of the function of the determination unit 525 and part of the processing of the remote control method are different from those of the first embodiment. The other configurations are the same as those of the first embodiment. The same steps as those of the first embodiment and the same configurations as those of the first embodiment are denoted by the same reference numerals and descriptions thereof are omitted.
[0088] As shown in FIG. 15 and FIG. 16, when the determination unit 525 determines that the captured image Im1 corresponds to the first case and not the second case (step S0: No, step S20: Yes), the remote control device 5 executes the following process. In this case, the remote control device 5 executes a deletion process, a perspective transformation process, and a coordinate point calculation process in this order for each of the two or more vehicles 10 included in the captured image Im1. At this time, for a vehicle 10 in which the positioning point 10e cannot be detected because another vehicle 10 overlaps the positioning point 10e so as to cover it, the image coordinate point P3 cannot be calculated, and an error code, for example, is generated. Therefore, the determination unit 525 determines whether each of the two or more vehicles 10 included in the captured image Im1 is classified into a detection target vehicle 10a or a non-target vehicle 10f depending on whether the image coordinate point P3 has been calculated. Specifically, the determination unit 525 determines whether the image coordinate point P3 has been calculated (step S32). Then, when the determination unit 525 determines that the image coordinate point P3 of the first vehicle 101 has been calculated (step S32: Yes), it determines that the first vehicle 101 is the detection target vehicle 10a (step S33). On the other hand, when the determination unit 525 determines that the image coordinate point P3 of the first vehicle 101 has not been calculated (step S32: No), it determines that the first vehicle 101 is the non-target vehicle 10f (step S34). Similarly, when the determination unit 525 determines that the image coordinate point P3 of the second vehicle 102 has been calculated (step S32: Yes), it determines that the second vehicle 102 is the detection target vehicle 10a (step S33). On the other hand, when the determination unit 525 determines that the image coordinate point P3 of the second vehicle 102 has not been calculated (step S32: No), it determines that the second vehicle 102 is the non-target vehicle 10f (step S34). Then, depending on the results of the determinations made in steps S32 to S34, the steps shown in FIG. 17 are executed.
[0089] According to the second embodiment described above, depending on whether the image coordinate point P3 can be calculated, it is possible to determine whether each of the two or more vehicles 10 included in the captured image Im1 should be classified as a detection target vehicle 10a or a non-target vehicle 10f.
[0090] C. Third embodiment: Fig. 18 is a first flowchart showing a remote control method in the third embodiment. Fig. 19 is a second flowchart showing a remote control method in the third embodiment. Fig. 19 illustrates a processing flow in the case where the captured image Im1 includes the first vehicle 101 and the second vehicle 102. Fig. 20 is a third flowchart showing the remote control method in the third embodiment. The remote control method shown in Figs. 18 to 20 is repeatedly executed at predetermined time intervals after the vehicle 10 starts traveling in the remote autonomous driving mode, for example.
[0091] In this embodiment, the determination unit 525 determines whether to classify each of the two or more vehicles 10 included in the captured image Im1 as a detection target vehicle 10a or a non-target vehicle 10f depending on whether the vehicle coordinate point Pv has been calculated. Therefore, part of the function of the determination unit 525 and part of the processing of the remote control method are different from those of the first embodiment. The other configurations are the same as those of the first embodiment. The same steps as those of the first embodiment and the same configurations as those of the first embodiment are denoted by the same reference numerals and description thereof will be omitted.
[0092] As shown in FIG. 18 and FIG. 19, when the determination unit 525 determines that the first case applies and not the second case (step S0: No, step S20: Yes), the remote control device 5 executes the following process. In this case, the remote control device 5 executes a deletion process, a perspective transformation process, a coordinate point calculation process, and a position conversion process in this order for each of the two or more vehicles 10 included in the captured image Im1. At this time, for a vehicle 10 in which the positioning point 10e cannot be detected because another vehicle 10 overlaps the positioning point 10e so as to cover it, the vehicle coordinate point Pv cannot be calculated, and an error code, for example, is generated. Therefore, the determination unit 525 determines whether each of the two or more vehicles 10 included in the captured image Im1 is classified into a detection target vehicle 10a or a non-target vehicle 10f depending on whether the vehicle coordinate point Pv has been calculated. Specifically, the determination unit 525 determines whether the vehicle coordinate point Pv has been calculated (step S35). Then, when the determination unit 525 determines that the vehicle coordinate point Pv of the first vehicle 101 has been calculated (step S35: Yes), it determines that the first vehicle 101 is the detection target vehicle 10a (step S36). On the other hand, when the determination unit 525 determines that the vehicle coordinate point Pv of the first vehicle 101 has not been calculated (step S35: No), it determines that the first vehicle 101 is the non-target vehicle 10f (step S37). Similarly, when the determination unit 525 determines that the vehicle coordinate point Pv of the second vehicle 102 has been calculated (step S35: Yes), it determines that the second vehicle 102 is the detection target vehicle 10a (step S36). On the other hand, when the determination unit 525 determines that the vehicle coordinate point Pv of the second vehicle 102 has not been calculated (step S35: No), it determines that the second vehicle 102 is the non-target vehicle 10f (step S37). Then, depending on the results of the determinations shown in steps S35 to S37, the steps in FIG. 20 are executed.
[0093] According to the third embodiment described above, depending on whether the vehicle coordinate point Pv can be calculated, it is possible to determine whether each of the two or more vehicles 10 included in the captured image Im1 should be classified as a detection target vehicle 10a or a non-target vehicle 10f.
[0094] D. Fourth embodiment: Fig. 21 is a first flowchart showing a remote control method in the fourth embodiment. Fig. 22 is a second flowchart showing a remote control method in the fourth embodiment. The remote control method shown in Fig. 21 and Fig. 22 is repeatedly executed at predetermined time intervals after the vehicle 10 starts traveling in the remote automatic driving mode, for example.
[0095] In this embodiment, the determination unit 525 determines whether or not the first case applies by using the captured image Im1 as well as the vehicle coordinate point Pv calculated in the position conversion process. Therefore, some functions of the determination unit 525 and some processes of the remote control method are different from those of the first embodiment. The other configurations are the same as those of the first embodiment. The same steps as those in the first embodiment and the same configurations as those in the first embodiment are denoted by the same reference numerals and descriptions thereof are omitted.
[0096] As shown in Fig. 21, when the determination unit 525 determines that the second case does not apply (step S0: No), the remote control device 5 executes each process from step S1 to step S9. When two or more vehicles 10 are included in the captured image Im1 (step S10: Yes), the determination unit 525 determines whether the first case applies using the captured image Im1 as well as the vehicle coordinate point Pv calculated in the position conversion process (step S20). Then, each step in Fig. 22 is executed according to the determination result in step S20.
[0097] According to the fourth embodiment, it is possible to determine whether or not the first case applies by using the vehicle coordinate point Pv indicating the position of the vehicle 10 calculated by using the captured image Im1.
[0098] E. Fifth embodiment: 23 is a diagram showing details of a CPU 52a mounted on a remote control device 5 in the fifth embodiment. In this embodiment, the external sensor 9 is an external LiDAR (Light Detection And Ranging) installed at a location different from the vehicle 10. Therefore, part of the configuration of the remote control device 5 and the method of calculating the position of the vehicle 10 are different from those in the first embodiment. The other configurations are the same as those in the first embodiment. The same reference numerals are used for the same configurations as those in the first embodiment, and the description thereof will be omitted.
[0099] The external lidar is a lidar that detects the detection target vehicle 10a from outside the detection target vehicle 10a. The external lidar irradiates a laser light in a predetermined detection range RF and detects the reflected light reflected by the object such as the detection target vehicle 10a, thereby detecting the distance and angle between the external lidar and the object, the shape of the object, and the like. The external lidar transmits the acquired external lidar information to the remote control device 5 together with lidar identification information that identifies the multiple external lidars and the acquisition time of the external lidar information. The installation positions and the number of external lidars are determined in consideration of the detection range of each external lidar and objects (obstacles) present in the surrounding area of the road 6, so that the entire road 6 can be detected by one or more external lidars.
[0100] The CPU 52a includes a position generating unit 528 shown in Fig. 23 instead of the position calculating unit 522 and the position converting unit 523 of the CPU 52 shown in Fig. 3. The position generating unit 528 generates the position of the detection target vehicle 10a by using external LIDAR information. The external LIDAR information is sensor information acquired by an external LIDAR. In this embodiment, the position generating unit 528 adopts the coordinate value of the positioning point 10e of the detection target vehicle 10a in the global coordinate system as the position of the detection target vehicle 10a.
[0101] FIG. 24 is a diagram showing an example of a method for calculating the position of a detection target vehicle 10a using external lidar information. In the example shown in FIG. 24, a reference object 8 capable of reflecting a laser beam emitted by an external lidar 95 is shown. The reference object 8 is, for example, a pole (support) to which a reflector that reflects a laser beam is attached. A plurality of reference objects 8 may be installed at different positions. The coordinate values (absolute positions) of one or more reference objects 8 in a global coordinate system are known. Information regarding the absolute position of the reference object 8 is, for example, stored in advance in the storage unit 53 of the remote control device 5.
[0102] In order to calculate the position of the detection target vehicle 10a, the position generating unit 528 calculates the position of the external lidar 95 by using the absolute position of the reference object 8 and the first detection vector Vs1. The first detection vector Vs1 is a vector directed from the external lidar 95 to the reference object 8. The first detection vector Vs1 has a distance Li1 between the external lidar 95 and the reference object 8 and a direction from the external lidar 95 to the reference object 8. In FIG. 24, the direction from the external lidar 95 to the reference object 8 is represented by an angle θ1 formed between the optical axis reference Ls of the external lidar 95 and the first detection vector Vs1. The optical axis reference Ls of the external lidar 95 corresponds to the center line of the detection range RF, which is the irradiation range of the laser light irradiated from the projector of the external lidar 95. The position generation unit 528 uses trigonometric functions to calculate a difference ΔX in coordinate values in the X direction between the external LIDAR 95 and a specific reference object 8, which is one specific reference object 8, and a difference ΔY in coordinate values in the Y direction between the external LIDAR 95 and the specific reference object 8. Then, the position generation unit 528 reflects the difference ΔX1 in the X direction and the difference ΔY1 in the Y direction in the coordinate values indicating the absolute position of the specific reference object 8, taking into account the angle difference θ1 between the optical axis reference Ls of the external LIDAR 95 and the first detection vector Vs1. In this way, the position generation unit 528 calculates the position of the external LIDAR 95 in the global coordinate system.
[0103] Next, the position generation unit 528 calculates the position of the detection target vehicle 10a using the calculated position of the external LIDAR 95 and the second detection vector Vs2. The second detection vector Vs2 is a vector directed from the external LIDAR 95 to the detection target vehicle 10a. The second detection vector Vs2 has a distance Li2 between the external LIDAR 95 and the detection target vehicle 10a, and a direction from the external LIDAR 95 to the detection target vehicle 10a. In FIG. 24, the direction from the external LIDAR 95 to the detection target vehicle 10a is represented by an angle θ2 between the optical axis reference Ls of the external LIDAR 95 and the second detection vector Vs2. The position generation unit 528 calculates a difference ΔX2 between the coordinate values in the X direction between the external LIDAR 95 and the detection target vehicle 10a, and a difference ΔY2 between the coordinate values in the Y direction between the external LIDAR 95 and the detection target vehicle 10a, using trigonometric functions. Then, the position generation unit 528 reflects the difference ΔX2 in the X direction and the difference ΔY2 in the Y direction in the coordinate value indicating the position of the external lidar 95, taking into consideration the angle difference θ2 between the optical axis reference Ls of the external lidar 95 and the second detection vector Vs2. As a result, the position generation unit 528 calculates the coordinate value in the global coordinate system of the specific part 10j of the detection target vehicle 10a detected by the external lidar 95 among the detection target vehicles 10a. Then, the position generation unit 528 calculates the coordinate value in the global coordinate system of the positioning point 10e of the detection target vehicle 10a from the calculated coordinate value in the global coordinate system of the specific part 10j based on the relative positional relationship between the specific part 10j of the detection target vehicle 10a and the positioning point 10e. Note that the method of calculating the position of the detection target vehicle 10a using the external lidar information is not limited to the above. Also, when the position of the detection target vehicle 10a is calculated using the external lidar information, it is not essential to use information on the reference object 8.
[0104] According to the fifth embodiment, the position of the detection target vehicle 10a can be calculated using the external lidar information acquired by the external lidar 95. Then, in at least either the first or second case, the position of the detection target vehicle 10a calculated based on the external lidar information, etc. can be used to create a third evacuation control value and transmit it to the control target vehicle 10b. This can reduce the possibility that two or more vehicles 10 overlap in the external lidar information when tracking the position of the vehicle 10 using the external lidar information. Therefore, the position of the vehicle 10 can be tracked more accurately using the external lidar information.
[0105] F. Sixth embodiment: FIG. 25 is a first flowchart showing a remote control method in the sixth embodiment. FIG. 26 is a second flowchart showing a remote control method in the sixth embodiment. By detecting one vehicle 10 by a plurality of external sensors 9, it is possible to improve the calculation accuracy when calculating the position of the vehicle 10, and to more accurately detect the vehicle 10 traveling in the boundary area between the detection ranges RG and RF of the adjacent plurality of external sensors 9. Therefore, in this embodiment, a remote control method for calculating the position of one vehicle 10 using a plurality of external lidar information acquired by a plurality of external sensors 9 at the same acquisition time will be described. In this embodiment, the detection target vehicle 10a is set in advance. The remote control method shown in FIG. 25 and FIG. 26 is repeatedly executed at predetermined time intervals, for example, after the vehicle 10 starts traveling in the remote automatic driving mode. In FIG. 25 and FIG. 26, the same steps as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.
[0106] As shown in FIG. 26, when the detection target vehicle 10a can be detected by any of the external sensors 9 included in the detection ranges RG and RF (step S91: No, step S93: No), the actual distance calculation unit 524 calculates the actual distance L1 between the detection target vehicle 10a and the vehicle 10 traveling without any other vehicle 10 between the detection target vehicle 10a and the detection target vehicle 10a (step S40). Then, the control value creation unit 526 sets the target distance L2 (step S50) and creates the evacuation control value (step S71). On the other hand, when the detection target vehicle 10a cannot be detected by any of the external sensors 9 included in the detection ranges RG and RF (step S93: Yes), the control value creation unit 526 creates a stop control value for safely stopping the detection target vehicle 10a (step S75). The transmission unit 527 transmits the control value created by the control value creation unit 526 to the control target vehicle 10b (step S80). When the vehicle 10 can be detected by all of the external sensors 9 that include the vehicle 10 in the detection ranges RG, RF (step S91: Yes), the remote control method ends.
[0107] According to the sixth embodiment, even when the position of one vehicle 10 is calculated using a plurality of pieces of external LIDAR information acquired by a plurality of external sensors 9 at the same acquisition time, it is possible to reduce the possibility that two or more vehicles 10 are overlapped on the sensor information. This makes it possible to calculate the position of the vehicle 10 more accurately using the sensor information.
[0108] Furthermore, according to the sixth embodiment, the position of one detection target vehicle 10a can be calculated using a plurality of external sensors 9. In this way, if the detection target vehicle 10a can be detected by any of the external sensors 9 that include the detection target vehicle 10a in the detection ranges RG, RF, there is no need to stop the detection target vehicle 10a from the viewpoint of ensuring safety. This allows the vehicle 10 to continue traveling in the remote automatic driving mode. This reduces the possibility that the movement of the vehicle 10 will be delayed.
[0109] G. Other embodiments: G-1. Other embodiment 1: The control value creation unit 526 may set the target distance L2 using the actual distance L1 between the first vehicle 101 and the second vehicle 102 that overlap or are expected to overlap in the sensor information, the sensor information, and the vehicle type database D2. In this case, the vehicle type database D2 further includes information on the required distance L3. In this case, the required distance L3 is a constant corresponding to the combination of the type of the first vehicle 101 and the type of the second vehicle 102. When setting the target distance L2 according to the predetermined required distance L3, the control value creation unit 526 acquires vehicle type identification information indicating the type of the first vehicle 101 and vehicle type identification information indicating the type of the second vehicle 102. Then, the control value creation unit 526 refers to the vehicle type database D2 to acquire the required distance L3 corresponding to the combination of the type of the first vehicle 101 and the type of the second vehicle 102 specified by the vehicle type identification information. When the actual distance L1 between the first vehicle 101 and the second vehicle 102, which overlap or are expected to overlap in the sensor information, is equal to or shorter than the required distance L3, the control value creation unit 526 creates an evacuation control value by setting the acquired required distance L3 as the target distance L2, for example. On the other hand, when the actual distance L1 between the first vehicle 101 and the second vehicle 102 is greater than the required distance L3, the control value creation unit 526 sets the target distance L2 to, for example, an arbitrary distance greater than the actual distance L1. In this manner, the target distance L2 can be easily set.
[0110] G-2. Other embodiment 2: The determination unit 525 may determine whether or not the second case applies, without acquiring shape information, based only on the type of the vehicle 10. Even in this form, it is possible to determine whether or not the second case applies.
[0111] G-3. Other embodiment 3: When acquiring the sensor information, the determination unit 525 may determine whether or not the second case applies, using only the actual distance L1 between the first vehicle 101 and the second vehicle 102 that are expected to overlap in the sensor information. In this case, the determination unit 525 determines that the second case applies when, for example, the actual distance L1 between the first vehicle 101 and the second vehicle 102 is less than a predetermined distance threshold. On the other hand, the determination unit 525 determines that the second case does not apply when the actual distance L1 between the first vehicle 101 and the second vehicle 102 is equal to or greater than the distance threshold. In this manner, it is easy to determine whether or not the second case applies.
[0112] G-4. Other embodiment 4: The determination unit 525 may determine whether the second case applies by using the actual distance L1 between the first vehicle 101 and the second vehicle 102 calculated at different times. In this case, the determination unit 525 arranges the actual distance L1 calculated at different times in chronological order, for example. Then, the determination unit 525 determines that the second case applies when the actual distance L1 approaches zero in the chronological order closer to the time when it is determined whether the second case applies. In this form, it is possible to determine whether the second case applies by grasping the time-series change of the actual distance L1 between the first vehicle 101 and the second vehicle 102 calculated at different times.
[0113] G-5. Other embodiment 5: The determination unit 525 may determine whether or not the second case applies by using the actual distance L1 between the first vehicle 101 and the second vehicle 102 calculated at a plurality of different time points. In this case, the determination unit 525 arranges, for example, a first difference, which is a difference between the actual distance L1 between the first time point and a calculation time point immediately before the first time point, in chronological order. The first difference is a value calculated by subtracting the actual distance L1 calculated at the calculation time point immediately before the first time point from the actual distance L1 calculated at the first time point. Then, the determination unit 525 determines that the second case applies when the difference in the actual distance L1 is smaller in the chronological order as it approaches the time point at which it is determined whether or not the second case applies. In this form, it is possible to determine whether or not the second case applies by grasping the time series change of the difference in the actual distance L1 between the first vehicle 101 and the second vehicle 102 calculated at a plurality of different time points.
[0114] G-6. Other embodiment 6: The determination unit 525 may determine whether or not the second case applies by using the actual distance L1 between the first vehicle 101 and the second vehicle 102 calculated at different times. In this case, the determination unit 525 calculates a function representing a time series graph generated by, for example, arranging the actual distance L1 calculated at different times in chronological order and connecting the actual distance L1 at each time. Then, the determination unit 525 calculates a slope in the time series graph by differentiating the function. Then, the determination unit 525 determines whether or not the second case applies by absolute comparison in which the slope in the time series graph is compared with a predetermined differentiation threshold value. In this form, the time series change of the difference in the actual distance L1 between the first vehicle 101 and the second vehicle 102 calculated at different times can be quantitatively represented by a differential value.
[0115] G-7. Other embodiment 7: The determination unit 525 may determine whether the second case applies by using the actual distance L1 between the first vehicle 101 and the second vehicle 102 calculated at a plurality of different time points. In this case, the determination unit 525 calculates, for example, a second difference. The second difference is a value calculated by subtracting the actual distance L1 calculated at the calculation time point immediately preceding the calculation time point closest to the determination time point at which it is determined whether the second case applies. Then, the determination unit 525 determines that the second case applies when the second difference is less than a predetermined difference threshold. On the other hand, when the second difference is equal to or greater than the predetermined difference threshold, it determines that the second case does not apply. With this embodiment, it is possible to determine whether the second case applies by absolute comparison in which the difference in the actual distance L1 between the first vehicle 101 and the second vehicle 102 calculated at a plurality of different time points is compared with a predetermined difference threshold.
[0116] G-8. Other embodiment 8: In each of the above embodiments, the vehicle 10 may have a configuration capable of moving by remote control, and may be in the form of a platform having the configuration described below, for example. Specifically, the vehicle 10 may have at least the drive unit 110, the steering unit 120, the braking unit 130, the vehicle communication unit 140, and the vehicle control unit 150 in order to perform the three functions of "running", "turning", and "stopping" by remote control. That is, the vehicle 10 that can be moved by remote control may not have at least some of the interior parts such as the driver's seat and the dashboard, may not have at least some of the exterior parts such as the bumper and the fenders, and may not have a body shell. In this case, the remaining parts such as the body shell may be attached to the vehicle 10 before the vehicle 10 is shipped from the factory, or the remaining parts such as the body shell may be attached to the vehicle 10 after the vehicle 10 is shipped from the factory in a state in which the remaining parts such as the body shell are not attached to the vehicle 10. Note that the position of the platform may be calculated in the same manner as the vehicle 10 in each embodiment.
[0117] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]
[0118] 1... remote control system, 5... remote control device, 6... running path, 8... reference object, 9... external sensor, 10... vehicle, 10a... detection target vehicle, 10b... control target vehicle, 10e... positioning point, 10f... non-target vehicle, 10g... upper rear end, 10i... front end, 10j... specific part, 51... remote communication unit, 52, 52a... CPU, 53... memory unit, 60... road surface, 61... grid line, 90... imaging device, 95... external lidar, 101... first vehicle, 102... second vehicle, 110... drive unit, 120... steering device, 130... braking device, 140... vehicle communication unit, 150... vehicle control device, 521... acquisition unit, 522 ...position calculation unit, 522a...distortion correction unit, 522b...rotation processing unit, 522c...crop processing unit, 522d...detection unit, 522e...deletion unit, 522f...perspective transformation unit, 522g...coordinate point calculation unit, 523...position conversion unit, 524...actual distance calculation unit, 525...determination unit, 526...control value creation unit, 527...transmission unit, 528...position generation unit, 901...first imaging device, 902...second imaging device, A1...unmoved area, A2...moved area, C...center of gravity, ΔD...observation error, D...first distance, D1...camera database, D2...vehicle model database, Dc...third distance, Do...observation distance, Dp...estimated Fixed distance, Dt...second distance, h...height of the positioning point of the detected vehicle from the road surface, H...height of the imaging device from the road surface, hg...height of the upper rear end of the first vehicle from the road surface, Im1...captured image, Im2...corrected image, Im3...rotated image, Im4...processed image, Im5, Im6...first mask image, Im7...second mask image, J1...first perpendicular line, J2...second perpendicular line, J3...third perpendicular line, L1...actual distance, L2...target distance, L3...required distance, Lf...total length, Li1...distance between the external lidar and the reference object, Li2...distance between the external lidar and the detected vehicle, Lp...total distance, Ls...optical axis reference, Lt... Distance, M1...first mask area, M2...second mask area, Md...position detection model, N...network, P0...base coordinate point, P1...first coordinate point, P2...second coordinate point, P3...image coordinate point, Pa1...distortion correction parameter, Pa2...perspective transformation parameter, Pc...image coordinate point, Pf...fixed coordinate point, Pv...vehicle coordinate point, Q1...first intersection, Q2...second intersection, Q3...third intersection, R0...base circumscribing rectangle, R1...first circumscribing rectangle, R2...second circumscribing rectangle, RF, RG, RG1, RG2...detection range, V...movement vector, Vs1...first detection vector, Vs2...second detection vector
Claims
1. A remote control device that remotely controls a traveling operation of at least some of a plurality of vehicles, an acquisition unit that acquires sensor information representing at least some of the plurality of vehicles, the sensor information being acquired by a sensor installed at a location different from the plurality of vehicles; a detection unit that detects the vehicle included in the sensor information; a control value creation unit that creates a control value that defines the driving operation of at least some of the vehicles among the plurality of vehicles, using the sensor information; a transmitter that transmits the control value to a control target vehicle that is a control target for the driving operation of the plurality of vehicles, The control value creation unit creates an evacuation control value as the control value that specifies the driving operation of at least one of the two or more vehicles, in at least one of a first case in which two or more of the multiple vehicles are overlapping in the sensor information, and a second case in which the two or more vehicles are expected to be overlapping when the sensor information is acquired before the sensor information is acquired, so that a target vehicle among the two or more vehicles that is to be detected by the sensor information can be detected by the detection unit without overlapping with other vehicles among the multiple vehicles.
2. 2. The remote control device according to claim 1, The first case is a case where a first vehicle and a second vehicle among the plurality of vehicles are present in an overlapping manner in the sensor information, The second case is a case where, before the sensor information is acquired, it is predicted that the first vehicle and the second vehicle will be overlapped when the sensor information is acquired, The remote control device further includes an actual distance calculation unit that calculates an actual distance between the first vehicle and the second vehicle, A remote control device, wherein the evacuation control value is a control value that specifies the driving operation of at least one of the first vehicle and the second vehicle so that the actual distance becomes a target distance that is longer than the actual distance at the time when at least one of the first case and the second case occurs.
3. 3. The remote control device according to claim 2, the target distance is set according to an external shape of the first vehicle and an external shape of the second vehicle, A remote control device, wherein the larger the external shape of the first vehicle and the external shape of the second vehicle are, the longer the target distance is, and the smaller the external shape of the first vehicle and the external shape of the second vehicle are, the shorter the target distance is.
4. 3. The remote control device according to claim 2, the second vehicle is a vehicle traveling behind the first vehicle in the same forward direction as the first vehicle, The evacuation control value is an acceleration control value for increasing the acceleration of the first vehicle in the forward direction; an acceleration control value for decreasing the acceleration of the second vehicle in the forward direction; a directional control value for causing the second vehicle to move backward in a direction opposite to the forward direction; a stop control value for stopping the traveling operation of the second vehicle.
5. 5. The remote control device according to claim 4, The avoidance control value further includes a steering angle control value that defines a steering angle of the control target vehicle.
6. 3. The remote control device according to claim 2, A remote control device, wherein the avoidance control value includes a steering angle control value that specifies a steering angle of the controlled vehicle in order to move either the first vehicle or the second vehicle in the width direction of a road.
7. 7. The remote control device according to claim 6, The evacuation control value further includes an acceleration control value that specifies an acceleration of the controlled vehicle.
8. 3. The remote control device according to claim 2, the second vehicle is a vehicle traveling behind the first vehicle in the same forward direction as the first vehicle, The evacuation control value is a control value for moving either the first vehicle or the second vehicle in the width direction of a road to cause the second vehicle to overtake the first vehicle, and includes an acceleration control value that specifies the acceleration of the controlled vehicle, and a steering angle control value that specifies the steering angle of the controlled vehicle.
9. 3. The remote control device according to claim 2, A remote control device, wherein the evacuation control value includes a trajectory control value that defines a travel trajectory by arranging target travel positions of the control target vehicle for each predetermined time in chronological order.
10. 2. The remote control device according to claim 1, A remote control device comprising a judgment unit that judges whether or not the second case applies depending on the type of each of the two or more vehicles that is expected to be included in the sensor information when the sensor information is acquired.
11. 2. The remote control device according to claim 1, a determination unit that determines whether or not the second case applies based on the exterior shape of each of the two or more vehicles that is expected to be included in the sensor information when the sensor information is acquired, the exterior shape of the vehicle differing depending on the type of the vehicle.
12. 1. A remote control system comprising: Multiple vehicles and A sensor that is installed at a location different from the plurality of vehicles and that acquires sensor information representing at least some of the plurality of vehicles; A remote control system comprising: a remote control device according to any one of claims 1 to 11.
13. A remote control method for remotely controlling a traveling operation of at least some of a plurality of vehicles, comprising: an acquisition step of acquiring sensor information representing at least some of the vehicles, the sensor information being acquired by a sensor installed at a location different from the vehicles; a detection step of detecting the vehicle included in the sensor information; a control value creation step of creating a control value that defines the driving operation of at least some of the vehicles among the plurality of vehicles, using the sensor information; a transmission step of transmitting the control value to a control target vehicle, the control value being a control target for the driving operation of the plurality of vehicles, The control value creation process, in at least one of a first case in which two or more of the multiple vehicles are overlapping in the sensor information, and a second case in which the two or more vehicles are expected to be overlapping when the sensor information is acquired before the sensor information is acquired, creates an evacuation control value as the control value that specifies the driving operation of at least one of the two or more vehicles, so that a target vehicle to be detected by the sensor information among the two or more vehicles is in a positional relationship that allows it to be detected in the detection process without overlapping with other vehicles among the multiple vehicles.This is a remote control method.
Citation Information
Patent Citations
Travel support control device
JP2016057677A
Vehicle control system and method for providing information to support overtaking maneuver
US20230091796A1
Method for operating a vehicle and method for operating a manufacturing system
JP2017538619A
