Calculation device
The calculation device improves unmanned vehicle navigation by using onboard and external sensors to enhance accuracy and safety by prioritizing accurate sensor data and stopping operations when thresholds are met, addressing the challenges of external sensor reliability and calibration.
Patent Information
- Application Number
- JP2023190098
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-17
- Filing Date
- 2023-11-07
- Publication Date
- 2024-11-28
- Estimated Expiration
- 2043-11-07
AI Technical Summary
Existing systems for unmanned vehicle operation face challenges in accurately calculating the position and orientation of moving objects due to reliance on external sensors that may decrease in accuracy with distance and require calibration, leading to potential errors in navigation.
A calculation device that utilizes onboard sensors like lidar and external sensors to determine the position and orientation of a target vehicle, selectively using the most accurate sensor information available and stopping calculations when accuracy thresholds are breached, and generating control signals to maintain safety and accuracy.
Enhances the accuracy of position and orientation calculations for unmanned vehicles by prioritizing onboard sensor data and stopping operations when sensor accuracy is compromised, ensuring safe and precise navigation.
Smart Images

Figure 2024166037000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a computing device. [Background technology]
[0002] 2. Description of the Related Art Conventionally, vehicles that run autonomously under remote control are known (see Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Special Publication No. 2017-538619 Summary of the Invention [Problem to be solved by the invention]
[0004] When moving a moving object such as a vehicle by unmanned driving, it is desirable to calculate the position and orientation of the moving object with high accuracy. [Means for solving the problem]
[0005] The present disclosure can be realized in the following forms.
[0006] (1) According to a first aspect of the present disclosure, a calculation device is provided. The calculation device used to move a moving body that can move by unmanned driving includes an acquisition unit that acquires sensor information acquired by a sensor, presence / absence information indicating whether or not a mounted moving body including a target moving body is present within a detection range of the mounted ranging device, and a calculation unit that calculates at least one of a position and an orientation of the target moving body using the sensor information acquired by the acquisition unit, and when the mounted moving body is present, the calculation unit calculates at least one of the position and the orientation of the target moving body using at least the mounted ranging device information acquired by the mounted ranging device as the sensor information, and when the mounted moving body is not present, calculates at least one of the position and the orientation of the target moving body by using at least one of image information acquired by an external camera as the sensor installed at a location different from the target moving body and the mounted moving body, and external ranging device information acquired by an external ranging device as the sensor installed at a location different from the target moving body and the mounted moving body as the sensor information. According to this embodiment, when an on-board moving body is present, at least one of the position and orientation of the target moving body can be calculated using on-board ranging device information acquired by the on-board ranging device mounted on the on-board moving body without using external ranging device information and image information. In this way, when an on-board moving body is present, for example, the greater the distance between the external camera and the external ranging device and the target moving body, the less accurate the calculation of the position and orientation of the target moving body can be. This makes it possible to improve the calculation accuracy of the position and orientation of the target moving body when the target moving body is moved by unmanned driving. (2) In the above embodiment, when there are multiple mounted mobile bodies, the calculation unit may use the mounted distance measuring device information acquired by the mounted distance measuring device capable of detecting the target mobile body most accurately, among the multiple mounted distance measuring devices possessed by the multiple mounted mobile bodies, as the sensor information. According to this embodiment, when there are multiple mounted mobile bodies, at least one of the position and orientation of the target mobile body can be calculated using the mounted distance measuring device information acquired by the mounted distance measuring device capable of detecting the target mobile body most accurately. This makes it possible to improve the calculation accuracy of the position and orientation of the target mobile body. (3) In the above embodiment, the acquisition unit may further acquire calibration information indicating whether or not the calibration of the on-board ranging device has been completed, and the calculation unit may calculate at least one of the position and the orientation of the target moving body by using at least the on-board ranging device information as the sensor information when the on-board moving body is present and the calibration of the on-board ranging device has been completed, and may calculate at least one of the position and the orientation of the target moving body by using at least one of the image information and the external ranging device information as the sensor information without using the on-board ranging device information when the calibration of the on-board ranging device has not been completed when the on-board moving body is present. According to this embodiment, when the calibration of the on-board ranging device has not been completed, at least one of the position and the orientation of the target moving body can be calculated using at least one of the image information and the external ranging device information without using the on-board ranging device information. This makes it possible to more reliably suppress a decrease in the calculation accuracy of the position and orientation of the target moving body. (4) In the above embodiment, when the mounted moving object is present, the acquisition unit acquires at least one of the image information and the external ranging device information as the sensor information in addition to the mounted ranging device information, and the calculation unit calculates at least one of the position and the orientation of the target moving object using the mounted ranging device information as the sensor information, and calculates at least one of the position and the orientation of the target moving object using at least one of the image information and the external ranging device information as the sensor information, and (i) calculates the target movement calculated using the mounted ranging device information. The calculation unit may stop the calculation of the position and the orientation of the target moving body using the sensor information when at least one of the following cases is met: (i) a difference between the position of the moving body and the position of the target moving body calculated using at least one of the image information and the external ranging device information is equal to or greater than a predetermined position threshold; and (ii) a difference between the orientation of the target moving body calculated using the mounted ranging device information and the orientation of the target moving body calculated using at least one of the image information and the external ranging device information is equal to or greater than a predetermined direction threshold. According to this embodiment, when there is a possibility that the detection accuracy of the sensor is degraded, the calculation of the position and the orientation of the target moving body can be stopped. This makes it possible to avoid calculating the position and the orientation of the target moving body based on sensor information whose accuracy may be degraded. (5) The above aspect further includes a signal generating unit that generates a control signal for controlling the operation of the moving body, and a transmitting unit that transmits the control signal generated by the signal generating unit to the moving body, and when the mounted moving body is present, the acquiring unit acquires at least one of the image information and the external ranging device information as the sensor information in addition to the mounted ranging device information, the calculating unit calculates at least one of the position and the orientation of the target moving body using the mounted ranging device information as the sensor information, and calculates at least one of the position and the orientation of the target moving body using at least one of the image information and the external ranging device information as the sensor information, (i) the position of the target moving body calculated using the mounted ranging device information, and the image information and the external ranging device information. In at least one of the cases where (i) a difference between the position of the target moving body calculated using at least one of the on-board ranging device information and the on-board ranging device information is equal to or greater than a predetermined position threshold, and (ii) a difference between the orientation of the target moving body calculated using the on-board ranging device information and the orientation of the target moving body calculated using at least one of the image information and the external ranging device information is equal to or greater than a predetermined direction threshold, the signal generating unit may generate, as the control signal, either (a) a stop control signal for stopping the moving body, or (b) a change control signal for changing a destination when the on-board moving body moves by the unmanned operation from a predetermined target location to a maintenance location where at least one of a repair process for repairing the on-board ranging device and a calibration process for calibrating the on-board ranging device is performed. According to this embodiment, when there is a possibility that the detection accuracy of the sensor is degraded, it is possible to stop the moving body more safely or perform maintenance on the on-board ranging device. The present disclosure can be realized in various forms other than the above-mentioned calculation device, for example, a method for calculating at least one of the position and orientation of a moving object, a method for manufacturing a calculation device, a method for controlling a calculation device, a computer program for realizing the control method, a non-transitory recording medium on which the computer program is recorded, etc. [Brief description of the drawings]
[0007] [Figure 1] FIG. 1 is a diagram illustrating a schematic configuration of a calculation system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a detailed configuration of a calculation system according to the first embodiment. [Diagram 3] 4 is a flowchart showing a processing procedure for vehicle travel control in the first embodiment. [Figure 4] 4 is a flowchart showing a calculation method and an operation control method according to the first embodiment. [Diagram 5] FIG. 4 is a diagram illustrating a method for calculating the position and orientation of a target vehicle. [Figure 6] FIG. 11 is a diagram illustrating a method for calculating the orientation of a target vehicle. [Figure 7] 10 is a flowchart showing a calculation method in the second embodiment. [Figure 8] 10 is a flowchart showing an operation control method in the second embodiment. [Figure 9] FIG. 13 is a diagram illustrating a configuration of a calculation system according to a third embodiment. [Figure 10] 13 is a flowchart showing a processing procedure for vehicle travel control in a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] A. First embodiment: FIG. 1 is a diagram showing a schematic configuration of a calculation system 1 in the first embodiment. The calculation system 1 calculates at least one of the position and the orientation of a target vehicle 101 as a moving body. The calculation system 1 includes one or more vehicles 10 including the target vehicle 101, one or more types of external sensors 300 installed at a location different from the vehicle 10, and a calculation device 5. FIG. 1 illustrates a case in which the target vehicle 101 and an onboard vehicle 102 are traveling on a road R. The onboard vehicle 102 is a vehicle 10 equipped with an onboard LiDAR (Light Detection And Ranging) 161 that includes the target vehicle 101 within a detection range RG1. The onboard LiDAR 161 has a detection range RG1. When the target vehicle 101 is located within the detection range RG1, the onboard LiDAR 161 detects the target vehicle 101. The onboard LiDAR 161 is an example of an onboard distance measuring device as a distance measuring device mounted on the vehicle 10. In other embodiments, the distance measuring device may be another sensor such as a stereo camera. When there is no need to distinguish between the vehicles 101 and 102, they will simply be referred to as "vehicle 10."
[0009] In this disclosure, a "mobile body" refers to an object that can move, such as a vehicle or an electric vertical take-off and landing aircraft (a so-called flying car). The vehicle may be a vehicle that runs on wheels or a vehicle that runs on caterpillar tracks, such as a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, a construction vehicle, etc. The vehicle includes an electric vehicle (BEV: Battery Electric Vehicle), a gasoline-powered vehicle, a hybrid vehicle, and a fuel cell vehicle. When the mobile body is something other than a vehicle, the expressions "vehicle" and "car" in this disclosure can be appropriately replaced with "mobile body", and the expression "running" can be appropriately replaced with "movement".
[0010] The vehicle 10 is configured to be capable of traveling by unmanned driving. "Unmanned driving" means driving without the driving operation of a passenger. Driving operation means at least one of the operations of "running", "turning" and "stopping" of the vehicle 10. Unmanned driving is realized by automatic or manual remote control using a device located outside the vehicle 10, or by autonomous control of the vehicle 10. The vehicle 10 traveling by unmanned driving may have a passenger who does not perform driving operation on board. The passenger who does not perform driving operation includes, for example, a person who simply sits in the seat of the vehicle 10, or a person who performs work other than driving operation, such as assembly, inspection, and operation of switches, while riding on the vehicle 10. Note that driving by a passenger who performs driving operation is sometimes called "manned driving".
[0011] In this specification, "remote control" includes "full remote control" in which all of the operations of the vehicle 10 are completely determined from outside the vehicle 10, and "partial remote control" in which some of the operations of the vehicle 10 are determined from outside the vehicle 10. In addition, "autonomous control" includes "full autonomous control" in which the vehicle 10 autonomously controls its own operations without receiving any information from devices external to the vehicle 10, and "partial autonomous control" in which the vehicle 10 autonomously controls its own operations using information received from devices external to the vehicle 10.
[0012] In this embodiment, the calculation system 1 is used in a factory that manufactures the vehicle 10. The reference coordinate system of the factory is a global coordinate system. That is, any position in the factory is expressed by X, Y, and Z coordinates in the global coordinate system. Note that in other embodiments, the calculation system 1 may be used in a place other than a factory.
[0013] 2 is a diagram showing a detailed configuration of the calculation system 1 in the first embodiment. The vehicle 10 includes an actuator group 120. The actuator group 120 includes an actuator of a drive device for accelerating the vehicle 10, an actuator of a steering device for changing the traveling direction of the vehicle 10, and an actuator of a braking device for decelerating the vehicle 10. Furthermore, the vehicle 10 includes a communication device 130 for communicating with other devices (e.g., the calculation device 5) other than the vehicle 10 and other vehicles 10 using wireless communication or the like, a vehicle control device 150 for controlling the operation of the vehicle 10, and an on-vehicle sensor group 160 having one or more types of internal sensors.
[0014] The vehicle control device 150 includes a vehicle CPU 111, a vehicle storage unit 112, an input / output interface 113, and an internal bus 114. The input / output interface 113 is used to communicate with various devices (e.g., a drive unit) mounted on the vehicle 10, an in-vehicle sensor group 160, and the like. The vehicle CPU 111, the vehicle storage unit 112, and the input / output interface 113 are connected via the internal bus 114 to enable bidirectional communication. The vehicle CPU 111 executes a program PG1 stored in the vehicle storage unit 112 to realize various functions including a function as a vehicle control unit 115.
[0015] The vehicle control unit 115 controls the actuator group 120 to cause the vehicle 10 to run. The vehicle control unit 115 controls the actuator group 120 using the running control signal received from the calculation device 5 to cause the vehicle 10 to run. The running control signal is a control signal for causing the vehicle 10 to run. In this embodiment, the running control signal includes the acceleration and steering angle of the vehicle 10 as parameters. In other embodiments, the running control signal may include the speed of the vehicle 10 as a parameter instead of or in addition to the acceleration of the vehicle 10.
[0016] The on-board sensor group 160 includes, for example, an on-board camera, an on-board radar, and an on-board lidar 161 as sensors that acquire surrounding information indicating the state of the surrounding area of the vehicle 10. The on-board camera captures images of the state of the surrounding area of the vehicle 10. The on-board radar and the on-board lidar 161 detect objects that exist in the surrounding area of the vehicle 10. Note that the configuration of the vehicle 10 is not limited to the above.
[0017] One or more types of external sensors 300 installed at a location different from the vehicle 10 acquire overhead information indicating the state of the vehicle 10 and the surrounding area of the vehicle 10. The external sensors 300 are sensors located outside the vehicle 10. In this embodiment, the calculation system 1 includes one or more external lidars 90 as the external sensors 300 that acquire the overhead information. The external lidars 90 are an example of an external ranging device as a ranging device installed at a location different from the vehicle 10.
[0018] The external lidar 90 is a lidar (Light Detection And Ranging: LiDAR) that detects the target vehicle 101 from outside the target vehicle 101. The external lidar 90 irradiates a predetermined detection range RG2 with laser light and detects the reflected light reflected by the object such as the target vehicle 101, thereby detecting the distance and angle between the external lidar 90 and the object, the shape of the object, and the like. The external lidar 90 transmits the acquired external lidar information to the calculation device 5. The installation positions and the number of external lidars 90 are determined in consideration of the detection range RG2 of each external lidar 90 and objects (obstacles) present in the peripheral area of the road R, in order to detect the entire road R by one or more external lidars 90. Note that the configuration of the external lidar 90 is not limited to the above.
[0019] The calculation device 5 calculates at least one of the position and the orientation of the target vehicle 101 using the sensor information. In this embodiment, the calculation device 5 is a server installed at a location different from the vehicle 10. In this embodiment, the calculation device 5 calculates the position and the orientation of the target vehicle 101. In addition, the calculation device 5 generates a driving control signal that specifies the driving operation of the target vehicle 101, and transmits the driving control signal to the target vehicle 101. The sensor information is acquired by a sensor that acquires at least one of bird's-eye view information and peripheral information. The sensor information is, for example, at least one of on-board LIDAR information that is mounted distance measuring device information acquired by an on-board LIDAR 161 as a mounted distance measuring device, and external LIDAR information that is external distance measuring device information acquired by an external LIDAR 90 as an external distance measuring device. The position of the vehicle 10 is the position of a positioning point 10e that is set in advance for a specific part of the vehicle 10. The position of the positioning point 10e is expressed by coordinate values in a global coordinate system. The heading of the vehicle 10 is a direction represented by a vector extending from the rear side Re to the front side Fr of the vehicle 10 along the fore-aft axis passing through the center of gravity of the vehicle 10.
[0020] The calculation device 5 includes a communication device 51, a device storage unit 53, a device CPU 52, an input / output interface 54, and an internal bus 55. The device storage unit 53, the device CPU 52, and the input / output interface 54 are connected via the internal bus 55 so as to be able to communicate bidirectionally. The input / output interface 54 is connected to the communication device 51 for communicating with various devices outside the calculation device 5. The communication device 51 is capable of communicating with the communication device 130 mounted on the vehicle 10 and the external rider 90, etc. The communication device 51 is, for example, a wireless communication device. The device storage unit 53 includes, for example, a RAM, a ROM, and a hard disk drive (HDD). The device CPU 52 develops various programs PG2 stored in the device storage unit 53 to realize various functions including functions as an acquisition unit 521, a calculation unit 522, a signal generation unit 523, and a transmission unit 524.
[0021] The acquisition unit 521 acquires various information. In this embodiment, the acquisition unit 521 acquires various information including presence / absence information and sensor information. The presence / absence information is information indicating whether or not an on-board vehicle 102 including the target vehicle 101 is present within the detection range RG1. The acquisition unit 521 acquires the presence / absence information, for example, by referring to a presence / absence database and schedule information previously stored in the device storage unit 53. The presence / absence database is, for example, a database that links vehicle identification information for identifying the vehicle 10, whether or not the on-board LIDAR 161 is installed, and the detection range RG1 of the on-board LIDAR 161. The schedule information is, for example, information indicating which vehicle 10 is scheduled to travel at which point in time and at which position on the reference route RR during unmanned driving. Note that the method of acquiring the presence / absence information is not limited to the above. For example, the acquisition unit 521 may acquire the presence / absence information by vehicle-to-vehicle communication between the target vehicle 101 and a vehicle 10 other than the target vehicle 101 equipped with the on-board LIDAR 161. The acquisition unit 521 may also acquire, as presence / absence information, whether or not the target vehicle 101 is included in the in-vehicle LIDAR information.
[0022] The calculation unit 522 calculates at least one of the position and orientation of the target vehicle 101 using the sensor information acquired by the acquisition unit 521. In this embodiment, the calculation unit 522 acquires vehicle position information about the target vehicle 101 by calculating the position and orientation of the target vehicle 101 using the sensor information. The vehicle position information is position information that is the basis for generating a travel control signal. In this embodiment, the vehicle position information includes the position and orientation of the vehicle 10 in the global coordinate system of the factory.
[0023] The signal generating unit 523 generates a driving control signal for controlling the vehicle 10 by defining the driving operation of the vehicle 10. The signal generating unit 523 generates, for example, a standard control signal which is a driving control signal for driving the vehicle 10 along a predetermined reference route RR. In addition, the signal generating unit 523 may generate the following driving control signal according to, for example, the accuracy of the position and orientation of the target vehicle 101 calculated by the calculation unit 522 and the driving situation of the vehicle 10. In this case, the signal generating unit 523 may generate a stop control signal which is a driving control signal for stopping the vehicle 10, or may generate a change control signal which is a driving control signal for changing the destination of the mounted vehicle 102.
[0024] The transmission unit 524 transmits various information. For example, the transmission unit 524 transmits a travel control signal generated by the signal generation unit 523 to the vehicle 10 to be controlled. Note that the configuration of the device CPU 52 is not limited to the above. At least a part of the function of the device CPU 52 may be realized as a function of another device (for example, the external LIDAR 90, the in-vehicle LIDAR 161, the vehicle control device 150).
[0025] Fig. 3 is a flowchart showing a processing procedure for driving control of the vehicle 10 in the first embodiment. The flow shown in Fig. 3 is repeatedly executed at predetermined time intervals from the time when the vehicle 10 starts to drive by remote control, for example.
[0026] In step 1, the calculation unit 522 of the calculation device 5 acquires vehicle position information by using sensor information.
[0027] In step 2, the signal generating unit 523 determines a target position to which the vehicle 10 should next head. In this embodiment, the target position is represented by X, Y, and Z coordinates in a global coordinate system. The device storage unit 53 prestores a reference route RR, which is a route along which the vehicle 10 should travel. The route is represented by nodes indicating the starting point, nodes indicating passing points, nodes indicating the destination, and links connecting the nodes. The signal generating unit 523 uses the vehicle position information and the reference route RR to determine a target position to which the vehicle 10 should next head. The signal generating unit 523 determines a target position on the reference route RR that is ahead of the current location of the vehicle 10.
[0028] In step 3, the signal generating unit 523 generates a travel control signal for driving the vehicle 10 toward the determined target position. The signal generating unit 523 calculates the travel speed of the vehicle 10 from the transition of the position of the vehicle 10, and compares the calculated travel speed with the target speed. When the travel speed is lower than the target speed as a whole, the signal generating unit 523 determines the acceleration so that the vehicle 10 accelerates, and when the travel speed is higher than the target speed, the signal generating unit 523 determines the acceleration so that the vehicle 10 decelerates. In addition, when the vehicle 10 is located on the reference route RR, the signal generating unit 523 determines the steering angle and acceleration so that the vehicle 10 does not deviate from the reference route RR, and when the vehicle 10 is not located on the reference route RR, in other words, when the vehicle 10 deviates from the reference route RR, the signal generating unit 523 determines the steering angle and acceleration so that the vehicle 10 returns to the reference route RR.
[0029] In step 4, the transmission unit 524 transmits the generated driving control signal to the vehicle 10. The calculation device 5 repeats, at a predetermined cycle, the acquisition of vehicle position information, the determination of a target position, the generation of a driving control signal, and the transmission of the driving control signal.
[0030] In step 5, the vehicle control unit 115 of the vehicle control device 150 mounted on the vehicle 10 receives the driving control signal transmitted from the calculation device 5. In step 6, the vehicle control unit 115 controls the actuator group 120 using the received driving control signal, thereby causing the vehicle 10 to drive at the acceleration and steering angle represented in the driving control signal. The vehicle control device 150 repeats receiving the driving control signal and controlling the actuator group 120 at a predetermined cycle. According to the calculation system 1 in this embodiment, the vehicle 10 can be driven by remote control, and the vehicle 10 can be moved without using transportation equipment such as a crane or a conveyor.
[0031] Fig. 4 is a flowchart showing a calculation method and an operation control method in the first embodiment. The calculation method is a method of calculating the position and orientation of the target vehicle 101. The operation control method is a method of generating and transmitting a traveling control signal using the position and orientation of the target vehicle 101 calculated by executing the calculation method, thereby controlling the traveling operation of the vehicle 10. The flowchart shown in Fig. 4 is executed, for example, when the power of the calculation device 5, the external lidar 90, the target vehicle 101, and the mounted vehicle 102 is turned on.
[0032] In step 101, the acquisition unit 521 of the calculation device 5 acquires presence / absence information. If the mounted vehicle 102 is present (step 102: Yes), in step 103, the transmission unit 524 transmits a request signal requesting sensor information to the mounted vehicle 102. On the other hand, if the mounted vehicle 102 is not present (step 102: No), in step 104, the transmission unit 524 transmits a request signal to the external LIDAR 90.
[0033] When the vehicle control device 150 of the onboard vehicle 102 receives the request signal (step 105: Yes), in step 106, the vehicle control device 150 of the onboard vehicle 102 acquires on-board LIDAR information from the on-board LIDAR 161. Then, in step 107, the vehicle control device 150 of the onboard vehicle 102 transmits the on-board LIDAR information to the calculation device 5. As a result, when the onboard vehicle 102 is present, the acquisition unit 521 of the calculation device 5 acquires the on-board LIDAR information as sensor information.
[0034] When the external LIDAR 90 receives the request signal (step 108: Yes), in step 109, the external LIDAR 90 transmits the external LIDAR information to the calculation device 5. As a result, the acquisition unit 521 of the calculation device 5 acquires the external LIDAR information as sensor information when the mounted vehicle 102 is not present.
[0035] If the sensor information acquired by the acquisition unit 521 is on-board sensor information (step 110: Yes), in step 111, the calculation unit 522 uses the on-board LIDAR information to calculate the position and orientation of the target vehicle 101. As a result, the calculation unit 522 acquires vehicle position information.
[0036] FIG. 5 is a diagram for explaining an example of a method for calculating the position and orientation of the target vehicle 101 using on-board lidar information acquired by the on-board lidar 161 mounted on the mounted vehicle 102. In this embodiment, the target vehicle 101 is described as traveling parallel to the mounted vehicle 102. In this disclosure, the X direction along the X axis is the front-rear (total length) direction of the vehicle 10, the +X direction side is the front side Fr of the vehicle 10, and the -X direction side is the rear side Re of the vehicle 10. Therefore, the forward direction of the vehicle 10 is the +X direction, and the backward direction of the vehicle 10 is the -X direction. In addition, the Y direction along the Y axis is the left-right (vehicle width) direction of the vehicle 10, the +Y direction side is the left side Lf of the vehicle 10, and the -Y direction side is the right side Ri of the vehicle 10. The Z direction along the Z axis is the up-down (vehicle height) direction of the vehicle 10, with the +Z direction side being the upper side Tp (ceiling side) of the vehicle 10 and the -Z direction side being the lower side Bt (floor side).
[0037] In the example shown in FIG. 5, a reference object 8 capable of reflecting a laser beam emitted by the on-board lidar 161 is illustrated. A plurality of reference objects 8 may be arranged at different positions. The coordinate values (absolute positions) of one or more reference objects 8 in the global coordinate system are known. The calculation unit 522 calculates the position of the on-board vehicle 102 by detecting one or more reference objects 8 capable of reflecting a laser beam by the on-board lidar 161. Then, based on the calculated position of the on-board vehicle 102 and the first detection vector Vs1, the calculation unit 522 calculates the position and orientation of the target vehicle 101. The first detection vector Vs1 is a vector from the on-board vehicle 102 toward the target vehicle 101. The first detection vector Vs1 has a distance L1 between the on-board vehicle 102 and the target vehicle 101, and a direction from the on-board vehicle 102 to the target vehicle 101. In FIG. 5, the orientation from the on-board vehicle 102 to the target vehicle 101 is represented by an angle θ1 between the optical axis reference Ls and the first detection vector Vs1. The optical axis reference Ls corresponds to the center line of the detection range RG1, which is the irradiation range of the laser light irradiated from the projector of the on-board lidar 161. The reference object 8 is, for example, a pole (support) to which a reflective material that reflects the laser light is attached. Information on the absolute position of the reference object 8 and information on the on-board lidar 161 are stored in advance in, for example, the device storage unit 53. The information on the on-board lidar 161 is, for example, the installation position of the on-board lidar 161 with respect to the on-board vehicle 102, the relative positional relationship between the on-board lidar 161 and the positioning point 10e of the on-board vehicle 102, and the angle difference θs between the longitudinal axis Cp1 of the on-board vehicle 102 and the optical axis reference Ls of the on-board lidar 161.
[0038] The vehicle-mounted lidar 161 acquires a distance L2 from the installation position of the vehicle-mounted lidar 161 to the reference object 8 by measuring the time from when the light projector emits a laser beam to when the light receiver receives the reflected light reflected by a reflector attached to the reference object 8. Similarly, the vehicle-mounted lidar 161 acquires a distance L1 from the installation position of the vehicle-mounted lidar 161 to the target vehicle 101 by receiving reflected light from the target vehicle 101. Furthermore, the vehicle-mounted lidar 161 acquires a transmission angle θ2 of the laser beam to the reference object 8 with respect to the optical axis reference Ls as a direction from the vehicle-mounted lidar 161 to the reference object 8. Similarly, the vehicle-mounted lidar 161 acquires a transmission angle θ1 of the laser beam to the target vehicle 101 with respect to the optical axis reference Ls as a direction from the vehicle-mounted lidar 161 to the target vehicle 101. Then, the distances L1, L2 and the angular differences θ1, θ2 between the optical axis reference Ls and the detection vectors Vs1, Vs2 acquired by the on-board lidar 161 are transmitted to the calculation device 5 as on-board lidar information, and the acquisition unit 521 of the calculation device 5 acquires the on-board lidar information.
[0039] The calculation unit 522 calculates the position of the mounted vehicle 102 in order to calculate the position of the target vehicle 101. Specifically, the calculation unit 522 executes the following process using the absolute position of the reference object 8, the distance L2 from the installation position of the on-board lidar 161 to the reference object 8, and the angle difference θ2 between the optical axis reference Ls of the on-board lidar 161 and the second detection vector Vs2. In this case, the calculation unit 522 uses trigonometric functions to calculate the difference ΔX1 in the coordinate values in the X direction between the on-board lidar 161 and a specific reference object 8 that is one specific reference object 8, and the difference ΔY1 in the coordinate values in the Y direction between the on-board lidar 161 and the specific reference object 8. Then, the calculation unit 522 reflects the difference ΔX1 in the X direction and the difference ΔY1 in the Y direction in the coordinate values indicating the absolute position of the specific reference object 8, taking into account the angle difference θ2 between the optical axis reference Ls of the on-board lidar 161 and the second detection vector Vs2. In this way, the calculation unit 522 calculates the coordinate values of the on-board rider 161 in the global coordinate system.
[0040] Next, the calculation unit 522 executes the following process using the calculated position of the on-board lidar 161, the distance L1 from the on-board lidar 161 to the target vehicle 101, and the angle difference θ1 between the optical axis reference Ls of the on-board lidar 161 and the first detection vector Vs1. In this case, the calculation unit 522 uses trigonometric functions to calculate the difference ΔX2 in the coordinate values in the X direction between the on-board lidar 161 and the target vehicle 101, and the difference ΔY2 in the coordinate values in the Y direction between the on-board lidar 161 and the target vehicle 101. Then, taking into account the angle difference θ1 between the optical axis reference Ls of the on-board lidar 161 and the first detection vector Vs1, the difference ΔX2 in the X direction and the difference ΔY2 in the Y direction are reflected in the coordinate values indicating the position of the on-board lidar 161. As a result, the calculation unit 522 calculates the coordinate values in the global coordinate system of the specific portion 10f of the target vehicle 101 by the on-board lidar 161. The calculation unit 522 calculates the coordinate value in the global coordinate system of the positioning point 10e of the target vehicle 101 from the calculated coordinate value in the global coordinate system of the specific part 10f based on the relative positional relationship between the specific part 10f and the positioning point 10e of the target vehicle 101. Then, the calculation unit 522 adopts the coordinate value in the global coordinate system of the positioning point 10e of the target vehicle 101 as the position of the target vehicle 101.
[0041] The calculation unit 522 calculates the orientation of the vehicle 10 using, for example, the calculated position of the specific part 10f of the target vehicle 101 and the relative positional relationship between the specific part 10f and the front center point Cf and the rear center point Cr. The front center point Cf is located at the center of the front side Fr of the target vehicle 101. The rear center point Cr is located at the center of the rear side Re of the target vehicle 101. The front center point Cf and the rear center point Cr are each present on the front-rear axis Cp2 of the target vehicle 101. In order to calculate the orientation of the target vehicle 101, the calculation unit 522 calculates the coordinate value of the front center point Cf and the coordinate value of the rear center point Cr based on the coordinate value indicating the position of the specific part 10f of the target vehicle 101. Next, the calculation unit 522 calculates the front-rear axis Cp2 of the target vehicle 101 by connecting the front center point Cf and the rear center point Cr. As a result, the calculation unit 522 determines that the direction vector Va, which is a vector that points in the forward direction along the calculated front-rear axis Cp2 of the target vehicle 101, is the direction of the target vehicle 101.
[0042] 5 may be applied to a case where the position and orientation of the target vehicle 101 are calculated using external lidar information. In addition, when the position and orientation of the target vehicle 101 are calculated using either the in-vehicle lidar information or the external lidar information, it is not essential to use information about the reference object 8.
[0043] 4, when the sensor information acquired by the acquisition unit 521 is external LIDAR information (step 110: No), in step 112, the calculation unit 522 uses the external LIDAR information to calculate the position and orientation of the target vehicle 101. As a result, the calculation unit 522 acquires vehicle position information.
[0044] FIG. 6 is a diagram for explaining an example of a method for calculating the orientation of the target vehicle 101 using in-vehicle LIDAR information or external LIDAR information. In the example shown in FIG. 6, the calculation unit 522 calculates the orientation of the target vehicle 101 using coordinate values indicating the positions of multiple detection points Dp1 to Dp4 of the target vehicle 101 detected from the three-dimensional point cloud information included in the external LIDAR information. In FIG. 6, the multiple detection points Dp1 to Dp4 of the target vehicle 101 are a first detection point Dp1, a second detection point Dp2, a third detection point Dp3, and a fourth detection point Dp4. The first detection point Dp1 is located at the left rear end of the target vehicle 101. The second detection point Dp2 is located at the left front end of the target vehicle 101. The third detection point Dp3 is located at the right front end of the target vehicle 101. The fourth detection point Dp4 is located at the right rear end of the target vehicle 101. In the following, a method for calculating the orientation of the target vehicle 101 using the above-mentioned four detection points Dp1 to Dp4 located at the four corners of the target vehicle 101 will be described.
[0045] First, the in-vehicle lidar 161 and the external lidar 90 acquire the relative positional relationship between the four detection points Dp1 to Dp4 of the target vehicle 101 and the specific reference object 8, and transmit it as sensor information to the calculation device 5. Next, the calculation device 5 calculates the coordinate values of each of the detection points Dp1 to Dp4 in the global coordinate system using the absolute position of the specific reference object 8 and the sensor information acquired by the acquisition unit 521. Next, the calculation unit 522 calculates intermediate points Mp1 and Mp2 on the front side Fr and the rear side Re of the target vehicle 101, respectively. Specifically, the calculation unit 522 calculates a first intermediate point Mp1 having a coordinate value of a middle position between the second detection point Dp2 and the third detection point Dp3, and a second intermediate point Mp2 having a coordinate value of a middle position between the first detection point Dp1 and the fourth detection point Dp4. When calculating each of the intermediate points Mp1, Mp2, the calculation unit 522 may calculate coordinate values at least in the X direction and the Y direction. Next, the calculation unit 522 calculates an intermediate axis Ci that is an axis connecting the first intermediate point Mp1 and the second intermediate point Mp2. Then, the calculation unit 522 calculates an intermediate vector Vi along the intermediate axis Ci based on the result of the determination as to whether the target vehicle 101 is traveling in the forward direction or the backward direction. The calculation unit 522 regards the direction of the intermediate vector Vi as the orientation of the target vehicle 101.
[0046] The number of the multiple detection points and their positions on the target vehicle 101 are not limited to those described above. The multiple detection points may be two, and for example, one detection point may be an antenna attached to the ceiling of the target vehicle 101, and the other detection point may be an emblem attached to the front grill or the like on the front side Fr of the target vehicle 101. In this case, the calculation unit 522 calculates the orientation of the target vehicle 101 using the coordinate values of the two detection points in the global coordinate system and the installation positions of the antenna and the emblem with respect to the target vehicle 101, i.e., the relative positional relationship of the antenna and the emblem with respect to the front-rear axis of the target vehicle 101.
[0047] As shown in FIG. 4, in step 113, the signal generating unit 523 of the calculation device 5 executes the following process using the vehicle position information and the reference route RR for the target vehicle 101. In this case, the signal generating unit 523 determines a target position to which the target vehicle 101 should next head, and generates a first control signal for driving the target vehicle 101 toward the determined target position. The first control signal is, for example, a standard control signal. Then, in step 114, the transmitting unit 524 transmits the first control signal to the target vehicle 101. When the vehicle control device 150 of the target vehicle 101 receives the first control signal (step 115: Yes), in step 116, the vehicle control device 150 of the target vehicle 101 controls the actuator group 120 using the received first control signal. Note that the signal generating unit 523 may generate a first control signal for controlling the operation of the mounted vehicle 102 using the vehicle position information and the reference route RR for the mounted vehicle 102.
[0048] According to the first embodiment, when the mounted vehicle 102 exists, the position and orientation of the target vehicle 101 can be calculated using the on-board lidar information acquired by the on-board lidar 161 mounted on the mounted vehicle 102. When a sensor provided at a location different from the vehicle 10 is used, such as the external lidar 90, the ratio of the vehicle 10 shown in the detection range RG2 decreases as the distance between the vehicle 10 and the sensor increases. Therefore, there is a risk that the calculation accuracy of the position and orientation of the target vehicle 101 will decrease. On the other hand, according to the first embodiment, when the target vehicle 101 is equipped with the on-board lidar 161, the position and orientation of the target vehicle 101 can be calculated without using sensor information acquired by a sensor provided at a location different from the vehicle 10, such as the external lidar 90. This makes it possible to suppress a decrease in the calculation accuracy of the position and orientation of the target vehicle 101 when the mounted vehicle 102 exists.
[0049] Moreover, according to the first embodiment, when the mounted vehicle 102 is not present, the position and orientation of the target vehicle 101 can be calculated using external lidar information. For example, when calculating the position and orientation of the target vehicle 101 by analyzing image information including the target vehicle 101, the calculation accuracy of the position and orientation of the target vehicle 101 may decrease due to differences in imaging conditions such as the weather, season, time of day, and imaging location at the time of imaging. On the other hand, according to the first embodiment, the position and orientation of the target vehicle 101 can be calculated without using image information. This makes it possible to suppress a decrease in the calculation accuracy of the position and orientation of the target vehicle 101 when the mounted vehicle 102 is not present.
[0050] Furthermore, according to the first embodiment, when the vehicle 10 is driven in an unmanned manner, the driving operation of the vehicle 10 can be controlled by generating a control signal using the calculated position and orientation of the vehicle 10.
[0051] B. Second embodiment: Fig. 7 is a flowchart showing a calculation method in the second embodiment. Fig. 8 is a flowchart showing an operation control method in the second embodiment. In this embodiment, a method of generating different driving control signals according to the detection accuracy of the sensor that acquires the sensor information will be described.
[0052] 7, in step 201, the acquisition unit 521 of the calculation device 5 acquires presence / absence information. If the mounted vehicle 102 is present (step 202: Yes), in step 203, the transmission unit 524 transmits a request signal to the mounted vehicle 102 and the external LIDAR 90. On the other hand, if the mounted vehicle 102 is not present (step 202: No), in step 204, the transmission unit 524 transmits a request signal to the external LIDAR 90.
[0053] When the vehicle control device 150 of the onboard vehicle 102 receives the request signal (step 205: Yes), in step 206, the vehicle control device 150 of the onboard vehicle 102 acquires on-board LIDAR information from the on-board LIDAR 161. Then, in step 207, the vehicle control device 150 of the onboard vehicle 102 transmits the on-board LIDAR information to the calculation device 5. When the external LIDAR 90 receives the request signal (step 208: Yes), in step 209, the external LIDAR 90 transmits the external LIDAR information to the calculation device 5.
[0054] If an on-board vehicle 102 is present, that is, if the acquisition unit 521 has acquired both the on-board LIDAR information and the external LIDAR information (step 210: Yes), the calculation unit 522 executes steps 211 and 212. In step 211, the calculation unit 522 acquires vehicle position information by calculating the position and orientation of the target vehicle 101 using the on-board LIDAR information. Furthermore, in step 212, the calculation unit 522 acquires vehicle position information by calculating the position and orientation of the target vehicle 101 using the external LIDAR information. Either step 211 or step 212 may be executed first, or they may be executed simultaneously in parallel.
[0055] As shown in FIG. 8, when the position difference is less than the predetermined position threshold (step 214: No), the signal generating unit 523 executes step 215. The position difference is the difference between the position of the positioning point 10e of the target vehicle 101 calculated using the on-board LIDAR information and the position of the positioning point 10e of the target vehicle 101 calculated using the external LIDAR information corresponding to the acquisition timing of the on-board LIDAR information. When the direction difference is less than the predetermined direction threshold (step 215: No), the signal generating unit 523 executes step 216. In step 216, the signal generating unit 523 generates a first control signal. The direction difference is the difference between a vector Va indicating the orientation of the target vehicle 101 calculated using the on-board LIDAR information and a vector Vi indicating the orientation of the target vehicle 101 calculated using the external LIDAR information corresponding to the acquisition timing of the on-board LIDAR information. In at least one of the cases where the position difference is equal to or greater than the position threshold (step 214: Yes) and where the direction difference is equal to or greater than the direction threshold (step 215: Yes), in step 217, the signal generating unit 523 generates a second control signal. The second control signal is, for example, a travel control signal, which is either a stop control signal or a change control signal. The stop control signal is a travel control signal for stopping at least one of the target vehicle 101 and the mounted vehicle 102. The change control signal is a travel control signal for changing the destination when the mounted vehicle 102 travels by remote control from the outside from a predetermined target location to a maintenance location. The maintenance location is a location where at least one of a repair process for repairing the on-board lidar 161 and a calibration process for calibrating the on-board lidar 161 is performed. The calibration process is, for example, a process for adjusting the optical axis reference Ls of the on-board lidar 161 shown in FIG. 5, or a process for correcting the distances L1, L2 and the angle differences θ1, θ2 acquired by the on-board lidar 161 so that they are correct. 8, in step 218, the transmission unit 524 transmits the driving control signal generated by the signal generation unit 523 to the vehicle 10 whose driving operation is to be controlled. If the vehicle control device 150 of the vehicle 10 receives the driving control signal (step 219: Yes), in step 220, the vehicle control device 150 controls the actuator group 120 using the received driving control signal.
[0056] On the other hand, as shown in Fig. 7, when the mounted vehicle 102 is not present, that is, when the acquisition unit 521 acquires external LIDAR information without acquiring on-board LIDAR information (step 210: No), the calculation unit 522 executes step 213. In step 213, the calculation unit 522 acquires vehicle position information by calculating the position and orientation of the target vehicle 101 using the external LIDAR information. Then, as shown in Fig. 8, in step 216, the signal generation unit 523 generates a first control signal. Note that when the mounted vehicle 102 is not present, each step from step 216 onwards is the same as when the mounted vehicle 102 is present.
[0057] According to the second embodiment, when the mounted vehicle 102 exists, the on-board LIDAR information and the external LIDAR information are acquired as sensor information, and the position and orientation of the target vehicle 101 are calculated from each piece of sensor information, thereby calculating the position difference and the direction difference. Then, when at least one of the position difference and the direction difference is equal to or greater than a threshold, a stop control signal is generated to stop the vehicle 10 more safely. In this way, it is possible to grasp that the detection accuracy of at least one of the on-board LIDAR 161 and the external LIDAR 90 as sensors may be degraded. Then, when at least one of the position difference and the direction difference is equal to or greater than a threshold, that is, when the detection accuracy of the sensor may be degraded, the vehicle 10 can be stopped more safely.
[0058] Furthermore, according to the second embodiment, when at least one of the position difference and the direction difference is equal to or greater than a threshold, a change control signal can be generated to drive the mounted vehicle 102 toward a maintenance location. In this way, when there is a possibility that the detection accuracy of the on-board LIDAR 161 has decreased, the on-board LIDAR 161 can be maintained. Furthermore, by inspecting whether or not the on-board LIDAR 161 has a defect such as optical axis deviation at the maintenance location, it can be estimated whether the defect is occurring in the on-board LIDAR 161 or the external LIDAR 90.
[0059] C. Third embodiment: 9 is a diagram showing the configuration of a calculation system 1v in the third embodiment. In this embodiment, the function of the calculation device 5 is realized by a vehicle control device 150v. As a result, the vehicle 10v in this embodiment can run by the autonomous control of the vehicle 10v.
[0060] The vehicle CPU 111v of the vehicle control device 150v executes the program PG1v stored in the vehicle memory unit 112v to realize various functions including an acquisition unit 116, a calculation unit 117, a signal generation unit 118, and the function of a vehicle control unit 115v.
[0061] Fig. 10 is a flowchart showing a processing procedure of the driving control of the vehicle 10v in the third embodiment. The flow shown in Fig. 10 is repeatedly executed at predetermined time intervals from the time when the vehicle 10v starts driving under autonomous control, for example.
[0062] In step 11, the calculation unit 117 of the vehicle control device 150v mounted on the vehicle 10v acquires vehicle position information using sensor information. In step 21, the signal generation unit 118 determines a target position to which the vehicle 10v should next head. In step 31, the signal generation unit 118 generates a driving control signal for driving the vehicle 10v toward the determined target position. In step 41, the vehicle control unit 115v controls the actuator group 120 using the generated driving control signal to drive the vehicle 10v according to the parameters represented in the driving control signal. The vehicle control device 150v repeats the acquisition of vehicle position information, the determination of the target position, the generation of the driving control signal, and the control of the actuators at a predetermined cycle. According to the calculation system 1v in this embodiment, the vehicle 10v can be driven by the autonomous control of the vehicle 10v without remotely controlling the vehicle 10v by the calculation device 5 installed at a location different from the vehicle 10v, such as a server.
[0063] D. Other embodiments: D-1. Other embodiment 1: When there are multiple mounted vehicles 102 for one target vehicle 101, the calculation unit 117, 522 may use the following sensor information as the on-board LIDAR information. In this case, the calculation unit 117, 522 may use the on-board LIDAR information acquired by the on-board LIDAR 161 capable of detecting the target vehicle 101 most accurately, among the multiple on-board LIDARs 161 possessed by the multiple mounted vehicles 102, as the sensor information. The detection accuracy of the target vehicle 101 is determined according to, for example, the distance L1 between the target vehicle 101 and the mounted vehicle 102, the shape and the size of the target vehicle 101, the angle difference θ1 between the front-rear axis Cp2 of the target vehicle 101 and the optical axis reference Ls of the on-board LIDAR 161, and the presence or absence of obstacles present in the surrounding area of the target vehicle 101 and the mounted vehicle 102. For example, when the angle difference θ1 between the target vehicle 101 and the optical axis reference Ls of the on-board lidar 161 mounted on each of the multiple mounted vehicles 102 is the same and no obstacle exists in the surrounding area, the calculation unit 117, 522 makes the following judgment. In this case, the calculation unit 117, 522 judges that the on-board lidar 161 mounted on the mounted vehicle 102 having the closest distance L1 to the target vehicle 101 can most accurately detect the target vehicle 101. In this form, when there are multiple mounted vehicles 102, the position and orientation of the target vehicle 101 can be calculated using on-board lidar information acquired by the on-board lidar 161 that can most accurately detect the target vehicle 101. This can improve the calculation accuracy of the position and orientation of the target vehicle 101.
[0064] D-2. Other embodiment 2: The calculation system 1, 1v may include an external camera as an external sensor 300 that acquires bird's-eye view information installed at a location different from the vehicle 10, 10v instead of the external lidar 90 or in addition to the external lidar 90. The external camera is an imaging device such as a CCD image sensor that acquires a captured image as image information including the target vehicle 101 as sensor information and outputs the captured image as a detection result. When the external sensor 300 is an external camera, the calculation unit 117, 522 acquires vehicle position information using the captured image acquired from the external camera that is the external sensor 300. In detail, the calculation unit 117, 522, for example, detects the outer shape of the vehicle 10, 10v from the captured image, calculates the coordinates of the positioning point 10e of the vehicle 10, 10v in the coordinate system of the captured image, i.e., the local coordinate system, and converts the calculated coordinates into coordinates in the global coordinate system to acquire the position of the vehicle 10, 10v. The external shape of the vehicle 10, 10v included in the captured image can be detected, for example, by inputting the captured image into a detection model utilizing artificial intelligence. The detection model is prepared, for example, in the calculation system 1, 1v or outside the calculation system 1, 1v, and is stored in advance in the storage unit 53, 112v. As the detection model, for example, a trained machine learning model trained to realize either semantic segmentation or instance segmentation can be used. As this machine learning model, for example, a convolutional neural network (hereinafter, CNN) trained by supervised learning using a training dataset can be used. The training dataset has, for example, a plurality of training images including the vehicle 10, 10v, and a label indicating whether each region in the training image is a region indicating the vehicle 10, 10v or a region indicating other than the vehicle 10, 10v. During learning of the CNN, it is preferable to update the parameters of the CNN by backpropagation (error backpropagation method) so as to reduce an error between the output result of the detection model and the label. In addition, the calculation unit 117, 522 can obtain the orientation of the vehicle 10, 10v by estimating it based on the orientation of the movement vector of the vehicle 10, 10v calculated from the positional change of the characteristic points of the vehicle 10, 10v between frames of the captured image, for example, using an optical flow method.In this embodiment, the position and orientation of the target vehicle 101 can be calculated by inputting image information into a pre-trained machine learning model that has been trained to detect the outer shape (outline) of the target vehicle 101 in the image.
[0065] D-3. Other embodiment 3: When the on-board vehicle 102 exists and the calibration of the on-board LIDAR 161 mounted on the on-board vehicle 102 is not completed, the calculation unit 117, 522 may calculate the position and orientation of the target vehicle 101 by using at least one of the image information and the external LIDAR information as the sensor information, without using the on-board LIDAR information as the sensor information. In this case, for example, the acquisition unit 116, 521 acquires calibration information indicating whether the calibration of the on-board LIDAR 161 is completed or not. Specifically, the acquisition unit 116, 521 acquires the calibration information about the on-board LIDAR 161 mounted on the on-board vehicle 102, for example, by referring to a calibration database. The calibration database is, for example, a database that links the vehicle identification information, the on-board LIDAR identification information that identifies the on-board LIDAR 161, and the calibration status of the on-board LIDAR 161. In this way, the calculation device 5 selects the type of sensor information to be used when calculating the position and orientation of the target vehicle 101 based on the calibration information. When the calibration of the on-board LIDAR 161 is not completed, the acquisition unit 116, 521 may acquire only at least one of the image information and the external LIDAR information as the sensor information without acquiring the on-board LIDAR information. Also, even when the calibration of the on-board LIDAR 161 is not completed, the acquisition unit 116, 521 may acquire the on-board LIDAR information as the sensor information in addition to at least one of the image information and the external LIDAR information. In this case, the calculation unit 117, 522 calculates the position and orientation of the target vehicle 101 using information other than the on-board LIDAR information among the acquired sensor information. Also, when the calibration of the on-board LIDAR 161 is not completed, the vehicle control device 150 of the mounted vehicle 102 may transmit an incompletion signal indicating that the calibration of the on-board LIDAR 161 is not completed to the calculation device 5 installed at a location different from the vehicle 10, 10v such as a server without acquiring the on-board LIDAR information. Even if the calibration of the on-board LIDAR 161 is not completed, the vehicle control device 150, 150v of the on-board vehicle 102 may acquire the on-board LIDAR information. In this case, the vehicle control device 150 of the on-board vehicle 102 may transmit an incompletion signal to the calculation device 5 such as a server instead of the acquired on-board LIDAR information.In this embodiment, the position and orientation of the target vehicle 101 can be calculated using the on-board lidar information acquired by the calibrated on-board lidar 161. In this way, it is possible to more reliably prevent the calculation accuracy of the position and orientation of the target vehicle 101 from decreasing. This reduces the possibility of generating a travel control signal based on sensor information whose detection accuracy may be degraded. Therefore, it is possible to prevent the target vehicle 101 from traveling a route different from the desired travel route.
[0066] D-4. Other embodiment 4: When the mounted vehicle 102 is present, the calculation unit 117, 522 may stop the calculation of the position and orientation of the target vehicle 101 using the sensor information in at least one of the cases where the position difference is equal to or greater than the position threshold and the direction difference is equal to or greater than the direction threshold. In this way, the calculation of the position and orientation of the target vehicle 101 can be stopped when there is a possibility that the detection accuracy of the sensor is degraded. This makes it possible to avoid calculating the position and orientation of the target vehicle 101 based on sensor information whose detection accuracy may be degraded. Therefore, it is possible to avoid generating a travel control signal based on an incorrect position and orientation of the target vehicle 101. Therefore, it is possible to prevent the target vehicle 101 from traveling along a route different from the desired travel route.
[0067] D-5. Other embodiment 5: When the onboard vehicle 102 is present, the calculation unit 117, 522 may estimate the sensor having a malfunction when at least one of the following cases is satisfied: the position difference is equal to or greater than the position threshold value, and the direction difference is equal to or greater than the direction threshold value. The calculation unit 117, 522 estimates the sensor having a malfunction by, for example, comparing the position difference and the direction difference calculated from the sensor information of the onboard LIDAR 161 and the external LIDAR 90 acquired at different times. Then, the calculation unit 117, 522 may stop the calculation of the position and the orientation of the target vehicle 101 using the sensor information from the sensor estimated to have a malfunction, without stopping the calculation of the position and the orientation of the target vehicle 101 using the sensor information from the sensor estimated to have a malfunction. In this manner, it is possible to estimate which sensor has a malfunction. As a result, it is possible to continue the calculation of the position and the orientation of the target vehicle 101 using the sensor information acquired by the sensor whose detection accuracy is unlikely to be deteriorated.
[0068] D-6. Other embodiment 6: The calculation device 5 may further include a notification control unit that notifies the user of the specific information when the mounted vehicle 102 exists and at least one of the following cases is found: when the position difference is equal to or greater than the position threshold value and when the direction difference is equal to or greater than the direction threshold value. The specific information is information indicating that the detection accuracy of the sensor that acquires the sensor information may be degraded. When the detection accuracy of the sensor may be degraded, the notification control unit notifies the user of the specific information, for example, by displaying a message indicating the specific information on the display device or by playing a sound indicating the specific information from the speaker. In this form, the user can be notified as soon as possible that the detection accuracy of the sensor may be degraded. This can reduce the possibility of generating a traveling control signal based on information whose detection accuracy may be degraded, or reduce the number of vehicles 10, 10v that stop by receiving a stop control signal.
[0069] D-7. Other embodiment 7: The calculation device 5 may calculate only one of the position and the orientation of the target vehicle 101. Even in this embodiment, at least one of the position and the orientation of the target vehicle 101 can be used as one piece of information when generating a control signal.
[0070] D-8. Other embodiment 8: In each of the first to third embodiments, the calculation device 5 selects the type of sensor information to be used when calculating the position and orientation of the target vehicle 101 based on the presence / absence information. In contrast, the calculation device 5 may include a determination unit that determines whether the mounted vehicle 102 is present based on information on whether the information of the on-board LIDAR 161 can be acquired from the mounted vehicle 102 and information on whether the on-board LIDAR information includes the target vehicle 101. In this embodiment, the type of sensor information to be used when calculating the position and orientation of the target vehicle 101 can be selected based on whether the on-board LIDAR information can be acquired and whether the on-board LIDAR information includes the target vehicle 101.
[0071] D-9. Other embodiment 9: In the calculation system 1, 1v, the function of the calculation device 5 may be realized by a plurality of devices. The calculation system 1, 1v may include, for example, a calculation device 5 including an acquisition unit 116, 521 and a calculation unit 117, 522, and a remote control device including a signal generation unit 118, 523 and a transmission unit 524. In this case, the calculation device 5 and the remote control device are provided in a location different from the vehicle 10, 10v. Even in such a form, the position and orientation of the target vehicle 101 can be calculated using sensor information according to the mounting status of the on-board lidar 161.
[0072] D-10. Other embodiment 10: A plurality of reference objects 8 may be installed near the road R. When a plurality of reference objects 8 are installed near the road R, reflective materials may be attached to each reference object 8 so that the arrangement patterns, such as the attachment position, attachment range, and attachment number, are different. In this way, when the reference object 8 is detected by the in-vehicle lidar 161 and the external lidar 90, differences occur in the three-dimensional point cloud information indicating each reference object 8 in the in-vehicle lidar information and the external lidar information. This allows the calculation device 5 to distinguish between the plurality of reference objects 8 included in the in-vehicle lidar information and the external lidar information. In addition, the calculation device 5 may distinguish between the plurality of reference objects 8, for example, based on the positional relationship (separation distance) between the position of the vehicle 10 and the absolute position of each reference object 8. In this form, the position and orientation of the target vehicle 101 can be calculated using the plurality of reference objects 8 whose absolute positions are known as indicators. Therefore, the calculation accuracy of the position and orientation of the target vehicle 101 can be improved.
[0073] D-11. Other embodiment 11: When calculating the position and orientation of the vehicle 10, 10v using three-dimensional point cloud information as external LIDAR information, the calculation unit 117, 522 may acquire the vehicle position information as follows. In this case, the calculation unit 117, 522 may acquire the vehicle position information by template matching using, for example, three-dimensional point cloud information as external LIDAR information output from the external LIDAR 90 as a detection result and reference point cloud information prepared in advance.
[0074] D-12. Other embodiment 12: In the first and second embodiments, the processes from obtaining vehicle position information to generating a driving control signal are executed by the calculation device 5. In contrast, at least a part of the processes from obtaining vehicle position information to generating a driving control signal may be executed by the vehicle 10. For example, the following forms (1) to (3) may be used.
[0075] (1) The calculation device 5 may acquire vehicle position information, determine a target position to which the vehicle 10 should next head, and generate a route from the current location of the vehicle 10 represented in the acquired vehicle position information to the target position. The calculation device 5 may generate a route to a target position between the current location and the destination, or may generate a route to the destination. The calculation device 5 may transmit the generated route to the vehicle 10. The vehicle 10 may generate a driving control signal so that the vehicle 10 drives on the route received from the calculation device 5, and control the actuator group 120 using the generated driving control signal.
[0076] (2) The calculation device 5 may acquire vehicle position information and transmit the acquired vehicle position information to the vehicle 10. The vehicle 10 may determine a target position to which the vehicle 10 should next head, generate a route from the current location of the vehicle 10 represented in the received vehicle position information to the target position, generate a driving control signal so that the vehicle 10 travels on the generated route, and control the actuator group 120 using the generated driving control signal.
[0077] (3) In the above-mentioned (1) and (2) embodiments, the vehicle 10 may be equipped with an internal sensor, and a detection result output from the internal sensor may be used for at least one of generating a route and generating a driving control signal. The internal sensor is a sensor equipped in the vehicle 10. The internal sensor may include, for example, a sensor for detecting a motion state of the vehicle 10, a sensor for detecting an operating state of each part of the vehicle 10, and a sensor for detecting an environment around the vehicle 10. Specifically, the internal sensor may include, for example, a camera, a LiDAR, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, a gyro sensor, and the like. For example, in the above-mentioned (1) embodiment, the calculation device 5 may acquire a detection result of the internal sensor, and may reflect the detection result of the internal sensor on the route when generating the route. In the above-mentioned (1) embodiment, the vehicle 10 may acquire a detection result of the internal sensor, and may reflect the detection result of the internal sensor on the driving control signal when generating the driving control signal. In the above-mentioned (2) embodiment, the vehicle 10 may acquire a detection result of the internal sensor, and may reflect the detection result of the internal sensor on the route when generating the route. In the above embodiment (2), the vehicle 10 may acquire the detection result of the internal sensor, and may reflect the detection result of the internal sensor in the driving control signal when generating the driving control signal.
[0078] D-13. Other embodiment 13: In the above third embodiment, the vehicle 10v may be equipped with an internal sensor, and a detection result output from the internal sensor may be used for at least one of generating a route and generating a driving control signal. For example, the vehicle 10v may acquire a detection result from the internal sensor, and when generating a route, may reflect the detection result from the internal sensor in the route. The vehicle 10v may acquire a detection result from the internal sensor, and when generating a driving control signal, may reflect the detection result from the internal sensor in the driving control signal.
[0079] D-14. Other embodiment 14: In the third embodiment, the vehicle 10v acquires the vehicle position information using the detection result of the external sensor 300. In contrast, the vehicle 10v may be equipped with an internal sensor, acquire vehicle position information using the detection result of the internal sensor, determine a target position to which the vehicle 10v should next head, generate a route from the current location of the vehicle 10v represented in the acquired vehicle position information to the target position, generate a travel control signal for traveling the generated route, and control the actuator group 120 using the generated travel control signal. In this case, the vehicle 10v can travel without using any of the detection results of the external sensor 300. The vehicle 10v may acquire a target arrival time or traffic congestion information from outside the vehicle 10v and reflect the target arrival time or traffic congestion information in at least one of the route and the travel control signal. In addition, all of the functional configurations of the calculation system 1, 1v may be provided in the vehicle 10v. That is, the processing realized by the calculation system 1, 1v in the present disclosure may be realized by the vehicle 10v alone. For example, an on-board vehicle 102 equipped with an on-board lidar 161 can realize all the functions of the calculation system 1, 1v by the vehicle 10v alone.
[0080] D-15. Other embodiment 15: In the first and second embodiments, the calculation device 5 automatically generates a driving control signal to be transmitted to the vehicle 10. In contrast, the calculation device 5 may generate a driving control signal to be transmitted to the vehicle 10 according to the operation of an external operator located outside the vehicle 10. For example, the external operator may operate a steering device including a display for displaying an image output from the external sensor 300, a steering wheel for remotely operating the vehicle 10, an accelerator pedal, a brake pedal, and a communication device for communicating with the calculation device 5 by wired communication or wireless communication, and the calculation device 5 may generate a driving control signal according to the operation applied to the steering device. In this case, the calculation device 5 may notify the external operator by, for example, displaying the calculated position and orientation of the vehicle 10 on the display of the steering device.
[0081] D-16. Other embodiment 16: In each of the above embodiments, the vehicle 10, 10v may have a configuration capable of moving by unmanned driving, and may be in the form of a platform having the configuration described below, for example. Specifically, the vehicle 10, 10v may have at least a vehicle control device 150, 150v and an actuator group 120 in order to perform the three functions of "running", "turning" and "stopping" by unmanned driving. When the vehicle 10, 10v acquires information from the outside for unmanned driving, the vehicle 10, 10v may further have a communication device 130. That is, the vehicle 10, 10v capable of moving by unmanned driving may not be equipped with at least a part of interior parts such as a driver's seat and a dashboard, may not be equipped with at least a part of exterior parts such as a bumper and a fender, and may not be equipped with a body shell. In this case, the remaining parts such as the body shell may be attached to the vehicle 10, 10v before the vehicle 10, 10v is shipped from the factory, or the remaining parts such as the body shell may be attached to the vehicle 10, 10v after the vehicle 10, 10v is shipped from the factory without the remaining parts such as the body shell being attached to the vehicle 10, 10v. Each part may be attached from any direction such as the upper side, lower side, front side, rear side, right side, or left side of the vehicle 10, and may be attached from the same direction or from different directions. Note that the position of the platform shape may be determined in the same manner as the vehicle 10, 10v in the first embodiment.
[0082] D-17. Other embodiment 17: The vehicle 10, 10v may be manufactured by combining a plurality of modules. The module means a unit composed of a plurality of parts grouped according to the part or function of the vehicle 10, 10v. For example, the platform of the vehicle 10, 10v may be manufactured by combining a front module constituting the front part of the platform, a central module constituting the central part of the platform, and a rear module constituting the rear part of the platform. The number of modules constituting the platform is not limited to three, and may be two or less or four or more. In addition to or instead of the parts constituting the platform, parts constituting parts of the vehicle 10, 10v other than the platform may be modularized. The various modules may include any exterior parts such as a bumper or a grille, or any interior parts such as a seat or a console. In addition, not limited to the vehicle 10, 10v, any type of moving body may be manufactured by combining a plurality of modules. Such a module may be manufactured, for example, by joining a plurality of parts by welding or a fastener, or may be manufactured by integrally molding at least a part of the parts constituting the module as one part by casting. The molding method for integrally molding a single component, particularly a relatively large component, is also called gigacast or megacast. For example, the front module, the center module, and the rear module may be manufactured using gigacast.
[0083] D-18. Other embodiment 18: Transporting the vehicles 10, 10v by using the unmanned driving of the vehicles 10, 10v is also called "self-propelled transport." Also, the configuration for realizing self-propelled transport is also called a "vehicle remote-controlled autonomous driving transport system." Also, the production method for producing the vehicles 10, 10v by using self-propelled transport is also called "self-propelled production." In self-propelled production, for example, at a factory where the vehicles 10, 10v are manufactured, at least a portion of the transport of the vehicles 10, 10v is realized by self-propelled transport.
[0084] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]
[0085] 1,1v...calculation system, 5...calculation device, 8...reference object, 10,10v...vehicle, 10e...positioning point, 10f...specific part, 51...communication device of calculation device, 52...device CPU, 53...device memory unit, 54...input / output interface of calculation device, 55...internal bus of calculation device, 90...external lidar, 101...target vehicle, 102...mounted vehicle, 111,111v...vehicle CPU, 112,112v...vehicle memory unit, 113...input / output interface of vehicle control device, 114...internal bus of vehicle control device, 115,115v...vehicle control unit, 116,521...acquisition unit, 117,522...calculation unit, 118,523...signal generation unit, 120...actuator group, 130...vehicle communication device, 150,150v...vehicle control control device, 160...vehicle-mounted sensor group, 161...vehicle-mounted lidar, 300...external sensor, 524...transmitter, Bt...lower side, Cf...front center point, Ci...middle axis, Cp1...front-rear axis of mounted vehicle, Cp2...front-rear axis of target vehicle, Cr...rear center point, Dp1...first detection point, Dp2...second detection point, Dp3...third detection point, Dp4...fourth detection point, Fr...forward side, L1, L2...distance, Lf...left side, Ls...optical axis reference, Mp1...first intermediate point, Mp2...second intermediate point, PG1, PG1v, PG2...program, R...road, RG1, RG2...detection range, Re...rear side, Ri...right side, RR...reference route, Tp...upper side, Va...direction vector, Vi...middle vector, Vs1...first detection vector, Vs2...second detection vector
Claims
1. A calculation device used to move a moving body that can be moved by unmanned operation, an acquisition unit that acquires sensor information acquired by a sensor and presence / absence information indicating whether or not a mounted mobile object including a target mobile object is present within a detection range of the mounted distance measuring device, the mounting mobile object being a mounted distance measuring device as the sensor; a calculation unit that calculates at least one of a position and an orientation of the target moving object using the sensor information acquired by the acquisition unit, The calculation unit is When the mounted moving object is present, at least one of the position and the orientation of the target moving object is calculated using mounted ranging device information acquired by at least the mounted ranging device as the sensor information; A calculation device that calculates at least one of the position and orientation of the target moving body by using as the sensor information at least one of image information acquired by an external camera as the sensor installed at a location different from the target moving body and the mounted moving body, and external ranging device information acquired by an external ranging device as the sensor installed at a location different from the target moving body and the mounted moving body when the mounted moving body is not present.
2. The calculation device according to claim 1 , A calculation device in which, when there are multiple mounted moving bodies, the calculation unit uses the mounted ranging device information acquired by the mounted ranging device that can most accurately detect the target moving body, among the multiple mounted ranging devices possessed by the multiple mounted moving bodies, as the sensor information.
3. The calculation device according to claim 1 , The acquisition unit further acquires calibration information indicating whether or not calibration of the on-board distance measuring device has been completed; The calculation unit is When the on-board moving object is present, if the calibration of the on-board ranging device is completed, at least the on-board ranging device information is used as the sensor information to calculate at least one of the position and the orientation of the target moving object; A calculation device that, when the mounted moving body is present and calibration of the mounted ranging device has not been completed, calculates at least one of the position and orientation of the target moving body by using at least one of the image information and the external ranging device information as the sensor information, without using the mounted ranging device information as the sensor information.
4. The calculation device according to claim 1 , When the mounted vehicle is present, the acquisition unit acquires at least one of the image information and the external ranging device information as the sensor information in addition to the mounted ranging device information; the calculation unit calculates at least one of the position and the orientation of the target moving object by using the mounted ranging device information as the sensor information, and calculates at least one of the position and the orientation of the target moving object by using at least one of the image information and the external ranging device information as the sensor information; A calculation device in which the calculation unit stops calculating the position and orientation of the target moving body using the sensor information when at least one of the following cases is true: (i) a difference between the position of the target moving body calculated using the mounted ranging device information and the position of the target moving body calculated using at least one of the image information and the external ranging device information is greater than or equal to a predetermined position threshold; and (ii) a difference between the orientation of the target moving body calculated using the mounted ranging device information and the orientation of the target moving body calculated using at least one of the image information and the external ranging device information is greater than or equal to a predetermined direction threshold.
5. The calculation device according to claim 1 , further comprising: A signal generating unit that generates a control signal for controlling an operation of the moving object; a transmitter that transmits the control signal generated by the signal generator to the moving object; When the mounted vehicle is present, the acquisition unit acquires at least one of the image information and the external ranging device information as the sensor information in addition to the mounted ranging device information; the calculation unit calculates at least one of the position and the orientation of the target moving object by using the mounted ranging device information as the sensor information, and calculates at least one of the position and the orientation of the target moving object by using at least one of the image information and the external ranging device information as the sensor information; A calculation device in which, in at least one of the following cases: (i) when a difference between the position of the target moving body calculated using the mounted ranging device information and the position of the target moving body calculated using at least one of the image information and the external ranging device information is equal to or greater than a predetermined position threshold, and (ii) when a difference between the orientation of the target moving body calculated using the mounted ranging device information and the orientation of the target moving body calculated using at least one of the image information and the external ranging device information is equal to or greater than a predetermined direction threshold, the signal generation unit generates, as the control signal, either (a) a stop control signal for stopping the moving body, or (b) a change control signal for changing the destination when the mounted moving body moves by the unmanned operation from a predetermined target location to a maintenance location where at least one of a repair process for repairing the mounted ranging device and a calibration process for calibrating the mounted ranging device is performed.
Citation Information
Patent Citations
Axis adjusting method of radar device and axis adjusting device
JP2009281862A
Vehicle-use radio communication apparatus and communication system
JP2013025423A
Navigation device, navigation method, and navigation program
JP2016105056A
Information generation system, information generation apparatus, information generation method, and computer program
JP2019079453A
Vehicle direction estimation device
JP2021196877A
