Calculation device, calculation system, and calculation method

Abstract

To provide a technology capable of suppressing the decline in accuracy of the position and orientation of a moving object when three-dimensional point cloud data is missing.SOLUTION: A calculation device is provided with: an acquisition unit for acquiring three-dimensional point cloud data; a calculation unit for calculating at least the position or orientation of a moving object by using the three-dimensional point cloud data; and at least one function unit among (i) a prediction unit for predicting that the three-dimensional point cloud data is missing, (ii) a detection unit for detecting that the three-dimensional point cloud data is missing, and (iii) a determination unit for determining whether or not a moving object exists in a specific area previously assumed to be missing the three-dimensional point cloud data. The calculation unit executes at least a second calculation process when at least any of a first case, a second case, and a third case occurs.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to a calculation device, a calculation system, and a calculation method.

Background Art

[0002] Conventionally, a technique of driving a vehicle autonomously or by remote control by monitoring the driving of the vehicle using a lidar outside the vehicle is known (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to move a moving object such as a vehicle by unmanned driving, the position and orientation of the moving object may be calculated using three-dimensional point cloud data representing the moving object output from a moving object detection device such as a lidar. However, if the three-dimensional point cloud data is missing due to an obstacle existing between the moving object detection device and the moving object, etc., the accuracy of the position and orientation of the moving object may decrease.

Means for Solving the Problems

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to the first embodiment of the present disclosure, a calculation device is provided. The calculation device includes an acquisition unit that acquires three-dimensional point cloud data representing a mobile body that can be moved by unmanned operation using a point cloud; a calculation unit that uses the three-dimensional point cloud data to calculate at least one of the position and orientation of the mobile body; and at least one functional unit comprising: (i) a prediction unit that predicts when the three-dimensional point cloud data is missing; (ii) a detection unit that detects when the three-dimensional point cloud data is missing; and (iii) a determination unit that determines whether or not the mobile body is located within a specific area where it is assumed in advance that the three-dimensional point cloud data is missing; the calculation unit uses the three-dimensional point cloud data and pre-prepared reference point cloud data, The calculation unit can perform a first calculation process that calculates at least one of the position and orientation of the moving body by comparing the three-dimensional point cloud data, and a second calculation process that calculates at least one of the position and orientation of the moving body by applying geometric data of a predetermined shape to the three-dimensional point cloud data. The calculation unit performs at least the second calculation process when at least one of the following cases is met: a first case in which the prediction unit predicts that the three-dimensional point cloud data is missing, a second case in which the detection unit detects that the three-dimensional point cloud data is missing, or a third case in which the determination unit determines that the moving body is located within the specified area. According to this configuration, the calculation device can calculate at least one of the position and orientation of the moving body using the three-dimensional point cloud data. At this time, the calculation device can accurately calculate the position and orientation of the moving body by comparing the three-dimensional point cloud data and the reference point cloud data in the first calculation process. However, in cases 1, 2, 3, or at least one of the above, the accuracy of the position and orientation of the moving object calculated by the first calculation process may decrease. In response to this, the calculation device can calculate at least one of the position and orientation of the moving object by performing at least the second calculation process in cases 1, 2, or 3.In this way, the calculation device can estimate information corresponding to missing parts of the point cloud that make up the 3D point cloud data by applying the geometric data to the 3D point cloud data through the second calculation process when at least one of the first, second, or third cases is met. As a result, the calculation device can suppress a decrease in the accuracy of the position and orientation of the moving object when at least one of the first, second, or third cases is met. (2) In the above configuration, the prediction unit may predict that the 3D point cloud data is missing by using at least one of the following: object information relating to at least one of an object in a first area where a first manufacturing process, which is a manufacturing process being performed on the moving body, is carried out, and an object in a second area where a second manufacturing process, which is a manufacturing process scheduled to be performed on the moving body, is carried out; and work information indicating whether or not a specific operation is performed in which at least one of a work object, which is a manufacturing apparatus used in the manufacturing process or a worker engaged in the work in the manufacturing process, enters at least one of the interior of the moving body or the surrounding area of ​​the moving body to perform the work. In this configuration, the calculation device can predict that the 3D point cloud data is missing by using at least one of the object information and work information. (3) In the above configuration, the specified area may be an area in which a specific operation is performed in which at least one of the working objects, which is a manufacturing device used in the manufacturing process of the mobile body and a worker engaged in the work in the manufacturing process, enters at least one of the interior of the mobile body and the surrounding area of ​​the mobile body to perform the operation. In this configuration, the calculation device can determine whether or not the mobile body is present in the area in which the specific operation is performed. This allows the calculation device to further suppress a decrease in the accuracy of the position and orientation of the mobile body in the third case. (4) In the above configuration, the detection unit may detect that the 3D point cloud data is incomplete when the number of points constituting the 3D point cloud data is less than a predetermined number. In this configuration, the calculation device can detect that the 3D point cloud data is incomplete when the number of points constituting the 3D point cloud data is less than a predetermined number. (5) In the above configuration, the detection unit may detect that the 3D point cloud data is missing when it detects an object between the moving object and the moving object detection device, which outputs the 3D point cloud data by detecting the moving object from the outside. In this configuration, the calculation device can detect that the 3D point cloud data is missing when it detects an object between the moving object and the moving object detection device. (6) In the above configuration, the acquisition unit acquires a plurality of three-dimensional point cloud data for the same moving object detected at different timings, the calculation unit calculates at least one of the position and orientation of the moving object at different timings by performing the first calculation process for each of the plurality of three-dimensional point cloud data, and generates time-series data of at least one of the position and orientation of the moving object by arranging them in chronological order, the detection unit uses the time-series data to detect that at least one of the plurality of three-dimensional point cloud data is missing, and if the detection unit detects that at least one of the plurality of three-dimensional point cloud data is missing, the calculation unit may perform the second calculation process without performing the first calculation process to calculate at least one of the position and orientation of the moving object. According to this configuration, the calculation device can acquire a plurality of three-dimensional point cloud data for the same moving object detected at different timings. The calculation device can generate time-series data by performing the first calculation process for each of the acquired plurality of three-dimensional point cloud data to calculate at least one of the position and orientation of the moving object at different timings. This allows the calculation device to use time-series data to detect if at least one of the multiple 3D point cloud data is missing. When the calculation device detects that at least one of the multiple 3D point cloud data is missing, it can calculate at least one of the position and orientation of the moving object by executing the second calculation process without executing the first calculation process. In this way, the calculation device can further suppress the decrease in accuracy of the position and orientation of the moving object in the second case. (7) In the above configuration, if at least one of the first case, the second case, or the third case is applicable, the calculation unit may perform the first calculation process and the second calculation process, respectively, and the calculation unit may calculate the position of the moving body by performing a predetermined calculation process using the position of the moving body calculated by performing the first calculation process and the position of the moving body calculated by performing the second calculation process, and the calculation unit may calculate the orientation of the moving body by performing a predetermined calculation process using the orientation of the moving body calculated by performing the first calculation process and the orientation of the moving body calculated by performing the second calculation process. According to this configuration, the calculation device can calculate at least one of the position and orientation of the moving body by performing the following process if at least one of the first case, the second case, or the third case is applicable. In this case, the calculation device can calculate the position of the moving object by performing a predetermined calculation process using the position of the moving object calculated by performing the first calculation process and the position of the moving object calculated by performing the second calculation process. Furthermore, the calculation device can calculate the orientation of the moving object by performing a predetermined calculation process using the orientation of the moving object calculated by performing the first calculation process and the orientation of the moving object calculated by performing the second calculation process. In this way, the calculation device can further suppress the decrease in accuracy of the position and orientation of the moving object when at least one of the first, second, or third cases is met. (8) According to a second embodiment of the present disclosure, a calculation system is provided. The calculation system comprises one or more mobile bodies that can be moved by unmanned operation; a mobile body detection device that detects the mobile bodies from the outside and outputs three-dimensional point cloud data representing the mobile bodies as a point cloud; an acquisition unit that acquires the three-dimensional point cloud data; a calculation unit that uses the three-dimensional point cloud data to calculate at least one of the position and orientation of the mobile bodies; and at least one functional unit comprising: (i) a prediction unit that predicts when the three-dimensional point cloud data is missing; (ii) a detection unit that detects when the three-dimensional point cloud data is missing; and (iii) a determination unit that determines whether or not the mobile bodies are located in a specific area where it is assumed in advance that the three-dimensional point cloud data is missing, wherein the calculation unit is The calculation unit can perform a first calculation process that calculates at least one of the position and orientation of the moving object by comparing 3D point cloud data with pre-prepared reference point cloud data, and a second calculation process that calculates at least one of the position and orientation of the moving object by applying geometric data of a predetermined shape to the 3D point cloud data. The calculation unit performs at least the second calculation process when at least one of the following cases is met: a first case in which the prediction unit predicts that the 3D point cloud data is missing, a second case in which the detection unit detects that the 3D point cloud data is missing, or a third case in which the determination unit determines that the moving object is located within the specified area. According to this configuration, the calculation system can calculate at least one of the position and orientation of the moving object using 3D point cloud data. At this time, the calculation system can accurately calculate the position and orientation of the moving object by comparing 3D point cloud data with reference point cloud data through the first calculation process. However, in at least one of the first, second, or third cases, the accuracy of the position and orientation of the moving object calculated by the first calculation process may decrease. In response to this, the calculation system can calculate at least one of the position and orientation of the moving object by performing at least the second calculation process in at least one of the first, second, or third cases.In this way, the calculation system can estimate information corresponding to the missing parts of the point cloud that make up the 3D point cloud data by applying the geometric data to the 3D point cloud data through the second calculation process when at least one of the first, second, or third cases is met. As a result, the calculation system can suppress a decrease in the accuracy of the position and orientation of the moving object when at least one of the first, second, or third cases is met. (9) A calculation method is provided according to a third embodiment of the present disclosure. The calculation method comprises: an acquisition step of acquiring three-dimensional point cloud data representing a mobile body that can be moved by unmanned operation; a calculation step of calculating at least one of the position and orientation of the mobile body using the three-dimensional point cloud data; and at least one functional step comprising: (i) a prediction step of predicting that the three-dimensional point cloud data is missing; (ii) a detection step of detecting that the three-dimensional point cloud data is missing; and (iii) a determination step of determining whether the mobile body is located in a specific area where it is assumed in advance that the three-dimensional point cloud data is missing, wherein the calculation step compares the three-dimensional point cloud data with pre-prepared reference point cloud data. The system can perform a first calculation process to calculate at least one of the position and orientation of the moving body, and a second calculation process to calculate at least one of the position and orientation of the moving body by applying geometric data of a predetermined shape to the 3D point cloud data. The calculation process performs at least the second calculation process if at least one of the following conditions is met: a first case in which the prediction process predicts that the 3D point cloud data is missing, a second case in which the detection process detects that the 3D point cloud data is missing, or a third case in which the determination process determines that the moving body exists within the specific area. According to this configuration, at least one of the position and orientation of the moving body can be calculated using the 3D point cloud data. In this case, the first calculation process can accurately calculate the position and orientation of the moving body by comparing the 3D point cloud data with reference point cloud data. However, in at least one of the first, second, or third cases, the accuracy of the position and orientation of the moving body calculated by the first calculation process may decrease. In contrast, with this configuration, the second calculation process is performed when at least one of the first, second, or third cases is met, and by fitting the geometric data to the 3D point cloud data, information corresponding to the missing parts of the point cloud constituting the 3D point cloud data can be estimated.This makes it possible to suppress a decrease in the accuracy of the position and orientation of the moving object in at least one of the first, second, or third cases. (10) According to a fourth embodiment of the present disclosure, a calculation device is provided. The calculation device includes an acquisition unit that acquires three-dimensional point cloud data representing a mobile body that can be moved by unmanned operation as a point cloud; a calculation unit that uses the three-dimensional point cloud data to calculate at least one of the position and orientation of the mobile body and outputs vehicle position information including at least one of the position and orientation of the mobile body; and a detection unit that detects when the three-dimensional point cloud data is missing, wherein the calculation unit is capable of performing a first calculation process that calculates at least one of the position and orientation of the mobile body by comparing the three-dimensional point cloud data with pre-prepared reference point cloud data, and a second calculation process that calculates at least one of the position and orientation of the mobile body by applying geometric data of a predetermined shape to the three-dimensional point cloud data, and the calculation unit is capable of performing the first calculation process and the By executing the first and second calculation processes, the device calculates at least one of the position and orientation of the moving body. The detection unit uses the first calculation result of at least one of the position and orientation of the moving body calculated by executing the first calculation process, and the second calculation result of at least one of the position and orientation of the moving body calculated by executing the second calculation process, to detect that the 3D point cloud data is missing. When the detection unit detects that the 3D point cloud data is missing, the calculation unit uses the first and second calculation results to select which of the first and second calculation processes to output as the vehicle position information, and outputs the selected calculation result as the vehicle position information. In this configuration, the calculation device can detect that the 3D point cloud data is missing using the first and second calculation results. When the calculation device detects that 3D point cloud data is missing, it can use the first and second calculation results to select which of the two calculation processes to execute to output the calculated result as vehicle position information. In other words, when the calculation device detects that 3D point cloud data is missing, it can determine and select which of the first and second calculation results provides a more accurate result.The calculation device can output the selected calculation result as vehicle position information. In this way, the calculation device can suppress the decrease in the accuracy of the moving object's position and orientation when it detects that 3D point cloud data is missing. (11) In the above configuration, the acquisition unit acquires a plurality of three-dimensional point cloud data for the same moving object detected at different timings, the calculation unit calculates at least one of the position and orientation of the moving object at different timings by performing the first calculation process and the second calculation process for each of the plurality of three-dimensional point cloud data, the calculation unit generates first time-series data of at least one of the position and orientation of the moving object by arranging the plurality of first calculation results at different timings in chronological order, and the calculation unit calculates the plurality of second calculation results at different timings By arranging the data in chronological order, a second time-series data is generated for at least one of the position and orientation of the moving object. The detection unit uses the first time-series data and the second time-series data to detect that at least one of the plurality of 3D point cloud data is missing. If the detection unit detects that at least one of the plurality of 3D point cloud data is missing, the calculation unit may use the first time-series data and the second time-series data to select whether to output the calculation result calculated by executing either the first calculation process or the second calculation process as the vehicle position information. In this configuration, the calculation device can acquire a plurality of 3D point cloud data for the same moving object detected at different timings. The calculation device can generate first time-series data by executing a first calculation process for each of the acquired plurality of 3D point cloud data and arranging the plurality of first calculation results at different timings in chronological order. The calculation device can generate second time-series data by executing a second calculation process for each of the acquired plurality of 3D point cloud data and arranging the plurality of second calculation results at different timings in chronological order. The calculation device can detect missing 3D point cloud data using the first and second time series data. When the calculation device detects missing 3D point cloud data, it can select whether to output the calculation result calculated by the first or second calculation process using the first and second time series data as vehicle position information.In other words, when the calculation device detects that at least one of the multiple 3D point cloud data is missing, it can use the first and second time series data to determine and select which of the first and second calculation results is more accurate. The calculation device can then output the selected calculation result as vehicle position information. In this way, the calculation device can suppress a decrease in the accuracy of the position and orientation of the moving object when it detects that at least one of the multiple 3D point cloud data is missing. (12) According to a fifth embodiment of the present disclosure, a calculation system is provided. The calculation system comprises one or more mobile bodies that can be moved by unmanned operation, a mobile body detection device that detects the mobile bodies from the outside and outputs three-dimensional point cloud data representing the mobile bodies as a point cloud, an acquisition unit that acquires the three-dimensional point cloud data, a calculation unit that uses the three-dimensional point cloud data to calculate at least one of the position and orientation of the mobile bodies and outputs vehicle position information including at least one of the position and orientation of the mobile bodies, and a detection unit that detects when the three-dimensional point cloud data is missing, wherein the calculation unit includes a first calculation process that calculates at least one of the position and orientation of the mobile bodies by comparing the three-dimensional point cloud data with pre-prepared reference point cloud data, and a second calculation process that calculates at least one of the position and orientation of the mobile bodies by applying geometric data of a predetermined shape to the three-dimensional point cloud data, The calculation unit can perform the first calculation process and the second calculation process, respectively, to calculate at least one of the position and orientation of the moving body. The detection unit uses the first calculation result of at least one of the position and orientation of the moving body calculated by performing the first calculation process, and the second calculation result of at least one of the position and orientation of the moving body calculated by performing the second calculation process, to detect that the 3D point cloud data is missing. When the detection unit detects that the 3D point cloud data is missing, the calculation unit uses the first calculation result and the second calculation result to select which of the first and second calculation processes to output as the vehicle position information, and outputs the selected calculation result as the vehicle position information. In this configuration, the calculation system can detect that the 3D point cloud data is missing using the first calculation result and the second calculation result. When the calculation system detects that 3D point cloud data is missing, it can use the first and second calculation results to select which of the two calculation processes—the first or the second—to execute to output the calculated result as vehicle position information.In other words, when the calculation system detects that 3D point cloud data is missing, it can determine and select which of the first and second calculation results is more accurate. The calculation system can then output the selected calculation result as vehicle position information. In this way, the calculation system can suppress the decrease in the accuracy of the position and orientation of the moving object when it detects that 3D point cloud data is missing. (13) According to the sixth embodiment of the present disclosure, a calculation method is provided. The calculation method comprises: an acquisition step of acquiring three-dimensional point cloud data representing a mobile body that can be moved by unmanned operation using a point cloud; a first calculation step of calculating at least one of the position and orientation of the mobile body using the three-dimensional point cloud data; a detection step of detecting that the three-dimensional point cloud data is missing; and a second calculation step of outputting vehicle position information including at least one of the position and orientation of the mobile body, wherein the first calculation step and the second calculation step can perform a first calculation process of calculating at least one of the position and orientation of the mobile body by comparing the three-dimensional point cloud data with pre-prepared reference point cloud data, and a second calculation process of calculating at least one of the position and orientation of the mobile body by applying geometric data of a predetermined shape to the three-dimensional point cloud data, wherein the first calculation step is By executing the first calculation process and the second calculation process, at least one of the position and orientation of the moving body is calculated. In the detection step, the first calculation result of at least one of the position and orientation of the moving body calculated by executing the first calculation process, and the second calculation result of at least one of the position and orientation of the moving body calculated by executing the second calculation process, are used to detect that the 3D point cloud data is missing. If the detection step detects that the 3D point cloud data is missing, the second calculation step uses the first and second calculation results to select which of the first and second calculation processes will be used to calculate the vehicle position information, and outputs the selected calculation result as the vehicle position information. According to this configuration, the absence of 3D point cloud data can be detected using the first and second calculation results. If missing 3D point cloud data is detected, the system can use the first and second calculation results to select which of the two calculation processes—the first or the second—to execute and output the calculated result as vehicle position information.In other words, when it is detected that 3D point cloud data is missing, the system can determine and select which of the first and second calculation results is more accurate. This allows the selected calculation result to be output as vehicle position information. In this way, when it is detected that 3D point cloud data is missing, the accuracy of the position and orientation of the moving object can be suppressed. This disclosure can be implemented in various forms other than the calculation device, calculation system, and mobile body described above. For example, it can be implemented in the form of a method for manufacturing the calculation device, calculation system, and mobile body, a method for controlling the calculation device, calculation system, and mobile body, a computer program for implementing the control method, and a non-temporary recording medium on which the computer program is stored. [Brief explanation of the drawing]

[0007] [Figure 1] A conceptual diagram showing the configuration of the driving system in the first embodiment. [Figure 2] A block diagram showing the configuration of the driving system in the first embodiment. [Figure 3] A diagram illustrating the second calculation process. [Figure 4] A flowchart illustrating the processing procedure for vehicle driving control in the first embodiment. [Figure 5] A flowchart illustrating the processing procedure in the first embodiment. [Figure 6] A block diagram showing the configuration of the driving system in the second embodiment. [Figure 7] A flowchart illustrating the processing procedure in the second embodiment. [Figure 8] A block diagram showing the configuration of the driving system in the third embodiment. [Figure 9] A flowchart illustrating the processing procedure in the third embodiment. [Figure 10] A block diagram showing the configuration of the driving system in the fourth embodiment. [Figure 11] A flowchart illustrating the processing procedure in the fourth embodiment. [Figure 12] Block diagram showing the configuration of the driving system in the fifth embodiment. [Figure 13] Flowchart showing the processing procedure in the fifth embodiment. [Figure 14] Block diagram showing the configuration of the driving system in the sixth embodiment. [Figure 15] Flowchart showing the processing procedure of vehicle driving control in the sixth embodiment.

Modes for Carrying Out the Invention

[0008] A. First Embodiment: FIG. 1 is a conceptual diagram showing the configuration of a driving system 50 in the first embodiment. The driving system 50 is a system for moving a moving body without depending on the driving operation of a passenger on board the moving body. The driving system 50 includes a calculation system 7 and a remote control device 80.

[0009] The calculation system 7 is a system for calculating at least one of the position and orientation of the vehicle 100. The calculation system 7 includes one or more vehicles 100 as moving bodies, a calculation device 70, and one or more vehicle detection devices. In this embodiment, the functions of the calculation device 70 and the remote control device 80 are realized by the server 200.

[0010] The vehicle detection device outputs, as a detection result, three-dimensional point cloud data representing the vehicle 100 by a point cloud by detecting the vehicle 100 from the outside. In this embodiment, the vehicle detection device is a LiDAR (Light Detection And Ranging) as an external sensor 300. The external sensor 300 is a sensor located outside the vehicle 100. The external sensor 300 includes a communication device (not shown) and can communicate with other devices such as the server 200 by wired communication or wireless communication. LiDAR is an example of a distance measuring device. In other embodiments, the vehicle detection device may be other sensors such as a stereo camera. Hereinafter, the LiDAR as the external sensor 300 is referred to as an external LiDAR 310.

[0011] In the present disclosure, a "mobile body" means an object that can move, for example, a vehicle or an electric vertical take-off and landing aircraft (so-called flying car). The vehicle may be a vehicle that travels by wheels or a vehicle that travels on an endless track, for example, a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, a construction vehicle, etc. The vehicle includes battery electric vehicles (BEVs), gasoline vehicles, hybrid vehicles, and fuel cell vehicles. When the mobile body is other than a vehicle, the expressions "vehicle" and "car" in the present disclosure can be appropriately replaced with "mobile body", and the expression "travel" can be appropriately replaced with "move".

[0012] Vehicle 100 is configured to be capable of traveling by autonomous driving. "Autonomous driving" means driving that does not depend on the driving operation of a passenger. The driving operation means an operation related to at least any one of "running", "turning", and "stopping" of vehicle 100. Autonomous driving is realized by automatic or manual remote control using a device located outside vehicle 100, or by autonomous control of vehicle 100. A passenger who does not perform a driving operation may board vehicle 100 while it is traveling by autonomous driving. Passengers who do not perform a driving operation include, for example, a person simply sitting in the seat of vehicle 100, or a person performing an operation different from the driving operation, such as assembly, inspection, or operation of switches, while boarding vehicle 100. Note that driving by the driving operation of a passenger is sometimes called "human driving".

[0013] In this specification, "remote control" includes "complete remote control" in which all of the operations of vehicle 100 are completely determined from outside vehicle 100, and "partial remote control" in which a part of the operations of vehicle 100 is determined from outside vehicle 100. Further, "autonomous control" includes "complete autonomous control" in which vehicle 100 autonomously controls its own operations without receiving any information from a device outside vehicle 100, and "partial autonomous control" in which vehicle 100 autonomously controls its own operations using information received from a device outside vehicle 100.

[0014] In this embodiment, the driving system 50 is used in a factory FC where the vehicle 100 is manufactured. The reference coordinate system of the factory FC is the global coordinate system GC, and any position within the factory FC can be represented by X, Y, Z coordinates in the global coordinate system GC. The factory FC comprises a first location PL1 and a second location PL2. The first location PL1 and the second location PL2 are connected by a track TR on which the vehicle 100 can travel. Multiple external sensors 300 are installed along the track TR in the factory FC. The positions of each external sensor 300 in the factory FC are pre-adjusted. The vehicle 100 moves from the first location PL1 to the second location PL2 via the track TR by unmanned operation.

[0015] Figure 2 is a block diagram showing the configuration of the driving system 50 in the first embodiment. The vehicle 100 includes a vehicle control device 110 for controlling various parts of the vehicle 100, an actuator group 120 including one or more actuators driven under the control of the vehicle control device 110, and a communication device 130 for communicating wirelessly with an external device such as a server 200. The actuator group 120 includes an actuator for a drive system to accelerate the vehicle 100, an actuator for a steering system to change the direction of travel of the vehicle 100, and an actuator for a braking system to decelerate the vehicle 100.

[0016] The vehicle control device 110 is composed of a computer comprising a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input / output interface 113 are connected via the internal bus 114 to enable bidirectional communication. The input / output interface 113 is connected to an actuator group 120 and a communication device 130. The processor 111 implements various functions, including those of a vehicle control unit 115, by executing a program PG1 stored in the memory 112.

[0017] The vehicle control unit 115 drives the vehicle 100 by controlling the actuator group 120. The vehicle control unit 115 can drive the vehicle 100 by controlling the actuator group 120 using the driving control signal received from the server 200. The driving control signal is a control signal for driving the vehicle 100. In this embodiment, the driving control signal includes the acceleration and steering angle of the vehicle 100 as parameters. In other embodiments, the driving control signal may include the speed of the vehicle 100 as a parameter instead of, or in addition to, the acceleration of the vehicle 100.

[0018] The server 200 is composed of a computer comprising a processor 201, memory 202, an input / output interface 203, and an internal bus 204. The processor 201, memory 202, and input / output interface 203 are connected via the internal bus 204 to enable bidirectional communication. A communication device 205 for communicating with various external devices of the server 200 is connected to the input / output interface 203. The communication device 205 can communicate with the vehicle 100 via wireless communication and can communicate with each external sensor 300 via wired or wireless communication. The processor 201 implements the following functions by executing the program PG2 stored in memory 202. The processor 201 implements various functions including those of an acquisition unit 211, a detection unit 212, a calculation unit 213, and a remote control unit 214.

[0019] The acquisition unit 211 acquires 3D point cloud data by detecting the vehicle 100 from the outside using an external LiDAR 310.

[0020] The detection unit 212 is one of the functional units that acquires information about missing data in the 3D point cloud data. The detection unit 212 detects that there are missing 3D point cloud data acquired by the acquisition unit 211.

[0021] The detection unit 212 detects, for example, that 3D point cloud data is missing when the actual number is less than a predetermined reference number. The actual number is the number of points that make up the 3D point cloud data actually acquired by the acquisition unit 211. The reference number is a threshold for detecting that 3D point cloud data is missing. The reference number is set, for example, using a planned number. The planned number is the number of points that are planned to be acquired as 3D point cloud data when the acquisition unit 211 acquires the 3D point cloud data. The reference number is set, for example, by multiplying the planned number by a predetermined multiplier. The multiplier is a number less than 1. The multiplier is determined, for example, according to the degree of influence on the control of the vehicle 100. The degree of influence on the control of the vehicle 100 is determined, for example, according to the correlation between the missing percentage of the 3D point cloud data and the accuracy of the position and orientation of the vehicle 100 calculated using the 3D point cloud data. The missing percentage is the ratio of the number of points corresponding to the missing portion to the total number of points that make up the 3D point cloud data. In other words, the reference number for detecting missing 3D point cloud data is set depending on whether the accuracy of the position and orientation of vehicle 100 calculated using partially missing 3D point cloud data is within an acceptable range. For example, if the planned number is 500, the reference number may be 300.

[0022] The reference number is set in advance for each predetermined detection area within the detection range of each external LiDAR 310. In this case, the detection unit 212 detects that 3D point cloud data is missing by performing the following process, for example. Specifically, the detection unit 212 first uses judgment information to identify which of the multiple detection areas of the multiple external LiDAR 310 the vehicle 100 is located in. The judgment information includes, for example, the transmission history of the driving control signal, the position and orientation of the vehicle 100 at a time prior to the detection timing, and the driving speed of the vehicle 100. Next, the detection unit 212 uses the count database DB1, which is pre-stored in the memory 202 of the server 200, to obtain the reference number of detection areas in which the vehicle 100 to be detected is located. The count database DB1 is a database that associates the reference number with each detection area in each of the multiple external LiDAR 310. Next, the detection unit 212 compares the actual number with the reference number and detects that 3D point cloud data is missing if the actual number is less than the reference number.

[0023] The detection unit 212 may also detect missing 3D point cloud data by other means. For example, the detection unit 212 may detect missing 3D point cloud data when it detects an obstacle between the vehicle 100 and the external LiDAR 310. The obstacle may be a moving object or a stationary object. A moving object is an object that can approach the vehicle 100 being detected by moving. Examples of moving objects include living organisms such as humans and animals, other vehicles 100 different from the vehicle 100 being detected, and manufacturing equipment that can be moved manually or automatically, such as automated guided vehicles (AGVs). A stationary object is an object that is placed naturally or artificially on the track TR on which the vehicle 100 travels. Examples of stationary objects include manufacturing equipment whose work arrangement can be changed as appropriate, equipment such as road cones and signs placed on the track TR, flying objects such as fallen leaves that land on the track TR, and plants such as trees whose size changes due to growth, etc.

[0024] The calculation unit 213 uses 3D point cloud data to calculate at least one of the position and orientation of the vehicle 100 and outputs vehicle position information. In this embodiment, the position of the vehicle 100 is the position of a preset positioning point for a specific part of the vehicle 100. The orientation of the vehicle 100 is the direction represented by a vector that points from the rear to the front of the vehicle 100 along the longitudinal axis of the vehicle 100. The vehicle position information is the position information that forms the basis for generating the driving control signal. In this embodiment, the vehicle position information includes the position and orientation of the vehicle 100 in the global coordinate system GC of the factory FC. The calculation unit 213 is capable of performing a first calculation process and a second calculation process.

[0025] The first calculation process is a process of calculating at least one of the position and orientation of the vehicle 100 by comparing the 3D point cloud data acquired by the acquisition unit 211 with pre-prepared reference point cloud data. In this embodiment, the calculation unit 213 calculates the position and orientation of the vehicle 100 represented by the 3D point cloud data by matching the 3D point cloud data with the reference point cloud data in the first calculation process. The reference point cloud data is point cloud data that virtually reproduces the vehicle 100. The reference point cloud data is, for example, 3D CAD data representing the vehicle 100. For matching the 3D point cloud data with the reference point cloud data, algorithms such as ICP (Iterative Closest Point) and NDT (Normal Distribution Transform) are used.

[0026] Figure 3 is a diagram illustrating the second calculation process. The second calculation process calculates at least one of the position and orientation of the vehicle 100 by applying a predetermined shape of graphic data FG to the 3D point cloud data PD acquired by the acquisition unit 211. The shape of the graphic data FG is such that when the graphic data FG is applied to surround the 3D point cloud data PD, the external shape of the vehicle 100 can be estimated. The shape of the graphic data FG is, for example, a rectangular parallelepiped. In this case, the graphic data FG is also called a bounding box. When the shape of the graphic data FG is a rectangular parallelepiped, for example, the ratio of the mutually orthogonal first sides SB1~SB4, second sides SB5~SB8, and third sides SB9~SB12 corresponds to the ratio of the vehicle width, overall length, and vehicle height. In other embodiments, the shape of the graphic data FG may be other than a rectangular parallelepiped. The shape of the graphic data FG may be, for example, rectangular.

[0027] In this embodiment, the calculation unit 213 calculates the position and orientation of the vehicle 100 represented by the 3D point cloud data PD by fitting the rectangular parallelepiped shape data FG to surround the 3D point cloud data PD in the second calculation process. Specifically, the calculation unit 213 first fits the rectangular parallelepiped shape data FG to surround the 3D point cloud data PD. Next, the calculation unit 213 performs the following process to calculate the position of the vehicle 100. The calculation unit 213 obtains the coordinates of the eight vertices VB1 to VB8 of the rectangular parallelepiped that constitute the shape data FG. Each coordinate of the shape data FG is associated with supplementary information indicating which of the eight vertices VB1 to VB8 of the rectangular parallelepiped that constitutes the shape data FG the coordinates belong to. Next, the calculation unit 213 uses the coordinate database DB2 stored in the server 200's memory 202 to calculate the coordinates of the vehicle 100's positioning point as the vehicle 100's position from the coordinates of the eight vertices VB1 to VB8 of the rectangular parallelepiped that constitutes the geometric data FG. The coordinate database DB2 is a database that shows the relative positional relationship between the eight vertices VB1 to VB8 of the rectangular parallelepiped that constitutes the geometric data FG and the vehicle 100's positioning point. The calculation unit 213 also performs the following processing to calculate the orientation of the vehicle 100. The calculation unit 213 uses the coordinates of the first central position CN1 and the coordinates of the second central position CN2 to calculate the orientation of the vehicle 100. The first central position CN1 is the central position of side SB1, which is aligned in the vehicle width direction on the front side of the rectangular parallelepiped that constitutes the geometric data FG, among the twelve sides SB1 to SB12 of the rectangular parallelepiped that constitutes the geometric data FG. The second central position CN2 is the central position of side SB2, which is aligned in the vehicle width direction on the rear side of the vehicle 100, among the 12 sides SB1 to SB12 of the rectangular parallelepiped that constitutes the geometric data FG.

[0028] If the detection unit 212 detects that the 3D point cloud data PD is missing, the calculation unit 213 calculates the position and orientation of the vehicle 100 by performing at least a second calculation process. In this embodiment, if the detection unit 212 detects that the 3D point cloud data PD is missing, the calculation unit 213 performs a first calculation process and a second calculation process, respectively. The calculation unit 213 performs an arithmetic mean operation on a first coordinate indicating the position of the vehicle 100 calculated by performing the first calculation process and a second coordinate indicating the position of the vehicle 100 calculated by performing the second calculation process. The calculation unit 213 performs an arithmetic mean operation on a first vector indicating the orientation of the vehicle 100 calculated by performing the first calculation process and a second vector indicating the orientation of the vehicle 100 calculated by performing the second calculation process. The calculation unit 213 outputs vehicle position information in which the coordinate obtained by calculating and averaging the first coordinate and the second coordinate is the position of the vehicle 100, and the vector obtained by arithmetic mean of the first vector and the second vector is the orientation of the vehicle 100.

[0029] The calculation unit 213 may output vehicle position information in which the coordinate obtained by weighting the first coordinate and the second coordinate is the position of the vehicle 100, and the vector obtained by weighting the first vector and the second vector is the orientation of the vehicle 100. In this case, the second coordinate is weighted according to the missing data percentage of the 3D point cloud data PD in the calculation process of weighting the first coordinate and the second coordinate, for example, so that the weight increases as the missing data percentage of the 3D point cloud data PD increases. The second vector is weighted according to the missing data percentage of the 3D point cloud data PD in the calculation process of weighting the first vector and the second vector, for example, so that the weight increases as the missing data percentage of the 3D point cloud data PD increases.

[0030] If the detection unit 212 does not detect that the 3D point cloud data PD is missing, the calculation unit 213 executes the first calculation process without executing the second calculation process. As a result, the calculation unit 213 outputs vehicle position information in which the first coordinate calculated by executing the first calculation process is the position of the vehicle 100, and the first vector calculated by executing the first calculation process is the orientation of the vehicle 100.

[0031] The remote control unit 214 acquires detection results from the sensors, generates a driving control signal to control the actuator group 120 of the vehicle 100 using the detection results, and transmits the driving control signal to the vehicle 100, thereby driving the vehicle 100 by remote control. In addition to the driving control signal, the remote control unit 214 may also generate and output control signals to control various auxiliary equipment and actuators that operate various devices such as wipers, power windows, and lamps, which are provided on the vehicle 100. In other words, the remote control unit 214 may operate these various devices and auxiliary equipment by remote control.

[0032] Figure 4 is a flowchart showing the processing procedure for controlling the driving of the vehicle 100 in the first embodiment. The flow shown in Figure 4 is executed repeatedly at predetermined intervals, for example, during the period when the vehicle 100 is being driven under the remote control of the server 200. In the processing procedure in Figure 4, the processor 201 of the server 200 functions as an acquisition unit 211, a detection unit 212, a calculation unit 213, and a remote control unit 214 by executing the program PG2. The processor 111 of the vehicle 100 functions as a vehicle control unit 115 by executing the program PG1.

[0033] In step S1, the processor 201 of the server 200 acquires vehicle position information using the detection results output from the external sensor 300. Specifically, in step S1, the processor 201 acquires vehicle position information using the 3D point cloud data PD acquired from the external LiDAR 310, which is the external sensor 300.

[0034] In step S2, the processor 201 of the server 200 determines the next target location that the vehicle 100 should head to. In this embodiment, the target location is represented by X, Y, Z coordinates in the global coordinate system GC. The memory 202 of the server 200 pre-stores a reference route RR, which is the path that the vehicle 100 should travel. The route is represented by a node indicating the starting point, nodes indicating waypoints, a node indicating the destination, and links connecting each node. The processor 201 uses the vehicle position information and the reference route RR to determine the next target location that the vehicle 100 should head to. The processor 201 determines the target location on the reference route RR beyond the vehicle 100's current location.

[0035] In step S3, the processor 201 of the server 200 generates a driving control signal to drive the vehicle 100 toward the determined target position. The processor 201 calculates the vehicle's speed from the change in the vehicle's position and compares the calculated speed with the target speed. Overall, the processor 201 determines the acceleration so that the vehicle 100 accelerates if the speed is lower than the target speed, and determines the acceleration so that the vehicle 100 decelerates if the speed is higher than the target speed. Furthermore, if the vehicle 100 is located on the reference path RR, the processor 201 determines the steering angle and acceleration so that the vehicle 100 does not deviate from the reference path RR, and if the vehicle 100 is not located on the reference path RR, in other words, if the vehicle 100 has deviated from the reference path RR, the processor 201 determines the steering angle and acceleration so that the vehicle 100 returns to the reference path RR.

[0036] In step S4, the processor 201 of the server 200 transmits the generated driving control signal to the vehicle 100. The processor 201 repeats the acquisition of vehicle position information, determination of target position, generation of driving control signal, and transmission of driving control signal at predetermined intervals.

[0037] In step S5, the processor 111 of the vehicle 100 receives a driving control signal transmitted from the server 200. In step S6, the processor 111 of the vehicle 100 controls the actuator group 120 using the received driving control signal, thereby driving the vehicle 100 at the acceleration and steering angle indicated in the driving control signal. The processor 111 repeats the reception of the driving control signal and the control of the actuator group 120 at predetermined intervals. According to the system 50 in this embodiment, the vehicle 100 can be driven by remote control, and the vehicle 100 can be moved without using transport equipment such as cranes or conveyors.

[0038] Figure 5 is a flowchart showing the processing procedure in the first embodiment. The flow shown in Figure 5 is executed repeatedly at predetermined intervals, for example, during the period when the vehicle 100 is being driven under the remote control of the server 200.

[0039] In step S101, the external LiDAR 310 acquires 3D point cloud data PD. In step S102, the external LiDAR 310 transmits the 3D point cloud data PD to the server 200.

[0040] In step S103, the acquisition unit 211 of the server 200 acquires 3D point cloud data PD. In step S104, the detection unit 212 detects that the 3D point cloud data PD acquired by the acquisition unit 211 is incomplete. If the detection unit 212 detects that the 3D point cloud data PD is incomplete (step S104: Yes), in step S105, the calculation unit 213 executes the first calculation process and the second calculation process, respectively. The calculation unit 213 outputs vehicle position information in which the coordinates calculated by performing predetermined calculations on the first coordinate and the second coordinate are the position of the vehicle 100, and the vectors calculated by performing predetermined calculations on the first vector and the second vector are the orientation of the vehicle 100. If the detection unit 212 does not detect that the 3D point cloud data PD is missing (step S104: No), in step S106, the calculation unit 213 executes the first calculation process without executing the second calculation process to calculate the position and orientation of the vehicle 100 and output vehicle position information. In step S107, the remote control unit 214 uses the vehicle position information and the reference path RR to determine the next target position to which the vehicle 100 should go. In step S108, the remote control unit 214 generates a driving control signal to drive the vehicle 100 toward the determined target position. In step S109, the remote control unit 214 transmits the generated driving control signal to the vehicle 100.

[0041] In step S110, the vehicle control unit 115 of the vehicle control device 110 controls the actuator group 120 using the received driving control signal, thereby causing the vehicle 100 to travel at the acceleration and steering angle represented by the driving control signal.

[0042] According to the first embodiment described above, the calculation system 7 can calculate the position and orientation of the vehicle 100 using 3D point cloud data PD representing the vehicle 100 output from the external LiDAR 310 in order to remotely control the vehicle 100. At this time, the calculation system 7 can accurately calculate the position and orientation of the vehicle 100 by matching the 3D point cloud data PD output from the external LiDAR 310 with pre-prepared reference point cloud data through a first calculation process. However, if the 3D point cloud data PD is incomplete, the matching accuracy between the 3D point cloud data PD and the reference point cloud data may decrease, potentially reducing the accuracy of the position and orientation of the vehicle 100. In contrast, according to the first embodiment described above, the calculation system 7 can detect when the acquired 3D point cloud data PD is incomplete. When the calculation system 7 detects that the 3D point cloud data PD is incomplete, it can calculate the position and orientation of the vehicle 100 by executing at least a second calculation process. In this way, when the calculation system 7 detects that the 3D point cloud data PD is missing, it can apply the geometric data FG to the 3D point cloud data PD through a second calculation process, as follows. In this case, the calculation system 7 can estimate the external shape of the vehicle 100, which could not be obtained due to the missing 3D point cloud data PD, from the geometric data FG, and can also estimate the dimensions of the vehicle 100, such as the vehicle width, overall length, and vehicle height. As a result, the calculation system 7 can supplement the information corresponding to the missing parts of the point cloud that make up the 3D point cloud data PD. Therefore, when the calculation system 7 detects that the 3D point cloud data PD is missing, it can suppress a decrease in the accuracy of the position and orientation of the vehicle 100.

[0043] Furthermore, according to the first embodiment described above, the calculation system 7 can detect that the 3D point cloud data PD is incomplete when the number of points constituting the 3D point cloud data PD is less than a predetermined number.

[0044] Furthermore, according to the first embodiment described above, the calculation system 7 can determine a threshold for detecting when 3D point cloud data PD is missing, depending on the degree of influence on the control of the vehicle 100.

[0045] Furthermore, according to the first embodiment described above, the calculation system 7 can detect when an object obstructing the 3D point cloud data PD is detected between the vehicle 100 and the external LiDAR 310, and when the 3D point cloud data PD is missing.

[0046] Furthermore, according to the first embodiment described above, the calculation system 7 can calculate the first coordinates and the first vector by executing a first calculation process when it detects that the 3D point cloud data PD is missing. Also, the calculation system 7 can calculate the second coordinates and the second vector by executing a second calculation process when it detects that the 3D point cloud data PD is missing. As a result, the calculation system 7 can output vehicle position information in which the coordinates calculated by performing predetermined calculations on the first and second coordinates are the position of the vehicle 100, and the vectors calculated by performing predetermined calculations on the first and second vectors are the orientation of the vehicle 100.

[0047] Furthermore, according to the first embodiment described above, when the calculation system 7 detects that the 3D point cloud data PD is missing, it can output vehicle position information in which the coordinate obtained by arithmetic mean of the first coordinate and the second coordinate is the position of the vehicle 100, and the coordinate obtained by arithmetic mean of the first vector and the second vector is the orientation of the vehicle 100.

[0048] Furthermore, according to the first embodiment described above, the calculation system 7 can perform the following processing when it detects that the 3D point cloud data PD is missing. In this case, the calculation system 7 can output vehicle position information in which the weighted average of the first and second coordinates is set as the position of the vehicle 100, and the weighted average of the first and second vectors is set as the orientation of the vehicle 100, with the weight of the second vector being greater than that of the first vector. In this way, the decrease in the accuracy of the position and orientation of the vehicle 100 can be further suppressed. Also, in this case, the second coordinate may be weighted according to the missing percentage of the 3D point cloud data PD in the calculation process of weighted averaging the first and second coordinates, such that the weight increases as the missing percentage of the 3D point cloud data PD increases. The second vector may be weighted according to the missing percentage of the 3D point cloud data PD in the calculation process of weighted averaging the first and second vectors, such that the weight increases as the missing percentage of the 3D point cloud data PD increases. This method further suppresses the decrease in the accuracy of the position and orientation of the vehicle 100.

[0049] Furthermore, the calculation system 7 only needs to be able to calculate at least one of the position and orientation of the vehicle 100. It may calculate the position of the vehicle 100 without calculating the orientation of the vehicle 100, and it may calculate the orientation of the vehicle 100 without calculating the position of the vehicle 100. In other words, the vehicle position information includes at least one of the position and orientation of the vehicle 100.

[0050] B. Second Embodiment: Figure 6 is a block diagram showing the configuration of the driving system 50a in the second embodiment. The driving system 50a comprises a calculation system 7a and a remote control device 80. The calculation system 7a comprises one or more vehicles 100, a calculation device 70a, and one or more external LiDARs 310. In this embodiment, the functions of the calculation device 70a and the remote control device 80 are realized by a server 200a. In this embodiment, the calculation system 7a differs from the first embodiment in its detection method for detecting missing 3D point cloud data PD and its method for calculating the position and orientation of the vehicle 100. The other configurations of the driving system 50a are the same as in the first embodiment unless otherwise specified. Components identical to those in the first embodiment are denoted by the same reference numerals and their descriptions are omitted.

[0051] Server 200a is composed of a computer comprising a processor 201a, memory 202a, input / output interface 203, and internal bus 204. The processor 201a implements the following functions by executing program PG2a stored in memory 202a. The processor 201a implements various functions including those of an acquisition unit 211a, a detection unit 212a, a calculation unit 213a, and a remote control unit 214.

[0052] The acquisition unit 211a acquires multiple 3D point cloud data PDs for the same vehicle 100 detected at different timings.

[0053] The calculation unit 213a calculates the position and orientation of the vehicle 100 at different time points by performing a first calculation process for each of the multiple 3D point cloud data PDs acquired by the acquisition unit 211a. The calculation unit 213a generates time-series data of the position of the vehicle 100 and time-series data of the orientation of the vehicle 100 by arranging the positions and orientations of the vehicle 100 at different time points in chronological order.

[0054] The detection unit 212a uses time-series data to detect that at least one of the multiple 3D point cloud data PDs acquired by the acquisition unit 211a is missing 3D point cloud data PD. If the 3D point cloud data PD is missing and the matching accuracy between the 3D point cloud data PD and the reference point cloud data decreases, the accuracy of the vehicle's position and orientation may decrease, which can cause variations in the vehicle's position and orientation at each timing in the time-series data. Therefore, in this embodiment, the detection unit 212a detects that the 3D point cloud data PD is missing if the variation in at least one of the position and orientation at each timing in the time-series data is greater than or equal to a predetermined variation threshold. The variation threshold is determined, for example, according to the degree of influence on the control of the vehicle 100.

[0055] If the detection unit 212a detects that at least one of the multiple 3D point cloud data PDs is missing, the calculation unit 213a executes the second calculation process without executing the first calculation process. As a result, the calculation unit 213a outputs vehicle position information in which the second coordinate calculated by executing the second calculation process is the position of the vehicle 100, and the second vector calculated by executing the second calculation process is the orientation of the vehicle 100. If the detection unit 212a does not detect that the 3D point cloud data PD is missing, the calculation unit 213a outputs vehicle position information in which the first coordinate calculated by executing the first calculation process is the position of the vehicle 100, and the first vector calculated by executing the first calculation process is the orientation of the vehicle 100.

[0056] Figure 7 is a flowchart showing the processing procedure in the second embodiment. The flow shown in Figure 7 is repeatedly executed at predetermined intervals, for example, during the period when the vehicle 100 is being driven under the remote control of the server 200a.

[0057] In step S201, the external LiDAR 310 detects the same vehicle 100 at multiple different timings. As a result, the external LiDAR 310 acquires multiple 3D point cloud data PDs for the same vehicle 100 detected at different timings. In step S202, the external LiDAR 310 transmits the multiple 3D point cloud data PDs to the server 200a.

[0058] In step S203, the acquisition unit 211a of the server 200a acquires multiple 3D point cloud data PDs for the same vehicle 100 detected at different times. In step S204, the calculation unit 213a calculates the position and orientation of the vehicle 100 at different times by performing a first calculation process for each of the multiple 3D point cloud data PDs acquired by the acquisition unit 211a. In step S205, the calculation unit 213a generates time-series data of the vehicle 100's position and time-series data of the vehicle 100's orientation by arranging the positions and orientations of the vehicle 100 at different times in chronological order. In step S206, the detection unit 212a uses the time-series data to detect that at least one of the multiple 3D point cloud data PDs acquired by the acquisition unit 211a is missing. If the detection unit 212a detects that at least one of the multiple 3D point cloud data PDs is missing (step S206: Yes), in step S207, the calculation unit 213a executes the second calculation process without executing the first calculation process to calculate the position and orientation of the vehicle 100 and outputs the vehicle position information. If the detection unit 212a does not detect that any of the multiple 3D point cloud data PDs are missing (step S206: No), in step S208, the calculation unit 213a outputs the position and orientation of the vehicle 100 calculated by executing the first calculation process as vehicle position information. In step S209, the remote control unit 214 uses the vehicle position information and the reference path RR to determine the next target position to which the vehicle 100 should go. In step S210, the remote control unit 214 generates a driving control signal to drive the vehicle 100 toward the determined target position. In step S211, the remote control unit 214 transmits the generated driving control signal to the vehicle 100.

[0059] In step S212, the vehicle control unit 115 of the vehicle control device 110 controls the actuator group 120 using the received driving control signal, thereby causing the vehicle 100 to travel at the acceleration and steering angle represented by the driving control signal.

[0060] According to the second embodiment described above, the calculation system 7a can acquire multiple 3D point cloud data PDs for the same vehicle 100 detected at different timings. The calculation system 7a can calculate the position and orientation of the vehicle 100 at different timings by executing a first calculation process for each of the acquired 3D point cloud data PDs. The calculation system 7a can generate time-series data of the vehicle 100's position and time-series data of the vehicle 100's orientation by arranging the positions and orientations of the vehicle 100 at different timings in chronological order. As a result, the calculation system 7a can use the time-series data to detect if at least one of the multiple 3D point cloud data PDs is missing. When the calculation system 7a detects that at least one of the multiple 3D point cloud data PDs is missing, it can calculate the position and orientation of the vehicle 100 by executing a second calculation process without executing the first calculation process. In this way, the calculation system 7a can suppress a decrease in the accuracy of the vehicle 100's position and orientation when 3D point cloud data PDs are missing.

[0061] Furthermore, according to the second embodiment described above, the calculation system 7a can detect that the 3D point cloud data PD is missing when the variation in at least one of the position and orientation at each timing of the time series data is greater than or equal to a predetermined threshold.

[0062] C. Third Embodiment: Figure 8 is a block diagram showing the configuration of the driving system 50b in the third embodiment. The driving system 50b comprises a calculation system 7b and a remote control device 80. The calculation system 7b comprises one or more vehicles 100, a calculation device 70b, and one or more external LiDARs 310. In this embodiment, the functions of the calculation device 70b and the functions of the remote control device 80 are realized by a server 200b. In this embodiment, the calculation system 7b differs from the first embodiment in its detection method for detecting missing 3D point cloud data PD and its method for calculating the position and orientation of the vehicle 100. The other configurations of the driving system 50b are the same as in the first embodiment unless otherwise specified. Components identical to those in each of the above embodiments are denoted by the same reference numerals and their descriptions are omitted.

[0063] Server 200b is composed of a computer comprising a processor 201b, memory 202b, input / output interface 203, and internal bus 204. The processor 201b implements the following functions by executing the program PG2b stored in memory 202b. The processor 201b implements various functions, including those of an acquisition unit 211a, a detection unit 212b, a calculation unit 213b, and a remote control unit 214.

[0064] The calculation unit 213b calculates the position and orientation of the vehicle 100 at different times by executing a first calculation process and a second calculation process for each of the multiple 3D point cloud data PDs acquired by the acquisition unit 211a. Hereinafter, the position and orientation of the vehicle 100 calculated by executing the first calculation process will also be called the first calculation result. The position and orientation of the vehicle 100 calculated by executing the second calculation process will also be called the second calculation result. The calculation unit 213b arranges the multiple first calculation results at different times in chronological order. As a result, the calculation unit 213b generates first time-series data, which is time-series data of the position and orientation of the vehicle 100 calculated by executing the first calculation process. The calculation unit 213b arranges the multiple second calculation results at different times in chronological order. As a result, the calculation unit 213b generates second time-series data, which is time-series data of the position and orientation of the vehicle 100 calculated by executing the second calculation process.

[0065] The detection unit 212b uses the first calculation result and the second calculation result to detect that at least one of the multiple 3D point cloud data PDs is missing. For example, the detection unit 212b uses the first time series data and the second time series data to detect that at least one of the multiple 3D point cloud data PDs acquired by the acquisition unit 211a is missing. In this embodiment, the detection unit 212b detects that the 3D point cloud data PD is missing if, in at least one of the time series data between the first time series data and the second time series data, the variation in at least one of the position and orientation at each timing is greater than or equal to a variation threshold.

[0066] If the detection unit 212b detects that at least one of the multiple 3D point cloud data PDs is missing, the calculation unit 213b performs the following processing using the first time series data and the second time series data. In this case, the calculation unit 213b selects whether to output the calculation result obtained by executing the first calculation process or the second calculation process as vehicle position information. If the 3D point cloud data PD is missing and the matching accuracy between the 3D point cloud data PD and the reference point cloud data decreases, the accuracy of the position and orientation of the vehicle 100 decreases, which may cause variations in the position and orientation of the vehicle 100 at each timing of the first time series data. Also, if the accuracy of the position and orientation of the vehicle 100 calculated by applying the geometric data FG to the 3D point cloud data PD decreases due to the detection status of the vehicle 100, etc., variations may occur in the position and orientation of the vehicle 100 at each timing of the second time series data. Therefore, in this embodiment, the calculation unit 213b, when comparing the first time series data and the second time series data between the first and second calculation processes, selects to output the calculation result corresponding to the time series data with smaller variation in position and orientation at each timing of the time series data as vehicle position information. For example, if the variation in position and orientation at each timing of the second time series data is smaller than the variation in position and orientation at each timing of the first time series data, the calculation unit 213b makes the following selection. In this case, the calculation unit 213b selects the second calculation result.

[0067] The calculation unit 213b outputs the selected calculation result as vehicle location information. At this time, the calculation unit 213b may output a predetermined calculation result as vehicle location information from among multiple calculation results used to generate time-series data. After selecting whether to output the calculation result obtained by executing the first calculation process or the second calculation process as vehicle location information, the calculation unit 213b may obtain a new calculation result by executing the selected calculation process again and output the newly obtained calculation result as vehicle location information.

[0068] Figure 9 is a flowchart showing the processing procedure in the third embodiment. The flow shown in Figure 9 is repeatedly executed at predetermined intervals, for example, during the period when the vehicle 100 is being driven under the remote control of the server 200b.

[0069] In step S301, the external LiDAR 310 detects the same vehicle 100 at multiple different timings. As a result, the external LiDAR 310 acquires multiple 3D point cloud data PDs for the same vehicle 100 detected at different timings. In step S302, the external LiDAR 310 transmits the multiple 3D point cloud data PDs to the server 200b.

[0070] In step S303, the acquisition unit 211a of the server 200b acquires multiple 3D point cloud data PDs for the same vehicle 100 detected at different times. In step S304, the calculation unit 213b calculates the position and orientation of the vehicle 100 at different times by executing a first calculation process and a second calculation process for each of the multiple 3D point cloud data PDs acquired by the acquisition unit 211a. In step S305, the calculation unit 213b generates first time-series data by arranging the multiple first calculation results from different times in chronological order. In step S306, the calculation unit 213b generates second time-series data by arranging the multiple second calculation results from different times in chronological order. In step S307, the detection unit 212b uses the first time-series data and the second time-series data to detect that at least one of the multiple 3D point cloud data PDs acquired by the acquisition unit 211a is missing. If the detection unit 212b detects that at least one of the multiple 3D point cloud data PDs is missing (step S307: Yes), the calculation unit 213 executes step S308. In step S308, the calculation unit 213b uses the first time series data and the second time series data to select whether to output the first calculation result or the second calculation result as vehicle position information. In step S309, the calculation unit 213b outputs the selected calculation result from the first and second calculation results as vehicle position information. If the detection unit 212b does not detect that any of the multiple 3D point cloud data PDs are missing (step S307: No), the calculation unit 213b executes step S310. In step S310, the calculation unit 213b outputs the first calculation result as vehicle position information. In step S311, the remote control unit 214 uses the vehicle position information and the reference path RR to determine the next target position that the vehicle 100 should head to. In step S312, the remote control unit 214 generates a driving control signal to move the vehicle 100 toward the determined target position. In step S313, the remote control unit 214 transmits the generated driving control signal to the vehicle 100.

[0071] In step S314, the vehicle control unit 115 of the vehicle control device 110 controls the actuator group 120 using the received driving control signal, thereby causing the vehicle 100 to travel at the acceleration and steering angle represented by the driving control signal.

[0072] According to the third embodiment described above, the calculation system 7b can use the first calculation result and the second calculation result to detect that at least one of the multiple 3D point cloud data PDs is missing. When the calculation system 7b detects that at least one of the multiple 3D point cloud data PDs is missing, it can use the first calculation result and the second calculation result to select which of the first and second calculation processes to execute to output the calculated result as vehicle position information. In other words, when the calculation system 7b detects that at least one of the multiple 3D point cloud data PDs is missing, it can determine and select which of the first and second calculation results is the more accurate calculation result. The calculation system 7b can output the selected calculation result as vehicle position information. In this way, when the calculation system 7b detects that at least one of the multiple 3D point cloud data PDs is missing, it can suppress a decrease in the accuracy of the position and orientation of the vehicle 100.

[0073] Furthermore, according to the third embodiment described above, the calculation system 7b can acquire multiple 3D point cloud data PDs for the same vehicle 100 detected at different timings. The calculation system 7b can calculate the position and orientation of the vehicle 100 at different timings by executing a first calculation process and a second calculation process for each of the acquired 3D point cloud data PDs. The calculation system 7b can generate first time-series data by arranging multiple first calculation results at different timings in chronological order. The calculation system 7b can generate second time-series data by arranging multiple second calculation results at different timings in chronological order. As a result, the calculation system 7b can use the first time-series data and the second time-series data to detect that at least one of the multiple 3D point cloud data PDs is missing. When the calculation system 7b detects that at least one of the multiple 3D point cloud data PDs is missing, it can execute the following process. In this case, the calculation system 7b can use the first time series data and the second time series data to select which of the first and second calculation processes to execute to output the calculated result as vehicle position information. In other words, when the calculation system 7b detects that at least one of the multiple 3D point cloud data PDs is missing, it can use the first and second time series data to determine and select which of the first and second calculation results is more accurate. The calculation system 7b can output the selected calculation result as vehicle position information. In this way, the calculation system 7b can suppress a decrease in the accuracy of the position and orientation of the vehicle 100 when it detects that at least one of the multiple 3D point cloud data PDs is missing.

[0074] Furthermore, according to the third embodiment described above, the calculation system 7b can select which of the first calculation process or the second calculation process to execute to output the calculated result as vehicle position information by comparing the variability between the first time series data and the second time series data.

[0075] D. Fourth Embodiment: Figure 10 is a block diagram showing the configuration of the driving system 50c in the fourth embodiment. The driving system 50c comprises a calculation system 7c and a remote control device 80. The calculation system 7c comprises one or more vehicles 100, a calculation device 70c, and one or more external LiDARs 310. In this embodiment, the functions of the calculation device 70c and the remote control device 80 are realized by a server 200c. In this embodiment, the calculation system 7c differs from the first embodiment in the type of functional unit that acquires information regarding missing data in the 3D point cloud data PD, and in the method for calculating the position and orientation of the vehicles 100. The other configurations of the driving system 50c are the same as in the first embodiment unless otherwise specified. Components identical to those in the first embodiment are denoted by the same reference numerals and their descriptions are omitted.

[0076] Server 200c is composed of a computer comprising a processor 201c, memory 202c, input / output interface 203, and internal bus 204. The processor 201c implements the following functions by executing the program PG2c stored in memory 202c. The processor 201c implements various functions, including those of an acquisition unit 211, a determination unit 215, a calculation unit 213c, and a remote control unit 214.

[0077] The determination unit 215 is one of the functional units that acquires information regarding missing data in the 3D point cloud data PD. The determination unit 215 determines whether or not the vehicle 100 is located within a specific area where it is anticipated that the 3D point cloud data PD acquired by the acquisition unit 211 may be missing data. The specific area is, for example, a work area where a manufacturing process is performed to carry out a specific task. In the specific task, a work object, which is at least one of the manufacturing equipment used in the manufacturing process or the workers engaged in the work in the manufacturing process, enters at least one of the interior of the vehicle 100 or the area surrounding the vehicle 100 to perform the work. The manufacturing equipment used in the manufacturing process is, for example, equipment used to assemble the vehicle 100, an automated guided vehicle that transports parts used in the manufacture of the vehicle 100, or an articulated robot that assembles parts onto the vehicle 100. If the vehicle 100 is located in the work area, the 3D point cloud data PD may be missing data due to work conditions such as the work object being located between the vehicle 100 and the external LiDAR 310. Therefore, the determination unit 215 determines whether or not the vehicle 100 is present within the work area, which is designated as a specific area.

[0078] In this embodiment, the determination unit 215 determines whether or not the vehicle 100 is present within the work area by acquiring process information indicating the manufacturing process being executed on the vehicle 100 and estimating the current location of the vehicle 100. The determination unit 215 acquires process information using, for example, management information MI pre-stored in the memory 202c of the server 200c. Management information MI is information indicating the manufacturing status of the vehicle 100. Management information MI is, for example, manufacturing time information indicating the timing at which multiple manufacturing processes are executed on the vehicle 100. In the manufacturing plan information, the vehicle identifier, the process identifier, and the execution timing of each manufacturing process are associated. The vehicle identifier is a unique identifier assigned to multiple vehicles 100 so as not to overlap among the vehicles 100 in order to identify multiple vehicles 100. The manufacturing time information is, for example, created according to the manufacturing plan of the vehicle 100 and updated as appropriate according to the progress toward the manufacturing plan. The determination unit 215 acquires, for example, the process identifier corresponding to the vehicle identification information of the target vehicle 100 as process information in the manufacturing time information.

[0079] The determination unit 215 may determine whether or not a vehicle 100 is present in a specific area by other means. For example, the determination unit 215 may determine whether or not a vehicle 100 is present in a specific area by acquiring sequence information and estimating the current location of the vehicle 100. The sequence information is information indicating the travel order of multiple vehicles 100 traveling within the detection range of multiple external LiDARs 310 installed in the factory FC. In the sequence information, a vehicle identifier and a sensor identifier are associated. The sensor identifier is a unique identifier assigned to multiple external sensors 300 installed in the factory FC so as not to overlap among the external sensors 300. The sequence information is created, for example, using vehicle position information, the transmission history of travel control signals to the vehicle 100, and the installation locations of the external sensors 300.

[0080] If the determination unit 215 determines that a vehicle 100 exists within a specific area, the calculation unit 213c calculates the position and orientation of the vehicle 100 by executing the second calculation process without executing the first calculation process. As a result, the calculation unit 213c obtains vehicle position information. If the determination unit 215 determines that a vehicle 100 does not exist within a specific area, the calculation unit 213c calculates the position and orientation of the vehicle 100 by executing the first calculation process without executing the second calculation process. As a result, the calculation unit 213c obtains vehicle position information.

[0081] Figure 11 is a flowchart showing the processing procedure in the fourth embodiment. The flow shown in Figure 11 is repeatedly executed at predetermined intervals, for example, during the period when the vehicle 100 is being driven under the remote control of the server 200c.

[0082] In step S401, the external LiDAR 310 acquires 3D point cloud data PD. In step S402, the external LiDAR 310 transmits the 3D point cloud data PD to the server 200c.

[0083] In step S403, the acquisition unit 211 of the server 200c acquires 3D point cloud data PD. In step S404, the determination unit 215 acquires process information. In step S405, the determination unit 215 uses the process information to estimate the current location of the vehicle 100 and determines whether or not the vehicle 100 is present in the work area. If the determination unit 215 determines that the vehicle 100 is present in the work area (step S405: Yes), in step S406, the calculation unit 213c executes the second calculation process without executing the first calculation process to calculate the position and orientation of the vehicle 100 and outputs the vehicle position information. If the determination unit 215 determines that the vehicle 100 is not present in the specific area (step S405: No), the calculation unit 213c executes step S407. In step S407, the calculation unit 213c executes the first calculation process without executing the second calculation process to calculate the position and orientation of the vehicle 100 and outputs the vehicle position information. In step S408, the remote control unit 214 uses the vehicle position information and the reference path RR to determine the next target location to which the vehicle 100 should go. In step S409, the remote control unit 214 generates a driving control signal to drive the vehicle 100 toward the determined target location. In step S410, the remote control unit 214 transmits the generated driving control signal to the vehicle 100.

[0084] In step S411, the vehicle control unit 115 of the vehicle control device 110 controls the actuator group 120 using the received driving control signal, thereby driving the vehicle 100 with the acceleration and steering angle represented by the driving control signal.

[0085] According to the fourth embodiment described above, the calculation system 7c can determine whether or not the vehicle 100 is located within a specific area where it is assumed that the 3D point cloud data PD is missing. If the calculation system 7c determines that the vehicle 100 is located within the specific area, it can perform at least a second calculation process to calculate the position and orientation of the vehicle 100. This allows the calculation system 7c to suppress a decrease in the accuracy of the position and orientation of the vehicle 100 when the presence of the vehicle 100 within the specific area may result in missing 3D point cloud data PD.

[0086] Furthermore, according to the fourth embodiment described above, the calculation system 7c can determine whether or not the vehicle 100 is located within a work area where there is a possibility that the 3D point cloud data PD may be missing when the 3D point cloud data PD is acquired.

[0087] Furthermore, according to the fourth embodiment described above, the calculation system 7c can determine whether or not a vehicle 100 is present in a specific area by acquiring process information using management information MI.

[0088] Furthermore, according to the fourth embodiment described above, the calculation system 7c can determine whether or not the vehicle 100 is located within a specific area by estimating the current location of the vehicle 100 using sequence information.

[0089] Furthermore, according to the fourth embodiment described above, when the calculation system 7c determines that a vehicle 100 is present within a specific area, it can calculate the position and orientation of the vehicle 100 by executing the second calculation process without executing the first calculation process. This reduces the processing load when calculating the position and orientation of the vehicle 100.

[0090] The specified area may include, in addition to or instead of, the work area, other areas other than the work area. The specified area may include, for example, a parking area such as a yard where manufactured vehicles 100 are parked and stored. If vehicles 100 are present in the parking area, the 3D point cloud data PD may be incomplete depending on the arrangement of vehicles 100, such as other vehicles 100 being located between vehicle 100 and the external LiDAR 310. Therefore, the determination unit 215 may determine whether or not vehicles 100 are present in the parking area designated as the specified area. In this way, the calculation system 7c can determine whether or not vehicles 100 are present in the parking area where there is a possibility that the 3D point cloud data PD may be incomplete when the 3D point cloud data PD is acquired.

[0091] E. Fifth Embodiment: Figure 12 is a block diagram showing the configuration of the driving system 50d in the fifth embodiment. The driving system 50d comprises a calculation system 7d and a remote control device 80. The calculation system 7d comprises one or more vehicles 100, a calculation device 70d, and one or more external LiDARs 310. In this embodiment, the functions of the calculation device 70d and the remote control device 80 are realized by a server 200d. In this embodiment, the calculation system 7d differs from the first embodiment in the type of functional unit that acquires information regarding missing data in the 3D point cloud data PD, and in the method for calculating the position and orientation of the vehicles 100. The other configurations of the driving system 50d are the same as in the first embodiment unless otherwise specified. Components identical to those in the first embodiment are denoted by the same reference numerals and their descriptions are omitted.

[0092] Server 200d is composed of a computer comprising a processor 201d, memory 202d, input / output interface 203, and internal bus 204. Processor 201d implements the following functions by executing program PG2d stored in memory 202d. Processor 201d implements various functions including those of an acquisition unit 211, a prediction unit 216, a calculation unit 213d, and a remote control unit 214.

[0093] The prediction unit 216 is one of the functional units that acquires information about missing data in the 3D point cloud data PD. The prediction unit 216 predicts that there will be missing data in the 3D point cloud data PD before the 3D point cloud data PD is acquired.

[0094] The prediction unit 216 predicts, for example, that the 3D point cloud data PD is missing using object information. The object information is information about at least one of the objects present in the first area where the first manufacturing process is performed, and the objects present in the second area where the second manufacturing process is performed. The first manufacturing process is a manufacturing process being performed on the vehicle 100. The second manufacturing process is a manufacturing process scheduled to be performed on the vehicle 100 after the first manufacturing process. The object information is, for example, information representing the number and arrangement of objects present in the first area and objects present in the second area. The object information may also be, for example, the number of workers engaged in the manufacturing process, the arrangement of manufacturing equipment that performs specific tasks, or the range of motion of manufacturing equipment such as articulated robots. The object information may also be, for example, information indicating the travel position of other vehicles 100.

[0095] The prediction unit 216 may also predict that the 3D point cloud data PD will be missing by other means. For example, the prediction unit 216 may predict that the 3D point cloud data PD will be missing using work information. Work information is information indicating whether or not a specific operation is performed in at least one of the first manufacturing process and the second manufacturing process.

[0096] If the prediction unit 216 predicts that the 3D point cloud data PD is incomplete, the calculation unit 213d performs at least the second calculation process. In this embodiment, if the prediction unit 216 predicts that the 3D point cloud data PD is incomplete, the calculation unit 213d performs the second calculation process without performing the first calculation process to calculate the position and orientation of the vehicle 100 and outputs the vehicle position information. If the prediction unit 216 predicts that the 3D point cloud data PD is not incomplete, the calculation unit 213d performs the first calculation process without performing the second calculation process to calculate the position and orientation of the vehicle 100 and outputs the vehicle position information.

[0097] Figure 13 is a flowchart showing the processing procedure in the fifth embodiment. The flow shown in Figure 13 is repeatedly executed at predetermined intervals, for example, during the period when the vehicle 100 is being driven under the remote control of the server 200d.

[0098] In step S501, the external LiDAR 310 acquires 3D point cloud data PD. In step S502, the external LiDAR 310 transmits the 3D point cloud data PD to the server 200c.

[0099] In step S503, the acquisition unit 211 of the server 200d acquires the 3D point cloud data PD. In step S504, the prediction unit 216 predicts that the 3D point cloud data PD will be incomplete. If the prediction unit 216 predicts that the 3D point cloud data PD will be incomplete (step S504: Yes), the calculation unit 213d executes step S505. In step S505, the calculation unit 213d executes the second calculation process without executing the first calculation process to calculate the position and orientation of the vehicle 100 and outputs the vehicle position information. If the prediction unit 216 predicts that the 3D point cloud data PD will not be incomplete (step S504: No), the calculation unit 213d executes step S506. In step S506, the calculation unit 213d executes the first calculation process without executing the second calculation process to calculate the position and orientation of the vehicle 100 and outputs the vehicle position information. In step S507, the remote control unit 214 uses the vehicle position information and the reference path RR to determine the next target location to which the vehicle 100 should go. In step S508, the remote control unit 214 generates a driving control signal to drive the vehicle 100 toward the determined target location. In step S509, the remote control unit 214 transmits the generated driving control signal to the vehicle 100.

[0100] In step S510, the vehicle control unit 115 of the vehicle control device 110 controls the actuator group 120 using the received driving control signal, thereby driving the vehicle 100 with the acceleration and steering angle represented by the driving control signal.

[0101] According to the fifth embodiment described above, the calculation system 7d can predict that the 3D point cloud data PD will be incomplete before the 3D point cloud data PD is acquired. When the calculation system 7d predicts that the 3D point cloud data PD will be incomplete, it can perform at least a second calculation process to calculate the position and orientation of the vehicle 100. In this way, the calculation system 7d can suppress a decrease in the accuracy of the position and orientation of the vehicle 100 when it is predicted that the 3D point cloud data PD will be incomplete.

[0102] Furthermore, according to the fifth embodiment described above, the calculation system 7d can predict that the 3D point cloud data PD is missing using at least one of the object information and the work information.

[0103] F. Sixth Embodiment: Figure 14 is a block diagram showing the configuration of the driving system 50v in the sixth embodiment. The driving system 50v includes a calculation system 7v. The calculation system 7v includes one or more vehicles 100 and one or more external LiDARs 310. In this embodiment, the driving system 50v differs from the first embodiment in that it does not include a server 200. The functions of the calculation device 70v are realized by the vehicle control device 110v. In addition, the vehicle 100v in this embodiment can be driven by autonomous control of the vehicle 100v. The other configurations are the same as in the first embodiment unless otherwise specified.

[0104] In this embodiment, the processor 111v of the vehicle control device 110v functions as an acquisition unit 116, a detection unit 117, a calculation unit 118, and a vehicle control unit 115v by executing a program PG1v stored in memory 112v. The acquisition unit 116 acquires 3D point cloud data PD by detecting the vehicle 100v from the outside using an external LiDAR 310. The detection unit 117 detects that the 3D point cloud data PD acquired by the acquisition unit 116 is incomplete. The calculation unit 118 calculates the position and orientation of the vehicle 100v using the 3D point cloud data PD and outputs vehicle position information. If the detection unit 117 detects that the 3D point cloud data PD is incomplete, the calculation unit 118 calculates the position and orientation of the vehicle 100v by executing at least a second calculation process and outputs vehicle position information. If the detection unit 117 does not detect that the 3D point cloud data PD is missing, the calculation unit 118 executes the first calculation process without executing the second calculation process to calculate the position and orientation of the vehicle 100v and output vehicle position information. The vehicle control unit 115v acquires the output results from the sensors, generates a driving control signal using the output results, and outputs the generated driving control signal to operate the actuator group 120, thereby enabling the vehicle 100v to be driven autonomously. In this embodiment, the memory 112v has the program PG1, as well as the detection model and reference path RR pre-stored in it.

[0105] Figure 15 is a flowchart showing the processing procedure for vehicle 100v's driving control in the sixth embodiment. The flow shown in Figure 15 is executed repeatedly at predetermined intervals, for example, during the period when vehicle 100v is driving under autonomous control. In the processing procedure in Figure 15, the vehicle 100v's processor 111v functions as an acquisition unit 116, a detection unit 117, a calculation unit 118, and a vehicle control unit 115v by executing the program PG1v.

[0106] In step S901, the processor 111v of the vehicle control device 110v acquires vehicle position information using the detection results output from the external LiDAR 310, which is an external sensor 300. In step S902, the processor 111v determines the target position to which the vehicle 100v should next go. In step S903, the processor 111v generates a driving control signal to drive the vehicle 100v toward the determined target position. In step S904, the processor 111v controls the actuator group 120 using the generated driving control signal to drive the vehicle 100v according to the parameters expressed in the driving control signal. The processor 111v repeats the acquisition of vehicle position information, determination of the target position, generation of the driving control signal, and control of the actuators at a predetermined cycle. According to the driving system 50v in this embodiment, the vehicle 100v can be driven by autonomous control of the vehicle 100v without remote control of the vehicle 100v by the server 200.

[0107] G. Other embodiments: G-1. Other Embodiments 1: At least some of the functions of servers 200, 200a to 200d may be functions of vehicle control devices 110, 110v, or functions of external sensors 300. Similarly, at least some of the functions of vehicle control devices 110, 110v may be functions of servers 200, 200a to 200d, or functions of external sensors 300. In other words, calculation devices 70, 70a to 70d, 70v, which include acquisition units 116, 211, 211a, prediction unit 216, detection units 117, 212, 212a, 212b, and determination unit 215, and calculation units 118, 213, 213a to 213d, may be servers 200, 200a to 200d, or vehicle control devices 110, 110v. In this configuration, the configuration of calculation systems 7, 7a to 7d, 7v can be changed as appropriate.

[0108] G-2. Other Embodiments 2: The calculation systems 7, 7a to 7d, and 7v only need to include at least one functional unit: a prediction unit 216, detection units 117, 212, 212a, and 212b, and a judgment unit 215. The calculation systems 7, 7a to 7d, and 7v may also include, for example, three functional units: a prediction unit 216, detection units 117, 212, 212a, and 212b, and a judgment unit 215. In this case, the calculation units 118, 213, 213a to 213d calculate at least one of the position and orientation of the vehicles 100 and 100v by performing at least the second calculation process when at least one of the first, second, or third cases is met. The first case is when the prediction unit 216 predicts that the 3D point cloud data PD is missing. The second case is when the detection units 117, 212, 212a, and 212b detect that the 3D point cloud data PD is missing. The third case is when the determination unit 215 determines that vehicles 100 and 100v exist within a specific area. In this configuration, the calculation systems 7, 7a to 7d, and 7v can suppress a decrease in the accuracy of the position and orientation of vehicles 100 and 100v in both cases: when it is predicted that the 3D point cloud data PD will be missing, and when the 3D point cloud data PD is actually missing.

[0109] G-3. Other Embodiments 3: In each of the above embodiments, the driving systems 50, 50a to 50d and 50v were equipped with an external LiDAR 310 as an external sensor 300. In contrast, the driving systems 50, 50a to 50d and 50v may also be equipped with a camera as an external sensor 300, for example. The camera as an external sensor 300 captures images of the vehicles 100 and 100v and outputs the captured images as detection results. When acquiring vehicle position information using the captured images acquired from the camera which is an external sensor 300, the calculation units 118, 213, 213a to 213d, for example, detect the outline of the vehicles 100 and 100v from the captured images, calculate the coordinates of the positioning points of the vehicles 100 and 100v in the coordinate system of the captured images, i.e., the local coordinate system, and acquire the position of the vehicles 100 and 100v by converting the calculated coordinates to coordinates in the global coordinate system GC. The outlines of vehicles 100 and 100v included in the captured images can be detected, for example, by inputting the captured images into a detection model utilizing artificial intelligence. The detection model is prepared, for example, within the driving systems 50, 50a~50d, 50v or outside of systems 50, 50a~50d, 50v, and is pre-stored in memory 112, 112v, 202, 202a~202d. Examples of detection models include pre-trained machine learning models that have been trained to achieve either semantic segmentation or instance segmentation. As for this machine learning model, for example, a convolutional neural network (CNN) trained by supervised learning using a training dataset can be used. The training dataset includes, for example, multiple training images containing vehicles 100 and 100v, and labels indicating whether each region in the training image represents a region of vehicle 100 or 100v or a region other than vehicle 100 or 100v. During CNN training, it is preferable to update the CNN parameters using backpropagation to reduce the error between the detection model's output and the labels.Furthermore, calculation units 118, 213, 213a to 213d can obtain the orientation of vehicles 100 and 100v by, for example, using the optical flow method, and estimating it based on the direction of the movement vector of vehicles 100 and 100v calculated from the positional changes of feature points of vehicles 100 and 100v between frames of the captured image.

[0110] G-4. Other Embodiments 4: In each of the embodiments from the first to the fifth embodiment described above, the servers 200, 200a to 200d perform the processing from acquiring vehicle position information to generating driving control signals. In contrast, the vehicle 100 may perform at least a part of the processing from acquiring vehicle position information to generating driving control signals. For example, the following forms (1) to (3) may also be used.

[0111] (1) Servers 200, 200a to 200d may acquire vehicle location information, determine the next target location that vehicle 100 should head to, and generate a route from the vehicle 100's current location to the target location as shown in the acquired vehicle location information. Servers 200, 200a to 200d may generate a route to the target location between the current location and the destination, or they may generate a route to the destination. Servers 200, 200a to 200d may transmit the generated route to vehicle 100. Vehicle 100 may generate a driving control signal so that vehicle 100 travels along the route received from servers 200, 200a to 200d, and may use the generated driving control signal to control the actuator group 120.

[0112] (2) Servers 200, 200a to 200d may acquire vehicle location information and transmit the acquired vehicle location information to vehicle 100. Vehicle 100 may determine the next target location to which vehicle 100 should go, generate a route from vehicle 100's current location to the target location as shown in the received vehicle location information, generate a driving control signal so that vehicle 100 travels along the generated route, and control the actuator group 120 using the generated driving control signal.

[0113] (3) In the embodiments of (1) and (2) above, the vehicle 100 is equipped with internal sensors, and the detection results output from the internal sensors may be used in at least one of the generation of a route and the generation of a driving control signal. The internal sensors are sensors mounted on the vehicle 100. The internal sensors may include, for example, sensors that detect the motion state of the vehicle 100, sensors that detect the operating state of each part of the vehicle 100, and sensors that detect the environment around the vehicle 100. Specifically, the internal sensors may include, for example, cameras, LiDAR, millimeter-wave radar, ultrasonic sensors, GPS sensors, acceleration sensors, gyro sensors, etc. For example, in the embodiment of (1) above, servers 200, 200a to 200d may acquire the detection results of the internal sensors and reflect the detection results of the internal sensors in the route when generating a route. In the embodiment of (1) above, the vehicle 100 may acquire the detection results of the internal sensors and reflect the detection results of the internal sensors in the driving control signal when generating a driving control signal. In the embodiment of (2) above, the vehicle 100 may acquire the detection results of the internal sensors and reflect the detection results of the internal sensors in the route when generating a route. In the embodiment described in (2) above, the vehicle 100 may acquire the detection results of the internal sensors and reflect the detection results of the internal sensors in the driving control signal when generating the driving control signal.

[0114] G-5. Other Embodiments 5: In the sixth embodiment described above, the vehicle 100v is equipped with an internal sensor, and the detection result output from the internal sensor may be used in at least one of the generation of the route and the generation of the driving control signal. For example, the vehicle 100v may acquire the detection result from the internal sensor and reflect the detection result from the internal sensor in the route when generating the route. The vehicle 100v may acquire the detection result from the internal sensor and reflect the detection result from the internal sensor in the driving control signal when generating the driving control signal.

[0115] G-6. Other Embodiments 6: In the sixth embodiment described above, the vehicle 100v acquires vehicle position information using the detection results of the external sensor 300. Alternatively, the vehicle 100v may be equipped with an internal sensor, which may acquire vehicle position information using the detection results of the internal sensor, determine the next target location to which the vehicle 100v should go, generate a route from the vehicle 100v's current location to the target location as shown in the acquired vehicle position information, generate a driving control signal for traveling along the generated route, and control the actuator group 120 using the generated driving control signal. In this case, the vehicle 100v can travel without using the detection results of the external sensor 300 at all. The vehicle 100v may also acquire target arrival time and congestion information from outside the vehicle 100v and reflect the target arrival time and congestion information in at least one of the route and the driving control signal. Furthermore, all the functional configurations of the driving system 50v may be provided in the vehicle 100v. In other words, the processing realized by the driving system 50v in this disclosure may be realized by the vehicle 100v alone.

[0116] G-7. Other Embodiments 7: In each of the embodiments from the first to the fifth described above, servers 200, 200a to 200d automatically generate driving control signals to be transmitted to the vehicle 100. Alternatively, servers 200, 200a to 200d may generate driving control signals to be transmitted to the vehicle 100 in accordance with the operations of an external operator located outside the vehicle 100. For example, an external operator may operate a control device that includes a display for displaying captured images output from an external sensor 300, a steering wheel for remotely controlling the vehicle 100, an accelerator pedal, a brake pedal, and a communication device for communicating with servers 200, 200a to 200d via wired or wireless communication, and servers 200, 200a to 200d may generate driving control signals in accordance with the operations applied to the control device.

[0117] G-8. Other Embodiments 8: In each of the above embodiments, the vehicles 100 and 100v only need to be configured to be able to move by unmanned operation, and may take the form of a platform having the configuration described below. Specifically, in order for the vehicles 100 and 100v to perform the three functions of "driving," "turning," and "stopping" by unmanned operation, they only need to be equipped with at least a vehicle control device 110 and 110v and an actuator group 120. When the vehicles 100 and 100v acquire information from the outside for unmanned operation, they may further be equipped with a communication device 130. That is, the vehicles 100 and 100v that can move by unmanned operation do not need to have at least some of the interior parts such as the driver's seat and dashboard installed, at least some of the exterior parts such as the bumper and fender installed, and do not need to have a body shell installed. In this case, the remaining parts such as the body shell may be attached to the vehicle 100, 100v before it is shipped from the factory FC, or the remaining parts such as the body shell may be attached to the vehicle 100, 100v after it has been shipped from the factory FC, while the remaining parts such as the body shell are not attached to the vehicle 100, 100v. Each part may be attached to the vehicle 100, 100v from any direction, such as the top, bottom, front, rear, right, or left, and each part may be attached from the same direction or from different directions. The positioning of the platform can also be determined in the same way as the vehicle 100, 100v in the first embodiment.

[0118] G-9. Other Embodiments 9: Vehicles 100 and 100v may be manufactured by combining multiple modules. A module refers to a unit composed of multiple parts grouped according to the part or function of the vehicle 100 and 100v. For example, the platform of vehicle 100 and 100v may be manufactured by combining a front module that constitutes the front part of the platform, a central module that constitutes the central part of the platform, and a rear module that constitutes the rear part of the platform. The number of modules that make up the platform is not limited to three, but may be two or fewer, or four or more. In addition to, or instead of, the parts that make up the platform, parts that make up parts of vehicle 100 and 100v that are different from the platform may be modularized. Furthermore, various modules may include any exterior parts such as bumpers and grilles, or any interior parts such as seats and consoles. Moreover, not limited to vehicles 100 and 100v, any type of mobile body may be manufactured by combining multiple modules. Such modules may be manufactured, for example, by joining multiple parts by welding or fasteners, or by integrally molding at least a part of the parts that make up the module as a single part by casting. A molding technique for integrally molding a single component, especially a relatively large component, is also called gigacast or megacast. For example, the front module, central module, and rear module mentioned above may be manufactured using gigacast.

[0119] G-10. Other Embodiments 10: The use of unmanned operation of the 100,100v vehicle to transport the 100,100v vehicle is also called "autonomous transport." The configuration for realizing autonomous transport is also called a "vehicle remote control autonomous driving transport system." Furthermore, a production method that uses autonomous transport to produce the 100,100v vehicle is also called "autonomous production." In autonomous production, for example, in a factory cluster (FC) that manufactures the 100,100v vehicle, at least a portion of the transport of the 100,100v vehicle is realized by autonomous transport.

[0120] G-11. Other Embodiments 11: In each of the above embodiments, some or all of the functions and processes implemented in software may be implemented in hardware. Conversely, some or all of the functions and processes implemented in hardware may be implemented in software. As hardware for implementing the various functions in each of the above embodiments, various circuits such as integrated circuits and discrete circuits may be used.

[0121] This disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from its spirit. For example, the technical features of the embodiments corresponding to the technical features in each form described in the summary of the invention can be replaced or combined as appropriate in order to solve some or all of the above-described problems, or to achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate. [Explanation of Symbols]

[0122] 7, 7a~7d, 7v…Calculation system, 50, 50a~50d, 50v…Driving system, 70, 70a~70d, 70v…Calculation device, 80…Remote control device, 100, 100v…Vehicle, 110, 110v…Vehicle control device, 111, 111v…Processor of vehicle control device, 112, 112v…Memory of vehicle control device, 113…Input / output interface of vehicle control device, 114…Internal bus of vehicle control device, 115, 115v…Vehicle control unit, 116, 211, 211a…Acquisition unit, 117, 212, 212a, 212b…Detection unit, 118, 213, 213a~213d…Calculation unit, 120…Actuator group, 130…Vehicle communication device, 200, 200a~200d…Server, 201, 201a~ 201d…Server processor, 202,202a~202d…Server memory, 203…Server input / output interface, 204…Server internal bus, 205…Server communication device, 214…Remote control unit, 215…Decision unit, 216…Prediction unit, 300…External sensor, 310…External LiDAR, CN1…First central position, CN2…Second central position, DB1…Quantity database, DB2…Coordinate database, FC…Factory, FG…Geometric data, GC…Global coordinate system, MI…Management information, PD…3D point cloud data, PG1,PG1v,PG2,PG2a~PG2d…Program, PL1…First location, PL2…Second location, RR…Reference path, SB1~SB12…Edge, TR…Track, VB1~VB8…Vertex

Claims

1. A calculation device, An acquisition unit that acquires 3D point cloud data representing a mobile object that can be moved by unmanned operation, A calculation unit that uses the three-dimensional point cloud data to calculate at least one of the position and orientation of the moving object, The system comprises at least one functional unit, which includes: (i) a prediction unit that predicts that the 3D point cloud data is missing; (ii) a detection unit that detects that the 3D point cloud data is missing; and (iii) a determination unit that determines whether or not the moving object is located within a specific area where it is assumed that the 3D point cloud data is missing. The calculation unit is capable of performing a first calculation process that calculates at least one of the position and orientation of the moving body by comparing the three-dimensional point cloud data with pre-prepared reference point cloud data, and a second calculation process that calculates at least one of the position and orientation of the moving body by applying geometric data of a predetermined shape to the three-dimensional point cloud data. The calculation unit is a calculation device that performs at least the second calculation process when at least one of the following cases is met: a first case in which the prediction unit predicts that the three-dimensional point cloud data is missing; a second case in which the detection unit detects that the three-dimensional point cloud data is missing; or a third case in which the determination unit determines that the moving object exists within the specific area.

2. A calculation device according to claim 1, The prediction unit, Object information relating to at least one of the following: an object located in a first area where a first manufacturing process, which is a manufacturing process performed on the moving body, is carried out; and an object located in a second area where a second manufacturing process, which is a manufacturing process scheduled to be performed on the moving body, is carried out; A calculation device that predicts that the three-dimensional point cloud data is missing, using at least one of the following: a manufacturing apparatus used in the first manufacturing process, a worker engaged in the work in the manufacturing process, and work information indicating whether a specific operation is performed in which a work object enters at least one of the interior of the moving body and the surrounding area of ​​the moving body.

3. A calculation device according to claim 1, The calculation device is an area in which a specific operation is performed in which at least one of the working objects, which is a manufacturing device used in the manufacturing process of the mobile body or a worker engaged in the work in the manufacturing process, enters at least one of the interior of the mobile body or the area surrounding the mobile body to perform the operation.

4. A calculation device according to claim 1, The detection unit is a calculation device that detects that the 3D point cloud data is incomplete when the number of points constituting the 3D point cloud data is less than a predetermined number.

5. A calculation device according to claim 1, The detection unit is a calculation device that detects when an object is detected between the moving object and a moving object detection device that outputs the three-dimensional point cloud data by detecting the moving object from the outside, in the three-dimensional point cloud data, and that the three-dimensional point cloud data is missing.

6. A calculation device according to claim 1, The acquisition unit acquires multiple 3D point cloud data for the same moving object detected at different timings. The calculation unit performs the first calculation process for each of the plurality of three-dimensional point cloud data to calculate at least one of the position and orientation of the moving object at different timings, and arranges them in chronological order to generate chronological data of at least one of the position and orientation of the moving object. The detection unit uses the time-series data to detect that at least one of the plurality of three-dimensional point cloud data is missing. If the detection unit detects that at least one of the plurality of three-dimensional point cloud data is missing, The calculation unit is a calculation device that calculates at least one of the position and orientation of the moving body by performing a second calculation process without performing the first calculation process.

7. A calculation device according to claim 1, In the case of at least one of the above cases 1, 2, and 3, The calculation unit performs the first calculation process and the second calculation process, respectively. The calculation unit calculates the position of the moving body by performing a predetermined calculation process using the position of the moving body calculated by performing the first calculation process and the position of the moving body calculated by performing the second calculation process. The calculation unit is a calculation device that calculates the orientation of the moving body by performing a predetermined calculation process using the orientation of the moving body calculated by performing the first calculation process and the orientation of the moving body calculated by performing the second calculation process.

8. A calculation system, One or more mobile units capable of moving by unmanned operation, A moving object detection device that detects the moving object from the outside and outputs three-dimensional point cloud data representing the moving object as a point cloud, The acquisition unit acquires the aforementioned three-dimensional point cloud data, A calculation unit that uses the three-dimensional point cloud data to calculate at least one of the position and orientation of the moving object, The system comprises at least one functional unit, which includes: (i) a prediction unit that predicts that the 3D point cloud data is missing; (ii) a detection unit that detects that the 3D point cloud data is missing; and (iii) a determination unit that determines whether or not the moving object is located within a specific area where it is assumed that the 3D point cloud data is missing. The calculation unit is capable of performing a first calculation process that calculates at least one of the position and orientation of the moving body by comparing the three-dimensional point cloud data with pre-prepared reference point cloud data, and a second calculation process that calculates at least one of the position and orientation of the moving body by applying geometric data of a predetermined shape to the three-dimensional point cloud data. The calculation system includes a calculation unit which performs at least the second calculation process when at least one of the following cases is met: a first case in which the prediction unit predicts that the three-dimensional point cloud data is missing; a second case in which the detection unit detects that the three-dimensional point cloud data is missing; or a third case in which the determination unit determines that the moving object is present within the specified area.

9. The calculation method, The acquisition process involves obtaining 3D point cloud data representing a mobile object that can be moved by unmanned operation, and A calculation step of calculating at least one of the position and orientation of the moving object using the three-dimensional point cloud data, The system comprises at least one functional step, which includes: (i) a prediction step of predicting that the three-dimensional point cloud data is missing; (ii) a detection step of detecting that the three-dimensional point cloud data is missing; and (iii) a determination step of determining whether or not the moving object is located within a specific area where it is assumed in advance that the three-dimensional point cloud data is missing. The calculation process can perform a first calculation process that calculates at least one of the position and orientation of the moving body by comparing the three-dimensional point cloud data with pre-prepared reference point cloud data, and a second calculation process that calculates at least one of the position and orientation of the moving body by applying geometric data of a predetermined shape to the three-dimensional point cloud data. A calculation method in which, in the calculation step, at least the second calculation process is performed if at least one of the following conditions is met: a first case in which the prediction step predicts that the three-dimensional point cloud data is missing; a second case in which the detection step detects that the three-dimensional point cloud data is missing; or a third case in which the determination step determines that the moving object exists within the specific area.

10. A calculation device, An acquisition unit that acquires 3D point cloud data representing a mobile object that can be moved by unmanned operation, A calculation unit that uses the three-dimensional point cloud data to calculate at least one of the position and orientation of the moving object and outputs vehicle position information including at least one of the position and orientation of the moving object, The system includes a detection unit that detects when the three-dimensional point cloud data is missing, The calculation unit is capable of performing a first calculation process that calculates at least one of the position and orientation of the moving body by comparing the three-dimensional point cloud data with pre-prepared reference point cloud data, and a second calculation process that calculates at least one of the position and orientation of the moving body by applying geometric data of a predetermined shape to the three-dimensional point cloud data. The calculation unit calculates at least one of the position and orientation of the moving body by performing the first calculation process and the second calculation process, The detection unit uses the first calculation result of at least one of the position and orientation of the moving body calculated by executing the first calculation process, and the second calculation result of at least one of the position and orientation of the moving body calculated by executing the second calculation process, to detect that the 3D point cloud data is missing. If the detection unit detects that the three-dimensional point cloud data is missing, The calculation unit selects, using the first calculation result and the second calculation result, which of the first calculation process and the second calculation process to execute to output the calculated result as the vehicle position information, and outputs the selected calculation result as the vehicle position information.

11. A calculation device according to claim 10, The acquisition unit acquires multiple 3D point cloud data for the same moving object detected at different timings. The calculation unit calculates at least one of the position and orientation of the moving object at different timings by performing the first calculation process and the second calculation process for each of the plurality of three-dimensional point cloud data, The calculation unit generates first time-series data of at least one of the position and orientation of the moving body by arranging a plurality of first calculation results at different timings in chronological order. The calculation unit generates second time-series data of at least one of the position and orientation of the moving body by arranging a plurality of the second calculation results at different timings in chronological order. The detection unit uses the first time series data and the second time series data to detect that at least one of the plurality of three-dimensional point cloud data is missing. If the detection unit detects that at least one of the plurality of three-dimensional point cloud data is missing, The calculation unit is a calculation device that uses the first time series data and the second time series data to select whether to output the calculation result obtained by executing the first calculation process or the second calculation process as the vehicle position information.

12. A calculation system, One or more mobile units capable of moving by unmanned operation, A moving object detection device that detects the moving object from the outside and outputs three-dimensional point cloud data representing the moving object as a point cloud, The acquisition unit acquires the aforementioned three-dimensional point cloud data, A calculation unit that uses the three-dimensional point cloud data to calculate at least one of the position and orientation of the moving object and outputs vehicle position information including at least one of the position and orientation of the moving object, The system includes a detection unit that detects when the three-dimensional point cloud data is missing, The calculation unit is capable of performing a first calculation process that calculates at least one of the position and orientation of the moving body by comparing the three-dimensional point cloud data with pre-prepared reference point cloud data, and a second calculation process that calculates at least one of the position and orientation of the moving body by applying geometric data of a predetermined shape to the three-dimensional point cloud data. The calculation unit calculates at least one of the position and orientation of the moving body by performing the first calculation process and the second calculation process, The detection unit uses the first calculation result of at least one of the position and orientation of the moving body calculated by executing the first calculation process, and the second calculation result of at least one of the position and orientation of the moving body calculated by executing the second calculation process, to detect that the 3D point cloud data is missing. If the detection unit detects that the three-dimensional point cloud data is missing, The calculation unit uses the first calculation result and the second calculation result to select which of the first calculation process or the second calculation process to execute to output the calculated result as the vehicle position information, and outputs the selected calculation result as the vehicle position information.

13. The calculation method, The acquisition process involves obtaining 3D point cloud data representing a mobile object that can be moved by unmanned operation, and A first calculation step of calculating at least one of the position and orientation of the moving object using the three-dimensional point cloud data, A detection step for detecting that the aforementioned 3D point cloud data is missing, The system includes a second calculation step that outputs vehicle position information including at least one of the position and orientation of the moving body, In the first and second calculation steps, it is possible to perform a first calculation process that calculates at least one of the position and orientation of the moving body by comparing the three-dimensional point cloud data with pre-prepared reference point cloud data, and a second calculation process that calculates at least one of the position and orientation of the moving body by applying geometric data of a predetermined shape to the three-dimensional point cloud data. In the first calculation step, the first calculation process and the second calculation process are performed to calculate at least one of the position and orientation of the moving body, In the detection step, the absence of the three-dimensional point cloud data is detected using the first calculation result of at least one of the position and orientation of the moving body calculated by executing the first calculation process, and the second calculation result of at least one of the position and orientation of the moving body calculated by executing the second calculation process. If the detection step detects that the three-dimensional point cloud data is missing, The calculation method comprising the second calculation step, which selects, using the first calculation result and the second calculation result, to output as the vehicle position information, and then outputs the selected calculation result as the vehicle position information.

Images