Apparatus, system, and method

Abstract

To prevent a situation where the position or the direction of a movable body cannot be correctly estimated due to the occurrence of a defect in measured appearance information.SOLUTION: An apparatus comprises: an appearance information acquisition unit that acquires measured appearance information on the appearance of a movable body from a detector that measures the appearance of the movable body; a movement information acquisition unit that acquires movement information on the movement of the movable body; a determination unit that determines whether a defect occurs in the measured appearance information; a complementation unit that, when the determination unit determines the occurrence of the defect, complements the defect in the measured appearance information by using the movement information; and an estimation unit that estimates at least one of the position and the direction of the movable body by comparing the measured appearance information with reference appearance information on the appearance of the movable body, and when the determination unit determines the occurrence of the defect, compares the measured appearance information complemented by the complementation unit with the reference appearance information.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present disclosure relates to an apparatus, system, and method for estimating the position and / or orientation of a moving object. [Background technology]

[0002] BACKGROUND ART In a vehicle manufacturing process, a technique for running a vehicle in an unmanned manner is known (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2017-538619 Summary of the Invention [Problem to be solved by the invention]

[0004] When a moving object such as a vehicle is driven autonomously, a process is executed to estimate the position and orientation of the moving object. The position and orientation of the moving object can be estimated by comparing measurement information acquired using a detector such as a camera or LiDAR (Light Detection and Ranging) with reference information prepared in advance. However, if there is a defect in the measurement information, an accurate estimation result may not be obtained if the estimation process is executed using the measurement information as is. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to a first aspect of the present disclosure, there is provided an apparatus comprising: an appearance information acquisition unit that acquires measured appearance information regarding an appearance of a moving object from a detector that measures the appearance of the moving object; a movement information acquisition unit that acquires movement information regarding the movement of the moving object; a determination unit that determines whether the measured appearance information has a defect; a completion unit that, when the determination unit determines that the defect has occurred, completes the defect in the measured appearance information using the movement information; and an estimation unit that estimates at least one of a position and an orientation of the moving object by comparing the measured appearance information with reference appearance information regarding the appearance of the moving object, and, when the determination unit determines that the defect has occurred, the estimation unit compares the measured appearance information completed by the completion unit with the reference appearance information. According to the device of this aspect, it is possible to compensate for the loss of the measured appearance information, and therefore it is possible to prevent the occurrence of a situation where the position and orientation of the moving object cannot be accurately estimated due to the loss of the measured appearance information. (2) In the device of the above aspect, the determining unit may determine that the defect occurs in the measured appearance information when the defect is detected from the measured appearance information. According to the device of this aspect, when a defect is detected in the measured appearance information, the defect in the measured appearance information can be complemented. (3) In the device of the above aspect, the determining unit may determine that the defect has occurred in the measurement appearance information when it is predicted that the defect will occur in the measurement appearance information. According to the device of this aspect, when it is predicted that there will be a loss in the measured appearance information, the loss in the measured appearance information can be compensated for. (4) In the device of the above form, the judgment unit may predict that the defect will occur in the measurement appearance information when the moving body is located at a predetermined location or when the moving body passes through the location. According to the device of this embodiment, it is possible to easily predict the occurrence of defects in the measured appearance information. (5) In the device of the above aspect, the location may be a location where a predetermined number of workers are present. According to the device of this form, it is possible to prevent the inability to obtain accurate estimation results for the position and orientation of a moving object in places where blind spots are likely to be created in the detector by workers. (6) In the device of the above aspect, the location may be a location where a worker gets on the moving body to perform work. According to the device of this form, it is possible to prevent the inability to obtain accurate estimation results for the position and orientation of a moving object in places where blind spots are likely to be created in the detector by workers. (7) The device of the above aspect may further include a control unit that slows down or stops the moving body when the completion of the defect in the measured appearance information has been repeated a predetermined number of times. According to the device of this aspect, it is possible to prevent the estimation accuracy from decreasing due to repeated interpolation. (8) In the device of the above aspect, the control unit may determine the number of times depending on a moving state of the moving object. According to the device of this aspect, it is possible to prevent the moving state of the moving body from becoming unstable. (9) In the device of the above aspect, the control unit may determine the number of times depending on the size of the defect. According to the device of this aspect, the estimation accuracy is more likely to decrease as the loss in the measured appearance information increases, so by determining the number of times according to the size of the loss, it is possible to prevent the estimation accuracy from decreasing significantly. (10) In the device of the above aspect, the control unit may determine the number of times depending on the distance between the detector and the moving object. According to this type of device, the estimation accuracy is more likely to decrease as the distance between the detector and the moving body increases, so by determining the above number of times according to the distance between the detector and the moving body, it is possible to prevent the estimation accuracy from decreasing significantly. (11) According to a second aspect of the present disclosure, there is provided a system comprising: a detector that measures an appearance of a moving object; an appearance information acquisition unit that acquires measured appearance information related to the appearance of the moving object from the detector; a movement information acquisition unit that acquires movement information related to the movement of the moving object; a determination unit that determines whether the measured appearance information has a defect; a completion unit that, when the determination unit determines that the defect has occurred, complements the defect in the measured appearance information using the movement information; and an estimation unit that estimates at least one of a position and an orientation of the moving object by comparing the measured appearance information with reference appearance information related to the appearance of the moving object, and, when the determination unit determines that the defect has occurred, compares the measured appearance information complemented by the completion unit with the reference appearance information. According to this embodiment, the system can compensate for the loss of the measured appearance information, thereby preventing the inability to obtain accurate estimation results for the position and orientation of the moving object due to the loss of the measured appearance information. (12) According to a third aspect of the present disclosure, there is provided a method comprising: an appearance information acquisition step of acquiring measured appearance information regarding the appearance of a moving body by measuring the appearance of the moving body; a movement information acquisition step of acquiring movement information regarding the movement of the moving body; a determination step of determining whether the measured appearance information has a defect; a complementation step of complementing the defect in the measured appearance information using the movement information if it is determined in the determination step that the defect has occurred; and an estimation step of estimating at least one of a position and an orientation of the moving body by comparing the measured appearance information with reference appearance information regarding the appearance of the moving body, wherein if it is determined in the determination step that the defect has occurred, an estimation step of comparing the measured appearance information complemented in the complementation step with the reference appearance information. According to the method of this aspect, it is possible to compensate for the loss of the measured appearance information, and therefore it is possible to prevent the position and orientation of the moving object from being incorrectly estimated due to the loss of the measured appearance information. The present disclosure may be realized in various forms other than an apparatus and a method, such as a system, a computer program, or a recording medium on which a computer program is recorded. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is an explanatory diagram showing the configuration of a system according to a first embodiment. [Figure 2] FIG. 1 is an explanatory diagram showing the configuration of a vehicle according to a first embodiment. [Figure 3] FIG. 2 is an explanatory diagram showing the configuration of a server device according to the first embodiment. [Figure 4] FIG. 1 is an explanatory diagram showing a state in which a vehicle moves by remote control in a factory. [Figure 5] 3 is a flowchart showing a processing procedure for vehicle travel control in the first embodiment. [Figure 6] FIG. 10 is an explanatory diagram showing a loss of measurement appearance information. [Figure 7] FIG. 4 is an explanatory diagram showing the contents of movement information. [Figure 8] FIG. 10 is an explanatory diagram showing a method for detecting a loss of measurement appearance information. [Figure 9] FIG. 10 is an explanatory diagram showing a method for complementing missing measurement appearance information. [Figure 10] FIG. 10 is an explanatory diagram showing the configuration of a system according to a second embodiment. [Figure 11] FIG. 10 is an explanatory diagram showing the configuration of a vehicle according to a second embodiment. [Figure 12] 10 is a flowchart showing a processing procedure for vehicle travel control according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] A. First embodiment: FIG. 1 is an explanatory diagram showing the configuration of a system 10 in a first embodiment. In this embodiment, the system 10 is used to move a mobile object in an unmanned manner in a factory that manufactures the mobile object. In this embodiment, the system 10 includes a vehicle 100, a server device 200, at least one external sensor 300, and a process control device 400. In this embodiment, the vehicle 100 corresponds to the "mobile object" in the present disclosure, the server device 200 corresponds to the "device" in the present disclosure, and the external sensor 300 corresponds to the "detector" in the present disclosure.

[0009] In this disclosure, a "mobile body" refers to an object that can move, such as a vehicle or an electric vertical take-off and landing aircraft (a so-called flying car). A vehicle may be a vehicle that runs on wheels or a vehicle that runs on tracks, such as a passenger car, truck, bus, motorcycle, automobile, tank, or construction vehicle. Vehicles include electric vehicles (BEVs: Battery Electric Vehicles), gasoline-powered vehicles, hybrid vehicles, and fuel cell vehicles. When a mobile body is something other than a vehicle, the terms "vehicle" and "car" in this disclosure may be replaced with "mobile body" as appropriate, and the term "traveling" may be replaced with "moving" as appropriate.

[0010] In this disclosure, "unmanned driving" means driving without the driver's control. Driving control means at least one of driving, turning, and stopping of the vehicle 100. Unmanned driving is achieved by automatic or manual remote control using a device located outside the vehicle 100, or by autonomous control of the vehicle 100. A vehicle 100 that is driving in an unmanned manner may have a driver on board who does not control the driving. A driver who does not control the driving includes, for example, a person who simply sits in a seat of the vehicle 100, or a person who is riding in the vehicle 100 and performing work other than driving operations, such as assembly, inspection, or operating switches. Note that driving with a driver controlling the driving is sometimes called "manned driving."

[0011] In this disclosure, "remote control" includes "full remote control" in which all of the operations of vehicle 100 are completely determined from outside vehicle 100, and "partial remote control" in which some of the operations of vehicle 100 are determined from outside vehicle 100. Furthermore, "autonomous control" includes "full autonomous control" in which vehicle 100 autonomously controls its own operations without receiving any information from devices external to vehicle 100, and "partial autonomous control" in which vehicle 100 autonomously controls its own operations using information received from devices external to vehicle 100.

[0012] 2 is an explanatory diagram showing the configuration of vehicle 100. In this embodiment, vehicle 100 is an electric vehicle that is configured to be able to run under remote control. Vehicle 100 includes a vehicle control device 110 for controlling each part of vehicle 100, a group of actuators 120 that are driven under the control of vehicle control device 110, a communication device 130 for communicating with server device 200 via wireless communication, and a group of internal sensors 140.

[0013] The actuator group 120 includes at least one actuator. In this embodiment, the actuator group 120 includes an actuator for a drive device that generates a propulsive force for the vehicle 100, an actuator for a steering device that changes the traveling direction of the vehicle 100, and an actuator for a braking device that generates a braking force for the vehicle 100. In this embodiment, the drive device includes a battery, a traction motor that is driven by power from the battery, and wheels that are rotated by the traction motor. The actuator for the drive device includes the traction motor.

[0014] The internal sensor group 140 includes at least one internal sensor. The internal sensor is a sensor mounted on the vehicle 100. In this embodiment, the internal sensor group 140 includes, as the internal sensors, a vehicle speed sensor for detecting the speed of the vehicle 100 and a steering angle sensor for detecting the steering angle of the vehicle 100.

[0015] The vehicle control device 110 is configured by a computer including a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input / output interface 113 are connected to each other via the internal bus 114 so as to be able to communicate bidirectionally. The input / output interface 113 is connected to an actuator group 120, a communication device 130, and an internal sensor group 140.

[0016] The processor 111 executes a computer program PG1 stored in advance in the memory 112, thereby functioning as a traveling control unit 196 and a movement information transmission unit 197.

[0017] The driving control unit 196 controls the actuator group 120. When a passenger is on board the vehicle 100, the driving control unit 196 controls the actuator group 120 in accordance with the operation of the passenger, thereby causing the vehicle 100 to drive. Regardless of whether a passenger is on board the vehicle 100 or not, the driving control unit 196 controls the actuator group 120 using a driving control signal received from the server device 200, thereby causing the vehicle 100 to drive. In this embodiment, the driving control signal includes the acceleration and steering angle of the vehicle 100 as parameters. Note that in other embodiments, the driving control signal may include the speed of the vehicle 100 as a parameter instead of or in addition to the acceleration of the vehicle 100.

[0018] The movement information transmission unit 197 acquires measurement results from the internal sensor group 140 and transmits the measurement results of the internal sensor group 140 to the server device 200. In this embodiment, the movement information transmission unit 197 repeatedly acquires and transmits the measurement results at a predetermined cycle. The measurement results of the internal sensor group 140 include the speed of the vehicle 100 measured by a vehicle speed sensor included in the internal sensor group 140, the steering angle of the vehicle 100 measured by a steering angle sensor included in the internal sensor group 140, and the times at which these were measured.

[0019] 3 is an explanatory diagram showing the configuration of the server device 200. The server device 200 is configured by a computer including a processor 201, a memory 202, an input / output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input / output interface 203 are connected via the internal bus 204 to enable bidirectional communication. A communication device 205 is connected to the input / output interface 203 for communicating with the vehicle 100 via wireless communication. In this embodiment, the communication device 205 can further communicate with an external sensor 300 and a process control device 400 via wired communication or wireless communication.

[0020] The processor 201 executes a computer program PG2 pre-stored in the memory 202, thereby functioning as a measurement appearance information acquisition unit 211, a movement information acquisition unit 212, a loss determination unit 213, a complementation unit 214, an estimation unit 215, and a remote control unit 216. In this embodiment, the remote control unit 216 corresponds to the "control unit" of the present disclosure.

[0021] The measured appearance information acquisition unit 211 acquires measured appearance information MG from the external sensor 300. The measured appearance information MG is information relating to the appearance of the vehicle 100, and is information obtained by measuring the vehicle 100. In this embodiment, the external sensor 300 is a LiDAR, and the measured appearance information is three-dimensional point cloud data.

[0022] The movement information acquisition unit 212 acquires the measurement results of the internal sensor group 140 from the vehicle 100 and records the measurement results of the internal sensor group 140 in the memory 202. In this embodiment, the movement information acquisition unit 212 records the measurement results of the internal sensor group 140, which are repeatedly transmitted from the vehicle 100 at a predetermined cycle, in chronological order in the memory 202. In the following description, information relating to the movement of the vehicle 100, such as the speed and steering angle of the vehicle 100, is referred to as movement information MV.

[0023] The loss determination unit 213 determines whether or not a loss occurs in the measured appearance information MG acquired by the measured appearance information acquisition unit 211. A loss occurring in the measured appearance information MG means that a part of the appearance of the vehicle 100 represented in the measured appearance information MG is missing.

[0024] When the loss determination unit 213 determines that a loss has occurred in the measured appearance information MG, the complementing unit 214 complements the loss in the measured appearance information MG using the movement information MV. In the present disclosure, complementing a loss in the measured appearance information MG includes not only complementing the entire loss in the measured appearance information MG, but also complementing a portion of the loss in the measured appearance information MG. In other words, in the present disclosure, complementing a loss in the measured appearance information MG means complementing at least a portion of the loss in the measured appearance information MG.

[0025] The estimation unit 215 estimates at least one of the position and orientation of the vehicle 100 by comparing the measured appearance information MG with reference appearance information RG pre-stored in the memory 202. The reference appearance information RG is information related to the appearance of the vehicle 100. In this embodiment, the reference appearance information RG is three-dimensional point cloud data. The reference appearance information RG can be generated using, for example, CAD data representing the appearance of the vehicle 100. The estimation unit 215 estimates both the position and orientation of the vehicle 100. If the loss determination unit 213 does not determine that a loss will occur in the measured appearance information MG, the estimation unit 215 uses the measured appearance information MG that has not been complemented by the complement unit 214 to compare with the reference appearance information RG. If the loss determination unit 213 determines that a loss will occur in the measured appearance information MG, the estimation unit 215 uses the measured appearance information MG that has been complemented by the complement unit 214 to compare with the reference appearance information RG.

[0026] The remote control unit 216 executes remote control of the vehicle 100. The remote control unit 216 uses the estimation result of the estimation unit 215 to generate a driving control signal for driving the vehicle 100 by remote control. The remote control unit 216 transmits the driving control signal to the vehicle 100.

[0027] As shown in FIG. 1 , the external sensor 300 is a sensor located outside the vehicle 100. The external sensor 300 measures the appearance of the vehicle 100 and generates measured appearance information including the appearance of the vehicle 100. In this embodiment, the external sensor 300 is a LiDAR, and the measured appearance information includes three-dimensional point cloud data and the measurement time of the three-dimensional point cloud data. The external sensor 300 includes a communication device (not shown) and can communicate with the server device 200 via wired or wireless communication. The external sensor 300 transmits the measured appearance information to the server device 200 via the communication device. In this embodiment, the external sensor 300 repeatedly generates and transmits the measured appearance information at a predetermined cycle.

[0028] The process control device 400 manages the overall manufacturing process of the vehicle 100 in the factory FC. The process control device 400 is configured with at least one computer. The process control device 400 has a database in which process information related to the manufacturing process of the vehicle 100 is recorded. The process information includes, for example, identification information of the vehicle 100, the details of the manufacturing process, and information related to the progress of the manufacturing process. The process control device 400 is equipped with a communication device (not shown) and can communicate with the server device 200 via wired or wireless communication.

[0029] FIG. 4 is an explanatory diagram showing how the vehicle 100 moves by remote control in a factory FC. In this embodiment, the vehicle 100 is remotely controlled in the factory FC where the vehicle 100 is manufactured. The factory FC includes a first location PL1 and a second location PL2. The first location PL1 is, for example, a location where the vehicle 100 is assembled, and the second location PL2 is, for example, a location where the vehicle 100 is inspected. The vehicle 100 assembled in the first location PL1 is remotely controlled so that it can be driven. The first location PL1 and the second location PL2 are connected by a track TR along which the vehicle 100 can travel. A plurality of external sensors 300 are installed around the track TR. The server device 200 can estimate the position and orientation of the vehicle 100 using the measurement results of the external sensors 300. The position and orientation of the vehicle 100 in the factory FC can be expressed using X, Y, and Z coordinates in a global coordinate system GC. Based on the estimated position and orientation of the vehicle 100, the server device 200 can generate a driving control signal for driving the vehicle 100 and transmit the driving control signal to the vehicle 100. The vehicle 100 can drive in accordance with the received driving control signal. Therefore, the system 10 can remotely move the vehicle 100 from the first location PL1 to the second location PL2 without using a transport device such as a crane or conveyor. The vehicle 100 that passes the inspection at the second location PL2 is then shipped from the factory FC.

[0030] Fig. 5 is a flowchart showing the processing procedure for driving control of the vehicle 100. Fig. 6 is an explanatory diagram showing a missing KS in the measured appearance information MG. Fig. 7 is an explanatory diagram showing the contents of the movement information MV. Fig. 8 is an explanatory diagram showing a method for detecting a missing KS in the measured appearance information MG. Fig. 9 is an explanatory diagram showing a method for complementing a missing KS in the measured appearance information MG.

[0031] 5 are repeatedly executed by the processor 201 of the server device 200. Steps S180 to S190 are repeatedly executed by the processor 111 of the vehicle control device 110. In step S110, the measured appearance information acquisition unit 211 acquires measured appearance information MG from the external sensor 300. In this embodiment, the measured appearance information MG is three-dimensional point cloud data. The measured appearance information MG includes a point cloud of the vehicle 100 to be controlled and a point cloud other than the vehicle 100. The point cloud other than the vehicle 100 is, for example, a point cloud of the road surface of the travel path TR, a point cloud of various pieces of equipment in the factory FC, a point cloud of workers in the factory FC, etc.

[0032] In step S120, the loss determination unit 213 determines whether or not a loss KS has occurred in the measured appearance information MG. If it is determined in step S120 that a loss KS has occurred in the measured appearance information MG, the loss determination unit 213 proceeds to step S130. If it is not determined in step S120 that a loss KS has occurred in the measured appearance information MG, the loss determination unit 213 skips step S130 and proceeds to step S140. As shown in FIG. 6 , if an obstacle OB such as a person or object is present between the external sensor 300 and the vehicle 100, the obstacle OB creates a blind spot for the external sensor 300, causing a loss KS in the measured appearance information MG. In this embodiment, the loss determination unit 213 determines whether or not a loss KS has occurred in the measured appearance information MG using at least one of the following determination methods A1 to A4.

[0033] <Judgment method A1> In determination method A1, the defect determination unit 213 determines whether a defect KS has occurred in the measured appearance information MG using process information acquired from the process control device 400. In a manufacturing process in which a predetermined number or more workers are present around the vehicle 100, there is a high possibility that a worker will get between the external sensor 300 and the vehicle 100, creating a blind spot for the external sensor 300. Therefore, the defect determination unit 213 determines that a defect KS has occurred in the measured appearance information MG when the current manufacturing process of the vehicle 100 is a manufacturing process in which a predetermined number or more workers are present around the vehicle 100. In this embodiment, the process information includes information about the current manufacturing process of the vehicle 100 and information about the number of workers present around the vehicle 100 in the current manufacturing process. The defect determination unit 213 obtains the number of workers present around the vehicle 100 using the process information acquired from the process control device 400, and determines that a defect KS has occurred in the measured appearance information MG when the number of workers present around the vehicle 100 is equal to or greater than the predetermined number. In addition, the defect determination unit 213 may obtain process information in advance from the process management device 400, including information regarding the location where the manufacturing process is performed on the vehicle 100 and information regarding the number of workers present around the vehicle 100 at the location where the manufacturing process is performed on the vehicle 100, and determine that a defect KS has occurred in the measurement appearance information MG when the vehicle 100 passes through a location where the manufacturing process is performed and there are more than a predetermined number of workers present around the vehicle 100.

[0034] <Judgment method A2> In determination method A2, the defect determination unit 213 determines whether a defect KS has occurred in the measured appearance information MG using process information acquired from the process control device 400. In a manufacturing process in which a worker gets into the vehicle 100 to perform work, there is a high possibility that a blind spot will be created for the external sensor 300 due to the worker getting into the vehicle 100. Therefore, the defect determination unit 213 determines that a defect KS has occurred in the measured appearance information MG when the current manufacturing process of the vehicle 100 is a manufacturing process in which a worker gets into the vehicle 100 to perform work. In this embodiment, the process information includes information about the current manufacturing process of the vehicle 100 and information about whether a worker gets into the vehicle 100 to perform work in the current manufacturing process. The defect determination unit 213 determines that a defect KS has occurred in the measured appearance information MG when the current manufacturing process of the vehicle 100 indicated in the process information acquired from the process control device 400 is a manufacturing process in which a worker gets into the vehicle 100 to perform work. In addition, the defect determination unit 213 may obtain process information in advance from the process management device 400, including information regarding the location where a manufacturing process is carried out in which workers get into the vehicle 100 to perform work, and determine that a defect KS has occurred in the measurement appearance information MG when the vehicle 100 passes through the location where a manufacturing process is carried out in which workers get into the vehicle 100 to perform work.

[0035] <Judgment method A3> In determination method A3, the defect determination unit 213 determines that a defect KS has occurred in the measured appearance information MG when it predicts that a defect KS will occur in the measured appearance information MG. For example, a camera is installed at a location where each manufacturing process of the vehicle 100 is performed, and the defect determination unit 213 analyzes images acquired from the camera to obtain the number of workers present around the vehicle 100 in the current manufacturing process, and predicts that a defect KS will occur in the measured appearance information MG if the number of workers present around the vehicle 100 is equal to or greater than a predetermined number. Alternatively, the defect determination unit 213 may analyze images acquired from the camera to determine whether a worker is in the vehicle 100, and predict that a defect KS will occur in the measured appearance information MG if the manufacturing process involves a worker in the vehicle 100 performing work.

[0036] <Judgment method A4> In determination method A4, when the defect determination unit 213 detects that a defect KS has occurred in the measured appearance information MG, the defect determination unit 213 determines that a defect KS has occurred in the measured appearance information MG. The defect determination unit 213 detects that a defect KS has occurred in the measured appearance information MG using the measured appearance information MG1 used in the current matching, the measured appearance information MG2 used in the Nth matching (N is a natural number), and the movement information MV stored in memory 202. Here, the measured appearance information MG1 used in the current matching is the measured appearance information MG measured by the external sensor 300 at time T1, and the measured appearance information MG2 used in the Nth matching is the measured appearance information MG measured by the external sensor 300 at time T2, which is before time T1. As shown in FIG. 7, the movement information MV includes time-series data of the speed and steering angle of the vehicle 100 measured by the internal sensor group 140 between time T2 and time T1. As shown in FIG. 8, first, the defect determination unit 213 determines a bounding box BB that encloses the portion of the measured appearance information MG2 used in the Nth matching operation corresponding to the vehicle 100. The portion of the measured appearance information MG2 corresponding to the vehicle 100 is the portion that matched with the reference appearance information RG in the matching operation. Next, the defect determination unit 213 calculates the movement and rotation amount of the vehicle 100 between time T2 and time T1 using the movement information MV stored in the memory 202. The defect determination unit 213 moves the bounding box BB by the movement amount of the vehicle 100 between time T2 and time T1, and rotates the bounding box BB by the rotation amount of the vehicle 100 between time T2 and time T1. The defect determination unit 213 obtains the number of points in the portion surrounded by the bounding box BB in the measured appearance information MG1 used in the current matching operation. If a defect KS occurs in the measured appearance information MG1, the number of points in the portion surrounded by the bounding box BB will be smaller than if a defect KS does not occur. Therefore, the loss determining unit 213 detects that a loss KS has occurred in the measured appearance information MG1 when the number of points in the portion surrounded by the bounding box BB is equal to or less than a predetermined number.

[0037] In determination method A4, the loss determination unit 213 can calculate the movement amount of the vehicle 100 using the following formula (1): In formula (1), x represents the movement amount of the vehicle 100 in the X direction, y represents the movement amount of the vehicle 100 in the Y direction, t represents time, v represents the speed of the vehicle 100, and θ represents the steering angle of the vehicle 100. Note that formula (1) represents the movement amount of the vehicle 100 over two seconds.

number

[0038] As shown in FIG. 5, if it is determined in step S120 that a missing portion KS exists in the measured appearance information MG, in step S130, the complementing unit 214 complements the missing portion of the measured appearance information MG. In this embodiment, the complementing unit 214 complements the missing portion of the measured appearance information MG1 to be used in the current matching using the measured appearance information MG2 used in the matching N times ago and the movement information MV stored in the memory 202. Here, the measured appearance information MG1 to be used in the current matching is the measured appearance information MG measured by the external sensor 300 at time T1, and the measured appearance information MG2 used in the matching N times ago is the measured appearance information MG measured by the external sensor 300 at time T2, which is before time T1. It is preferable that the number N is small. It is preferable that the number N is 1. As shown in FIG. 7, the movement information MV includes time-series data of the speed and steering angle of the vehicle 100 measured by the internal sensor group 140 between time T2 and time T1. As shown in FIG. 9 , first, the complementing unit 214 calculates the movement amount and rotation amount of the vehicle 100 between time T2 and time T1 using the movement information MV stored in the memory 202. Next, the complementing unit 214 moves the measured appearance information MG2 by the movement amount of the vehicle 100 between time T2 and time T1, and rotates the measured appearance information MG2 by the rotation amount of the vehicle 100 between time T2 and time T1. By moving and rotating the measured appearance information MG2, the portion of the measured appearance information MG2 corresponding to the vehicle 100 overlaps with the missing portion of the measured appearance information MG1, thereby complementing the missing portion KS of the measured appearance information MG1. The complementing unit 214 may move and rotate the entire measured appearance information MG2, or may move and rotate only the portion of the measured appearance information MG2 corresponding to the vehicle 100, or may move and rotate only the portion of the measured appearance information MG2 corresponding to the missing portion of the measured appearance information MG1. When complementing the missing KS in the measurement appearance information MG1, the complementing unit 214 may move the measurement appearance information MG2 without rotating the measurement appearance information MG2.

[0039] In step S140, the estimation unit 215 estimates the position and orientation of the vehicle 100 by matching the measured appearance information MG with the reference appearance information RG. If the missing KS was not interpolated in step S130, the estimation unit 215 estimates the position and orientation of the vehicle 100 by matching the measured appearance information MG in which the missing KS was not interpolated with the reference appearance information RG. If the missing KS was interpolated in step S130, the estimation unit 215 estimates the position and orientation of the vehicle 100 by matching the measured appearance information MG in which the missing KS was interpolated with the reference appearance information RG. Examples of matching methods that can be used include the normal distribution transform (NDT) and iterative closest point (ICP). The estimation unit 215 can estimate the position and orientation of the vehicle 100 in the coordinate system of the external sensor 300 by matching. Since the position and orientation of the external sensor 300 are fixed and the positional relationship between the coordinate system of the external sensor 300 and the global coordinate system GC is known, the estimation unit 215 can convert the position and orientation of the vehicle 100 in the coordinate system of the external sensor 300 into the position and orientation of the vehicle 100 in the global coordinate system GC. Note that the process of acquiring the measured appearance information MG may be referred to as an appearance information acquisition process. The process of acquiring the movement information MV may be referred to as a movement information acquisition process. The process of determining whether or not there is a defect in the measured appearance information MG may be referred to as a determination process. The process of complementing the defect KS in the measured appearance information MG may be referred to as a complementation process. The process of estimating at least one of the position and orientation of the vehicle 100 may be referred to as an estimation process.

[0040] In step S150, the remote control unit 216 determines a target position to which the vehicle 100 should next head. In this embodiment, the target position is represented by X, Y, and Z coordinates in the global coordinate system GC. A reference route RR, which is the route the vehicle 100 should travel, is stored in advance in the memory 202 of the server device 200. The route is represented by nodes indicating the departure point, nodes indicating passing points, nodes indicating the destination, and links connecting the nodes. The remote control unit 216 uses the vehicle position information and the reference route RR to determine a target position to which the vehicle 100 should next head. The remote control unit 216 determines a target position on the reference route RR that is ahead of the current location of the vehicle 100.

[0041] In step S160, the remote control unit 216 generates a travel control signal for driving the vehicle 100 toward the determined target position. The remote control unit 216 calculates the travel speed of the vehicle 100 from the change in the position of the vehicle 100 and compares the calculated travel speed with the target speed. When the travel speed is lower than the target speed, the remote control unit 216 determines an acceleration rate so that the vehicle 100 accelerates. When the travel speed is higher than the target speed, the remote control unit 216 determines an acceleration rate so that the vehicle 100 decelerates. Furthermore, when the vehicle 100 is located on the reference route RR, the remote control unit 216 determines a steering angle and acceleration rate so that the vehicle 100 does not deviate from the reference route RR. When the vehicle 100 is not located on the reference route RR, in other words, when the vehicle 100 has deviated from the reference route RR, the remote control unit 216 determines a steering angle and acceleration rate so that the vehicle 100 returns to the reference route RR.

[0042] In step S170, remote control unit 216 transmits the generated driving control signal to vehicle 100. Server device 200 repeats, at a predetermined cycle, the acquisition of vehicle position information, determination of a target position, generation of a driving control signal, and transmission of the driving control signal.

[0043] In step 180, the driving control unit 196 receives the driving control signal transmitted from the server device 200. In step 190, the driving control unit 196 controls the actuator group 120 using the received driving control signal, thereby causing the vehicle 100 to drive at the acceleration and steering angle indicated in the driving control signal. The vehicle control device 110 repeats receiving the driving control signal and controlling the actuator group 120 at a predetermined cycle.

[0044] The movement and rotation amounts of the vehicle 100 calculated from the movement information MV, which is the measurement result of the internal sensor group 140 mounted on the vehicle 100, may deviate from the actual movement and rotation amounts of the vehicle 100. Therefore, when the measured appearance information MG2 used in the Nth previous matching is moved and rotated by the movement and rotation amounts calculated from the movement information MV to complement the missing KS in the measured appearance information MG1 used in the current matching, a deviation may occur between the measured appearance information MG1 and the measured appearance information MG2. Therefore, if the missing KS is complemented in the measured appearance information MG2 used to complement the missing KS in the measured appearance information MG1, the accuracy of estimating the position and orientation of the vehicle 100 may decrease. Thus, repeated complementation of the measured appearance information MG using the complemented measured appearance information MG may result in a significant decrease in the accuracy of estimating the position and orientation of the vehicle 100. Therefore, in this embodiment, the remote control unit 216 decelerates or stops the vehicle 100 when complementation of the missing KS in the measured appearance information MG is repeated a predetermined number of times. The number of times may be fixed in advance, or may be arbitrarily determined by the remote control unit 216. The remote control unit 216 can determine the number of times using, for example, at least one of the following number-determining methods B1 to B3.

[0045] <Number of times determination method B1> In count determination method B1, the remote control unit 216 determines the above count according to the movement state of the vehicle 100. Moving the vehicle 100 at a high speed when the estimation accuracy of the position and orientation of the vehicle 100 has decreased is undesirable because it increases the possibility that the vehicle 100 will come into contact with an obstacle or the like. Therefore, the remote control unit 216 may decrease the above count as the speed of the vehicle 100 increases. In this case, the possibility that the vehicle 100 will come into contact with an obstacle or the like can be reduced. Furthermore, when the vehicle 100 is turning, the behavior of the vehicle 100 is likely to become unstable. Therefore, when the vehicle 100 is turning, the remote control unit 216 may decrease the above count compared to when the vehicle 100 is traveling straight. In this case, it is possible to prevent the behavior of the vehicle 100 from becoming unstable.

[0046] <Number of times determination method B2> In count determination method B2, the remote control unit 216 determines the above count according to the size of the missing KS in the measured appearance information MG. The larger the missing KS in the measured appearance information MG, the lower the accuracy of estimating the position and orientation of the vehicle 100. Therefore, the remote control unit 216 may decrease the above count as the missing KS in the measured appearance information MG increases. In this case, it is possible to prevent a significant decrease in the accuracy of estimating the position and orientation of the vehicle 100.

[0047] <Number of times determination method B3> In count determination method B3, the remote control unit 216 determines the above count according to the distance between the external sensor 300 and the vehicle 100. The greater the distance between the external sensor 300 and the vehicle 100, the lower the accuracy of estimating the position and orientation of the vehicle 100. Therefore, the remote control unit 216 may decrease the above count as the distance between the external sensor 300 and the vehicle 100 increases. In this case, it is possible to prevent a significant decrease in the accuracy of estimating the position and orientation of the vehicle 100.

[0048] According to the system 10 of the present embodiment described above, even if a missing KS occurs in the measured appearance information MG acquired from the external sensor 300, the missing KS in the measured appearance information MG can be complemented. Therefore, it is possible to prevent a situation in which the position and orientation of the vehicle 100 cannot be accurately estimated due to the missing KS occurring in the measured appearance information MG.

[0049] B. Second embodiment: FIG. 10 is an explanatory diagram showing the configuration of a system 10b in the second embodiment. FIG. 11 is an explanatory diagram showing the configuration of a vehicle 100 in the second embodiment. As shown in FIG. 10, the second embodiment differs from the first embodiment in that the system 10b does not include a server device 200 and that the vehicle 100 is configured to be able to travel by autonomous control of the vehicle 100 rather than by remote control. Unless otherwise specified, the other configurations are the same as those in the first embodiment. Note that in this embodiment, the vehicle control device 110 corresponds to the "device" in the present disclosure, and the travel control unit 196 corresponds to the "control unit" in the present disclosure.

[0050] 11 , in this embodiment, the communication device 130 communicates with the external sensor 300 and the process control device 400 via wireless communication. A reference route RR and reference appearance information RG are pre-stored in the memory 112 of the vehicle control device 110. In this embodiment, the processor 111 of the vehicle control device 110 executes a computer program PG1 pre-stored in the memory 112, thereby functioning as a measured appearance information acquisition unit 191, a movement information acquisition unit 192, a defect determination unit 193, a complementation unit 194, an estimation unit 195, and a travel control unit 196.

[0051] The measured appearance information acquisition unit 191 acquires measured appearance information MG from the external sensor 300. The movement information acquisition unit 192 acquires the measurement results of the internal sensor group 140, in other words, movement information MV, and records the movement information MV in the memory 112. The defect determination unit 193 determines whether a defect occurs in the measured appearance information MG acquired by the measured appearance information acquisition unit 191. If it is determined that a defect KS occurs in the measured appearance information MG, the complement unit 194 complements the defect KS in the measured appearance information MG using the movement information MV. The estimation unit 195 estimates at least one of the position and orientation of the host vehicle 100 by comparing the measured appearance information MG with the reference appearance information RG. The estimation unit 195 estimates both the position and orientation of the host vehicle 100. If the defect determination unit 193 does not determine that a defect KS occurs in the measured appearance information MG, the estimation unit 195 uses the measured appearance information MG, in which the defect KS has not been complemented by the complement unit 194, to compare with the reference appearance information RG. When the defect determination unit 193 determines that a defect KS occurs in the measured appearance information MG, the estimation unit 195 uses the measured appearance information MG in which the defect KS has been complemented by the complementation unit 194 to compare with the reference appearance information RG. In this embodiment, the driving control unit 196 uses the estimation result of the estimation unit 195 to generate a driving control signal for driving the host vehicle 100. The driving control unit 196 controls the actuator group 120 using the driving control signal that it has generated.

[0052] 12 is a flowchart showing the processing procedure for driving control of the vehicle 100 in the second embodiment. Steps S210 to S270 are repeatedly executed by the processor 111 of the vehicle control device 110. In step S210, the measured appearance information acquisition unit 191 acquires measured appearance information MG from the external sensor 300.

[0053] In step S220, the defect determination unit 213 determines whether or not a defect KS occurs in the measurement appearance information MG. The defect determination unit 193 determines whether or not a defect KS occurs in the measurement appearance information MG using at least one of the determination methods A1 to A4 described above.

[0054] If it is determined in step S220 that a defect KS has occurred in the measurement appearance information MG, the defect determination unit 193 proceeds to step S230. If it is not determined in step S220 that a defect KS has occurred in the measurement appearance information MG, the defect determination unit 193 skips step S230 and proceeds to step S240.

[0055] In step S230, the complementing unit 194 complements the missing measurement appearance information MG. In this embodiment, the complementing unit 214 complements the missing measurement appearance information MG to be used in the current matching by using the measurement appearance information MG used in the Nth matching operation before and the movement information MV stored in the memory 112.

[0056] In step S240, the estimation unit 195 estimates the position and orientation of the host vehicle 100 by matching the measured appearance information MG with the reference appearance information RG. If the missing KS was not complemented in step S230, the estimation unit 195 estimates the position and orientation of the host vehicle 100 by matching the measured appearance information MG in which the missing KS has not been complemented with the reference appearance information RG. If the missing KS was complemented in step S230, the estimation unit 195 estimates the position and orientation of the host vehicle 100 by matching the measured appearance information MG in which the missing KS has been complemented with the reference appearance information RG.

[0057] In step S250, the driving control unit 196 determines a target position to which the host vehicle 100 should next head. In step S260, the driving control unit 196 generates a driving control signal for driving the host vehicle 100 toward the determined target position. In step S270, the driving control unit 196 controls the actuator group 120 using the generated driving control signal, thereby causing the host vehicle 100 to drive in accordance with parameters represented in the driving control signal. The processor 111 of the vehicle control device 110 repeats, at a predetermined cycle, acquisition of vehicle position information, determination of a target position, generation of a driving control signal, and control of the actuator group 120.

[0058] According to the system 10b of the present embodiment described above, the vehicle 100 can be driven by autonomous control of the vehicle 100 without remotely controlling the vehicle 100 using the server device 200. Furthermore, in the present embodiment, as in the first embodiment, even if a missing KS occurs in the measured appearance information MG acquired from the external sensor 300, the missing KS in the measured appearance information MG can be complemented. Therefore, it is possible to prevent a situation in which the position and orientation of the vehicle 100 cannot be accurately estimated due to the missing KS occurring in the measured appearance information MG.

[0059] C. Other Embodiments: (C1) In the first and second embodiments, the external sensor 300 is a LiDAR. However, the external sensor 300 may be a camera. In this case, the estimation units 215, 195 may estimate the position and orientation of the vehicle 100 by image matching using an image of the vehicle 100 output from a camera, rather than point cloud matching using three-dimensional point cloud data of the vehicle 100 output from the LiDAR.

[0060] (C2) In the first and second embodiments, the server device 200 executes the processes from acquiring the position information of the vehicle 100 to generating the driving control signal. However, at least a part of the processes from acquiring the position information of the vehicle 100 to generating the driving control signal may be executed by the vehicle 100. For example, the following forms (1) to (3) may be used.

[0061] (1) The server device 200 may acquire position information of the vehicle 100, determine a target position to which the vehicle 100 should next head, and generate a route from the current location of the vehicle 100 indicated in the acquired position information to the target position. The server device 200 may generate a route to a target position between the current location and the destination, or may generate a route to the destination. The server device 200 may transmit the generated route to the vehicle 100. The vehicle 100 may generate a driving control signal so that the vehicle 100 drives on the route received from the server device 200, and control the actuator group 120 using the generated driving control signal.

[0062] (2) The server device 200 may acquire location information of the vehicle 100 and transmit the acquired location information to the vehicle 100. The vehicle 100 may determine a target location to which the vehicle 100 should next travel, generate a route from the current location of the vehicle 100 indicated in the received location information to the target location, generate a travel control signal so that the vehicle 100 travels on the generated route, and control the actuator group 120 using the generated travel control signal. Note that in each of the above-described embodiments, the vehicle operation information may be a route from the current location of the vehicle 100 to the target location.

[0063] (3) In the above embodiments (1) and (2), the vehicle 100 may be equipped with an internal sensor, and detection results output from the internal sensor may be used for at least one of generating a route and generating a driving control signal. The internal sensor may include, for example, a camera, LiDAR, millimeter-wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, and a gyro sensor. For example, in the above embodiment (1), the server device 200 may acquire the detection results of the internal sensor and reflect the detection results of the internal sensor in the route when generating a route. In the above embodiment (1), the vehicle 100 may acquire the detection results of the internal sensor and reflect the detection results of the internal sensor in the driving control signal when generating a driving control signal. In the above embodiment (2), the vehicle 100 may acquire the detection results of the internal sensor and reflect the detection results of the internal sensor in the route when generating a route. In the above embodiment (2), the vehicle 100 may acquire the detection results of the internal sensor and reflect the detection results of the internal sensor in the route when generating a route.

[0064] (C3) In the first and second embodiments, the vehicle 100 may be equipped with an internal sensor, and the detection results output from the internal sensor may be used for at least one of generating a route and generating a driving control signal. For example, the vehicle 100 may acquire the detection results of the internal sensor, and when generating a route, may reflect the detection results of the internal sensor in the route. The vehicle 100 may acquire the detection results of the internal sensor, and when generating a driving control signal, may reflect the detection results of the internal sensor in the driving control signal.

[0065] (C4) In the second embodiment, the vehicle 100 acquires vehicle position information using the detection results of the external sensor 300. In contrast, the vehicle 100 may be equipped with an internal sensor, and the vehicle 100 may acquire vehicle position information using the detection results of the internal sensor, determine a target position to which the vehicle 100 should next head, generate a route from the current location of the vehicle 100 represented in the acquired vehicle position information to the target position, generate a driving control signal for traveling along the generated route, and control the actuator group 120 using the generated driving control signal. In this case, the vehicle 100 can travel without using any detection results of the external sensor 300. Note that the vehicle 100 may acquire a target arrival time and congestion information from outside the vehicle 100, and reflect the target arrival time and congestion information in at least one of the route and the driving control signal.

[0066] (C5) In the first and second embodiments, the server device 200 automatically generates the driving control signal to be transmitted to the vehicle 100. However, the server device 200 may generate the driving control signal to be transmitted to the vehicle 100 in accordance with the operation of an external operator located outside the vehicle 100. For example, the external operator may operate a control device including a display that displays an image output from a camera that is the external sensor 300, a steering wheel for remotely controlling the vehicle 100, an accelerator pedal, a brake pedal, and a communication device for communicating with the server device 200 via wired or wireless communication, and the server device 200 may generate the driving control signal in accordance with the operation applied to the control device.

[0067] (C6) In the first and second embodiments, the vehicle 100 may be configured to be able to travel by unmanned driving, and may be in the form of a platform having the configuration described below, for example. Specifically, the vehicle 100 may be equipped with at least a vehicle control device 110 and an actuator group 120 to perform the three functions of "running," "turning," and "stopping" by unmanned driving. When the vehicle 100 acquires information from the outside for unmanned driving, the vehicle 100 may further be equipped with a communication device 130. In other words, the vehicle 100 that can travel by unmanned driving may not be equipped with at least some of the interior parts such as a driver's seat and a dashboard, may not be equipped with at least some of the exterior parts such as a bumper and a fender, and may not be equipped with a body shell. In this case, the remaining parts such as the body shell may be attached to the vehicle 100 before the vehicle 100 is shipped from the factory FC, or the remaining parts such as the body shell may be attached to the vehicle 100 after the vehicle 100 is shipped from the factory FC without the remaining parts such as the body shell attached. Each component may be attached from any direction, such as the upper, lower, front, rear, right or left side of the vehicle 100, and may be attached from the same direction or from different directions. Note that the position of the platform configuration may also be determined in the same manner as for the vehicle 100 in the first embodiment.

[0068] (C7) The vehicle 100 may be manufactured by combining multiple modules. A module refers to a unit composed of multiple parts grouped according to the location or function of the vehicle 100. For example, the platform of the vehicle 100 may be manufactured by combining a front module that forms the front part of the platform, a central module that forms the center part of the platform, and a rear module that forms the rear part of the platform. The number of modules that form the platform is not limited to three, and may be two or less, or four or more. In addition to or instead of the parts that form the platform, parts that form parts of the vehicle 100 other than the platform may be modularized. The various modules may include any exterior parts such as a bumper or a grille, or any interior parts such as a seat or a console. In addition to the vehicle 100, any type of mobile object may be manufactured by combining multiple modules. Such a module may be manufactured, for example, by joining multiple parts using welding or fasteners, or by integrally molding at least some of the parts that form the module into a single part by casting. The molding method for integrally molding a single component, particularly a relatively large component, is also called gigacasting or megacasting. For example, the front module, center module, and rear module described above may be manufactured using gigacasting.

[0069] (C8) Transporting the vehicle 100 by using the unmanned driving of the vehicle 100 is also called "self-propelled transport." The configuration for realizing self-propelled transport is also called a "vehicle remote-controlled autonomous transport system." The production method for producing the vehicle 100 by using self-propelled transport is also called "self-propelled production." In self-propelled production, for example, at a factory FC where the vehicle 100 is manufactured, at least a portion of the transport of the vehicle 100 is realized by self-propelled transport.

[0070] (C9) In the first to third embodiments, some or all of the functions and processes implemented by software may be implemented by hardware. Furthermore, some or all of the functions and processes implemented by hardware may be implemented by software. Hardware for implementing the various functions in each of the above embodiments may be implemented by various circuits, such as integrated circuits and discrete circuits.

[0071] The present disclosure is not limited to the above-described embodiments and can be realized in various configurations without departing from the spirit thereof. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention section can be appropriately replaced or combined to solve some or all of the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0072] 10, 10b...system, 100...vehicle, 110...vehicle control device, 111...processor, 112...memory, 113...input / output interface, 114...internal bus, 120...actuator group, 130...communication device, 140...internal sensor group, 191...measured appearance information acquisition unit, 192...movement information acquisition unit, 193...loss determination unit, 194...complementation unit, 195...estimation unit, 196...travel control unit, 197...movement information transmission unit, 200...server device, 201...processor, 202...memory, 203...input / output interface, 204...internal bus, 205...communication device, 211...measured appearance information acquisition unit, 212...movement information acquisition unit, 213...loss determination unit, 214...complementation unit, 215...estimation unit, 216...remote control unit, 300...external sensor, 400...process management device

Claims

1. 1. An apparatus comprising: an appearance information acquisition unit that acquires measured appearance information relating to the appearance of the moving object from a detector that measures the appearance of the moving object; a movement information acquisition unit that acquires movement information related to the movement of the moving object; a determination unit that determines whether or not a defect occurs in the measurement appearance information; a complementing unit that complements the loss of the measurement appearance information using the movement information when the determining unit determines that the loss has occurred; an estimation unit that estimates at least one of a position and an orientation of the moving object by comparing reference appearance information regarding the appearance of the moving object with the measured appearance information, and when the determination unit determines that the defect has occurred, the estimation unit compares the measured appearance information complemented by the complement unit with the reference appearance information; An apparatus comprising:

2. 10. The apparatus of claim 1, The determination unit determines that the defect occurs in the measurement appearance information when the defect is detected from the measurement appearance information.

3. 10. The apparatus of claim 1, The determination unit determines that the defect has occurred in the measurement appearance information when it predicts that the defect will occur in the measurement appearance information.

4. 4. The apparatus of claim 3, The determination unit predicts that the defect will occur in the measured appearance information when the moving object is located at a predetermined location or when the moving object passes through the location.

5. 5. The apparatus of claim 4, The location is a location where a predetermined number of workers are present.

6. 5. The apparatus of claim 4, The location is a location where a worker gets on the mobile body to perform work.

7. 10. The apparatus of claim 1, The apparatus further comprises a control unit that slows down or stops the moving body when the completion of the defect in the measured appearance information has been repeated a predetermined number of times.

8. 8. The apparatus of claim 7, The control unit determines the number of times depending on a moving state of the moving object.

9. 8. The apparatus of claim 7, The control unit determines the number of times depending on the size of the defect.

10. 8. The apparatus of claim 7, The control unit determines the number of times depending on the distance between the detector and the moving object.

11. 1. A system comprising: a detector for measuring the appearance of a moving object; an appearance information acquisition unit that acquires measured appearance information regarding the appearance of the moving object from the detector; a movement information acquisition unit that acquires movement information related to the movement of the moving object; a determination unit that determines whether or not a defect occurs in the measurement appearance information; a complementing unit that complements the loss of the measurement appearance information using the movement information when the determining unit determines that the loss has occurred; an estimation unit that estimates at least one of a position and an orientation of the moving object by comparing reference appearance information regarding the appearance of the moving object with the measured appearance information, and when the determination unit determines that the defect has occurred, the estimation unit compares the measured appearance information complemented by the complement unit with the reference appearance information; A system comprising:

12. 1. A method comprising: an appearance information acquisition step of acquiring measured appearance information relating to the appearance of the moving object by measuring the appearance of the moving object; a movement information acquisition step of acquiring movement information relating to the movement of the moving object; a determination step of determining whether or not a defect occurs in the measurement appearance information; a complementation step of complementing the defect in the measurement appearance information using the movement information when it is determined that the defect has occurred in the determination step; an estimation step of estimating at least one of a position and an orientation of the moving object by comparing reference appearance information relating to the appearance of the moving object with the measured appearance information, and when it is determined in the determination step that the missing portion has occurred, comparing the measured appearance information complemented in the complement step with the reference appearance information; A method comprising:

Images