System and mobile body

Abstract

To appropriately control movement of a moving body by means of remote control or autonomous control.SOLUTION: A system comprises: an acquisition unit which acquires a moving-body situation indicating at least one of a state of a moving body movable by unmanned driving and acquires an environment around the moving body; an inference unit performing an inference process of inferring at least one of a position and a direction of the moving body by using sensor information acquired by means of a sensor; a control command generation unit generating and outputting a control command for controlling movement of the moving body by using the inference result of the inference process; and a determination unit determining which of a first process and a second process is performed as the inference process, according to the acquired moving-body situation. The first process is a process in which accuracy of an inference result is higher than that in the second process, and the second process is a process in which a processing time is shorter than that in the first process.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a system and a moving body.

Background Art

[0002] Patent Document 1 discloses a technique for driving a vehicle autonomously or by remote control in a vehicle manufacturing process.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to appropriately control the movement of a moving body such as a vehicle by remote control or autonomous control, it is desirable to accurately detect the position and orientation of the moving body. On the other hand, if the time spent on detecting the position and orientation increases, there may be a case where the movement of the moving body cannot be appropriately controlled due to the time lag from the start of detection to the completion of detection.

Means for Solving the Problems

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to a first embodiment of the present disclosure, a system is provided. The system comprises: an acquisition unit that acquires a mobile body status representing at least one of the state of a mobile body that can be moved by unmanned operation and the environment surrounding the mobile body; an estimation unit that performs an estimation process to estimate at least one of the position and orientation of the mobile body using sensor information acquired using sensors; a control command generation unit that generates and outputs a control command for controlling the movement of the mobile body using the estimation result of the estimation process; and a determination unit that determines whether to perform a first process or a second process as the estimation process according to the acquired mobile body status. The first process is a process that provides higher accuracy of the estimation result than the second process, and the second process is a process that takes less time than the first process. In this type of system, depending on the situation of the moving object, it is determined whether to execute a first process that produces more accurate estimation results or a second process that takes less time. This increases the likelihood that the movement of the moving object will be appropriately controlled by remote control or autonomous control. (2) In the above embodiment, the device comprises the acquisition unit, the estimation unit, the control command generation unit, and the determination unit, and includes a server device provided outside the mobile body, the sensor information is external information acquired using an external sensor provided outside the mobile body, the determination unit decides to execute the first process when the mobile body status matches a predetermined first condition, and decides to execute the second process when the mobile body status does not match the first condition, and the first condition may include at least one of the following: the mobile body is waiting to move, the mobile body's speed is less than or equal to a predetermined first reference speed, the communication speed between the server device and the mobile body is less than or equal to a predetermined reference communication speed, the cumulative amount of external force applied to the mobile body is greater than or equal to a predetermined amount, and the time the mobile body has been continuously operated is greater than or equal to a predetermined operating time. This type of system allows for more accurate estimation of position and orientation in situations where there is a relatively large time window for estimation processing, or in situations where it is preferable to estimate position and orientation with greater accuracy. Therefore, it can increase the likelihood that the movement of a moving object can be appropriately controlled by remote control. (3) In the above embodiment, the determination unit decides to execute the second process when the moving body condition matches a predetermined second condition, and decides to execute the first process when the moving body condition does not match the second condition, wherein the second condition includes at least one of the following: a condition in which the moving speed of the moving body is greater than a predetermined second reference moving speed, and a condition in which a complementary sensor different from the sensor is used to complement the estimation result with a weight greater than or equal to a predetermined weight, and the control command generation unit may generate the control command using the estimation result complemented by the complementary sensor when the complementary sensor is used to complement the estimation result. According to this embodiment of the system, the position and orientation can be estimated more quickly by the estimation process in situations where there is relatively little time to estimate the position and orientation of the moving body by the estimation process, or in situations where the estimation result is complemented by a complementary sensor. (4) According to a second embodiment of the present disclosure, a system is provided. This system includes: an acquisition unit that acquires a mobile body status representing at least one of the state of a mobile body that can be moved by unmanned operation and the surrounding environment of the mobile body; an estimation unit that performs an estimation process to estimate at least one of the position and orientation of the mobile body using sensor information acquired using sensors; a control command generation unit that generates and outputs a control command for controlling the movement of the mobile body using the estimation result of the estimation process; and a determination unit that determines whether to perform a first process or a second process as the estimation process according to the acquired mobile body status. The first process is a process that calculates at least one of the position and orientation of the mobile body a predetermined first number of times using the sensor information and a predetermined algorithm, and outputs the estimation result based on each of the calculated calculation results; the second process is a process that calculates at least one of the position and orientation of the mobile body a predetermined second number of times using the sensor information and the algorithm, and outputs the estimation result based on each of the calculated calculation results, wherein the first number of times is greater than the second number of times. In this type of system, depending on the status of the moving object, it is determined whether to execute a first process that calculates position and orientation more times, or a second process that calculates position and orientation less often. This increases the likelihood that the movement of the moving object will be appropriately controlled by remote control or autonomous control. (5) According to a third embodiment of the present disclosure, a mobile body that can be moved by unmanned operation is provided. The mobile body comprises an actuator that controls the movement of the mobile body; an acquisition unit that acquires a mobile body status representing at least one of the state of the mobile body and the environment surrounding the mobile body; an estimation unit that performs an estimation process to estimate at least one of the position and orientation of the mobile body using sensor information acquired using sensors; a control command generation unit that generates and outputs a control command for operating the actuator using the estimation result of the estimation process; and a determination unit that determines whether to execute a first process or a second process as the estimation process according to the mobile body status. The first process is a process that provides higher accuracy of the estimation result than the second process, and the second process is a process that takes less time than the first process. With this embodiment of the mobile body, it is determined whether to execute the first process, which provides higher accuracy of the estimation result, or the second process, which takes less time, according to the mobile body status, thereby increasing the possibility that the movement of the mobile body can be appropriately controlled by remote control or autonomous control. This disclosure can be implemented in forms other than the systems and mobile devices described above, such as server devices, control methods, programs for implementing the control methods, non-temporary recording media on which the programs are recorded, and program products. The program product may be provided, for example, as a recording medium on which the programs are recorded, or as a program product that can be distributed via a network. Furthermore, for example, the systems and mobile devices described above may use a pre-trained machine learning model to estimate the position and orientation of the mobile device during the estimation process. [Brief explanation of the drawing]

[0007] [Figure 1] This is an explanatory diagram showing the system configuration in the first embodiment. [Figure 2] This is an explanatory diagram showing how vehicles move autonomously within a factory. [Figure 3A] This is a flowchart of the first unmanned operation process. [Figure 3B]This is a flowchart of the location information acquisition process in the first embodiment. [Figure 4] This is an explanatory diagram showing the system configuration in the second embodiment. [Figure 5] This is a flowchart of the location information acquisition process in the second embodiment. [Figure 6] This is a flowchart of the location information acquisition process in the third embodiment. [Figure 7] This is an explanatory diagram showing the system configuration in the fifth embodiment. [Figure 8] This is the flowchart for the second unmanned operation process. [Modes for carrying out the invention]

[0008] A. First Embodiment: Figure 1 is an explanatory diagram showing the configuration of system 50 in the first embodiment. System 50 in this embodiment is configured as an unmanned operation system and is used in a factory that manufactures mobile objects to move them by unmanned operation.

[0009] In this embodiment, the mobile entity is a vehicle 100. More specifically, the vehicle 100 is an electric vehicle (BEV: Battery Electric Vehicle). However, the mobile entity is not limited to an electric vehicle; for example, it may be a gasoline car, a hybrid car, or a fuel cell vehicle. Furthermore, the mobile entity may be a vehicle with wheels or a vehicle with tracks; for example, it may be any vehicle such as a passenger car, truck, bus, motorcycle, car, tank, or construction vehicle. Moreover, the mobile entity is not limited to a vehicle; it may be an electric vertical take-off and landing aircraft (a so-called flying car).

[0010] The term "unmanned operation" above refers to operation without driver intervention by an onboard passenger. Driver intervention refers to operations related to at least one of the following: "driving," "turning," or "stopping" of the vehicle 100. Unmanned operation is achieved by automatic or manual remote control using devices installed outside the vehicle 100, or by autonomous control of the vehicle 100. A vehicle 100 operating under unmanned operation may have onboard passengers who do not perform driver intervention. Passengers who do not perform driver intervention include, for example, people simply sitting in the seats of the vehicle 100, or people performing tasks other than driver intervention, such as assembly, inspection, or operating switches, while on board the vehicle 100. Note that operation with driver intervention by an onboard passenger is sometimes called "manned operation."

[0011] As shown in Figure 1, the system 50 in this embodiment includes a vehicle 100, a server device 200 for remotely controlling the vehicle 100, a group of external sensors 300 installed in the factory, and a process control device 500 for managing the manufacturing process of the vehicle 100 in the factory.

[0012] Vehicle 100 includes a vehicle control device 110 for controlling various parts of vehicle 100, a drive device 120 for accelerating vehicle 100, a steering device 130 for changing the direction of travel of vehicle 100, a braking device 140 for decelerating vehicle 100, a communication device 150 for communicating with server device 200 via wireless communication, and an internal sensor group 160. In this embodiment, the drive device 120 includes a battery, a driving motor driven by the battery's power, and drive wheels rotated by the driving motor. The drive device 120, steering device 130, and braking device 140 each include actuators for operating vehicle 100. In particular, the drive device 120, steering device 130, and braking device 140 include actuators for controlling the movement of vehicle 100. Hereinafter, these various actuators provided in vehicle 100 will also be referred to as the actuator group.

[0013] The internal sensor group 160 includes at least one internal sensor 161. Note that Figure 1 shows only one internal sensor 161 included in the internal sensor group 160 as an example. The internal sensor 161 is a sensor mounted on the vehicle 100. In this embodiment, the internal sensor 161 is an internal state sensor that detects the state of the vehicle 100. The state of the vehicle 100 is a motion state such as the vehicle 100 being stopped, driving, accelerating, decelerating, moving straight, or turning. In this embodiment, the internal sensor group 160 includes, as internal sensors 161, a rotation speed sensor for measuring the rotation speed of the driving motor, an acceleration sensor for measuring the acceleration of the vehicle 100, a vehicle speed sensor for measuring the speed of the vehicle 100, and a yaw rate sensor for measuring the yaw axis angular velocity (yaw rate) of the vehicle 100. In other embodiments, the internal sensor group 160 may include, for example, a camera or LiDAR (Light Detection And Ranging) as internal sensors 161. In this case, the internal sensors 161, such as a camera or LiDAR, may be used as internal environment sensors to detect the environment around the vehicle 100. Hereinafter, the information detected by the internal sensors 161 will also be referred to as internal sensor information. Each piece of internal sensor information is transmitted from the internal sensors 161 to the vehicle control device 110, and also transmitted to the server device 200 via the communication device 150.

[0014] The vehicle control device 110 is comprised of a computer comprising a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, memory 112, and input / output interface 113 are connected via the internal bus 114 to enable bidirectional communication. The input / output interface 113 is connected to a drive unit 120, a steering unit 130, a braking unit 140, a communication unit 150, and an internal sensor group 160. The memory 112 stores the computer program PG1.

[0015] The processor 111 functions as a vehicle control unit 115 by executing a computer program PG1. The vehicle control unit 115 controls an actuator group. In particular, the vehicle control unit 115 controls the drive device 120, the steering device 130, and the braking device 140 to make the vehicle 100 travel. When a driver is on board the vehicle 100, the vehicle control unit 115 can make the vehicle 100 travel by controlling the drive device 120, the steering device 130, and the braking device 140 according to the driver's operation. Regardless of whether a driver is on board the vehicle 100 or not, the vehicle control unit 115 can make the vehicle 100 travel by controlling the drive device 120, the steering device 130, and the braking device 140 according to a control command described later.

[0016] The external sensor group 300 includes at least one external sensor 301. The external sensor group 300 in the present embodiment has a plurality of external sensors 301, but in FIG. 1, only one external sensor 301 included in the external sensor group 300 is illustrated. The external sensor 301 is a sensor located outside the vehicle 100. The external sensor 301 can acquire external information by detecting the motion state of the vehicle 100 from outside the vehicle 100. The external information is information representing the motion state of the vehicle 100 detected from outside the vehicle 100, and is used for an estimation process described later. In the present embodiment, each external sensor 301 is constituted by a camera installed in a factory, and captures a captured image including the vehicle 100 from outside the vehicle 100. This captured image is external information that optically captures the outer shape of the vehicle 100, and is external information representing the motion state of the vehicle 100 optically detected from outside the vehicle 100. Each external sensor 301 includes a communication device 302 and can communicate with the server device 200 by wired communication or wireless communication. Each external information is transmitted to the server device 200 via the communication device 302. Note that the external sensor 301 may be used as a sensor for detecting the environment around the vehicle 100. Hereinafter, the information detected by the external sensor 301 is also collectively referred to as external sensor information. Further, the external sensor information and the internal sensor information are also collectively referred to as sensor information.

[0017] As shown in FIG. 1, the server device 200 is constituted by a computer including a processor 201, a memory 202, an input / output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input / output interface 203 are communicably connected bidirectionally via the internal bus 204. A communication device 205 for communicating with the vehicle 100 by wireless communication is connected to the input / output interface 203. In the present embodiment, the communication device 205 can communicate with the external sensor group 300 and the process management device 500 by wired communication or wireless communication.

[0018] By executing the computer program PG2 stored in the memory 202, the processor 201 functions as a remote control unit 210, an acquisition unit 215, an estimation unit 220, a determination unit 235, and a determination unit 240. In addition to the computer program PG2, the memory 202 stores situation data SD, a first analysis program AP1, a second analysis program AP2, and an outer shape detection model Md, which will be described later.

[0019] The acquisition unit 215 acquires the moving body situation. The moving body situation represents at least one of the state of the moving body and the environment around the moving body. The moving body situation is used to determine which of the first process and the second process is to be executed as an estimation process described later. Hereinafter, the moving body situation regarding the vehicle 100 is also referred to as the vehicle situation. The vehicle situation includes, for example, the motion state of the vehicle 100, the location where the vehicle 100 travels, the time zone when the vehicle 100 travels, the type of the manufacturing process being performed on the vehicle 100, the communication state between the vehicle 100 and the server device 200 (for example, the presence or absence of communication delay and communication speed), the cumulative degree of external force applied to the vehicle 100, and the driving time of the vehicle 100. The acquisition unit 215 acquires such vehicle situations directly or indirectly from, for example, the internal sensor 161, the external sensor 301, the process management device 500, and each communication device.

[0020] In this embodiment, the determination unit 240 determines whether the vehicle status satisfies a predetermined first status. The first status in this embodiment includes a waiting status in which the vehicle 100 is waiting to move, a low-speed movement status in which the vehicle 100's movement speed is less than or equal to a predetermined first reference movement speed, a communication delay status in which the communication speed between the server device 200 and the vehicle 100 is less than or equal to a predetermined reference communication speed, an external force accumulation status in which the cumulative degree of external force applied to the vehicle 100 is greater than or equal to a predetermined reference level, and a long-duration operation status in which the vehicle 100 is continuously operated for a continuous operating time greater than or equal to a predetermined reference operating time. The status data SD described above stores various reference values ​​and various information that are referenced by the determination unit 240 when determining the vehicle status.

[0021] The fact that a vehicle 100 is in a standby state can be obtained, for example, from internal sensor information indicating that the vehicle speed of the vehicle 100 is zero, external sensor information indicating that the vehicle 100 is in a stopped state, sensor information indicating that the distance between the vehicle 100 and another vehicle 100 traveling further ahead is less than or equal to a predetermined reference distance, or from manufacturing information indicating that a process to put the vehicle 100 into standby is being executed. The process to put the vehicle 100 into standby is, for example, a process to perform a predetermined process on the vehicle 100 while the vehicle 100 is stopped, or a process to stop the vehicle 100 in order to adjust the manufacturing cycle time of the vehicle 100 according to the takt time. The fact that a vehicle is in a low-speed moving state can be obtained, for example, from internal sensor information indicating that the vehicle speed is less than or equal to a first reference moving speed, external sensor information indicating that the vehicle 100 is moving at or below the first reference moving speed, or from manufacturing information indicating that a process to drive the vehicle 100 at or below the reference moving speed is being executed. The vehicle status being a communication delay can be obtained, for example, from the communication log between the server device 200 and the vehicle 100. The vehicle status being an accumulation of external forces can be obtained, for example, from an acceleration sensor as an internal sensor 161, an external sensor 301, or manufacturing information indicating that a process is being performed that applies external force to the vehicle 100. These external forces include, for example, the force applied to the vehicle 100 when a person gets into the vehicle 100, the force applied to the vehicle 100 when the vehicle 100 travels on an uneven road, the force applied to the vehicle 100 when the vehicle 100 comes into contact with a guide installed on or around the road SR, or the force resulting from a manufacturing process performed on the vehicle 100. The presence of these external forces on the vehicle 100 can be detected, for example, using the external sensor 301 or manufacturing information. Therefore, the cumulative degree of such external forces may be calculated, for example, as the cumulative value of the sensor values ​​of the internal sensor 161 related to the external forces, using the external sensor 301 and the internal sensor 161. Alternatively, the cumulative degree of external forces may be calculated as the sensor value of the acceleration sensor when the vehicle 100 is moving straight on a flat road, using the external sensor 301 and the internal sensor 161. This sensor value of the acceleration sensor reflects the cumulative error caused by the external forces as the cumulative degree of external forces.Furthermore, the cumulative degree of external force may be calculated as the number of times the external force was applied to the vehicle 100 using an external sensor 301 or manufacturing information. The vehicle status being in a continuous operation state can be obtained, for example, by using the elapsed time from when the vehicle 100 started running unmanned until the present time.

[0022] The estimation unit 220 performs an estimation process. The estimation process is a process that estimates at least one of the position and orientation of the vehicle 100 using sensor information. In this embodiment, the estimation process is a process that estimates the position and orientation of the vehicle 100 using external information as sensor information. Therefore, in this embodiment, the position and orientation of the vehicle 100 are output as the estimation result of the estimation process. This estimation result is used to obtain the basic position information of the vehicle 100. The basic position information is the information that forms the basis for generating control commands and includes the position and orientation of the vehicle 100. Hereinafter, the position estimated by the estimation process will also be called the estimated position, and the orientation estimated by the estimation process will also be called the estimated orientation. The estimation unit 220 selectively performs either the first process or the second process, which will be described later, as the estimation process.

[0023] Furthermore, the estimation unit 220 can calculate the position and orientation of the vehicle 100 using complementary sensors different from those used in the estimation process, without relying on the estimation process. In this case, the estimation unit 220 obtains the position and orientation of the vehicle 100 by using, for example, the vehicle speed, acceleration, and yaw rate detected by the internal state sensor as a complementary sensor. In addition to the sensor information obtained by the complementary sensor, the vehicle 100's driving history and the history of control values ​​used to control the vehicle 100's movement may also be used to calculate this position and orientation. The position and orientation thus calculated can be used to complement the estimation results. Hereinafter, the position and orientation calculated using the internal sensor 161 as a complementary sensor will also be referred to as the first complementary information. In this embodiment, the first complementary information includes orientation but does not include position. In other embodiments, the first complementary information may include orientation and position, or it may include position but not orientation. Note that the above vehicle status is obtained by the acquisition unit 215 prior to the execution of the estimation process. Therefore, the vehicle status does not include the estimated position or estimated orientation. Furthermore, in this embodiment, the vehicle status does not include the first supplementary information.

[0024] The remote control unit 210 has a control command generation unit 225, which generates control commands and transmits them to the vehicle 100, thereby enabling remote control of the vehicle 100. The control command generation unit 225 generates and outputs control commands to control the movement of the vehicle 100 using basic position information. As described above, the basic position information is obtained using the estimation results of the estimation process. Therefore, it can be said that at least the estimation results of the estimation process are used to generate the control commands. In this embodiment, the control command generation unit 225 generates and outputs a driving control signal, which will be described later, as a control command.

[0025] In this embodiment, the basic position information is obtained by appropriately supplementing the estimation results with first supplementary information. Therefore, in this embodiment, it can be said that, in addition to the estimation results, the first supplementary information is used as appropriate in generating control commands. More specifically, the basic position information in this embodiment includes the estimated position and the estimated orientation, which is appropriately supplemented by the orientation as the first supplementary information. In this embodiment, the first weight, which represents the weight to which the first supplementary information is used to supplement the estimation results, is changed according to the vehicle situation, as will be described later. The first weight is expressed as a weighting index defined such that, for example, when the first weight is zero, the estimated orientation is not supplemented by the first supplementary information, and when the first weight is 1, the estimated orientation is 100% supplemented by the first supplementary information. Note that in other embodiments, when the first supplementary information includes both position and orientation, the first weight may be expressed by weights set for position and orientation respectively, or it may be expressed as a statistic for each weight or as an index representing the magnitude of each weight.

[0026] The decision unit 235 determines whether to execute the first process or the second process as an estimation process, depending on the moving object status acquired by the acquisition unit 215. The first process is a process that provides higher accuracy in estimation results than the second process. The second process is a process that takes less time to process than the first process. In this specification, when the position and orientation are estimated by the estimation process as in this embodiment, "high accuracy in estimation results" means that the estimation accuracy of both position and orientation is high. The comparison (evaluation) of the accuracy of the estimation results of the two estimation processes is performed by comparing the estimation results of each estimation process using the same external information 200 times. In this embodiment, for each estimation process, the number of times in which the estimation accuracy of both position and orientation was higher than that of other estimation processes is recorded, and the estimation process with the most recorded counts at the end of the 200 comparisons is defined as the estimation process with higher accuracy. Similarly, the comparison (evaluation) of the processing times of the two estimation processes is performed by comparing the processing times of each estimation process using the same external information 200 times. In this case, the processing time is calculated as the time from when the estimation process started until when it was completed.

[0027] The process control device 500 is a device for managing the manufacturing process of vehicles 100 in a factory. The process control device 500 consists of at least one computer. The process control device 500 is equipped with a communication device (not shown) and can communicate with the server device 200 and various factory equipment via wired or wireless communication. By communicating with various factory equipment, the process control device 500 can determine when, where, who, and which vehicle 100 is scheduled to perform what work, and when, where, who, and which vehicle 100 has performed what work. In this embodiment, when a vehicle 100 starts traveling along a target route, individual information such as an identification number and model that identifies the vehicle 100 is transmitted from the process control device 500 to the server device 200. This individual information corresponds to manufacturing information used to manage the manufacturing process of the vehicle 100. The server device 200 can use this individual information to identify the vehicle (target vehicle) 100 that is subject to unmanned operation.

[0028] Figure 2 is an explanatory diagram showing how a vehicle 100 moves unmanned within the factory FC. The factory FC has a first location PL1 where the first operation is performed and a second location PL2 where the second operation is performed. The first location PL1 and the second location PL2 are connected by a travel path SR on which the vehicle 100 can travel. Multiple external sensors 301 are provided around the travel path SR. In this embodiment, the first location PL1 is where the vehicle 100 is assembled. The second location PL2 is where the vehicle 100 is inspected. Vehicles 100 that pass the inspection at the second location PL2 are shipped out of the factory FC. Each vehicle 100 assembled at the first location PL1 is transported to a predetermined starting point at the first location PL1 with the vehicle control device 110, drive unit 120, steering unit 130, braking unit 140, and communication device 150 installed. In other words, in this embodiment, each vehicle 100 is positioned at the starting point of the first location PL1 in a state where it can be driven by remote control. Each vehicle 100 travels from the starting point of the first location PL1 to a predetermined goal point of the second location PL2 by remote control by the server device 200.

[0029] Figure 3A is a flowchart of the unmanned operation process performed in this embodiment. The unmanned operation process is the process of driving the vehicle 100 by unmanned operation. Figure 3A can also be said to represent the method of driving the vehicle 100 by unmanned operation in this embodiment.

[0030] In S10, the remote control unit 210 of the server device 200 acquires basic position information of the vehicle 100 using the detection result output from the external sensor 301. In this embodiment, the basic position information of the vehicle 100 includes the position and orientation of the vehicle 100 in the global coordinate system of the factory FC. In this embodiment, the external sensor 301 is a camera installed in the factory FC, and in S10, the captured image is output as the detection result. The position and orientation of the external sensor 301 in the factory FC are pre-adjusted. The remote control unit 210 detects the vehicle 100 from the image acquired from the external sensor 301 and acquires basic position information from the position of the vehicle 100 in that image.

[0031] In detail, in S10 of this embodiment, the remote control unit 210 detects the outline of the vehicle 100 from the captured image, calculates the coordinates of the positioning point of the vehicle 100 in the image coordinate system, which is the coordinate system of the captured image, and obtains the position of the vehicle 100 as basic position information by converting the calculated coordinates to coordinates in the global coordinate system. In this embodiment, the positioning point is the left rear corner of the vehicle 100, but in other embodiments, it may be any part of the vehicle 100. In this embodiment, the outline of the vehicle 100 included in the captured image is detected by inputting the captured image into the outline detection model Md. The outline detection model Md is a detection model that utilizes artificial intelligence. Specifically, the outline detection model Md is, for example, a trained machine learning model that has been trained to realize either semantic segmentation or instance segmentation. As this machine learning model, for example, a convolutional neural network (hereinafter simply referred to as CNN) trained by supervised learning using a training dataset can be used. The training dataset includes, for example, multiple training images containing vehicle 100, and labels indicating whether each region in the training images represents vehicle 100 or something other than vehicle 100. During CNN training, it is preferable that the CNN parameters are updated using backpropagation to reduce the error between the output of the detection model and the labels. In other embodiments, the outline detection model Md is not limited to a CNN, but may be, for example, a pre-trained machine learning model other than a neural network.

[0032] Furthermore, in S10, the remote control unit 210 can obtain the orientation of the vehicle 100 as basic position information by, for example, using the optical flow method, and estimating it based on the direction of the vehicle 100's movement vector calculated from the positional changes of the vehicle 100's feature points between frames of the captured image. The orientation of the vehicle 100 is calculated as a direction vector that can identify the orientation of the vehicle 100 in at least the horizontal direction, for example, as the direction of a vector pointing from the rear to the front of the vehicle 100.

[0033] In this embodiment, specifically in S10, the basic position information of the vehicle 100 is acquired by executing a position information acquisition process, which will be described later.

[0034] In S20, the remote control unit 210 determines the next target location that the vehicle 100 should head to. In this embodiment, the target location is represented by X, Y, Z coordinates in the global coordinate system. The memory 202 pre-stores a reference route, which is the path that the vehicle 100 should travel. The route is represented by a node indicating the starting point, nodes indicating waypoints, a node indicating the destination, and links connecting each node. The remote control unit 210 uses the vehicle 100's base position information and the reference route to determine the next target location that the vehicle 100 should head to. The remote control unit 210 determines the target location on the reference route beyond the vehicle 100's current location.

[0035] In S30, the remote control unit 210 generates a driving control signal from the control command generation unit 225 to drive the vehicle 100 toward the determined target position. In this embodiment, the driving control signal includes the acceleration and steering angle of the vehicle 100 as parameters. The remote control unit 210 calculates the driving speed of the vehicle 100 from the change in the position of the vehicle 100 and compares the calculated driving speed with a predetermined target speed for the vehicle 100. If the driving speed is lower than the target speed, the remote control unit 210 determines the acceleration so that the vehicle 100 accelerates, and if the driving speed is higher than the target speed, it determines the acceleration so that the vehicle 100 decelerates. If the vehicle 100 is located on the reference path, the remote control unit 210 determines the steering angle so that the vehicle 100 does not deviate from the reference path, and if the vehicle 100 is not located on the reference path, in other words, if the vehicle 100 has deviated from the reference path, it determines the steering angle so that the vehicle 100 returns to the reference path.

[0036] In S40, the remote control unit 210 transmits the generated driving control signal to the vehicle 100. The remote control unit 210 repeats the process of acquiring the vehicle 100's position information, determining the target position, generating the driving control signal, and transmitting the driving control signal at predetermined intervals.

[0037] In S50, the vehicle control unit 115 receives a driving control signal from the remote control unit 210. In S60, the vehicle control unit 115 uses the received driving control signal to control the actuator group, thereby driving the vehicle 100 at the acceleration and steering angle indicated in the driving control signal. The vehicle control unit 115 repeats the reception of the driving control signal and the control of the actuator group at a predetermined cycle. According to the system 50 in this embodiment, since the vehicle 100 can be driven by remote control, the vehicle 100 can be moved without using transport equipment such as cranes or conveyors.

[0038] Figure 3B is a flowchart of the location information acquisition process in this embodiment. The location information acquisition process is for acquiring the basic location information of the vehicle 100. As described above, the location information acquisition process in Figure 3B is executed in S10 of Figure 3A to acquire the basic location information.

[0039] In S110, the acquisition unit 215 acquires the vehicle status of the target vehicle 100. In S115, the determination unit 240 determines whether the vehicle status acquired in S110 matches the first condition. If the first condition is met in S115, in S120, the decision unit 235 decides to execute the first process as an estimation process.

[0040] From S125 to S140, the estimation unit 220 executes the first process determined in S120 as an estimation process. First, in S125, the estimation unit 220 acquires a captured image as external information from the external sensor 301. In S130, the estimation unit 220 calculates the position and orientation of the target vehicle 100 by analyzing the captured image acquired in S125 using the first analysis program AP1. The first analysis program AP1 is a program that enables the computer to perform an outline detection process to detect the outline of the target vehicle 100 in the captured image, a polygonal approximation process to approximate the outline detected by the outline detection process with polygons, and a first calculation process to calculate the position and orientation of the vehicle 100 using the approximation result from the polygonal approximation process. Hereafter, the process of calculating the position and orientation of the vehicle 100 by analyzing the captured image, as in S130, will also be called the analysis process.

[0041] As described above, in the outline detection process of this embodiment, the outline of the target vehicle 100 is detected when the captured image is input to the outline detection model Md. The outline detection model Md detects the area of ​​the vehicle 100 included in the input image by segmentation and detects the outline of the target vehicle 100 by masking the area of ​​the vehicle 100 included in the input image. In addition, in the first calculation process, the coordinates of the positioning points in the image coordinate system are calculated using the approximation result from the polygon approximation process, and the calculated coordinates are converted to coordinates in the global coordinate system using a predetermined transformation matrix and transformation formula.

[0042] In S135, the estimation unit 220 determines whether the number of times the analysis process in S130 has been executed is equal to or greater than a predetermined first number. In this embodiment, the first number is two or more times. If the number of times S130 has been executed is less than the first number, the estimation unit 220 returns to S125. In the re-executed S125, the estimation unit 220 acquires images captured by the external sensor 301 that were captured later in time than the previously executed S125. That is, in this embodiment, the sensor information used in each repeatedly executed analysis process is different sensor information. In S137, the estimation unit 220 outputs a statistical quantity obtained by statistically processing each calculation result of the first number of analysis processes as the estimation result. In this embodiment, this statistical quantity is the median. In other embodiments, this statistical quantity may be, for example, the arithmetic mean. In S140, the estimation unit 220 outputs the estimation result. In this embodiment, in S140, the estimation unit 220 outputs the statistical quantity calculated in S137 as the estimation result. That is, in S137, the estimation result is output based on each calculation result of the analysis process in S130 which is executed repeatedly. By outputting the statistical quantity as the estimation result in this way, the influence of disturbances in the captured image can be reduced and the accuracy of the estimation result can be further improved.

[0043] If the vehicle condition does not meet the first condition in S115, the decision unit 235 decides in S145 to execute the second process as an estimation process.

[0044] From S150 to S160, the estimation unit 220 executes the second process determined in S150 as an estimation process. First, in S150, the estimation unit 220 acquires a captured image as external information from the external sensor 301. In S155, the estimation unit 220 calculates the position and orientation of the target vehicle 100 by analyzing the captured image acquired in S150 using the second analysis program AP2. The second analysis program AP2 is a program that enables the computer to perform the same outline detection process as in S130, an outline rectangle approximation process that approximates the detected outline with an outline rectangle, and a second calculation process that calculates the position and orientation using the approximation result of the outline rectangle approximation process. Therefore, in this embodiment, in the outline detection processes in S155 and S130, the outline of the target vehicle 100 is detected using the same outline detection model Md and the same algorithm. Furthermore, in the second calculation process, the coordinates of the positioning points in the image coordinate system are calculated using the approximation result from the circumscribed rectangle approximation process, and the calculated coordinates are converted to coordinates in the global coordinate system using predetermined transformation matrices and transformation formulas.

[0045] The bounding rectangle approximation process in the S155 analysis process has lower accuracy in its approximation results and is faster than the polygon approximation process in the S130 analysis process. As a result, the accuracy of the position and orientation calculated by the S155 analysis process is lower than that of the S130 analysis process. Furthermore, the processing time for the S155 analysis process is shorter than that for the S130 analysis process. In other words, in this embodiment, the S130 analysis process has higher accuracy in calculating position and orientation, and a longer processing time compared to the S155 analysis process. Note that the accuracy and processing speed of approximation processes such as polygon approximation and bounding rectangle approximation, as well as the accuracy and processing speed of the analysis process, can be evaluated in the same way as the accuracy and processing time of estimation processes, for example.

[0046] In S160, the estimation unit 220 outputs the estimation result based on the calculation result of the analysis process in S155. In this embodiment, the calculation result of the analysis process in S155 is output as is in S160. Hereafter, the number of times the analysis process is executed in the second process will also be referred to as the second count. In this embodiment, the second count is fewer than the first count, specifically, it is 1 time.

[0047] As described above, in this embodiment, the analysis process in the first process is more accurate and takes longer to process than the analysis process in the second process. Furthermore, in the first process in this embodiment, the analysis process is executed a greater number of times than in the second process. Therefore, the first process in this embodiment is more accurate and takes longer to process than the second process.

[0048] In S165, the control command generation unit 225 determines the first weight to a predetermined weight greater than zero, thereby supplementing the estimation result output in S160 with the first supplementary information. In other words, in this embodiment, the estimation result is supplemented with the first supplementary information when the vehicle condition does not match the first condition. The first condition in this embodiment can also be described as a situation in which the supplementation of the estimation result with the first supplementary information is not performed. The time required from the start of S150 to the completion of S165 is shorter than the time required from the start of S125 to the completion of S140b. Also, as described above, in this embodiment, the estimation result is not supplemented when the vehicle condition matches the first condition. Therefore, in this case, it can be said that the first weight is zero. Then, in S20 and S30 of Figure 3A above, the control command generation unit 225 uses the estimation result output in S140, or the estimation result supplemented in S165, as basic position information to generate and output a driving control signal as a control command.

[0049] According to the system 50 of this embodiment described above, depending on the vehicle conditions, it is determined whether the first process, which has higher accuracy in estimation results than the second process, or the second process, which has a shorter processing time than the first process, will be executed as the estimation process. This increases the likelihood that an appropriate control command will be generated using the estimation results from the estimation process, thereby increasing the likelihood that the movement of the vehicle 100 will be appropriately controlled by remote control.

[0050] Furthermore, in this embodiment, the execution of the first process is determined when the vehicle status of the target vehicle 100 satisfies a first condition. This first condition includes a standby state, a low-speed movement state, a communication delay state, an external force accumulation state, and a long-duration driving state. This allows for more accurate estimation of the position and orientation of the vehicle 100 in situations where there is a relatively large time leeway for estimating the position and orientation of the vehicle 100 by the estimation process, and in situations where it is preferable to estimate the position and orientation more accurately by the estimation process. Therefore, the possibility of the vehicle 100's movement being appropriately controlled by remote control can be increased. Specifically, in a low-speed movement state, there is a larger time leeway for estimating the position and orientation of the vehicle 100 compared to a situation where the vehicle 100 is traveling at a speed faster than the first reference movement speed. In a standby state, this time leeway is even greater. In a communication delay state, it takes more time for the server device 200 to receive internal sensor information compared to a situation where the communication speed is faster than the reference communication speed, so it is preferable to estimate the position and orientation more accurately using an estimation process that utilizes external information. Furthermore, in situations where external forces accumulate, compared to when the degree of external force accumulation is below a standard level, there is a higher possibility that errors in the internal state sensor (e.g., measurement errors and drift errors) due to the external forces applied to the vehicle 100 will be larger. Therefore, it is preferable to estimate the position and orientation with greater accuracy using estimation processing with external information. Also, in situations where the continuous operating time is shorter than the standard operating time, there is a higher possibility that errors in the internal state sensor will be larger due to the rise in temperature of the internal state sensor. Therefore, it is preferable to estimate the position and orientation with greater accuracy using estimation processing with external information.

[0051] Furthermore, if the first condition is not met, the decision of the second process is executed, allowing the position and orientation to be estimated more quickly in the estimation process. Therefore, the time lag from the start to the completion of position and orientation estimation in the estimation process can be suppressed, increasing the likelihood that the movement of vehicle 100 can be appropriately controlled by remote control.

[0052] B. Second Embodiment: Figure 4 is an explanatory diagram showing the configuration of system 50b in the second embodiment. In this embodiment, unlike the first embodiment, the decision unit 235b decides to execute the second process when the vehicle condition matches a predetermined second condition, and decides to execute the first process when the vehicle condition does not match the second condition.

[0053] System 50b includes an external complementary sensor 400. The external complementary sensor 400 is located outside the vehicle 100 and is used to complement the estimation results. The external complementary sensor 400 detects the vehicle 100 from outside the vehicle 100. In this embodiment, the external complementary sensor 400 is composed of a LiDAR installed at the factory FC. The external complementary sensor 400 measures the distance of the vehicle 100 located in a predetermined section of the travel path SR and acquires three-dimensional point cloud data of the vehicle 100. This three-dimensional point cloud data is information representing the motion state of the vehicle 100 as optically detected from outside the vehicle 100. The external complementary sensor 400 is equipped with a communication device 402 and can communicate with the server device 200b via wired or wireless communication. The information acquired by the external complementary sensor 400 is transmitted to the server device 200 via the communication device 402.

[0054] The estimation unit 220 can obtain the position and orientation of the vehicle 100 using the distance measurement results from the external complementary sensor 400 and the three-dimensional point cloud data acquired by the external complementary sensor 400. When using three-dimensional point cloud data to obtain the position and orientation, the estimation unit 220 obtains the position and orientation of the vehicle 100 by performing template matching using the three-dimensional point cloud data and the template point cloud data TP stored in memory 202b, for example. Various algorithms such as ICP (Iterative Closest Point) and NDT (Normal Distributions Transform) can be used for template matching. The position and orientation thus obtained can be used to complement the estimation results. That is, the external complementary sensor 400 can be used as a complementary sensor to complement the estimation results. Hereinafter, the position and orientation obtained using the external complementary sensor 400 will also be referred to as second complementary information. In this embodiment, the basic position information is obtained by appropriately complementing the estimation results with the first complementary information and the second complementary information. In this embodiment, the second complementary information includes position and orientation. In this embodiment, the second weight, which represents the weight used to complete the estimation result by using the second supplementary information, is changed according to the vehicle conditions, as described later. The second weight is represented, for example, in the same way as the first weight. In other embodiments, the second weight may include, for example, position but not orientation, or orientation but not position. In this embodiment, the vehicle conditions are not included in the second supplementary information.

[0055] In this embodiment, the determination unit 240b determines whether the vehicle condition matches a second condition. The second condition in this embodiment includes a high-speed movement condition in which the vehicle 100's movement speed is greater than a predetermined second reference movement speed, and a supplementation condition in which a supplementation sensor is used to supplement the estimation result with a weight greater than or equal to a predetermined reference weight. The supplementation condition includes an internal supplementation condition in which the internal sensor 161 is used to supplement the estimation result with a weight greater than or equal to a predetermined first reference weight, and an external supplementation condition in which the external supplementation sensor 400 is used to supplement the estimation result with a weight greater than or equal to a predetermined second reference weight. The first reference weight and the second reference weight are each weights greater than zero. The internal supplementation condition in this embodiment includes a first supplementation condition in which the communication speed between the server device 200 and the vehicle 100 is greater than the reference communication speed, the cumulative degree of external force applied to the vehicle 100 is less than a reference level, and the continuous operating time is less than the reference operating time, and a second supplementation condition in which the vehicle 100 is in a turning state.

[0056] The acquisition unit 215 acquires that the vehicle status is a high-speed movement status in substantially the same way as it acquires that the vehicle status is a low-speed movement status. The acquisition unit 215 also acquires that the vehicle status is a first complementary status in substantially the same way as it acquires that the vehicle status is a communication delay status, an external force accumulation status, or a continuous operation status. The acquisition unit 215 also acquires that the vehicle status is a second complementary status from, for example, internal sensor information indicating that the vehicle 100 is turning, external sensor information indicating that the motion state of the vehicle 100 is a turning state, or manufacturing information indicating that a process to turn the vehicle 100 is being executed. The acquisition unit 215 also acquires that the vehicle status is an external complementary status from, for example, internal sensor information, external sensor information, or manufacturing information indicating the location where the vehicle 100 is traveling.

[0057] Figure 5 is a flowchart of the location information acquisition process in the second embodiment. In Figure 5, the same reference numerals are used for steps similar to those in Figure 3B. At S115b, the determination unit 240b determines whether the vehicle condition matches the second condition. If the vehicle condition does not match the second condition at S115b, at S120, the decision unit 235b decides to execute the first process. If the vehicle condition matches the second condition at S115b, at S120, the decision unit 235b decides to execute the second process.

[0058] In S165b, the control command generation unit 225 complements the estimation result output in S160. In this embodiment, in S165b, if the vehicle condition matches the internal complementation condition, the control command generation unit 225 complements the estimation result with the first complementation information by setting the first weight to a weight equal to or greater than the first criterion. Also in S165b, if the vehicle condition matches the external complementation condition, the control command generation unit 225 complements the estimation result with the second complementation information by setting the second weight to a weight equal to or greater than the second criterion. If the vehicle information matches both the internal and external complementation conditions, the estimation result may be complemented by, for example, both the first and second complementation information. In this embodiment, in S20 and S30 of Figure 3A, the control command generation unit 225 uses the estimation result output in S140, or the estimation result complemented in S165b, as basic position information to generate and output a driving control signal as a control command.

[0059] According to system 50b in the second embodiment described above, the execution of the second process is decided when the vehicle status of the target vehicle 100 satisfies the second condition. This second condition includes a high-speed movement condition and a complementary condition. In this way, the position and orientation are estimated faster in the estimation process in situations where there is relatively little time to estimate the position and orientation of the vehicle 100 by the estimation process, or in situations where the estimation results are complemented using complementary sensors. Therefore, the time lag from the start to the completion of the estimation of position and orientation in the estimation process can be suppressed, and the possibility that the movement of the vehicle 100 can be appropriately controlled by remote control can be increased.

[0060] C. Third Embodiment: Figure 6 is a flowchart of the location information acquisition process in the third embodiment. In Figure 6, steps similar to those in Figure 3B are denoted by the same reference numerals as in Figure 3B. In the third embodiment, the number of times the first calculation process is executed in the first process and the number of times the second calculation process is executed in the second process are the same, specifically, one time. The configuration of the system 50, server device 200, and vehicle 100 in the third embodiment is the same as in the first embodiment unless otherwise described.

[0061] Unlike the first embodiment, the first process in this embodiment does not include S135 and S137. Also, in S140b shown in Figure 6, the estimation unit 220 outputs the calculation result of S130 as the estimation result. Note that the analysis process in S130 is more accurate and takes longer to process than the analysis process in S155, so the first process in this embodiment is more accurate and takes longer to process than the second process. In this embodiment, in S20 and S30 in Figure 3A, the control command generation unit 225 uses the estimation result output in S140b or the estimation result supplemented in S165 as basic position information to generate and output a driving control signal as a control command.

[0062] In the third embodiment described above, the system 50 also determines, depending on the moving object's status, whether the first process, which provides a higher accuracy of estimation results than the second process, or the second process, which has a shorter processing time than the first process, will be executed as the estimation process. This increases the likelihood that the movement of the vehicle 100 will be appropriately controlled by remote control.

[0063] D. Fourth Embodiment: In the fourth embodiment, the system 50 performs a location information acquisition process substantially similar to that shown in Figure 3. Unlike the first embodiment, the same second analysis program AP2 is used in both the first and second processes in this embodiment. The configuration of the system 50, server device 200, and vehicle 100 in the fourth embodiment is the same as in the first embodiment unless otherwise described.

[0064] In this embodiment, in steps S130 and S155 of Figure 3, the estimation unit 220 calculates the position and orientation of the target vehicle 100 by analyzing the captured image using the second analysis program AP2. That is, in this embodiment, the same algorithm is used to analyze the captured image and estimate the position and orientation in both the analysis process included in the first process and the analysis process included in the second process. In this embodiment, the accuracy and processing time of the analysis process in the first process are the same as those of the analysis process in the second process. On the other hand, in the first process, the analysis process is executed a first time, which is more than the second time. Therefore, the first process has higher accuracy and a longer processing time compared to the second process. In this embodiment, the first analysis program AP1 does not need to be stored in the memory 202.

[0065] According to the system 50 in the fourth embodiment described above, depending on the status of the moving object, it is determined whether to execute a first process that calculates the position and orientation more times, or a second process that calculates the position and orientation less times. Therefore, the possibility that the movement of the vehicle 100 can be appropriately controlled by remote control can be increased.

[0066] In the fourth embodiment, for example, the determination unit 235 and the judgment unit 240 may determine whether to execute the first or second process as the estimation process by changing the number of times the analysis process is executed according to the vehicle conditions. In this case, determining the number of times the analysis process is executed to the first number of times corresponds to determining that the first process will be executed. Also, determining the number of times the analysis process is executed to the second number of times corresponds to determining that the second process will be executed.

[0067] E. Fifth Embodiment: Figure 7 is an explanatory diagram showing the configuration of system 50c in the fifth embodiment. Unlike the first embodiment, system 50c in this embodiment does not include a server device 200. Also, in this embodiment, the operation of vehicle 100b without a driver is achieved by autonomous control of vehicle 100b. The configuration of system 50c and vehicle 100b in the fifth embodiment is the same as in the first embodiment unless otherwise described.

[0068] In this embodiment, the memory 112b of the vehicle control device 110b stores the computer program PG1, the situation data SD, the external shape detection model Md, the first analysis program AP1, and the second analysis program AP2. The processor 111b of the vehicle control device 110 functions as a vehicle control unit 115b by executing the computer program PG1, and also functions as an acquisition unit 215, an estimation unit 220, a control command generation unit 225, a determination unit 235, and a judgment unit 240.

[0069] Figure 8 is a flowchart of the unmanned operation process in this embodiment. In S11, the vehicle control unit 115b acquires the basic position information of the vehicle 100b using the detection results output from the external sensor 301. In S21, the vehicle control unit 115b determines the target position to which the vehicle 100b should next go. In this embodiment, a reference path is pre-stored in the memory 112b. In S31, the vehicle control unit 115b generates a driving control signal from the control command generation unit 225b to drive the vehicle 100b toward the determined target position. In S41, the vehicle control unit 115b drives the vehicle 100b with the acceleration and steering angle expressed in the driving control signal by controlling the actuator group using the generated driving control signal. The vehicle control unit 115b repeats the acquisition of the basic position information of the vehicle 100b, determination of the target position, generation of the driving control signal, and control of the actuator group at predetermined intervals. According to system 50c in this embodiment, the vehicle 100b can be driven by autonomous control of the vehicle 100b without remote control of the vehicle 100b from an external source.

[0070] In this embodiment, the same position information acquisition process as in Figure 3A is performed by the vehicle control device 110b of vehicle 100b. In this embodiment, the target vehicle 100b refers to the vehicle 100b itself that performs the position information acquisition process, i.e., the vehicle itself. In S21 and S31 of Figure 8, the control command generation unit 225b uses the estimation result output in S140 of Figure 3, or the estimation result supplemented in S165, as basic position information to generate and output a driving control signal as a control command.

[0071] In the fifth embodiment of vehicle 100b described above, the decision of whether to execute the first or second process as an estimation process is made according to the vehicle status, thereby increasing the likelihood that the movement of vehicle 100 will be appropriately controlled by autonomous control.

[0072] Furthermore, in a configuration where unmanned operation of the vehicle 100b is achieved without the server device 200, similar to the fifth embodiment, the execution of the second process may be decided when the vehicle condition matches the second condition, similar to the second embodiment. Also, in this configuration, similar to the third embodiment, the number of times the first calculation process is executed in the first process and the number of times the second calculation process is executed in the second process may be the same. Also, in this configuration, similar to the fourth embodiment, the same algorithm may be used for the analysis process in the first process and the analysis process in the second process.

[0073] F. Other embodiments: (F1) In each of the above embodiments, the estimation unit 220 may estimate at least one of the position and orientation of the vehicle 100 using internal sensor information instead of external information during the estimation process. In this case, the internal sensor 161 can be, for example, at least one of a camera and a LiDAR mounted on the vehicle 100, and the internal sensor information can be images of the surrounding environment of the vehicle 100 taken by the camera, distance measurement results of targets around the vehicle 100 acquired by the LiDAR, or point cloud data of targets. In this case, the estimation process can be, for example, a process that estimates at least one of the position and orientation of the vehicle 100 by analyzing images, distance measurement results, or point cloud data acquired as internal sensor information. In this case, it is preferable that the first situation includes at least one of the standby situation, low-speed movement situation, external force accumulation situation, and continuous operation situation. It is also preferable that the second situation includes at least one of the high-speed movement situation and the complementation situation.

[0074] (F2) In each of the above embodiments, the determination unit 235 may change the first count depending on the vehicle condition. In this case, for example, the vehicle condition may be represented by a statistical value that represents the deviation of the vehicle 100 traveling on the road SR from the target route, and the determination unit 235 may increase the first count as this statistical value increases.

[0075] (F3) In each of the above embodiments, the estimation process outputs both the position and orientation of the vehicle 100 as estimation results, but it is sufficient if at least one of the position and orientation of the vehicle 100 is output as an estimation result. In this case, the other that is not output as an estimation result may be supplemented using, for example, first supplementary information or second supplementary information.

[0076] (F4) In each of the above embodiments, the determination unit 235 may selectively determine the process to be executed as the estimation process from three or more processes that differ in the accuracy of the estimation result and the processing time.

[0077] (F5) In each of the above embodiments, the approximation process in the first process has higher accuracy and a longer processing time compared to the approximation process in the second process. Conversely, if the first process has higher accuracy and a longer processing time compared to the second process, the approximation process in the first process may have lower accuracy, the same accuracy, a shorter processing time, or the same processing time compared to the approximation process in the second process. In this case, the fact that the first process has higher accuracy and a longer processing time compared to the second process may be achieved, for example, by having higher accuracy and a longer processing time for the approximation process in the first process, or by having more first iterations than second iterations. Similarly, if the first process has higher accuracy and a longer processing time compared to the second process, the analysis process in the first process may have lower accuracy, the same accuracy, a shorter processing time, or the same processing time compared to the analysis process in the second process. In this case, the fact that the first process has higher accuracy and a longer processing time compared to the second process may be achieved, for example, by having more first iterations than second iterations. Furthermore, the estimation process is not limited to the processes described in each of the above embodiments, but may be any process as long as it is possible to estimate the position and orientation of the vehicle 100 using the sensor information.

[0078] (F6) In each of the above embodiments, the first situation includes a standby situation, a low-speed movement situation, a communication delay situation, an external force accumulation situation, and a long-term operation situation, but it does not have to include these situations. However, it is preferable that the first situation includes at least one of the standby situation, a low-speed movement situation, a communication delay situation, an external force accumulation situation, and a long-term operation situation. In addition, the first situation may include other situations in place of these situations, or in addition to these situations. For example, the first situation may be a situation in which the vehicle 100 is traveling in a predetermined area within the factory FC. In this case, the acquisition unit 215 may acquire the vehicle situation in which the vehicle 100 is traveling in this area using, for example, internal sensor information, external sensor information, or manufacturing information that represents the location of the vehicle 100. In addition, this area may be, for example, an area in which the estimation accuracy tends to be lower compared to other areas, or an area in which the distance between the vehicle 100 and the target route tends to be larger compared to other areas. In this case, the decision unit 235 may, for example, refer to map data that records the correspondence between each area within the factory FC and the processes used as estimation results, using information representing the acquired driving location, to determine whether to execute the first process or the second process. Even in this case, the decision unit 235 can decide to execute the first process if the vehicle condition matches the first condition, and decide to execute the second process if the vehicle condition does not match the second condition. In this case, the determination unit 240 may not be provided.

[0079] (F7) In the second embodiment described above, the second situation includes a high-speed movement situation, an internal interpolation situation, and an external interpolation situation, but it does not have to include these situations. However, it is preferable that the second situation includes at least one of the high-speed movement situation, the internal interpolation situation, and the external interpolation situation. In addition, the second situation may include other situations in place of these situations, or in addition to these situations. For example, the second situation may be a situation in which the vehicle 100 is traveling in a predetermined area within the factory FC. This area may be, for example, an area that tends to have a higher estimation accuracy compared to other areas, or an area that tends to have a smaller distance between the vehicle 100 and the target route compared to other areas. When map data is used in substantially the same manner as described above in such a form, the determination unit 240 does not have to be provided.

[0080] (F8) In each of the above embodiments, the external sensor 301 is a camera. In contrast, the external sensor 301 may be a LiDAR. In this case, the sensor information output from the external sensor 301 is not a captured image, but three-dimensional point cloud data. In this case, the remote control unit 210 and the vehicle control unit 115 may use the three-dimensional point cloud data to estimate at least one of the position and orientation of the vehicle 100 in the estimation process. This three-dimensional point cloud data is external information in which the outline of the vehicle 100 is optically captured, and is external information representing the motion state of the vehicle 100 as optically detected from outside the vehicle 100. The remote control unit 210 can estimate the position and orientation of the vehicle 100 by, for example, performing template matching using the three-dimensional point cloud data. In this case, the external complementary sensor 400 is preferably a camera, for example. Also, if the external sensor group 300 includes a camera and a LiDAR as external sensors 301, the captured image and point cloud data may be fused and used by, for example, fusion processing.

[0081] (F9) In each of the above embodiments, the control command generation unit 225 generates a driving control signal as a control command. However, the control command generation unit 225 does not have to generate a driving control signal as a control command. Specifically, the control command only needs to include at least one of a driving control signal and generated information for generating the driving control signal. For example, if the control command generation unit 225 provided in the server device 200 generates generated information as a control command, the vehicle control device 110b of the vehicle 100b that receives the generated information may use the generated information to generate a driving control signal. As generated information, for example, basic position information, route, and target position can be used.

[0082] (F10) In the first to fourth embodiments described above, the server device 200 performs the processing from acquiring the basic position information of the vehicle 100 to generating the driving control signal. In contrast, the vehicle 100 may perform at least a part of the processing from acquiring the basic position information of the vehicle 100 to generating the driving control signal. For example, the following forms (1) to (3) may be used.

[0083] (1) The server device 200 may acquire basic position information of the vehicle 100, determine the next target location to which the vehicle 100 should go, and generate a route from the vehicle 100's current location, as shown in the acquired basic position information, to the target location. The server device 200 may generate a route to the target location between the current location and the destination, or it may generate a route to the destination. The server device 200 may transmit the generated route to the vehicle 100. The vehicle 100 may generate a driving control signal so that the vehicle 100 travels along the route received from the server device 200, and may use the generated driving control signal to control the group of actuators.

[0084] (2) The server device 200 may acquire basic position information of the vehicle 100 and transmit the acquired basic position information to the vehicle 100. The vehicle 100 may determine the next target location to which the vehicle 100 should go, generate a route from the vehicle 100's current location shown in the received position information to the target location, generate a driving control signal so that the vehicle 100 travels along the generated route, and control the actuator group using the generated driving control signal.

[0085] (3) In the embodiments of (1) and (2) above, detection results output from an internal sensor 161 mounted on the vehicle 100 may be used as sensor information in at least one of the route generation and the driving control signal generation. The internal sensor 161 may include, for example, a camera, LiDAR, millimeter-wave radar, ultrasonic sensor, GPS sensor, acceleration sensor, gyro sensor, yaw rate sensor, etc. For example, in the embodiment of (1) above, the server device 200 may acquire the detection results of the internal sensor 161 and reflect the detection results of the internal sensor 161 in the route when generating the route. In the embodiment of (1) above, the vehicle 100 may acquire the detection results of the internal sensor 161 and reflect the detection results of the internal sensor 161 in the driving control signal when generating the driving control signal. In the embodiment of (2) above, the vehicle 100 may acquire the detection results of the internal sensor 161 and reflect the detection results of the internal sensor 161 in the route when generating the route. In the embodiment of (2) above, the vehicle 100 may acquire the detection results of the internal sensor 161 and reflect the detection results of the internal sensor 161 in the driving control signal when generating the driving control signal.

[0086] (F11) In the fifth embodiment described above, detection results output from an internal sensor 161 mounted on the vehicle 100 may be used in at least one of the route generation and the driving control signal generation. For example, the vehicle 100 may acquire the detection results from the internal sensor 161 and reflect the detection results from the internal sensor 161 in the route when generating the route. The vehicle 100 may acquire the detection results from the internal sensor 161 and reflect the detection results from the internal sensor 161 in the driving control signal when generating the driving control signal.

[0087] (F12) In the fifth embodiment described above, the vehicle 100 acquires its position information using the detection results of the external sensor 301. Alternatively, the vehicle 100 may acquire basic position information using the detection results of the internal sensor 161 mounted on the vehicle 100, determine the next target position to which the vehicle 100 should go, generate a route from the vehicle 100's current location to the target position as shown in the acquired basic position information, generate a driving control signal for traveling along the generated route, and control the actuator group using the generated driving control signal. In this case, the vehicle 100 can travel without using the detection results of the external sensor 301 at all. The vehicle 100 may also acquire target arrival time and congestion information from outside the vehicle 100 and reflect the target arrival time and congestion information in at least one of the route and the driving control signal. Furthermore, all the functional configurations of the system 50 may be provided on the vehicle 100. That is, the processing realized by the system 50 shown in this disclosure may be realized by the vehicle 100 alone.

[0088] (F13) In the first embodiment described above, the server device 200 automatically generates a driving control signal to be transmitted to the vehicle 100. In addition to this, or instead, the server device 200 may generate a driving control signal according to manual operation by an operator located outside the vehicle 100. For example, the operator may operate a control device that includes a display for displaying an image output from an external sensor 301, a steering wheel for remotely controlling the vehicle 100, an accelerator pedal, a brake pedal, and a communication device for communicating with the server device 200 via wired or wireless communication, and the server device 200 may generate a driving control signal according to the operation applied to the control device.

[0089] (F14) In each of the above embodiments, the vehicle 100 only needs to have a configuration that allows it to move by unmanned operation, and may take the form of a platform having the configuration described below. Specifically, in order for the vehicle 100 to perform the three functions of "driving," "turning," and "stopping" by unmanned operation, it is sufficient to have at least a vehicle control device 110 and a group of actuators. If the vehicle 100 acquires information from the outside for unmanned operation, the vehicle 100 may further have a communication device 150. That is, the vehicle 100 that can move by unmanned operation does not need to have at least some of the interior parts such as seats and dashboards attached, at least some of the exterior parts such as bumpers and fenders attached, and does not need to have a body shell attached. In this case, the remaining parts such as the body shell may be attached to the vehicle 100 before the vehicle 100 is shipped from the factory FC, or the remaining parts such as the body shell may be attached to the vehicle 100 after the vehicle 100 has been shipped from the factory FC without the remaining parts such as the body shell being attached. Each component may be attached to the vehicle 100 from any direction, such as the top, bottom, front, rear, right, or left side, and may be attached from the same direction or from different directions. The positioning of the platform can also be determined in the same way as for the vehicle 100 in each of the above embodiments.

[0090] (F15) In each of the above embodiments, the vehicle 100 may be manufactured by combining multiple modules. A module means a unit composed of multiple parts grouped together according to the part or function of the vehicle 100. For example, the platform of the vehicle 100 may be manufactured by combining a front module that constitutes the front part of the platform, a central module that constitutes the central part of the platform, and a rear module that constitutes the rear part of the platform. The number of modules that constitute the platform is not limited to three, but may be two or fewer, or four or more. In addition to, or instead of, the parts that constitute the platform may be modularized, as well as parts that constitute parts of the vehicle 100 that are different from the platform. Furthermore, various modules may include any exterior parts such as bumpers and grilles, or any interior parts such as seats and consoles. Moreover, not limited to the vehicle 100, any form of mobile body may be manufactured by combining multiple modules. Such modules may be manufactured, for example, by joining multiple parts by welding or fasteners, or by integrally molding at least a part of the parts that constitute the module as a single part by casting. A molding technique for integrally molding a single component, especially a relatively large component, is also called gigacast or megacast. For example, the front module, central module, and rear module mentioned above may be manufactured using gigacast.

[0091] (F16) Transporting vehicle 100 using the unmanned operation of vehicle 100 is also called "autonomous transport." The configuration for realizing autonomous transport is also called a "vehicle remote control autonomous driving transport system." Furthermore, a production method that uses autonomous transport to produce vehicle 100 is also called "autonomous production." In autonomous production, for example, at a factory that manufactures vehicle 100, at least a portion of the transport of vehicle 100 is realized by autonomous transport.

[0092] This disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from its spirit. For example, the technical features in the embodiments corresponding to the technical features in each form described in the summary of the invention can be replaced or combined as appropriate in order to solve some or all of the above-described problems, or to achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate. [Explanation of Symbols]

[0093] 50, 50b, 50c…System, 100, 100b…Vehicle, 110, 110b…Vehicle control device, 111, 111b…Processor, 112, 112b…Memory, 113…Input / Output interface, 114…Internal bus, 115, 115b…Vehicle control unit, 120…Drive unit, 130…Steering unit, 140…Brake unit, 150…Communication device, 160…Internal sensor group, 161…Internal sensor, 200, 200b…Server device, 2 01…Processor, 202,202b…Memory, 203…Input / Output Interface, 204…Internal Bus, 205…Communication Device, 210…Remote Control Unit, 215…Acquisition Unit, 220…Estimation Unit, 225,225b…Control Command Generation Unit, 235,235b…Decision Unit, 240,240b…Judgment Unit, 300…External Sensor Group, 301…External Sensor, 302…Communication Device, 400…External Complementary Sensor, 402…Communication Device, 500…Process Control Device

Claims

1. An acquisition unit that acquires the status of a mobile body, which is capable of moving by unmanned operation, and the environment surrounding the mobile body, An estimation unit that performs an estimation process to estimate at least one of the position and orientation of the moving object using sensor information acquired using a sensor, A control command generation unit generates and outputs a control command for controlling the movement of the moving body using the estimation results of the estimation process, The system includes a determination unit that determines whether to execute the first process or the second process as the estimation process, depending on the acquired status of the moving object, The first process is a process that produces a higher accuracy of the estimation result than the second process, and the second process is a process that takes less time than the first process. The sensor information is external information acquired using an external sensor located outside the moving body. The decision unit decides to execute the first process if the condition of the moving body matches a predetermined first condition, and decides to execute the second process if the condition of the moving body does not match the first condition. The first situation described above is: The situation in which the aforementioned moving object is waiting to move, A situation in which the moving speed of the aforementioned moving body is less than or equal to a predetermined first reference moving speed, A situation in which the communication speed between a server device located outside the mobile body and the mobile body is less than or equal to a predetermined reference communication speed, having the acquisition unit, estimation unit, control command generation unit, and determination unit, A situation in which the cumulative degree of external force applied to the moving body exceeds a predetermined level, This includes at least one of the following situations: the time the mobile body is continuously operated is equal to or greater than a predetermined operating time; system.

2. An acquisition unit that acquires a mobile body status representing at least one of the state of a mobile body that can be moved by unmanned operation and the environment surrounding the mobile body, An estimation unit that performs an estimation process to estimate at least one of the position and orientation of the moving object using sensor information acquired using a sensor, A control command generation unit generates and outputs a control command for controlling the movement of the moving body using the estimation results of the estimation process, The system includes a determination unit that determines whether to execute the first process or the second process as the estimation process, depending on the acquired status of the moving object, The first process is a process that produces a higher accuracy of the estimation result than the second process, and the second process is a process that takes less time than the first process. The determination unit decides to execute the second process if the moving body condition matches a predetermined second condition, and decides to execute the first process if the moving body condition does not match the second condition. The second situation described above is, A situation in which the moving speed of the aforementioned moving body is greater than a predetermined second reference moving speed, This includes at least one of the following situations: a complementary sensor different from the aforementioned sensor is used to complement the estimation result with a weight greater than or equal to a predetermined weight; The control command generation unit generates the control command using the estimation result that has been supplemented using the supplementing sensor when the supplementing sensor is used to supplement the estimation result.

3. An acquisition unit that acquires the status of a mobile body, which is capable of moving by unmanned operation, and the surrounding environment of the mobile body, An estimation unit that performs an estimation process to estimate at least one of the position and orientation of the moving object using sensor information acquired using a sensor, A control command generation unit generates and outputs a control command for controlling the movement of the moving body using the estimation results of the estimation process, The system includes a determination unit that determines whether to execute the first process or the second process as the estimation process, depending on the acquired status of the moving object, The first process involves using the sensor information and a predetermined algorithm to calculate at least one of the position and orientation of the moving object a predetermined first number of times, and outputting the estimation result based on each of the calculated results. The second process involves using the sensor information and the algorithm to calculate at least one of the position and orientation of the moving object a predetermined second number of times, and outputting the estimation result based on each of the calculated results. A system in which the first count is greater than the second count.

4. A mobile vehicle that can be moved by unmanned operation, An actuator that controls the movement of the moving body, An acquisition unit that acquires the state of the moving body and the environment surrounding the moving body, An estimation unit that performs an estimation process to estimate at least one of the position and orientation of the moving object using sensor information acquired using a sensor, A control command generation unit generates and outputs a control command for operating the actuator using the estimation results of the estimation process, The system includes a determination unit that determines whether to execute the first process or the second process as the estimation process, depending on the status of the moving object, The first process is a process that produces a higher accuracy of the estimation result than the second process, and the second process is a process that takes less time than the first process. The decision unit decides to execute the first process if the moving body condition matches a predetermined first condition, and decides to execute the second process if the moving body condition does not match the first condition. The first situation described above is: The situation in which the aforementioned moving object is waiting to move, A situation in which the moving speed of the aforementioned moving body is less than or equal to a predetermined first reference moving speed, A situation in which the cumulative degree of external force applied to the moving body exceeds a predetermined level, This includes at least one of the following situations: the time the mobile body is continuously operated is equal to or greater than a predetermined operating time; A mobile object.

Images