System, and, moving body

Abstract

To appropriately control movement of a moving body by means of remote control or autonomous control.SOLUTION: A system comprises: an acquisition unit which acquires a moving-body situation indicating at least one of a state of a moving body movable by unmanned driving and acquires an environment around the moving body; an inference unit performing an inference process of inferring at least one of a position and a direction of the moving body by using sensor information acquired by means of a sensor; a control command generation unit generating and outputting a control command for controlling movement of the moving body by using the inference result of the inference process; and a determination unit determining which of a first process and a second process is performed as the inference process, according to the acquired moving-body situation. The first process is a process in which accuracy of an inference result is higher than that in the second process, and the second process is a process in which a processing time is shorter than that in the first process.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a system and a mobile object. [Background technology]

[0002] Patent Document 1 discloses a technique for running a vehicle autonomously or by remote control during the vehicle manufacturing process. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2017-538619 Summary of the Invention [Problem to be solved by the invention]

[0004] In order to appropriately control the movement of a moving object such as a vehicle by remote control or autonomous control, it is desirable to detect the position and orientation of the moving object with high accuracy. However, if the time spent on detecting the position and orientation increases, the movement of the moving object may not be appropriately controlled due to the time lag between the start of detection and the completion of detection. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to a first aspect of the present disclosure, a system is provided. The system includes an acquisition unit that acquires a mobile body status indicating at least one of a state of a mobile body that can move by unmanned driving and an environment around the mobile body, an estimation unit that executes an estimation process to estimate at least one of a position and an orientation of the mobile body using sensor information acquired by a sensor, a control command generation unit that generates and outputs a control command for controlling the movement of the mobile body using an estimation result of the estimation process, and a determination unit that determines whether a first process or a second process is to be executed as the estimation process according to the acquired mobile body status. The first process is a process that produces an estimation result with higher accuracy than the second process, and the second process is a process that has a shorter processing time than the first process. According to this form of system, a determination is made as to whether a first process, which has a higher estimation accuracy, or a second process, which has a shorter processing time, will be executed depending on the status of the moving body, thereby increasing the likelihood that the movement of the moving body will be appropriately controlled by remote control or autonomous control. (2) In the above form, the vehicle may include a server device having the acquisition unit, the estimation unit, the control command generation unit, and the determination unit and provided outside the moving body, wherein the sensor information is external information acquired using an external sensor provided outside the moving body, and the determination unit decides to execute the first process when the moving body status matches a predetermined first status, and decides to execute the second process when the moving body status does not match the first status, and the first status may include at least one of a status in which the moving body is waiting to move, a status in which the moving speed of the moving body is equal to or less than a predetermined first reference moving speed, a status in which the communication speed between the server device and the moving body is equal to or less than a predetermined reference communication speed, a status in which the cumulative degree of external force applied to the moving body is equal to or greater than a predetermined degree, and a status in which the time the moving body is continuously operated is equal to or greater than a predetermined operating time. According to the system of this embodiment, the position and orientation can be estimated more accurately in a situation where there is a relatively large time margin for estimating the position and orientation by the estimation process, or in a situation where it is preferable to estimate the position and orientation more accurately by the estimation process, thereby increasing the possibility that the movement of the moving object can be appropriately controlled by remote control. (3) In the above embodiment, the determination unit determines to execute the second process when the moving object situation matches a predetermined second situation, and determines to execute the first process when the moving object situation does not match the second situation, and the second situation includes at least one of a situation in which the moving speed of the moving object is higher than a predetermined second reference moving speed and a situation in which a complementary sensor different from the sensor is used with a weight equal to or higher than a predetermined weight to complement the estimation result, and the control command generation unit may generate the control command using the estimation result complemented by the complementary sensor when the complementary sensor is used to complement the estimation result. According to the system of this embodiment, the position and orientation are estimated more quickly by the estimation process in a situation in which the time allowance for estimating the position and orientation of the moving object by the estimation process is relatively small or in a situation in which the estimation result is complemented by a complementary sensor. Therefore, the time lag from the start to the completion of the estimation of the position and orientation in the estimation process can be suppressed, and the possibility that the movement of the moving object is appropriately controlled by remote control or autonomous control can be increased. (4) According to a second aspect of the present disclosure, a system is provided. The system includes an acquisition unit that acquires a mobile body status indicating at least one of a state of a mobile body that can move by unmanned driving and an environment surrounding the mobile body, an estimation unit that executes an estimation process to estimate at least one of a position and an orientation of the mobile body using sensor information acquired by a sensor, a control command generation unit that generates and outputs a control command for controlling the movement of the mobile body using an estimation result of the estimation process, and a determination unit that determines whether a first process or a second process is to be executed as the estimation process according to the acquired mobile body status. The first process is a process of calculating at least one of a position and an orientation of the mobile body a predetermined first number of times using the sensor information and a predetermined algorithm, and outputting the estimation result based on each calculated calculation result, and the second process is a process of calculating at least one of a position and an orientation of the mobile body a predetermined second number of times using the sensor information and the algorithm, and outputting the estimation result based on each calculated calculation result, the first number of times being greater than the second number of times. According to this form of system, depending on the status of the moving body, it is determined whether a first process, which involves a greater number of calculations of position and orientation, or a second process, which involves a smaller number of calculations of position and orientation, will be executed, thereby increasing the likelihood that the movement of the moving body will be appropriately controlled by remote control or autonomous control. (5) According to a third embodiment of the present disclosure, a moving body capable of moving by unmanned driving is provided. The moving body includes an actuator for controlling the movement of the moving body, an acquisition unit for acquiring a moving body status indicating at least one of a state of the moving body and an environment around the moving body, an estimation unit for executing an estimation process for estimating at least one of a position and an orientation of the moving body using sensor information acquired by a sensor, a control command generation unit for generating and outputting a control command for operating the actuator using an estimation result of the estimation process, and a determination unit for determining whether a first process or a second process is to be executed as the estimation process according to the moving body status. The first process is a process in which the estimation result is more accurate than the second process, and the second process is a process in which the processing time is shorter than the first process. According to the moving body of this embodiment, it is determined whether the first process, which has a higher accuracy of the estimation result, or the second process, which has a shorter processing time, is to be executed according to the moving body status, so that the possibility that the movement of the moving body is appropriately controlled by remote control or autonomous control can be increased. The present disclosure can be realized in the form of, for example, a server device, a control method, a program for implementing the control method, a non-transient recording medium on which the program is recorded, a program product, etc. The program product may be provided, for example, as a recording medium on which the program is recorded, or as a program product that can be distributed via a network. In addition, for example, the above-mentioned system and mobile body may estimate the position and orientation of the mobile body using a machine learning model that has been machine-learned in advance in the estimation process. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is an explanatory diagram showing a configuration of a system in a first embodiment. [Diagram 2] FIG. 2 is an explanatory diagram showing a state in which a vehicle moves in an unmanned manner in a factory. [Figure 3A] 11 is a flowchart of a first unmanned operation process. [Figure 3B]4 is a flowchart of a position information acquisition process in the first embodiment. [Figure 4] FIG. 11 is an explanatory diagram showing a configuration of a system in a second embodiment. [Diagram 5] 13 is a flowchart of a position information acquisition process in the second embodiment. [Figure 6] 13 is a flowchart of a position information acquisition process in the third embodiment. [Figure 7] FIG. 13 is an explanatory diagram showing the configuration of a system according to a fifth embodiment. [Figure 8] 13 is a flowchart of a second unmanned operation process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A. First embodiment: 1 is an explanatory diagram showing the configuration of a system 50 in a first embodiment. The system 50 in this embodiment is configured as an unmanned driving system, and is used to move moving objects by unmanned driving in a factory that manufactures moving objects.

[0009] In this embodiment, the moving body is a vehicle 100. More specifically, the vehicle 100 is an electric vehicle (BEV: Battery Electric Vehicle). The moving body is not limited to an electric vehicle, and may be, for example, a gasoline-powered vehicle, a hybrid vehicle, or a fuel cell vehicle. The moving body may be a vehicle having wheels or a vehicle having tracks, and may be any vehicle, such as a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, or a construction vehicle. The moving body is not limited to a vehicle, and may be an electric vertical take-off and landing aircraft (a so-called flying car).

[0010] The above-mentioned "unmanned driving" means driving without the driving operation of a passenger. The driving operation means at least one of the operations of "running", "turning" and "stopping" of the vehicle 100. The unmanned driving is realized by automatic or manual remote control using a device provided outside the vehicle 100, or by autonomous control of the vehicle 100. The vehicle 100 traveling by unmanned driving may have a passenger who does not perform driving operations on board. The passenger who does not perform driving operations includes, for example, a person who simply sits in the seat of the vehicle 100, and a person who performs work other than driving operations, such as assembly, inspection, and operation of switches, while riding in the vehicle 100. Note that driving by a passenger's driving operation is sometimes called "manned driving".

[0011] As shown in FIG. 1, the system 50 in this embodiment includes a vehicle 100, a server device 200 for remotely controlling the vehicle 100, a group of external sensors 300 installed in a factory, and a process management device 500 for managing the manufacturing process of the vehicle 100 in the factory.

[0012] The vehicle 100 includes a vehicle control device 110 for controlling each part of the vehicle 100, a driving device 120 for accelerating the vehicle 100, a steering device 130 for changing the traveling direction of the vehicle 100, a braking device 140 for decelerating the vehicle 100, a communication device 150 for communicating with the server device 200 by wireless communication, and an internal sensor group 160. In this embodiment, the driving device 120 includes a battery, a running motor driven by the power of the battery, and a driving wheel rotated by the running motor. In addition, the driving device 120, the steering device 130, and the braking device 140 each include an actuator for operating the vehicle 100. In particular, the driving device 120, the steering device 130, and the braking device 140 include an actuator for controlling the movement of the vehicle 100. Hereinafter, the various actuators provided in the vehicle 100 are also referred to as an actuator group.

[0013] The internal sensor group 160 includes at least one internal sensor 161. Note that FIG. 1 illustrates only one internal sensor 161 included in the internal sensor group 160. The internal sensor 161 is a sensor mounted on the vehicle 100. The internal sensor 161 in this embodiment is an internal state sensor that detects the state of the vehicle 100. The state of the vehicle 100 is, for example, a motion state in which the vehicle 100 is in a stopped state, a running state, an accelerating state, a decelerating state, a straight running state, or a turning state. The internal sensor group 160 in this embodiment includes, as the internal sensor 161, a rotation speed sensor for measuring the rotation speed of a driving motor, an acceleration sensor for measuring the acceleration of the vehicle 100, a vehicle speed sensor for measuring the speed of the vehicle 100, and a yaw rate sensor for measuring the yaw axis angular velocity (yaw rate) of the vehicle 100. In other embodiments, the internal sensor group 160 may include, for example, a camera or LiDAR (Light Detection And Ranging) as the internal sensor 161. In this case, the camera or LiDAR that is the internal sensor 161 may be used as an internal environment sensor that detects the environment around the vehicle 100. Hereinafter, information detected by the internal sensor 161 is also referred to as internal sensor information. Each piece of internal sensor information is transmitted from the internal sensor 161 to the vehicle control device 110, and is also transmitted to the server device 200 via the communication device 150.

[0014] The vehicle control device 110 is configured by a computer including a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input / output interface 113 are connected via the internal bus 114 to enable bidirectional communication. The input / output interface 113 is connected to the drive device 120, the steering device 130, the braking device 140, the communication device 150, and the internal sensor group 160. The memory 112 stores a computer program PG1.

[0015] The processor 111 executes the computer program PG1 to function as the vehicle control unit 115. The vehicle control unit 115 controls the actuator group. In particular, the vehicle control unit 115 controls the driving device 120, the steering device 130, and the braking device 140 to make the vehicle 100 run. When a driver is on board the vehicle 100, the vehicle control unit 115 controls the driving device 120, the steering device 130, and the braking device 140 in response to the driver's operation to make the vehicle 100 run. Regardless of whether a driver is on board the vehicle 100 or not, the vehicle control unit 115 controls the driving device 120, the steering device 130, and the braking device 140 in response to a control command to be described later to make the vehicle 100 run.

[0016] The external sensor group 300 includes at least one external sensor 301. The external sensor group 300 in this embodiment has a plurality of external sensors 301, but FIG. 1 illustrates only one external sensor 301 included in the external sensor group 300. The external sensor 301 is a sensor located outside the vehicle 100. The external sensor 301 can acquire external information by detecting the motion state of the vehicle 100 from outside the vehicle 100. The external information is information representing the motion state of the vehicle 100 detected from outside the vehicle 100, and is used in the estimation process described later. In this embodiment, each external sensor 301 is configured by a camera installed in a factory, and captures a captured image including the vehicle 100 from outside the vehicle 100. This captured image is external information optically capturing the outer shape of the vehicle 100, and is external information representing the motion state of the vehicle 100 optically detected from outside the vehicle 100. Each external sensor 301 includes a communication device 302 and can communicate with the server device 200 by wired communication or wireless communication. Each piece of external information is transmitted to the server device 200 via the communication device 302. Note that the external sensor 301 may be used as a sensor that detects the environment around the vehicle 100. Hereinafter, information detected by the external sensor 301 is also collectively referred to as external sensor information. Furthermore, the external sensor information and the internal sensor information are also collectively referred to as sensor information.

[0017] 1, the server device 200 is configured by a computer including a processor 201, a memory 202, an input / output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input / output interface 203 are connected via the internal bus 204 to be able to communicate bidirectionally. A communication device 205 for communicating with the vehicle 100 by wireless communication is connected to the input / output interface 203. In this embodiment, the communication device 205 can communicate with the external sensor group 300 and the process management device 500 by wired communication or wireless communication.

[0018] The processor 201 executes a computer program PG2 stored in the memory 202, thereby functioning as a remote control unit 210, an acquisition unit 215, an estimation unit 220, a decision unit 235, and a judgment unit 240. In addition to the computer program PG2, the memory 202 also stores situation data SD, a first analysis program AP1, a second analysis program AP2, and an external shape detection model Md, which will be described later.

[0019] The acquisition unit 215 acquires the moving object status. The moving object status indicates at least one of the state of the moving object and the environment around the moving object. The moving object status is used to determine whether to execute a first process or a second process as an estimation process described later. Hereinafter, the moving object status regarding the vehicle 100 is also referred to as the vehicle status. The vehicle status includes, for example, the motion state of the vehicle 100, the location where the vehicle 100 runs, the time period during which the vehicle 100 runs, the type of manufacturing process being performed on the vehicle 100, the communication status between the vehicle 100 and the server device 200 (for example, the presence or absence of communication delay and communication speed), the cumulative degree of external force applied to the vehicle 100, and the driving time of the vehicle 100. The acquisition unit 215 acquires such vehicle status directly or indirectly from, for example, the internal sensor 161, the external sensor 301, the process management device 500, and each communication device.

[0020] In this embodiment, the determination unit 240 determines whether the vehicle situation satisfies a predetermined first situation. The first situation in this embodiment includes a waiting situation in which the vehicle 100 is waiting to move, a low-speed moving situation in which the moving speed of the vehicle 100 is equal to or lower than a predetermined first reference moving speed, a communication delay situation in which the communication speed between the server device 200 and the vehicle 100 is equal to or lower than a predetermined reference communication speed, an external force accumulation situation in which the accumulated degree of the external force applied to the vehicle 100 is equal to or higher than a predetermined reference level, and a long-term driving situation in which the continuous driving time during which the vehicle 100 is continuously driven is equal to or higher than a predetermined reference driving time. The situation data SD stores various reference values ​​and various information referred to by the determination unit 240 when determining the vehicle situation.

[0021] The fact that the vehicle status of a certain vehicle 100 is a waiting status is acquired, for example, from internal sensor information indicating that the vehicle speed of the vehicle 100 is zero, external sensor information indicating that the motion state of the vehicle 100 is a stopped state, sensor information indicating that the distance between the vehicle 100 and another vehicle 100 traveling ahead is equal to or less than a predetermined reference distance, or manufacturing information indicating that a process of making the vehicle 100 wait is being executed. The process of making the vehicle 100 wait is, for example, a process of performing a predetermined process on the vehicle 100 while the vehicle 100 is stopped, or a process of stopping the vehicle 100 to adjust the cycle time of manufacturing the vehicle 100 according to the takt time. The fact that the vehicle status is a low-speed moving status is acquired, for example, from internal sensor information indicating that the vehicle speed is equal to or less than a first reference moving speed, external sensor information indicating that the vehicle 100 is moving at or less than a first reference moving speed, or manufacturing information indicating that a process of making the vehicle 100 travel at or less than a reference moving speed is being executed. The fact that the vehicle status is a communication delay status is acquired, for example, from a communication log between the server device 200 and the vehicle 100. The fact that the vehicle status is an external force accumulation status is acquired, for example, by using an acceleration sensor as the internal sensor 161, the external sensor 301, or manufacturing information indicating that a process in which an external force is applied to the vehicle 100 is being performed. The external force referred to here is, for example, an external force applied to the vehicle 100 when a person gets into the vehicle 100, an external force applied to the vehicle 100 when the vehicle 100 runs on an uneven running road, an external force applied to the vehicle 100 when the vehicle 100 comes into contact with a guide installed on or around the running road SR, or an external force resulting from a manufacturing process performed on the vehicle 100. The fact that these external forces are applied to the vehicle 100 can be detected, for example, by using the external sensor 301 or manufacturing information. Therefore, such a cumulative degree of external force may be calculated as an integrated value of the sensor value of the internal sensor 161 related to the external force, for example, using the external sensor 301 and the internal sensor 161. The cumulative degree of external force may also be calculated as a sensor value of an acceleration sensor when the vehicle 100 of the external force is traveling straight on a flat road, using the external sensor 301 and the internal sensor 161. The sensor value of the acceleration sensor reflects a cumulative error caused by the external force as the cumulative degree of external force.Furthermore, the cumulative degree of the external force may be calculated as the number of times the external force is applied to the vehicle 100, using the external sensor 301 or manufacturing information. The fact that the vehicle state is a continuous driving state is acquired, for example, using the elapsed time from when the vehicle 100 starts traveling in an unmanned driving state until the current time.

[0022] The estimation unit 220 executes an estimation process. The estimation process is a process of estimating at least one of the position and the orientation of the vehicle 100 using sensor information. The estimation process in this embodiment is a process of estimating the position and the orientation of the vehicle 100 using external information as sensor information. Therefore, in this embodiment, the position and the orientation of the vehicle 100 are output as an estimation result of the estimation process. This estimation result is used to obtain basic position information of the vehicle 100. The basic position information is information that is the basis for generating a control command, and includes the position and the orientation of the vehicle 100. Hereinafter, the position estimated by the estimation process is also referred to as an estimated position, and the orientation estimated by the estimation process is also referred to as an estimated orientation. The estimation unit 220 selectively executes either a first process or a second process, which will be described later, as the estimation process.

[0023] Furthermore, the estimation unit 220 can calculate the position and orientation of the vehicle 100 using a complementary sensor different from the sensor used in the estimation process, without relying on the estimation process. In this case, the estimation unit 220 acquires the position and orientation of the vehicle 100 by calculating them using, for example, the vehicle speed, acceleration, and yaw rate detected by an internal state sensor serving as a complementary sensor. In addition to the sensor information acquired by the complementary sensor, the travel history of the vehicle 100 and the history of the control value used to control the movement of the vehicle 100 may be used to calculate the position and orientation. The position and orientation calculated in this manner can be used to complement the estimation result. Hereinafter, the position and orientation calculated using the internal sensor 161 serving as a complementary sensor are also referred to as first complementary information. The first complementary information in this embodiment includes the orientation and does not include the position. In other embodiments, the first complementary information may include the orientation and the position, or may include the position and not include the orientation. The above vehicle status is acquired by the acquisition unit 215 prior to the execution of the estimation process. Therefore, the vehicle status does not include the estimated position or the estimated orientation. In the present embodiment, the vehicle situation does not include the first complementary information.

[0024] The remote control unit 210 has a control command generation unit 225, which generates a control command and transmits it to the vehicle 100, thereby realizing the movement of the vehicle 100 by remote control. The control command generation unit 225 uses the basic position information to generate and output a control command for controlling the movement of the vehicle 100. As described above, the basic position information is acquired using the estimation result of the estimation process. Therefore, it can be said that at least the estimation result of the estimation process is used to generate the control command. In this embodiment, the control command generation unit 225 generates and outputs a travel control signal, which will be described later, as a control command.

[0025] The basic position information in this embodiment is obtained by appropriately complementing the estimation result with the first complementary information. Therefore, in this embodiment, it can be said that the first complementary information is appropriately used in addition to the estimation result to generate the control command. More specifically, the basic position information in this embodiment includes an estimated position and an estimated orientation appropriately complemented by the orientation as the first complementary information. In this embodiment, the first weight indicating the weight with which the first complementary information is used to complement the estimation result is changed according to the vehicle situation as described later. For example, the first weight is expressed as a weighting index defined such that when the first weight is zero, the estimated orientation is not complemented by the first complementary information, and when the first weight is 1, the estimated orientation is 100% complemented by the first complementary information. Note that, when the first complementary information in other embodiments includes the position and the orientation, the first weight may be expressed, for example, by weights set for each of the position and the orientation, or may be expressed as an index indicating the statistics of each weight or the magnitude of each weight.

[0026] The determination unit 235 determines whether to execute the first process or the second process as the estimation process according to the moving object status acquired by the acquisition unit 215. The first process is a process that produces a higher accuracy of the estimation result than the second process. The second process is a process that has a shorter processing time than the first process. In this specification, when the position and orientation are estimated by the estimation process as in this embodiment, "the estimation result has a high accuracy" means that the estimation accuracy of both the position and the orientation is high. The comparison (evaluation) of the accuracy of the estimation results of the two estimation processes is performed by performing 200 comparisons of the estimation results of each estimation process using the same external information. In this embodiment, for each estimation process, the number of times that the estimation accuracy of both the position and the orientation was higher than the other estimation process is recorded, and the estimation process that has the greater number of times recorded at the time when the 200 comparisons are completed may be defined as the estimation process with the higher accuracy. Similarly, the comparison (evaluation) of the processing time of the two estimation processes is performed by comparing the processing time of each estimation process using the same external information 200 times. In this case, the processing time is calculated as the time from when the estimation process is started to when the estimation process is completed.

[0027] The process management device 500 is a device for managing the manufacturing process of the vehicle 100 in a factory. The process management device 500 is configured with at least one computer. The process management device 500 is equipped with a communication device (not shown) and can communicate with the server device 200 and various equipment in the factory by wired communication or wireless communication. The process management device 500 grasps when, where, who, and which vehicle 100 are scheduled to perform what work, and when, where, who, and which vehicle 100 have performed what work. In this embodiment, when one vehicle 100 starts traveling along a target route, individual information indicating an identification number, model, etc. that identifies the vehicle 100 is transmitted from the process management device 500 to the server device 200. This individual information corresponds to manufacturing information used to manage the manufacturing process of the vehicle 100. The server device 200 can identify the vehicle (target vehicle) 100 that is the target of unmanned driving by using this individual information.

[0028] FIG. 2 is an explanatory diagram showing a state in which the vehicle 100 moves in an unmanned manner in a factory FC. The factory FC has a first location PL1 where a first task is performed, and a second location PL2 where a second task is performed. The first location PL1 and the second location PL2 are connected by a travel path SR on which the vehicle 100 can travel. A plurality of external sensors 301 are provided around the travel path SR. In this embodiment, the first location PL1 is a location where the vehicle 100 is assembled. The second location PL2 is a location where the vehicle 100 is inspected. The vehicle 100 that has passed the inspection at the second location PL2 is shipped from the factory FC. Each vehicle 100 assembled at the first location PL1 is transported to a predetermined start point at the first location PL1 with the vehicle control device 110, the drive device 120, the steering device 130, the braking device 140, and the communication device 150 attached. That is, in this embodiment, each vehicle 100 is placed at a start point in the first location PL1 in a state in which it can run by remote control. Each vehicle 100 runs from the start point in the first location PL1 to a predetermined goal point in the second location PL2 by remote control by the server device 200.

[0029] Fig. 3A is a flowchart of the unmanned driving process executed in this embodiment. The unmanned driving process is a process for driving the vehicle 100 by unmanned driving. Fig. 3A can also be said to represent a driving method of the vehicle 100 by unmanned driving in this embodiment.

[0030] In S10, the remote control unit 210 of the server device 200 acquires basic position information of the vehicle 100 using the detection result output from the external sensor 301. In this embodiment, the basic position information of the vehicle 100 includes the position and orientation of the vehicle 100 in the global coordinate system of the factory FC. In this embodiment, the external sensor 301 is a camera installed in the factory FC, and in S10, a captured image is output as the detection result. Note that the position and orientation of the external sensor 301 in the factory FC are adjusted in advance. The remote control unit 210 detects the vehicle 100 from the image acquired from the external sensor 301, and acquires basic position information from the position of the vehicle 100 in the image.

[0031] In detail, in S10 in this embodiment, the remote control unit 210 detects the outer shape of the vehicle 100 from the captured image, calculates the coordinates of the positioning point of the vehicle 100 in the image coordinate system, which is the coordinate system of the captured image, and converts the calculated coordinates into coordinates in the global coordinate system to obtain the position of the vehicle 100 as basic position information. The positioning point in this embodiment is the left rear corner of the vehicle 100, but in other embodiments, it may be any part of the vehicle 100. In this embodiment, the outer shape of the vehicle 100 included in the captured image is detected by inputting the captured image to the outer shape detection model Md. The outer shape detection model Md is a detection model that utilizes artificial intelligence. Specifically, the outer shape detection model Md is, for example, a trained machine learning model that has been trained to realize either semantic segmentation or instance segmentation. As this machine learning model, for example, a convolutional neural network (hereinafter, simply referred to as CNN) trained by supervised learning using a learning dataset can be used. The learning dataset has, for example, a plurality of training images including the vehicle 100, and labels indicating whether each region in the training images is a region indicating the vehicle 100 or a region indicating something other than the vehicle 100. When learning the CNN, it is preferable to update the parameters of the CNN by backpropagation (error backpropagation method) so as to reduce the error between the output result by the detection model and the label. In other embodiments, the outer shape detection model Md is not limited to the CNN, and may be, for example, a trained machine learning model other than a neural network.

[0032] Also, in S10, the remote control unit 210 can obtain the orientation of the vehicle 100 as basic position information by, for example, utilizing an optical flow method to estimate based on the orientation of a movement vector of the vehicle 100 calculated from position changes of feature points of the vehicle 100 between frames of captured images. The orientation of the vehicle 100 is calculated as a direction vector that can identify the orientation of the vehicle 100 at least in the horizontal direction, for example, as the orientation of a vector going from the rear side to the front side of the vehicle 100.

[0033] In addition, in S10 in this embodiment, specifically, a location information acquisition process, which will be described later, is executed to acquire basic location information of the vehicle 100.

[0034] At S20, the remote control unit 210 determines a target position to which the vehicle 100 should next head. In this embodiment, the target position is represented by X, Y, and Z coordinates in a global coordinate system. A reference route, which is a route along which the vehicle 100 should travel, is stored in advance in the memory 202. The route is represented by nodes indicating a departure point, nodes indicating passing points, nodes indicating a destination, and links connecting the nodes. The remote control unit 210 uses the basic position information of the vehicle 100 and the reference route to determine a target position to which the vehicle 100 should next head. The remote control unit 210 determines a target position on the reference route that is ahead of the current location of the vehicle 100.

[0035] At S30, the remote control unit 210 generates a travel control signal for driving the vehicle 100 toward the determined target position by the control command generation unit 225. In this embodiment, the travel control signal includes the acceleration and steering angle of the vehicle 100 as parameters. The remote control unit 210 calculates the travel speed of the vehicle 100 from the transition of the position of the vehicle 100, and compares the calculated travel speed with a predetermined target speed of the vehicle 100. When the travel speed is lower than the target speed, the remote control unit 210 determines the acceleration so that the vehicle 100 accelerates, and when the travel speed is higher than the target speed, the remote control unit 210 determines the acceleration so that the vehicle 100 decelerates. When the vehicle 100 is located on the reference route, the remote control unit 210 determines the steering angle so that the vehicle 100 does not deviate from the reference route, and when the vehicle 100 is not located on the reference route, in other words, when the vehicle 100 deviates from the reference route, the remote control unit 210 determines the steering angle so that the vehicle 100 returns to the reference route.

[0036] In S40, remote control unit 210 transmits the generated driving control signal to vehicle 100. Remote control unit 210 repeats, at a predetermined cycle, obtaining position information of vehicle 100, determining a target position, generating the driving control signal, and transmitting the driving control signal.

[0037] At S50, the vehicle control unit 115 receives a driving control signal from the remote control unit 210. At S60, the vehicle control unit 115 controls the actuator group using the received driving control signal to cause the vehicle 100 to drive at the acceleration and steering angle represented by the driving control signal. The vehicle control unit 115 repeats receiving the driving control signal and controlling the actuator group at a predetermined cycle. According to the system 50 of this embodiment, the vehicle 100 can be driven by remote control, and therefore the vehicle 100 can be moved without using transportation equipment such as a crane or a conveyor.

[0038] Fig. 3B is a flowchart of the location information acquisition process in this embodiment. The location information acquisition process is a process for acquiring basic location information of the vehicle 100. As described above, the location information acquisition process in Fig. 3B is executed in order to acquire the basic location information in S10 in Fig. 3A.

[0039] At S110, the acquisition unit 215 acquires a vehicle status of the target vehicle 100. At S115, the determination unit 240 determines whether or not the vehicle status acquired at S110 matches a first status. If the first condition is satisfied at S115, the decision unit 235 decides to execute the first process as the estimation process at S120.

[0040] From S125 to S140, the estimation unit 220 executes the first process determined in S120 as the estimation process. First, in S125, the estimation unit 220 acquires a captured image as external information from the external sensor 301. In S130, the estimation unit 220 calculates the position and orientation of the target vehicle 100 by analyzing the captured image acquired in S125 using the first analysis program AP1. The first analysis program AP1 is a program for causing a computer to realize an outer shape detection process for detecting the outer shape of the target vehicle 100 in the captured image, a polygon approximation process for approximating the outer shape detected by the outer shape detection process with a polygon, and a first calculation process for calculating the position and orientation of the vehicle 100 using the approximation result of the polygon approximation process. Hereinafter, the process of calculating the position and orientation of the vehicle 100 by analyzing the captured image as in S130 is also referred to as an analysis process.

[0041] As described above, in the outer shape detection process in this embodiment, the captured image is input to the outer shape detection model Md to detect the outer shape of the target vehicle 100. The outer shape detection model Md detects the area of ​​the vehicle 100 included in the input image by segmentation, and detects the outer shape of the target vehicle 100 by masking the area of ​​the vehicle 100 included in the input image. In addition, in the first calculation process, the coordinates of the positioning point in the image coordinate system are calculated using the approximation result of the polygon approximation process, and the calculated coordinates are converted into coordinates in the global coordinate system using a predetermined conversion matrix or conversion formula.

[0042] In S135, the estimation unit 220 judges whether the number of times the analysis process in S130 has been performed is equal to or greater than a first number of times. In this embodiment, the first number of times is equal to or greater than two times. If the number of times S130 has been performed is less than the first number of times, the estimation unit 220 returns the process to S125. In S125 that is executed again, the estimation unit 220 acquires a captured image captured by the external sensor 301 after the time of the previously executed S125. That is, in this embodiment, the sensor information used in each of the repeatedly executed analysis processes is different sensor information. In S137, the estimation unit 220 outputs a statistic obtained by statistically processing each calculation result of the first number of analysis processes as an estimation result. In this embodiment, the statistic is a median. In other embodiments, the statistic may be, for example, an arithmetic mean value. In S140, the estimation unit 220 outputs the estimation result. In S140 in this embodiment, the estimation unit 220 outputs the statistics calculated in S137 as an estimation result. That is, in S137, an estimation result is output based on each calculation result of the analysis process in S130 that is repeatedly executed. By outputting the statistics as an estimation result in this way, it is possible to reduce the influence of disturbances in the captured image and further improve the accuracy of the estimation result.

[0043] If the vehicle situation does not match the first condition in S115, the decision unit 235 decides to execute the second process as the estimation process in S145.

[0044] From S150 to S160, the estimation unit 220 executes the second process determined in S150 as the estimation process. First, in S150, the estimation unit 220 acquires a captured image as external information from the external sensor 301. In S155, the estimation unit 220 calculates the position and orientation of the target vehicle 100 by analyzing the captured image acquired in S150 using the second analysis program AP2. The second analysis program AP2 is a program for causing a computer to realize an outline detection process similar to S130, a circumscribing rectangle approximation process that approximates the detected outline by a circumscribing rectangle, and a second calculation process that calculates the position and orientation using the approximation result of the circumscribing rectangle approximation process. Therefore, in this embodiment, in the outline detection processes in S155 and S130, the outline of the target vehicle 100 is detected by the same algorithm using the same outline detection model Md. In addition, in the second calculation process, the coordinates of the positioning point in the image coordinate system are calculated using the approximation results from the circumscribing rectangle approximation process, and the calculated coordinates are converted into coordinates in the global coordinate system using a predetermined transformation matrix or transformation formula.

[0045] The circumscribing rectangle approximation process in the analysis process of S155 is a process that has a lower accuracy of the approximation result and a faster processing speed than the polygon approximation process in the analysis process of S130. As a result, the accuracy of the position and orientation calculated by the analysis process of S155 is lower than the accuracy of the position and orientation calculated by the analysis process of S130. Also, the processing time of the analysis process of S155 is shorter than the processing time of the analysis process of S130. That is, in this embodiment, the analysis process of S130 is a process that has a higher calculation accuracy of the position and orientation and a longer processing time than the analysis process of S155. Note that the accuracy and processing speed of the approximation process such as the polygon approximation process and the circumscribing rectangle approximation process, and the accuracy and processing speed of the analysis process can be evaluated in the same manner as the accuracy and processing time of the estimation process, for example.

[0046] In S160, the estimation unit 220 outputs an estimation result based on the calculation result of the analysis process in S155. In S160 in this embodiment, the calculation result of the analysis process in S155 is output as is. In the following, the number of times the analysis process is executed in the second process is also referred to as the second number of times. In this embodiment, the second number of times is less than the first number of times, and specifically, is one time.

[0047] As described above, in this embodiment, the analysis process in the first process is more accurate and takes longer than the analysis process in the second process. Moreover, in the first process in this embodiment, the analysis process is executed a first number of times, which is greater than the second number of times. Therefore, the first process in this embodiment is more accurate and takes longer than the second process.

[0048] In S165, the control command generating unit 225 determines the first weight to be a predetermined weight greater than zero, thereby complementing the estimation result output in S160 with the first complement information. That is, in this embodiment, when the vehicle situation does not match the first situation, the estimation result is complemented with the first complement information. The first situation in this embodiment can also be said to be a situation in which complementation of the estimation result with the first complement information is not performed. Note that the time required from the start of S150 to the completion of S165 is shorter than the time required from the start of S125 to the completion of S140b. Also, as described above, in this embodiment, when the vehicle situation matches the first situation, the estimation result is not complemented. Therefore, in this case, it can also be said that the first weight is zero. Then, in S20 and S30 of the above FIG. 3A, the control command generating unit 225 uses the estimation result output in S140 or the estimation result complemented in S165 as basic position information to generate and output a traveling control signal as a control command.

[0049] According to the system 50 in the present embodiment described above, it is determined whether the first process, which has a higher accuracy of the estimation result than the second process, or the second process, which has a shorter processing time than the first process, is to be executed as the estimation process, depending on the vehicle situation. In this way, it is possible to increase the possibility that an appropriate control command is generated using the estimation result from the estimation process, and therefore it is possible to increase the possibility that the movement of the vehicle 100 is appropriately controlled by remote control.

[0050] In addition, in this embodiment, when the vehicle situation of the target vehicle 100 satisfies the first situation, the execution of the first process is determined. This first situation includes a waiting situation, a low-speed moving situation, a communication delay situation, an external force accumulation situation, and a long-time driving situation. In this way, the position and orientation are estimated with higher accuracy in a situation in which the time allowance for estimating the position and orientation of the vehicle 100 by the estimation process is relatively large, or in a situation in which it is preferable to estimate the position and orientation more accurately by the estimation process. Therefore, it is possible to increase the possibility that the movement of the vehicle 100 is appropriately controlled by remote control. Specifically, in a low-speed moving situation, the time allowance for estimating the position and orientation of the vehicle 100 is large compared to a situation in which the vehicle 100 is traveling at a speed faster than the first reference moving speed. In addition, in a waiting situation, this time allowance is larger. In addition, in a communication delay situation, it takes more time for the server device 200 to receive internal sensor information compared to a situation in which the communication speed is faster than the reference communication speed, so it is preferable to estimate the position and orientation with higher accuracy by the estimation process using external information. In addition, in an external force accumulation situation, compared to a case where the accumulated degree of external force is less than a reference level, there is a higher possibility that an error (e.g., measurement error or drift error) of the internal state sensor will be larger due to an external force applied to the vehicle 100, so it is preferable to estimate the position and orientation with higher accuracy by an estimation process using external information. In addition, in a long-term driving situation, compared to a situation where the continuous driving time is shorter than the reference driving time, there is a higher possibility that an error of the internal state sensor will be larger due to an increase in the temperature of the internal state sensor, so it is preferable to estimate the position and orientation with higher accuracy by an estimation process using external information.

[0051] Moreover, when the first condition is not satisfied, the second process is determined, so that the position and orientation are estimated more quickly in the estimation process. This reduces the time lag from when the estimation of the position and orientation is started to when it is completed in the estimation process, and increases the possibility that the movement of the vehicle 100 is appropriately controlled by remote control.

[0052] B. Second embodiment: 4 is an explanatory diagram showing the configuration of a system 50b in the second embodiment. In this embodiment, unlike the first embodiment, the decision unit 235b decides to execute the second process when the vehicle situation matches a predetermined second situation, and decides to execute the first process when the vehicle situation does not match the second situation.

[0053] The system 50b includes an external complementary sensor 400. The external complementary sensor 400 is a sensor located outside the vehicle 100 and used to complement the estimation result. The external complementary sensor 400 detects the vehicle 100 from outside the vehicle 100. In this embodiment, the external complementary sensor 400 is configured by a LiDAR installed in a factory FC. The external complementary sensor 400 measures the distance to the vehicle 100 located in a predetermined section of the travel path SR and acquires three-dimensional point cloud data of the vehicle 100. This three-dimensional point cloud data is information representing the motion state of the vehicle 100 optically detected from outside the vehicle 100. The external complementary sensor 400 includes a communication device 402 and can communicate with the server device 200b by wired communication or wireless communication. The information acquired by the external complementary sensor 400 is transmitted to the server device 200 via the communication device 402.

[0054] The estimation unit 220 can acquire the position and orientation of the vehicle 100 using the result of distance measurement by the external complementary sensor 400 and the three-dimensional point cloud data acquired by the external complementary sensor 400. When the three-dimensional point cloud data is used to acquire the position and orientation, the estimation unit 220 acquires the position and orientation of the vehicle 100 by performing template matching using, for example, the three-dimensional point cloud data and the template point cloud data TP stored in the memory 202b. For the template matching, various algorithms such as ICP (Iterative Closest Point) and NDT (Normal Distributions Transform) can be used. The position and orientation acquired in this way can be used to complement the estimation result. That is, the external complementary sensor 400 can be used as a complementary sensor for complementing the estimation result. Hereinafter, the position and orientation acquired by using the external complementary sensor 400 are also referred to as second complementary information. The basic position information in this embodiment is acquired by appropriately complementing the estimation result with the first complementary information and the second complementary information. The second complementary information in this embodiment includes the position and the orientation. In this embodiment, the second weight, which indicates the weight with which the second complementary information is used to complement the estimation result, is changed according to the vehicle situation as described below. The second weight is expressed, for example, in the same manner as the first weight. In other embodiments, the second weight may include, for example, a position and not include an orientation, or may include an orientation and not include a position. Note that, in this embodiment, the vehicle situation does not include the second complementary information.

[0055] In this embodiment, the determination unit 240b determines whether the vehicle situation matches the second situation. The second situation in this embodiment includes a high-speed movement situation in which the moving speed of the vehicle 100 is greater than a predetermined second reference moving speed, and a complementary situation in which the complementary sensor is used to complement the estimation result with a weight equal to or greater than a predetermined reference weight. The complementary situation includes an internal complementary situation in which the internal sensor 161 is used to complement the estimation result with a weight equal to or greater than a predetermined first reference weight, and an external complementary situation in which the external complementary sensor 400 is used to complement the estimation result with a weight equal to or greater than a predetermined second reference weight. The first reference weight and the second reference weight are each greater than zero. The internal complementary situation in this embodiment includes a first complementary situation in which the communication speed between the server device 200 and the vehicle 100 is greater than a reference communication speed, the cumulative degree of the external force applied to the vehicle 100 is less than a reference level, and the continuous driving time is less than a reference driving time, and a second complementary situation in which the vehicle 100 is in a turning state.

[0056] The acquisition unit 215 acquires that the vehicle situation is a high-speed movement situation, for example, in substantially the same manner as when acquiring that the vehicle situation is a low-speed movement situation. The acquisition unit 215 also acquires that the vehicle situation is a first complementary situation, for example, in substantially the same manner as when acquiring that the vehicle situation is a communication delay situation, an external force accumulation situation, or a continuous driving situation. The acquisition unit 215 also acquires that the vehicle situation is a second complementary situation, for example, from internal sensor information indicating that the vehicle 100 is turning, external sensor information indicating that the motion state of the vehicle 100 is a turning state, or manufacturing information indicating that a process of turning the vehicle 100 is being executed. The acquisition unit 215 also acquires that the vehicle situation is an external complementary situation, for example, from internal sensor information, external sensor information, or manufacturing information indicating a place where the vehicle 100 is traveling.

[0057] Fig. 5 is a flowchart of the location information acquisition process in the second embodiment. In Fig. 5, the same steps as in Fig. 3B are denoted by the same reference numerals as in Fig. 3B. In S115b, the determination unit 240b determines whether the vehicle situation matches the second situation. If the vehicle situation does not match the second situation in S115b, the decision unit 235b decides to execute the first process in S120. If the vehicle situation matches the second situation in S115b, the decision unit 235b decides to execute the second process in S120.

[0058] In S165b, the control command generating unit 225 complements the estimation result output in S160. In S165b in this embodiment, when the vehicle situation matches the internal complement situation, the control command generating unit 225 complements the estimation result with the first complement information by determining the first weight to be equal to or greater than the first criterion. In addition, in S165b, when the vehicle situation matches the external complement situation, the control command generating unit 225 complements the estimation result with the second complement information by determining the second weight to be equal to or greater than the second criterion. Note that, when the vehicle information matches both the internal complement situation and the external complement situation, the estimation result may be complemented, for example, by both the first complement information and the second complement information. Note that, in this embodiment, in S20 and S30 of FIG. 3A, the control command generating unit 225 uses the estimation result output in S140 or the estimation result complemented in S165b as basic position information to generate and output a traveling control signal as a control command.

[0059] According to the system 50b in the second embodiment described above, when the vehicle situation of the target vehicle 100 satisfies the second situation, it is determined to execute the second process. This second situation includes a high-speed movement situation and a complementary situation. In this way, the position and orientation are estimated more quickly in the estimation process in a situation where there is a relatively small time margin for estimating the position and orientation of the vehicle 100 by the estimation process or in a situation where the estimation result is complemented using a complementary sensor. Therefore, the time lag from the start to the completion of the estimation of the position and orientation in the estimation process can be suppressed, and the possibility that the movement of the vehicle 100 is appropriately controlled by remote control can be increased.

[0060] C. Third embodiment: FIG. 6 is a flowchart of the location information acquisition process in the third embodiment. In FIG. 6, the same steps as in FIG. 3B are denoted by the same reference numerals as in FIG. 3B. In the third embodiment, the number of times the first calculation process is executed in the first process and the number of times the second calculation process is executed in the second process are the same, specifically, once. Parts of the configurations of the system 50, the server device 200, and the vehicle 100 in the third embodiment that are not particularly described are the same as those in the first embodiment.

[0061] Unlike the first embodiment, the first process in this embodiment does not include S135 and S137. In addition, in S140b shown in FIG. 6, the estimation unit 220 outputs the calculation result of S130 as an estimation result as it is. Note that the analysis process of S130 is more accurate and takes longer than the analysis process of S155, so the first process in this embodiment is more accurate and takes longer than the second process. Note that in this embodiment, in S20 and S30 of FIG. 3A, the control command generation unit 225 uses the estimation result output in S140b or the estimation result complemented in S165 as basic position information to generate and output a traveling control signal as a control command.

[0062] The system 50 in the third embodiment described above also determines whether the first processing, which has a higher accuracy of the estimation result than the second processing, or the second processing, which has a shorter processing time than the first processing, will be executed as the estimation processing depending on the moving body status, thereby increasing the likelihood that the movement of the vehicle 100 will be appropriately controlled by remote control.

[0063] D. Fourth embodiment: In the system 50 in the fourth embodiment, a location information acquisition process is executed substantially similarly to that in Fig. 3. In this embodiment, unlike the first embodiment, the same second analysis program AP2 is used in each of the first process and the second process. Among the configurations of the system 50, the server device 200, and the vehicle 100 in the fourth embodiment, parts that are not particularly described are similar to those in the first embodiment.

[0064] In this embodiment, in S130 and S155 of FIG. 3, the estimation unit 220 calculates the position and orientation of the target vehicle 100 by analyzing the captured image using the second analysis program AP2. That is, in this embodiment, the captured image is analyzed using the same algorithm in the analysis process included in the first process and the analysis process included in the second process, and the position and orientation are estimated. In this embodiment, the accuracy and processing time of the analysis process in the first process are similar to the accuracy and processing time of the analysis process in the second process. On the other hand, in the first process, the analysis process is executed a first number of times, which is greater than the second number of times. Therefore, the first process has higher accuracy and longer processing time than the second process. In this embodiment, the first analysis program AP1 does not have to be stored in the memory 202.

[0065] According to the system 50 in the fourth embodiment described above, it is determined whether the first process, which calculates the position and orientation more frequently, or the second process, which calculates the position and orientation less frequently, is to be executed depending on the moving body situation. Therefore, it is possible to increase the possibility that the movement of the vehicle 100 is appropriately controlled by remote control.

[0066] In the fourth embodiment, for example, the decision unit 235 and the judgment unit 240 may determine whether to execute the first process or the second process as the estimation process by changing the number of times to execute the analysis process according to the vehicle situation. In this case, determining the number of times to execute the analysis process to the first number of times corresponds to determining that the first process is executed. Also, determining the number of times to execute the analysis process to the second number of times corresponds to determining that the second process is executed.

[0067] E. Fifth embodiment: 7 is an explanatory diagram showing the configuration of a system 50c in the fifth embodiment. In this embodiment, unlike the first embodiment, the system 50c does not include a server device 200. In this embodiment, the unmanned driving of the vehicle 100b is realized by the autonomous control of the vehicle 100b. Parts of the configuration of the system 50c and the vehicle 100b in the fifth embodiment that are not particularly described are similar to those in the first embodiment.

[0068] In this embodiment, the memory 112b of the vehicle control device 110b stores a computer program PG1, situation data SD, a contour detection model Md, a first analysis program AP1, and a second analysis program AP2. The processor 111b of the vehicle control device 110 executes the computer program PG1 to function as the vehicle control unit 115b, and also to function as an acquisition unit 215, an estimation unit 220, a control command generation unit 225, a decision unit 235, and a judgment unit 240.

[0069] FIG. 8 is a flowchart of the unmanned driving process in this embodiment. In S11, the vehicle control unit 115b acquires basic position information of the vehicle 100b using the detection result output from the external sensor 301. In S21, the vehicle control unit 115b determines a target position to which the vehicle 100b should next head. In this embodiment, a reference route is stored in advance in the memory 112b. In S31, the vehicle control unit 115b generates a driving control signal for driving the vehicle 100b toward the determined target position by the control command generation unit 225b. In S41, the vehicle control unit 115b controls the actuator group using the generated driving control signal to drive the vehicle 100b at the acceleration and steering angle represented by the driving control signal. The vehicle control unit 115b repeats the acquisition of basic position information of the vehicle 100b, the determination of the target position, the generation of the driving control signal, and the control of the actuator group at a predetermined cycle. According to the system 50c of this embodiment, the vehicle 100b can be driven by autonomous control of the vehicle 100b without remote control of the vehicle 100b from the outside.

[0070] In this embodiment, the vehicle control device 110b of the vehicle 100b executes the same position information acquisition process as that in FIG. 3A. In this embodiment, the target vehicle 100b means the vehicle 100b itself that executes the position information acquisition process, that is, the host vehicle. In S21 and S31 in FIG. 8, the control command generation unit 225b uses the estimation result output in S140 in FIG. 3 or the estimation result complemented in S165 as basic position information to generate and output a traveling control signal as a control command.

[0071] In the vehicle 100b in the fifth embodiment described above, either the first process or the second process is determined to be executed as the estimation process depending on the vehicle conditions, thereby increasing the likelihood that the movement of the vehicle 100 will be appropriately controlled by autonomous control.

[0072] In addition, in a form in which unmanned driving of the vehicle 100b is realized without the server device 200 as in the fifth embodiment, execution of the second process may be determined when the vehicle situation matches the second situation as in the second embodiment. Also, in this form, the number of times the first calculation process is executed in the first process may be the same as the number of times the second calculation process is executed in the second process as in the third embodiment. Also, in this form, the same algorithm may be used in the analysis process in the first process and the analysis process in the second process as in the fourth embodiment.

[0073] F. Other Embodiments: (F1) In each of the above embodiments, the estimation unit 220 may estimate at least one of the position and the orientation of the vehicle 100 using internal sensor information instead of external information in the estimation process. In this case, at least one of a camera and a LiDAR mounted on the vehicle 100 can be used as the internal sensor 161, and at least one of an image of the surrounding environment of the vehicle 100 captured by a camera, a distance measurement result of a target around the vehicle 100 acquired by a LiDAR, and point cloud data of the target can be used as the internal sensor information. In this case, the estimation process may be, for example, a process of estimating at least one of the position and the orientation of the vehicle 100 by analyzing an image, a distance measurement result, or a point cloud data acquired as the internal sensor information. In this case, it is preferable that the first situation includes at least one of a waiting situation, a low-speed moving situation, an external force accumulation situation, and a continuous driving situation. In addition, it is preferable that the second situation includes at least one of a high-speed moving situation and a supplementary situation.

[0074] (F2) In each of the above embodiments, the determination unit 235 may change the first number of times depending on the vehicle situation. In this case, for example, the vehicle situation may be represented by a statistic of a value representing a deviation from the target route of the vehicle 100 traveling on the travel road SR, and the determination unit 235 may increase the first number of times as the statistic increases.

[0075] (F3) In each of the above embodiments, in the estimation process, both the position and the orientation of the vehicle 100 are output as the estimation result, but it is sufficient if at least one of the position and the orientation of the vehicle 100 is output as the estimation result. In this case, the other, which is not output as the estimation result, may be complemented using, for example, the first complementary information or the second complementary information.

[0076] (F4) In each of the above embodiments, the determination unit 235 may selectively determine the process to be executed as the estimation process from three or more processes each having different accuracy of the estimation result and different processing time.

[0077] (F5) In each of the above embodiments, the approximation process in the first process is more accurate and takes longer than the approximation process in the second process. On the other hand, if the first process is more accurate and takes longer than the second process, the approximation process in the first process may be less accurate, may have the same accuracy, may have a shorter processing time, or may have the same processing time as the approximation process in the second process. In this case, the fact that the first process is more accurate and takes longer than the second process may be realized, for example, by the approximation process in the first process being more accurate and taking longer, or by the first number of times being greater than the second number of times. Similarly, if the first process is more accurate and takes longer than the second process, the analysis process in the first process may be less accurate, may have the same accuracy, may have a shorter processing time, or may have the same processing time as the analysis process in the second process. In this case, the fact that the first process is more accurate and takes longer than the second process may be realized, for example, by the first number of times being greater than the second number of times. Furthermore, the estimation process is not limited to the processes described in the above embodiments, and may be any process as long as the position and orientation of the vehicle 100 can be estimated using sensor information.

[0078] (F6) In each of the above embodiments, the first situation includes a waiting situation, a low-speed moving situation, a communication delay situation, an external force accumulation situation, and a long-time driving state, but may not include these situations. However, it is preferable that the first situation includes at least one of the waiting situation, the low-speed moving situation, the communication delay situation, the external force accumulation situation, and the long-time driving state. The first situation may include other situations instead of or in addition to these situations. For example, the first situation may be a situation in which the vehicle 100 is traveling in a predetermined area in the factory FC. In this case, the acquisition unit 215 may acquire the vehicle situation in which the vehicle 100 is traveling in this area, for example, by using internal sensor information, external sensor information, or manufacturing information that indicates the traveling location of the vehicle 100. In addition, this area may be, for example, an area that tends to have lower estimation accuracy compared to other areas, or an area that tends to have a larger distance between the vehicle 100 and the target route compared to other areas. In this case, the determination unit 235 may determine whether to execute the first process or the second process by, for example, referring to map data that records the correspondence between each area in the factory FC and the process used as the estimation result, using the information indicating the acquired traveling location. Even in this case, the determination unit 235 can determine to execute the first process when the vehicle situation matches the first situation, and can determine to execute the second process when the vehicle situation does not match the second situation. In this case, the judgment unit 240 may not be provided.

[0079] (F7) In the second embodiment, the second situation includes a high-speed movement situation, an internal complement situation, and an external complement situation, but may not include these situations. However, it is preferable that the second situation includes at least one of a high-speed movement situation, an internal complement situation, and an external complement situation. The second situation may include other situations instead of or in addition to these situations. For example, the second situation may be a situation in which the vehicle 100 is traveling in a predetermined area in the factory FC. This area may be, for example, an area in which the estimation accuracy tends to be higher than other areas, or an area in which the distance between the vehicle 100 and the target route tends to be smaller than other areas. In such a form, when map data is used in a manner similar to the above, the determination unit 240 may not be provided.

[0080] (F8) In each of the above embodiments, the external sensor 301 is a camera. In contrast, the external sensor 301 may be a LiDAR. In this case, the sensor information output from the external sensor 301 is not a captured image but three-dimensional point cloud data. In this case, the remote control unit 210 and the vehicle control unit 115 may estimate at least one of the position and orientation of the vehicle 100 using the three-dimensional point cloud data in the estimation process. This three-dimensional point cloud data is external information that optically captures the outer shape of the vehicle 100 and is external information that represents the motion state of the vehicle 100 optically detected from outside the vehicle 100. The remote control unit 210 can estimate the position and orientation of the vehicle 100, for example, by performing template matching using the three-dimensional point cloud data. In this case, the external complement sensor 400 is preferably, for example, a camera. In addition, when the external sensor group 300 includes a camera and a LiDAR as the external sensor 301, the captured image and the point cloud data may be fused and used by, for example, a fusion process.

[0081] (F9) In each of the above embodiments, the control command generating unit 225 generates a traveling control signal as a control command. In contrast, the control command generating unit 225 may not generate a traveling control signal as a control command. Specifically, the control command may include at least one of a traveling control signal and generation information for generating the traveling control signal. For example, when the control command generating unit 225 provided in the server device 200 generates the generation information as a control command, the vehicle control device 110b of the vehicle 100b that receives the generation information may generate a traveling control signal using the generation information. For example, the base position information, the route, or the target position may be used as the generation information.

[0082] (F10) In the above first to fourth embodiments, the server device 200 executes the processes from obtaining basic position information of the vehicle 100 to generating the traveling control signal. In contrast, at least a part of the processes from obtaining basic position information of the vehicle 100 to generating the traveling control signal may be executed by the vehicle 100. For example, the following forms (1) to (3) may be used.

[0083] (1) The server device 200 may acquire basic position information of the vehicle 100, determine a target position to which the vehicle 100 should next head, and generate a route from the current location of the vehicle 100 represented in the acquired basic position information to the target position. The server device 200 may generate a route to a target position between the current location and the destination, or may generate a route to the destination. The server device 200 may transmit the generated route to the vehicle 100. The vehicle 100 may generate a driving control signal so that the vehicle 100 drives on the route received from the server device 200, and control the actuator group using the generated driving control signal.

[0084] (2) Server device 200 may acquire basic position information of vehicle 100 and transmit the acquired basic position information to vehicle 100. Vehicle 100 may determine a target position to which vehicle 100 should next head, generate a route from the current location of vehicle 100 represented in the received position information to the target position, generate a driving control signal so that vehicle 100 travels on the generated route, and control the actuator group using the generated driving control signal.

[0085] (3) In the above-mentioned embodiments (1) and (2), the detection result output from the internal sensor 161 mounted on the vehicle 100 may be used as sensor information for at least one of generating the route and generating the travel control signal. The internal sensor 161 may include, for example, a camera, a LiDAR, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, a gyro sensor, and a yaw rate sensor. For example, in the above-mentioned embodiment (1), the server device 200 may acquire the detection result of the internal sensor 161 and reflect the detection result of the internal sensor 161 in the route when generating the route. In the above-mentioned embodiment (1), the vehicle 100 may acquire the detection result of the internal sensor 161 and reflect the detection result of the internal sensor 161 in the travel control signal when generating the travel control signal. In the above-mentioned embodiment (2), the vehicle 100 may acquire the detection result of the internal sensor 161 and reflect the detection result of the internal sensor 161 in the route when generating the route ... travel control signal when generating the travel control signal.

[0086] (F11) In the above fifth embodiment, a detection result output from the internal sensor 161 mounted on the vehicle 100 may be used for at least one of generating a route and generating a traveling control signal. For example, the vehicle 100 may acquire the detection result of the internal sensor 161, and when generating a route, may reflect the detection result of the internal sensor 161 in the route. The vehicle 100 may acquire the detection result of the internal sensor 161, and when generating a traveling control signal, may reflect the detection result of the internal sensor 161 in the traveling control signal.

[0087] (F12) In the above fifth embodiment, the vehicle 100 acquires the position information of the vehicle 100 using the detection result of the external sensor 301. In contrast, the vehicle 100 may acquire basic position information using the detection result of the internal sensor 161 mounted on the vehicle 100, determine a target position to which the vehicle 100 should next head, generate a route from the current location of the vehicle 100 represented in the acquired basic position information to the target position, generate a travel control signal for traveling the generated route, and control the actuator group using the generated travel control signal. In this case, the vehicle 100 can travel without using any detection result of the external sensor 301. The vehicle 100 may acquire a target arrival time or traffic congestion information from outside the vehicle 100, and reflect the target arrival time or traffic congestion information in at least one of the route and the travel control signal. In addition, all of the functional configurations of the system 50 may be provided in the vehicle 100. In other words, the processing realized by the system 50 shown in the present disclosure may be realized by the vehicle 100 alone.

[0088] (F13) In the first embodiment, the server device 200 automatically generates a driving control signal to be transmitted to the vehicle 100. In addition to this, or instead of this, the server device 200 may generate a driving control signal in accordance with manual operation of an operator located outside the vehicle 100. For example, the operator may operate a control device including a display for displaying an image output from the external sensor 301, a steering wheel for remotely operating the vehicle 100, an accelerator pedal, a brake pedal, and a communication device for communicating with the server device 200 by wired communication or wireless communication, and the server device 200 may generate a driving control signal in accordance with the operation applied to the control device.

[0089] (F14) In each of the above embodiments, the vehicle 100 may have a configuration capable of moving by unmanned driving, and may be in the form of a platform having the configuration described below, for example. Specifically, the vehicle 100 may have at least the vehicle control device 110 and a group of actuators in order to perform the three functions of "running", "turning", and "stopping" by unmanned driving. When the vehicle 100 acquires information from the outside for unmanned driving, the vehicle 100 may further have a communication device 150. That is, the vehicle 100 capable of moving by unmanned driving may not have at least a part of interior parts such as seats and dashboards, may not have at least a part of exterior parts such as bumpers and fenders, and may not have a body shell. In this case, the remaining parts such as the body shell may be attached to the vehicle 100 before the vehicle 100 is shipped from the factory FC, or the remaining parts such as the body shell may be attached to the vehicle 100 after the vehicle 100 is shipped from the factory FC in a state in which the remaining parts such as the body shell are not attached to the vehicle 100. Each part may be attached from any direction, such as the upper side, lower side, front side, rear side, right side, or left side of the vehicle 100, and may be attached from the same direction or from different directions. Note that the position of the platform shape may be determined in the same manner as for the vehicle 100 in each of the above embodiments.

[0090] (F15) In each of the above embodiments, the vehicle 100 may be manufactured by combining a plurality of modules. The module means a unit composed of a plurality of parts grouped according to the location or function of the vehicle 100. For example, the platform of the vehicle 100 may be manufactured by combining a front module that configures the front part of the platform, a central module that configures the central part of the platform, and a rear module that configures the rear part of the platform. The number of modules that configure the platform is not limited to three, and may be two or less, or four or more. In addition to or instead of the parts that configure the platform, parts that configure parts of the vehicle 100 that are different from the platform may be modularized. The various modules may include any exterior parts such as a bumper or a grille, or any interior parts such as a seat or a console. In addition, not limited to the vehicle 100, any type of moving body may be manufactured by combining a plurality of modules. Such a module may be manufactured, for example, by joining a plurality of parts by welding or a fastener, or may be manufactured by integrally molding at least a part of the parts that configure the module as one part by casting. The molding method for integrally molding a single component, particularly a relatively large component, is also called gigacast or megacast. For example, the front module, the center module, and the rear module may be manufactured using gigacast.

[0091] (F16) Transporting the vehicle 100 by using the driving of the vehicle 100 through unmanned driving is also called "self-propelled transport." Also, a configuration for realizing self-propelled transport is also called a "vehicle remote-controlled autonomous driving transport system." Also, a production method for producing the vehicle 100 using self-propelled transport is also called "self-propelled production." In self-propelled production, for example, at a factory where the vehicle 100 is manufactured, at least a part of the transportation of the vehicle 100 is realized by self-propelled transport.

[0092] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention column can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0093] 50, 50b, 50c... system, 100, 100b... vehicle, 110, 110b... vehicle control device, 111, 111b... processor, 112, 112b... memory, 113... input / output interface, 114... internal bus, 115, 115b... vehicle control unit, 120... drive device, 130... steering device, 140... braking device, 150... communication device, 160... internal sensor group, 161... internal sensor, 200, 200b... server device, 2 01...processor, 202, 202b...memory, 203...input / output interface, 204...internal bus, 205...communication device, 210...remote control unit, 215...acquisition unit, 220...estimation unit, 225, 225b...control command generation unit, 235, 235b...determination unit, 240, 240b...judgment unit, 300...external sensor group, 301...external sensor, 302...communication device, 400...external complementary sensor, 402...communication device, 500...process management device

Claims

1. An acquisition unit that acquires a mobile object status that represents at least one of a state of a mobile object that can be moved by unmanned driving and an environment surrounding the mobile object; an estimation unit that executes an estimation process to estimate at least one of a position and an orientation of the moving object using sensor information acquired by a sensor; a control command generation unit that generates and outputs a control command for controlling movement of the moving object using an estimation result of the estimation process; a determination unit that determines whether a first process or a second process is to be executed as the estimation process according to the acquired moving object status, A system, wherein the first process is a process that produces an estimation result with higher accuracy than the second process, and the second process is a process that has a shorter processing time than the first process.

2. 2. The system of claim 1, a server device including the acquisition unit, the estimation unit, the control command generation unit, and the determination unit, the server device being provided outside the moving object; the sensor information is external information acquired using an external sensor located outside the moving object, the determination unit determines to execute the first process when the moving object status matches a predetermined first status, and determines to execute the second process when the moving object status does not match the first status; The first situation is: A state in which the moving body is waiting to move; and a situation in which the moving speed of the moving object is equal to or lower than a predetermined first reference moving speed; a situation in which the communication speed between the server device and the mobile object is equal to or lower than a predetermined reference communication speed; A situation in which a cumulative degree of an external force applied to the moving body is equal to or greater than a predetermined degree; and a situation in which the time during which the moving body is continuously driven is equal to or longer than a predetermined driving time. system.

3. 2. The system of claim 1, the determination unit determines to execute the second process when the moving object status matches a predetermined second status, and determines to execute the first process when the moving object status does not match the second status; The second situation is: A situation in which the moving speed of the moving object is greater than a predetermined second reference moving speed; a situation in which a complementary sensor different from the sensor is used to complement the estimation result with a weight equal to or greater than a predetermined weight; A system, wherein when the complementary sensor is used to complement the estimation result, the control command generation unit generates the control command using the estimation result complemented by the complementary sensor.

4. An acquisition unit that acquires a mobile object status that represents at least one of a state of a mobile object that can be moved by unmanned driving and a surrounding environment of the mobile object; an estimation unit that executes an estimation process to estimate at least one of a position and an orientation of the moving object using sensor information acquired by a sensor; a control command generation unit that generates and outputs a control command for controlling movement of the moving object using an estimation result of the estimation process; a determination unit that determines whether a first process or a second process is to be executed as the estimation process according to the acquired moving object status, the first processing is a processing of calculating at least one of a position and an orientation of the moving object a predetermined first number of times by using the sensor information and a predetermined algorithm, and outputting the estimation result based on each calculated result; the second processing is a processing of calculating at least one of a position and an orientation of the moving object a predetermined second number of times by using the sensor information and the algorithm, and outputting the estimation result based on each calculated result; The first number is greater than the second number.

5. A mobile body capable of moving by unmanned operation, An actuator for controlling the movement of the moving body; An acquisition unit that acquires a moving object status that represents at least one of a state of the moving object and an environment surrounding the moving object; an estimation unit that executes an estimation process to estimate at least one of a position and an orientation of the moving object using sensor information acquired by a sensor; a control command generation unit that generates and outputs a control command for operating the actuator using an estimation result of the estimation process; a determination unit that determines whether a first process or a second process is to be executed as the estimation process in accordance with the moving object status; The first process is a process that produces an estimation result with higher accuracy than the second process, and the second process is a process that has a shorter processing time than the first process.

Images