Calculation device, calculation system, information processing device, factory, and calculation method

Abstract

To provide a technology which can calculate a position etc. of a movable body at a low cost, while reducing a load required for preparation for calculating the position etc. of the movable body.SOLUTION: A calculation device comprises: a sensor information acquisition unit which acquires first sensor information acquired by detecting a movable body classified into a first state from the outside by a first sensor, and acquires second sensor information acquired by detecting the movable body classified into a second state from the outside by a second sensor; and a calculation unit which calculates at least one of the position and a direction of the movable body by using the first sensor information without executing pre-processing, and calculates at least one of the position and the direction of the movable body by using the second sensor information by executing the pre-processing.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a calculation device, a calculation system, an information processing device, a factory, and a calculation method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, vehicles that run unmanned by remote control are known (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2017-538619 Summary of the Invention [Problem to be solved by the invention]

[0004] In some cases, at least one of the position and orientation of a moving body is calculated using sensor information output from a sensor that detects a moving body from the outside. In some cases, the external shape of the moving body is detected from the sensor information in order to calculate the position of the moving body using the sensor information. For example, when detecting the external shape of a moving body contained in a captured image output from a camera, a method is considered in which a machine learning model is trained as a preprocessing and the external shape of the moving body is detected using the trained machine learning model. However, when a moving body moves in a factory where moving bodies are manufactured, the appearance of the moving body may differ depending on the manufacturing process. For example, when a moving body is a vehicle, the appearance of a vehicle in the form of a platform in which various devices such as wheels and a vehicle control device are assembled to a chassis is significantly different from that of a vehicle in which a body such as a body shell is assembled to a chassis. Due to such differences in the appearance of the moving body depending on the manufacturing process, there is a risk that the detection accuracy when detecting the external shape of the moving body from the sensor information will decrease. If the detection accuracy when detecting the external shape of the moving body from the sensor information decreases, there is a risk that the calculation accuracy of the position of the moving body will decrease. Therefore, when detecting the external shape of a moving body using a trained machine learning model, it is desirable to train the machine learning model for each external state of the moving body. Therefore, when a moving object moves in a factory, if the external shape of the moving object contained in the sensor information is detected using a trained machine learning model, it may be necessary to prepare learning data for each external state of the moving object, or the learning time may be longer by the number of external states of the moving object. On the other hand, for example, when detecting the external shape of a moving object contained in measurement point cloud data output from a LiDAR, a method of calculating the position of the moving object by matching the measurement point cloud data with reference point cloud data prepared in advance is conceivable. In this case, the position of the moving object can be calculated without performing pre-processing such as training a machine learning model, so that the load required for pre-processing to calculate the position of the moving object can be reduced. However, LiDAR is generally more expensive than a camera.In this way, when calculating the position, etc. of a moving body without performing preprocessing, depending on the type of sensor, the cost required to calculate the position, etc. of a moving body may be higher than when performing preprocessing to calculate the position, etc. of the moving body. In view of the above, there is a demand for a technology that can calculate the position, etc. of a moving body at low cost while reducing the load required for preprocessing that is performed to calculate the position, etc. of the moving body. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to a first aspect of the present disclosure, there is provided a calculation device for calculating at least one of a position and an orientation of a moving object, the moving object being capable of moving in an unmanned manner in a factory where a plurality of manufacturing processes are performed to manufacture the moving object, the moving object being classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes, the calculation device including: a sensor information acquisition unit for acquiring first sensor information representing the moving object classified into the first state output from a first sensor by detecting from the outside the moving object classified into a first state among the plurality of states by the first sensor; acquiring second sensor information representing the moving object classified into the second state output from the second sensor by detecting from the outside the moving object classified into a second state different from the first state among the plurality of states by a second sensor of a different type from the first sensor; and a calculation unit for calculating at least one of the position and the orientation of the moving object using the first sensor information without performing preprocessing, and for calculating at least one of the position and the orientation of the moving object using the second sensor information by performing the preprocessing. According to this embodiment, the calculation device can calculate at least one of the position and orientation of the moving object classified into the first state using the first sensor information without performing preprocessing. Also, the calculation device can perform preprocessing and calculate at least one of the position and orientation of the moving object classified into the second state using the second sensor information. In this way, when calculating at least one of the position and orientation of the moving object classified into the first state, it is not necessary to perform preprocessing. Therefore, it is possible to reduce the load required for preprocessing performed to calculate the position, etc. of the moving object. Furthermore, according to this embodiment, the moving object classified into the first state and the moving object classified into the second state can be detected by different types of sensors depending on the price of the sensor. This can reduce the possibility that the cost required for calculating the position, etc. of the moving object will be high. (2) According to a second aspect of the present disclosure, there is provided a calculation system for calculating at least one of a position and an orientation of a moving body, the moving body being capable of moving in an unmanned manner in a factory where a plurality of manufacturing processes are performed to manufacture the moving body, the moving body being classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes, the calculation system including: a first sensor that detects from outside the moving body classified into a first state among the plurality of states and outputs first sensor information representing the moving body classified into the first state, a second sensor of a different type from the first sensor that detects from outside the moving body classified into a second state different from the first state among the plurality of states and outputs second sensor information representing the moving body classified into the second state, and a calculation unit that calculates at least one of the position and the orientation of the moving body using the first sensor information without performing preprocessing, and calculates at least one of the position and the orientation of the moving body using the second sensor information by performing the preprocessing. According to this embodiment, the calculation system can calculate at least one of the position and orientation of the moving object classified into the first state using the first sensor information without performing preprocessing. Also, the calculation system can perform preprocessing and calculate at least one of the position and orientation of the moving object classified into the second state using the second sensor information. In this way, it is not necessary to perform preprocessing when calculating at least one of the position and orientation of the moving object classified into the first state. Therefore, it is possible to reduce the load required for preprocessing performed to calculate the position, etc. of the moving object. Furthermore, according to this embodiment, the moving object classified into the first state and the moving object classified into the second state can be detected by different types of sensors depending on the price of the sensor. This can reduce the possibility that the cost required for calculating the position, etc. of the moving object will be high. (3) According to a third aspect of the present disclosure, there is provided a calculation device for calculating at least one of a position and a direction of a moving object, the moving object being capable of moving in an unmanned manner in a factory where a plurality of manufacturing processes are performed to manufacture the moving object, the moving object being classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes, the calculation device including a sensor information acquisition unit that acquires first sensor information representing the moving object classified into a first state among the plurality of states, and acquires second sensor information representing the moving object classified into a second state among the plurality of states different from the first state, and a calculation unit that calculates the moving object represented by the first sensor information using the first sensor information. and a calculation unit that calculates at least one of the position and the orientation of the moving body represented by the second sensor information using the second sensor information, wherein the first sensor information is information output from one of the sensors, a camera and a LiDAR, by detecting the moving body classified in the first state from the outside, and the second sensor information is information output from the other sensor, by detecting the moving body classified in the second state from the outside, using the other sensor of the camera and the LiDAR. According to this embodiment, the calculation device can calculate at least one of the position and the orientation of the moving body classified in the first state by detecting the moving body classified in the first state using one of the sensors, a camera and a LiDAR. The calculation device can calculate at least one of the position and the orientation of the moving body classified in the second state by detecting the moving body classified in the second state using a sensor of a type different from the sensor that detects the moving body classified in the first state, of the camera and the LiDAR. This can reduce the cost required for calculating at least one of the position and the orientation of the moving body. (4) According to a fourth aspect of the present disclosure, there is provided a calculation system for calculating at least one of a position and a direction of a moving object, the moving object is capable of moving in an unmanned manner within a factory where a plurality of manufacturing processes are carried out to manufacture the moving object, the moving object is classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes, the calculation system includes a first sensor that detects from outside the moving object classified into a first state among the plurality of states and outputs first sensor information representing the moving object classified into the first state, and a second sensor that detects from outside the moving object classified into a second state among the plurality of states different from the first state and outputs first sensor information representing the moving object classified into the first state. The present invention includes a second sensor that outputs second sensor information representing the moving object classified in the second state, and a calculation unit that uses the first sensor information to calculate at least one of the position and the orientation of the moving object represented by the first sensor information and uses the second sensor information to calculate at least one of the position and the orientation of the moving object represented by the second sensor information, wherein the first sensor is either a camera or a LiDAR, and when the first sensor is the camera, the second sensor is the LiDAR, and when the first sensor is the LiDAR, the second sensor is the camera. According to this embodiment, the calculation system can calculate at least one of the position and the orientation of the moving object classified in the first state by detecting the moving object classified in the first state using either a camera or a LiDAR sensor. The calculation system can calculate at least one of the position and the orientation of the moving object classified in the second state by detecting the moving object classified in the second state using a sensor of a type different from the sensor that detects the moving object classified in the first state, either a camera or a LiDAR. This makes it possible to reduce the cost required to calculate at least one of the position and orientation of the moving object. (5) According to a fifth embodiment of the present disclosure, a factory is provided. In the factory for manufacturing a moving body, a plurality of manufacturing processes are performed in the factory to manufacture the moving body, the moving body can move in the factory by unmanned operation, the moving body has different appearances depending on the plurality of manufacturing processes and is classified into a plurality of states, the factory includes a first sensor installed in a first area where the moving body classified into a first state among the plurality of states moves, and a second sensor installed in a second area where the moving body classified into a second state different from the first state among the plurality of states moves, the first sensor being either a camera or a LiDAR, and when the first sensor is the camera, the second sensor is the LiDAR, and when the first sensor is the LiDAR, the second sensor is the camera. According to this embodiment, different types of sensors can be installed in the first area and the second area. (6) In the above embodiment, the first sensor may detect the moving object classified in the first state from the outside, and output first sensor information representing the moving object classified in the first state; the second sensor may detect the moving object classified in the second state from the outside, and output second sensor information representing the moving object classified in the second state; the first sensor information may be used to calculate at least one of the position and orientation of the moving object represented by the first sensor information; and the second sensor information may be used to calculate at least one of the position and orientation of the moving object represented by the second sensor information. According to this embodiment, at least one of the position and orientation of the moving object classified in the first state may be calculated by detecting the moving object classified in the first state using one of the sensors, a camera and a LiDAR. At least one of the position and orientation of the moving object classified in the second state may be calculated by detecting the moving object classified in the second state using a sensor of a type different from the sensor that detects the moving object classified in the first state, out of the camera and the LiDAR. This may reduce the cost required for calculating at least one of the position and orientation of the moving object. (7) According to a sixth aspect of the present disclosure, an information processing device is provided. In the information processing device used to move a moving body, the moving body can move in an unmanned manner in a factory where a plurality of manufacturing processes are performed to manufacture the moving body, and the moving body is classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes. The information processing device includes a determination information acquisition unit that acquires at least one of state information indicating which of the plurality of states the moving body is in and process information indicating which of the plurality of manufacturing processes the manufacturing process being performed on the moving body is in, and a determination unit that determines which of a first sensor as the sensor and a second sensor as the sensor of a different type from the first sensor is to be used to detect the moving body, using at least one of the state information and the process information, in order to calculate at least one of the position and orientation of the moving body using sensor information representing the moving body output from a sensor of a type corresponding to the state of the moving body. According to this aspect, the information processing device can determine which of the first sensor and the second sensor is to be used to detect the moving body, using at least one of the state information and the process information. (8) According to a seventh aspect of the present disclosure, there is provided a calculation system for calculating at least one of a position and an orientation of a moving body, the moving body being capable of moving in an unmanned manner in a factory where a plurality of manufacturing processes are performed to manufacture the moving body, the moving body being classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes, the calculation system including: a first sensor as a sensor that detects the moving body from the outside and outputs first sensor information representing the moving body; a second sensor as the sensor of a different type from the first sensor, the second sensor being the sensor that detects the moving body from the outside and outputs second sensor information representing the moving body; state information indicating which of the plurality of states the state of the moving body is; and process information indicating which of the plurality of manufacturing processes the manufacturing process being performed on the moving body is. a determination unit that determines which of the first sensor and the second sensor is to be used to detect the moving body using at least one of the state information and the process information in order to calculate at least one of the position and the orientation of the moving body using sensor information representing the moving body output from the sensor of a type corresponding to the state of the moving body, and a calculation unit that calculates at least one of the position and the orientation of the moving body using the first sensor information without performing preprocessing when the determination unit determines that the moving body is to be detected using the first sensor, and that performs the preprocessing and calculates at least one of the position and the orientation of the moving body using the second sensor information when the determination unit determines that the moving body is to be detected using the second sensor. According to this embodiment, the calculation system can calculate at least one of the position and the orientation of the moving body using the sensor information obtained by detecting the moving body using a sensor of a type corresponding to the state of the moving body. (9) According to an eighth aspect of the present disclosure, there is provided a calculation system for calculating at least one of a position and an orientation of a moving body, the moving body being capable of moving in an unmanned manner in a factory where a plurality of manufacturing processes are performed to manufacture the moving body, the moving body being classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes, the calculation system acquiring at least one of a first sensor as a sensor that detects the moving body from the outside and outputs first sensor information representing the moving body, a second sensor as the sensor of a different type from the first sensor and that detects the moving body from the outside and outputs second sensor information representing the moving body, state information indicating which of the plurality of states the moving body is in, and process information indicating which of the plurality of manufacturing processes the manufacturing process being performed on the moving body is. The system includes a determination information acquisition unit, a determination unit that determines which of the first sensor and the second sensor is to be used to detect the moving body using at least one of the state information and the process information in order to calculate at least one of the position and the orientation of the moving body using sensor information representing the moving body output from the sensor of a type corresponding to the state of the moving body, and a calculation unit that calculates at least one of the position and the orientation of the moving body using the sensor information output from the sensor determined by the determination unit to be used to detect the moving body, wherein the first sensor is either a camera or a LiDAR, and when the first sensor is the camera, the second sensor is the LiDAR, and when the first sensor is the LiDAR, the second sensor is the camera. According to this embodiment, the calculation system can calculate at least one of the position and the orientation of the moving body using sensor information acquired by detecting the moving body using a sensor of a type corresponding to the state of the moving body. (10) In the above embodiment, the pre-processing may include a learning process for training a machine learning model. According to this embodiment, in the learning process, it is not necessary to learn the feature amount of the moving body classified into the first state. Therefore, it is possible to save the effort of preparing learning data for the moving body classified into the first state. Furthermore, out of the time required for machine learning, the time required for learning the feature amount of the moving body classified into the first state can be reduced. This makes it possible to shorten the learning time. Therefore, it is possible to reduce the load required for the pre-processing executed to calculate the position, etc. of the moving body in a state where the possibility of a decrease in detection accuracy when detecting the external shape of the moving body from the first sensor information is reduced. (11) In the above embodiment, the calculation unit may input the second sensor information to the machine learning model, which is trained by executing the learning process, and is trained to output shape data indicating an external shape of the moving body when the second sensor information is input, to acquire the shape data, and calculate at least one of the position and the orientation of the moving body using the shape data. According to this embodiment, the calculation unit can acquire the shape data by inputting the second sensor information to the trained machine learning model. Then, the calculation unit can calculate at least one of the position and the orientation of the moving body using the shape data. (12) In the above-mentioned form, the machine learning model is prepared for each of the states of the moving body, and each of the multiple machine learning models prepared for each of the states has been pre-trained to output the shape data by inputting a plurality of training images including M (M is an integer of 2 or more) first training images each including the moving body classified into one of the multiple states, and N (N is an integer of 0 or more and less than M) second training images each including the moving body classified into the remaining other states, and the calculation unit may acquire the shape data by inputting the second sensor information to the machine learning model selected according to the state of the moving body from the multiple machine learning models prepared for the states, calculate coordinates in an image coordinate system of a positioning point previously set for the moving body using the shape data, and convert the calculated coordinates into coordinates in a global coordinate system, thereby calculating at least one of the position and the orientation of the moving body represented by the second sensor information. According to this embodiment, a machine learning model suitable for detecting a moving object classified into each state is prepared in advance by learning more first training images including a moving object classified into one state than second training images including moving objects classified into the remaining other states. As a result, the calculation unit can accurately detect a moving object from the second sensor information by inputting the second sensor information to a machine learning model selected according to the state of the moving object from among a plurality of machine learning models prepared for each state of the moving object. (13) In the above embodiment, the machine learning model is trained in advance to output the shape data by inputting a plurality of training images including the moving object and a correct answer label associated with each of the plurality of training images, the correct answer label indicating the state of the moving object included in the training image, and the calculation unit may obtain the shape data by inputting the second sensor information and the state of the moving object to the machine learning model, calculate coordinates in an image coordinate system of a positioning point set in advance for the moving object using the shape data, and convert the calculated coordinates into coordinates in a global coordinate system to calculate at least one of the position and the orientation of the moving object represented by the second sensor information. According to this embodiment, a machine learning model is prepared in advance, which is trained by associating a plurality of training images including the moving object with a correct answer label indicating the state of the moving object included in each training image. As a result, the moving object can be accurately detected from the second sensor information by inputting the second sensor information associated with the state of the moving object to the machine learning model. (14) In the above embodiment, the calculation unit may detect the moving object from the captured image output from the camera, output shape data indicating an external shape of the moving object, calculate coordinates in an image coordinate system of a positioning point preset for the moving object using the shape data, and convert the calculated coordinates into coordinates in a global coordinate system, thereby calculating at least one of the position and the orientation of the moving object represented by the captured image. According to this embodiment, the calculation unit can calculate at least one of the position and the orientation of the moving object represented by the captured image by converting the coordinates in the image coordinate system into coordinates in a global coordinate system using the shape data. (15) In the above-mentioned embodiment, the calculation unit may calculate at least one of the position and the orientation of the moving body represented by the measurement point cloud data by matching the measurement point cloud data output from the LiDAR with reference point cloud data prepared in advance. According to this embodiment, the calculation unit can calculate at least one of the position and the orientation of the moving body represented by the measurement point cloud data by matching the measurement point cloud data output from the LiDAR with reference point cloud data prepared in advance. (16) According to a ninth aspect of the present disclosure, there is provided a calculation method for calculating at least one of a position and an orientation of a moving body, the moving body being capable of moving in an unmanned manner in a factory where a plurality of manufacturing processes are performed to manufacture the moving body, the moving body being classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes, the calculation method including: a sensor information acquisition step of acquiring first sensor information representing the moving body classified into the first state output from the first sensor by detecting from the outside the moving body classified into a first state among the plurality of states by the first sensor, acquiring second sensor information representing the moving body classified into the second state output from the second sensor by detecting from the outside the moving body classified into a second state different from the first state among the plurality of states by a second sensor of a different type from the first sensor, and a calculation step of calculating at least one of the position and the orientation of the moving body using the first sensor information without performing a preprocessing, and performing the preprocessing to calculate at least one of the position and the orientation of the moving body using the second sensor information. According to this embodiment, at least one of the position and orientation of the moving object classified into the first state can be calculated using the first sensor information without performing pre-processing. Also, at least one of the position and orientation of the moving object classified into the second state can be calculated using the second sensor information by performing pre-processing. In this way, when calculating at least one of the position and orientation of the moving object classified into the first state, it is not necessary to perform pre-processing. Therefore, it is possible to reduce the load required for pre-processing performed to calculate the position, etc. of the moving object. Furthermore, according to this embodiment, the moving object classified into the first state and the moving object classified into the second state can be detected by different types of sensors depending on the price of the sensor. This can reduce the possibility that the cost required for calculating the position, etc. of the moving object will be high. (17) According to a tenth aspect of the present disclosure, there is provided a calculation method for calculating at least one of a position and a direction of a moving object, the moving object being capable of moving in an unmanned manner in a factory where a plurality of manufacturing processes are performed to manufacture the moving object, the moving object being classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes, the calculation method including a sensor information acquisition step of acquiring first sensor information representing the moving object classified into a first state among the plurality of states, and acquiring second sensor information representing the moving object classified into a second state among the plurality of states different from the first state, and calculating the movement represented by the first sensor information using the first sensor information. and a calculation step of calculating at least one of the position and the orientation of the moving body represented by the second sensor information using the second sensor information, wherein the first sensor information is information output from one of the sensors, a camera and a LiDAR, by detecting the moving body classified in the first state from the outside, and the second sensor information is information output from the other sensor, by detecting the moving body classified in the second state from the outside, using the other sensor of the camera and the LiDAR. According to this embodiment, at least one of the position and the orientation of the moving body classified in the first state can be calculated by detecting the moving body classified in the first state using one of the sensors, a camera and a LiDAR. In addition, at least one of the position and the orientation of the moving body classified in the second state can be calculated by detecting the moving body classified in the second state using a sensor of a type different from the sensor that detects the moving body classified in the first state, a sensor of a type different from the sensor that detects the moving body classified in the first state, a camera and a LiDAR. This makes it possible to reduce the cost required for calculating at least one of the position and the orientation of the moving body. (18) According to an eleventh aspect of the present disclosure, there is provided a calculation method for calculating at least one of a position and an orientation of a moving body, the moving body being capable of moving in an unmanned manner in a factory where a plurality of manufacturing processes are performed to manufacture the moving body, the moving body being classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes, the calculation method including a determination information acquisition step of acquiring at least one of state information indicating which of the plurality of states the moving body is in and process information indicating which of the plurality of manufacturing processes the manufacturing process being performed on the moving body is, and a step of outputting first sensor information representing the moving body by detecting the moving body from outside in order to calculate at least one of the position and the orientation of the moving body using sensor information representing the moving body output from the sensor of a type corresponding to the state of the moving body. The method includes a determination step of determining, using at least one of the state information and the process information, which of a first sensor as the sensor and a second sensor as the sensor of a type different from the first sensor, which is a sensor that detects the moving body from the outside and outputs second sensor information representing the moving body, to use the sensor to detect the moving body, and a calculation step of calculating at least one of the position and the orientation of the moving body using the first sensor information without performing preprocessing when it is determined in the determination step that the moving body is to be detected using the first sensor, and calculating at least one of the position and the orientation of the moving body using the second sensor information by performing the preprocessing when it is determined in the determination step that the moving body is to be detected using the second sensor. According to this embodiment, at least one of the position and the orientation of the moving body can be calculated using sensor information acquired by detecting the moving body using a sensor of a type corresponding to the state of the moving body. (19) According to a twelfth aspect of the present disclosure, there is provided a calculation method for calculating at least one of a position and an orientation of a moving body, the moving body being capable of moving in an unmanned manner in a factory where a plurality of manufacturing processes are performed to manufacture the moving body, the moving body being classified into a plurality of states by having different appearances depending on the plurality of manufacturing processes, the calculation method including a determination information acquisition step of acquiring at least one of state information indicating which of the plurality of states the moving body is in and process information indicating which of the plurality of manufacturing processes the manufacturing process being performed on the moving body is, and a determination information acquisition step of outputting first sensor information representing the moving body by detecting the moving body from outside in order to calculate at least one of the position and the orientation of the moving body using sensor information representing the moving body outputted from the sensor of a type corresponding to the state of the moving body. The method includes a determination step of determining which of a first sensor to use to detect the moving body, the first sensor being a sensor of a type different from the first sensor, and a second sensor being a sensor of a type different from the first sensor, the second sensor being a sensor of a type different from the first sensor, and outputting second sensor information representing the moving body by detecting the moving body from the outside, using at least one of the state information and the process information, and a calculation step of calculating at least one of the position and the orientation of the moving body using the sensor information output from the sensor determined to be used to detect the moving body in the determination step, the first sensor being either a camera or a LiDAR, and when the first sensor is the camera, the second sensor is the LiDAR, and when the first sensor is the LiDAR, the second sensor is the camera. According to this embodiment, at least one of the position and orientation of the moving body can be calculated using sensor information acquired by detecting the moving body using a sensor of a type corresponding to the state of the moving body. The present disclosure may be realized in various forms other than the above-described calculation device, calculation system, information processing device, factory, and calculation method. For example, the present disclosure may be realized in the form of a calculation device, an information processing device, a calculation system, a method for manufacturing a moving body, a calculation device, an information processing device, a calculation system, a method for controlling a moving body, a computer program for realizing the control method, a non-transitory recording medium on which the computer program is recorded, etc. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a configuration of a traveling system in a first embodiment. [Diagram 2] FIG. 1 is a diagram showing a configuration of a vehicle control device in a first embodiment. [Diagram 3] FIG. 2 is a diagram showing a state in which a vehicle travels within a factory in the first embodiment. [Figure 4] FIG. 1 is a diagram showing the configuration of a calculation device. [Diagram 5] FIG. 2 is a diagram showing the configuration of a remote control device. [Figure 6] 4 is a flowchart showing a processing flow in the first embodiment. [Figure 7] 6 is a flowchart showing an example of a method for calculating the vehicle position and the like represented by the measurement point cloud data. [Figure 8] 6 is a flowchart showing an example of a method for calculating the position of a vehicle represented by a captured image. [Figure 9] 9A to 9C are schematic diagrams showing examples of various images when the calculation method shown in FIG. 8 is executed. [Figure 10] FIG. 11 is a diagram for explaining details of a coordinate point calculation step. [Figure 11] FIG. 4 is a diagram for explaining a method of calculating a base coordinate point. [Figure 12] FIG. 1 is a diagram for explaining details of a position conversion process. [Figure 13] FIG. 2 for explaining the details of the position conversion process. [Figure 14] FIG. 11 is a diagram showing a state in which a vehicle travels within a factory in a second embodiment. [Figure 15]FIG. 11 is a diagram showing the configuration of a calculation system according to a second embodiment. [Figure 16] FIG. 11 is a diagram showing a state in which a vehicle travels within a factory in a third embodiment. [Figure 17] FIG. 13 is a diagram showing the configuration of a calculation system according to a third embodiment. [Figure 18] 10 is a flowchart showing a vehicle position calculation method using a second detection model. [Figure 19] FIG. 13 is a diagram showing the configuration of a calculation system according to a fourth embodiment. [Figure 20] FIG. 13 is a diagram showing a state in which a vehicle travels through a factory in a fifth embodiment. [Figure 21] FIG. 13 is a diagram showing the configuration of a calculation system according to a fifth embodiment. [Figure 22] 13 is a first flowchart showing a processing flow in the fifth embodiment. [Figure 23] 23 is a second flowchart showing the processing flow in the fifth embodiment. [Figure 24] FIG. 13 is a diagram showing the configuration of a traveling system according to a sixth embodiment. [Diagram 25] 13 is a flowchart showing a processing flow in the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A. First embodiment: 1 is a diagram showing the configuration of a traveling system 1 in the first embodiment. The traveling system 1 is a system for moving a moving body in a factory where the moving body is manufactured, without the need for a driver operating the moving body. The traveling system 1 includes one or more moving bodies, a calculation system 5 having a first sensor Se1, a second sensor Se2, and a calculation device 6, and a remote control device 7.

[0009] The term "mobile body" refers to an object that can move, such as a vehicle 10 or an electric vertical take-off and landing aircraft (a so-called flying car). In this embodiment, the mobile body is the vehicle 10. The vehicle 10 may be a vehicle that runs on wheels or a vehicle that runs on caterpillar tracks, such as a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, a construction vehicle, etc. The vehicle 10 includes an electric vehicle (BEV: Battery Electric Vehicle), a gasoline vehicle, a hybrid vehicle, and a fuel cell vehicle. In addition, when the mobile body is other than the vehicle 10, the expressions "vehicle" and "car" in this disclosure can be appropriately replaced with "mobile body", and the expression "running" can be appropriately replaced with "movement".

[0010] The vehicle 10 includes a driving device 11, a steering device 12, a braking device 13, a vehicle communication device 14, and a vehicle control device 15. The driving device 11 accelerates the vehicle 10. The driving device 11 includes wheels, a driving force source that rotates the wheels, and a power supply device that supplies power to the driving force source. For example, if the vehicle 10 is an electric vehicle, the driving force source is a driving motor, and the power supply device is a battery such as a lithium-ion battery. The steering device 12 changes the traveling direction of the vehicle 10. The braking device 13 decelerates the vehicle 10. The vehicle communication device 14 communicates with an external device using wireless communication or the like. The external device is a device such as a remote control device 7 provided in a location different from the vehicle 10, and another vehicle 10. The vehicle communication device 14 is, for example, a wireless communication device.

[0011] FIG. 2 is a diagram showing a configuration of the vehicle control device 15 in the first embodiment. The vehicle control device 15 controls the operation of the vehicle 10. The vehicle control device 15 includes an input / output interface 159, a storage unit 157, and a CPU 150. In the vehicle control device 15, the input / output interface 159, the storage unit 157, and the CPU 150 are connected to each other via a bus or the like so as to be able to communicate bidirectionally. The input / output interface 159 is used to communicate with various devices mounted on the vehicle 10. The storage unit 157 of the vehicle control device 15 stores various information including various programs P15 that control the operation of the vehicle control device 15. The storage unit 157 of the vehicle control device 15 includes, for example, a RAM, a ROM, and a hard disk drive (HDD). The CPU 150 of the vehicle control device 15 functions as an operation control unit 154 by developing the various programs P15 stored in the storage unit 157. The operation control unit 154 executes driving control of the vehicle 10 by controlling the operation of actuators that change the acceleration / deceleration and steering angle of the vehicle 10. "Driving control" refers to various controls for driving actuators that perform the three functions of the vehicle 10, "running," "turning," and "stopping," such as adjusting the acceleration, speed, and steering angle of the vehicle 10. In this embodiment, the actuators include the actuators of the drive device 11, the actuators of the steering device 12, and the actuators of the braking device 13. At least a part of the functions of the CPU 150 of the vehicle control device 15 may be realized by a hardware circuit.

[0012] The vehicle 10 can run by unmanned driving. "Unmanned driving" means driving without the driving operation of a passenger. Driving operation means at least one of the operations of "running", "turning" and "stopping" of the vehicle 10. Unmanned driving is realized by automatic / manual remote control using a device provided outside the vehicle 10, or autonomous control of the vehicle 10. The vehicle 10 running by unmanned driving may have a passenger who does not perform driving operation on board. The passenger who does not perform driving operation includes, for example, a person who simply sits in the seat of the vehicle 10, or a person who performs a task other than driving operation, such as assembly, inspection, and operation of switches, while riding on the vehicle 10. Note that driving by a passenger's driving operation is sometimes called "manned driving".

[0013] In this embodiment, the vehicle 10 is remotely controlled by the remote control device 7 to travel in an unmanned manner at least in a part of the factory Fc1. FIG. 3 is a diagram showing the vehicle 10 traveling in the factory Fc1 in the first embodiment. The factory Fc1 is a place where manufacturing equipment for the vehicle 10 is installed. The factory Fc1 is also called a manufacturing line for the vehicle 10. In the factory Fc1, a plurality of manufacturing processes Pr1 to Pr7 are performed to manufacture the vehicle 10. In this embodiment, the plurality of manufacturing processes Pr1 to Pr7 are a platform manufacturing process Pr1, a first transfer process Pr2, an assembly process Pr3, a second transfer process Pr4, and an inspection process Pr5, which are performed in this order, to manufacture the vehicle 10 as a finished product. Then, the vehicle 10 as a finished product is stored in a storage location PL7 in a storage process Pr7 after passing through a third transfer process Pr6 until it is shipped.

[0014] The platform manufacturing process Pr1 is a process for manufacturing a platform vehicle 10p. The platform vehicle 10p is a vehicle 10 in the form of a platform. The platform vehicle 10p is a vehicle 10 in the middle of manufacturing, and is configured to be able to run by unmanned driving. As shown in FIG. 1, the platform vehicle 10p is equipped with at least a drive unit 11, a steering unit 12, a braking unit 13, and a vehicle control unit 15 in order to perform three functions of "running", "turning", and "stopping" by unmanned driving. In this embodiment, the vehicle 10 acquires information from a remote control unit 7 provided at a location different from the vehicle 10 in order to run by remote control. Therefore, the platform vehicle 10p further includes a vehicle communication unit 14. The platform manufacturing process Pr1 is performed at the manufacturing site PL1.

[0015] 3 is a process of assembling interior parts such as seats and exterior parts such as a painted body shell 18 to the platform vehicle 10p. The assembling process Pr3 is performed at an assembling location PL3.

[0016] The inspection process Pr5 is a process for inspecting the manufactured vehicle 10. The inspection process Pr5 is performed at an inspection place PL5.

[0017] In this embodiment, the manufacturing location PL1, the assembly location PL3, the inspection location PL5, and the storage location PL7 are each located in a different building. Therefore, the manufacturing location PL1 and the assembly location PL3 are connected by a first runway Ro1. The assembly location PL3 and the inspection location PL5 are connected by a second runway Ro2. The inspection location PL5 and the storage location PL7 are connected by a third runway Ro3. The first runway Ro1, the second runway Ro2, and the third runway Ro3 may each be a private road or a public road. The locations PL1 to PL4 where the manufacturing processes Pr1, Pr3, Pr5, and Pr7 are performed may be located on one site or across multiple sites. In addition, the locations PL1 to PL4 where the manufacturing processes Pr1, Pr3, Pr5, and Pr7 are performed may be located on one address or across multiple addresses.

[0018] The first transportation process Pr2 is a process of moving the vehicle 10 from the manufacturing location PL1 to the assembly location PL3. The second transportation process Pr4 is a process of moving the vehicle 10 from the assembly location PL3 to the inspection location PL5. The third transportation process Pr6 is a process of moving the vehicle 10 from the inspection location PL5 to the storage location PL7.

[0019] As shown in FIG. 1, the platform vehicle 10p and the assembled vehicle 10m have different external shapes, and therefore the appearances of the vehicle 10 are different. The assembled vehicle 10m is the vehicle 10 in which at least the body shell 18 is assembled to the platform vehicle 10p. In this embodiment, as shown in FIG. 3, the vehicle 10 after the assembly process Pr3 has started is the assembled vehicle 10m. In this way, the vehicle 10 has different appearances depending on the multiple manufacturing processes Pr1 to Pr7, and is therefore classified into multiple states St1, St2 according to the appearance of the vehicle 10. In this embodiment, the vehicle 10 is classified into two states St1, St2, a first state St1 and a second state St2, before and after the body shell 18 is assembled. In this embodiment, the first state St1 refers to a case in which the states St1, St2 of the vehicle 10 are the platform state Sp. The platform state Sp means a state in which the vehicle 10 is in the form of a platform. The second state St2 refers to a case where the states St1, St2 of the vehicle 10 are the assembled state Sm. The assembled state Sm means a state in which at least the body shell 18 is assembled to the platform vehicle 10p. Note that the number and types of the manufacturing processes Pr1 to Pr7 executed in the factory Fc1 and the number and types of the states St1, St2 of the vehicle 10 classified according to the appearance of the vehicle 10 are not limited to those described above.

[0020] In the following, the vehicle 101 classified into the first state St1 is referred to as the "first vehicle 101." The vehicle 102 classified into the second state St2 is referred to as the "second vehicle 102." In this embodiment, the first vehicle 101 is the platform vehicle 10p. The second vehicle 102 is the assembled vehicle 10m. When there is no need to distinguish between the vehicles 101, 102 in the multiple states St1, St2, they will simply be referred to as "vehicle 10."

[0021] As shown in FIG. 1, the calculation system 5 includes a first sensor Se1 and a second sensor Se2 serving as external sensors Se provided at a location different from that of the vehicle 10, and a calculation device 6.

[0022] As shown in FIG. 3, the first sensor Se1 is installed in a first area Ar11 in a factory Fc1. The first area Ar11 is an area in which the first vehicle 101 travels. In this embodiment, the first area Ar11 includes a manufacturing location PL1 and a first road Ro1. The installation position, orientation, and number of the first sensors Se1 in the first area Ar11 are determined in consideration of the detection range RG1 of the first sensor Se1 and the like. The first sensor Se1 detects the first vehicle 101 from the outside, and outputs first sensor information representing the first vehicle 101 as a detection result. In this embodiment, the first sensor Se1 is a LiDAR (Light Detection And Ranging: LiDAR). Therefore, in this embodiment, the LiDARs Li1 and Li2 as the first sensor Se1 detect the platform vehicle 10p from the outside, and output measurement point cloud data representing the external shape of the platform vehicle 10p in a three-dimensional manner as the first sensor information.

[0023] The second sensor Se2 is installed in a second area Ar21 in the factory Fc1. The second area Ar21 is an area where the second vehicle 102 travels. In this embodiment, the second area Ar21 includes an assembly location PL3, a second runway Ro2, an inspection location PL5, a third runway Ro3, and a storage location PL7. The installation position, orientation, and number of the second sensors Se2 in the second area Ar21 are determined in consideration of the detection range RG2 of the second sensor Se2, etc. The second sensor Se2 detects the second vehicle 102 from the outside, and outputs second sensor information representing the second vehicle 102 as a detection result. In this embodiment, the second sensor Se2 is a .Cm having an imaging element and an optical system that are sensitive to visible light. In the following, the visible light camera Cm is also simply referred to as a "camera Cm". The camera Cm is, for example, either a CCD image sensor or a CMOS image sensor. Therefore, in this embodiment, the cameras Cm1 to Cm5 serving as the second sensor Se2 capture an image of the assembled vehicle 10m from the outside, and output an original image including the assembled vehicle 10m as the second sensor information.

[0024] FIG. 4 is a diagram showing the configuration of the calculation device 6. The calculation device 6 calculates at least one of the position and the orientation of the vehicle 10 by using sensor information, which is the detection result of the vehicle 10 output from an external sensor Se installed in the factory Fc1. The position of the vehicle 10 is the position of a positioning point 10f that is preset for a specific part of the vehicle 10. The position of the vehicle 10 is represented by X, Y, and Z coordinates in the global coordinate system of the factory Fc1. The orientation of the vehicle 10 is a direction represented by a vector that runs from the rear side to the front side of the vehicle 10 along the longitudinal axis of the vehicle 10 that passes through the center of gravity of the vehicle 10.

[0025] The calculation device 6 includes a communication unit 61, a storage unit 63, and a CPU 62. In the calculation device 6, the communication unit 61, the storage unit 63, and the CPU 62 are each connected to each other via a bus or the like so as to be able to communicate bidirectionally.

[0026] The communication unit 61 of the calculation device 6 communicably connects the calculation device 6 to devices other than the calculation device 6. The communication unit 61 of the calculation device 6 is, for example, a wireless communication device.

[0027] The storage unit 63 of the calculation device 6 stores various information including various programs P63 that control the operation of the calculation device 6, management information In, and a first detection model Md1 as the detection model Md. The storage unit 63 of the calculation device 6 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0028] The management information In is information indicating the travel order of multiple vehicles 10 traveling in the detection ranges RG1, RG2 of the external sensors Se. In the management information In, vehicle identifiers and sensor identifiers Li1, Li2, Cm1 to Cm5 are associated with each other. The vehicle identifiers are unique identifiers assigned so as not to overlap among the vehicles 10 in order to identify the multiple vehicles 10. The sensor identifiers Li1, Li2, Cm1 to Cm5 are unique identifiers assigned so as not to overlap among the external sensors Se in order to identify the multiple external sensors Se installed in the factory Fc1. The management information In is created using, for example, the position and orientation of the vehicle 10, the transmission history of travel control signals to the vehicle 10, and the installation locations of the external sensors Se.

[0029] The detection model Md is a trained machine learning model that utilizes artificial intelligence to detect the exterior shape of the vehicle 10 represented by a captured image. When a captured image is input, the detection model Md detects the exterior shape of the vehicle 10 from the captured image and outputs shape data indicating the exterior shape of the vehicle 10. For example, a convolutional neural network (hereinafter, CNN) that realizes either semantic segmentation or instance segmentation can be used for the detection model Md.

[0030] In this embodiment, the storage unit 157 of the calculation device 6 stores one first detection model Md1 as the detection model Md. When a captured image is input, the first detection model Md1 detects the vehicle 10 from the captured image and outputs shape data. The captured image is either an original image as raw data output from the camera Cm or a processed image obtained by processing the original image. The first detection model Md1 is trained in advance by supervised learning using a first learning dataset. The first learning dataset has a plurality of training images including the vehicle 10 and a region correct answer label indicating whether each region in the training image is a region indicating the vehicle 10 or a region indicating something other than the vehicle 10. During training of the CNN, it is preferable that the parameters of the CNN are updated by backpropagation (error backpropagation method) so as to reduce an error between the output result by the first detection model Md1 and the region correct answer label.

[0031] As shown in FIG. 4, the CPU 62 of the calculation device 6 deploys various programs P63 stored in the storage unit 63, thereby functioning as a sensor information acquisition unit 621 and a calculation unit 622.

[0032] The sensor information acquisition unit 621 acquires sensor information output from an external sensor Se whose detection ranges RG1, RG2 include the vehicle 10 that is the calculation target. In this embodiment, the sensor information acquisition unit 621 identifies the external sensor Se whose detection ranges RG1, RG2 include the vehicle 10 that is the calculation target, using the management information In stored in the storage unit 63 of the calculation device 6. Then, the sensor information acquisition unit 621 acquires the sensor information output from the identified external sensor Se.

[0033] The calculation unit 622 calculates at least one of the position and the orientation of the vehicle 10 using the sensor information acquired by the sensor information acquisition unit 621. Specifically, when the first sensor information is acquired by the sensor information acquisition unit 621, the calculation unit 622 calculates at least one of the position and the orientation of the first vehicle 101 represented by the first sensor information using the first sensor information without performing pre-processing. When the second sensor information is acquired by the sensor information acquisition unit 621, the calculation unit 622 performs pre-processing and calculates at least one of the position and the orientation of the second vehicle 102 represented by the second sensor information using the second sensor information. In this embodiment, the pre-processing is a learning process that trains the first detection model Md1 as a machine learning model. Therefore, when the sensor information acquisition unit 621 acquires the measurement point cloud data, the calculation unit 622 calculates the position and the orientation of the platform vehicle 10p represented by the measurement point cloud data using the measurement point cloud data without performing learning process. When the sensor information acquisition unit 621 acquires an original image, the calculation unit 622 calculates the position and orientation of the assembled vehicle 10m represented by the captured image using the first detection model Md1 learned by executing a learning process and the captured image. Then, the calculation unit 622 transmits position and orientation information indicating the calculation results of the position and orientation of the vehicle 10 to the remote control device 7. Note that at least a part of the functions of the CPU 62 of the calculation device 6 may be realized by a hardware circuit.

[0034] 5 is a diagram showing a configuration of the remote control device 7. The remote control device 7 generates a driving control signal that defines the driving operation of the vehicle 10, and transmits the signal to the vehicle 10. The remote control device 7 includes a communication unit 71, a storage unit 73, and a CPU 72. In the remote control device 7, the communication unit 71, the storage unit 73, and the CPU 72 are each connected via a bus or the like so as to be able to communicate in both directions.

[0035] The communication unit 71 of the remote control device 7 communicably connects the remote control device 7 to devices other than the remote control device 7. The communication unit 71 of the remote control device 7 is, for example, a wireless communication device.

[0036] The storage unit 73 of the remote control device 7 stores various information including various programs P73 that control the operation of the remote control device 7 and an ideal route Ip. The ideal route Ip is a route along which the vehicle 10 should travel. The ideal route Ip is represented by nodes indicating the departure point, nodes indicating passing points, nodes indicating the destination, and links connecting the nodes. The storage unit 73 of the remote control device 7 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0037] The CPU 72 of the remote control device 7 functions as a remote control unit 721 by developing various programs P73 stored in the storage unit 73. The remote control unit 721 generates a travel control signal and transmits it to the vehicle 10 to remotely control the travel operation of the vehicle 10. In order to generate the travel control signal, the remote control unit 721 first determines a target position to which the vehicle 10 should next head. In this embodiment, the target position is represented by X, Y, and Z coordinates in the global coordinate system of the factory Fc1. The remote control unit 721 uses the position direction information and the ideal route Ip to determine a target position to which the vehicle 10 should next head. The remote control unit 721 determines a target position on the ideal route Ip ahead of the current position of the vehicle 10. The remote control unit 721 generates a travel control signal for driving the vehicle 10 toward the determined target position. In this embodiment, the travel control signal includes the acceleration and steering angle of the vehicle 10 as parameters. The remote control unit 721 calculates the running speed of the vehicle 10 from the transition of the position of the vehicle 10, and compares the calculated running speed with a predetermined target speed of the vehicle 10. When the running speed is lower than the target speed, the remote control unit 721 determines the acceleration so that the vehicle 10 accelerates, and when the running speed is higher than the target speed, the remote control unit 721 determines the acceleration so that the vehicle 10 decelerates. When the vehicle 10 is located on the ideal route Ip, the remote control unit 721 determines the steering angle so that the vehicle 10 does not deviate from the ideal route Ip, and when the vehicle 10 is not located on the ideal route Ip, in other words, when the vehicle 10 deviates from the ideal route Ip, the remote control unit 721 determines the steering angle so that the vehicle 10 returns to the ideal route Ip. In this way, the remote control unit 721 generates a running control signal including the acceleration and steering angle of the vehicle 10 as parameters. Then, the remote control unit 721 transmits the generated running control signal to the vehicle 10. The remote control unit 721 repeats the acquisition of position and direction information, the determination of a target position, the generation of a driving control signal, and the transmission of the driving control signal at a predetermined cycle. At least a part of the functions of the CPU 72 of the remote control device 7 may be realized by a hardware circuit.

[0038] Fig. 6 is a flowchart showing a processing flow in the first embodiment. The processing flow shown in Fig. 6 is repeatedly executed at predetermined time intervals from the time when the vehicle 10 starts traveling by unmanned driving, for example.

[0039] In step S101, the sensor information acquisition unit 621 of the calculation device 6 uses the management information In to identify an external sensor Se that includes the vehicle 10 to be calculated in its detection range RG1, RG2 among a plurality of external sensors Se installed in the factory Fc1. In step S102, the sensor information acquisition unit 621 transmits a request signal for acquiring sensor information to the identified external sensor Se. In step S103, the external sensor Se that has received the request signal transmits sensor information to the calculation device 6. When the sensor information acquisition unit 621 of the calculation device 6 acquires the measurement point cloud data, in step S104, the calculation unit 622 calculates the position and orientation of the platform vehicle 10p represented by the measurement point cloud data using the measurement point cloud data without using the first detection model Md1. When the sensor information acquisition unit 621 of the calculation device 6 acquires the original image, in step S105, the calculation unit 622 calculates the position and orientation of the assembled vehicle 10m represented by the captured image using the first detection model Md1 and the captured image. In step S106, the calculation unit 622 transmits the position and direction information to the remote control device 7.

[0040] In step S107, the remote control unit 721 of the remote control device 7 uses the position / direction information and the ideal route Ip to determine a target position to which the vehicle 10 should next head. In step S108, the remote control unit 721 generates a driving control signal for driving the vehicle 10 toward the determined target position. In step S109, the remote control unit 721 transmits the driving control signal to the vehicle 10. In step S110, the operation control unit 154 of the vehicle control device 15 mounted on the vehicle 10 uses the received driving control signal to control the actuator, thereby causing the vehicle 10 to drive at the acceleration and steering angle represented by the driving control signal. The vehicle control device 15 repeats receiving the driving control signal and controlling the actuator at a predetermined cycle.

[0041] Fig. 7 is a flowchart showing an example of a calculation method for calculating the position and orientation of the platform vehicle 10p represented by the measurement point cloud data using the measurement point cloud data without using the first detection model Md1. In the example shown in Fig. 7, the calculation unit 622 calculates the position and orientation of the platform vehicle 10p represented by the measurement point cloud data by matching the measurement point cloud data with reference point cloud data prepared in advance. The reference point cloud data is, for example, three-dimensional CAD data of the platform vehicle 10p. For example, an algorithm such as ICP (Iterative Closest Point) or NDT (Normal Distribution Transform) is used for matching the measurement point cloud data with the reference point cloud data.

[0042] In step S104a, the calculation unit 622 acquires the measurement point cloud data. In step S104b, the calculation unit 622 matches the measurement point cloud data with the reference point cloud data to calculate the position and orientation of the platform vehicle 10p represented by the measurement point cloud data.

[0043] The method of calculating the position and orientation of the vehicle 10 represented by the measurement point cloud data is not limited to the above. The calculation unit 622 may calculate the position and orientation of the vehicle 10, for example, by using the distance between the lidar Li and the vehicle 10 and the angle difference between the optical axis reference of the lidar Li and the vector from the lidar Li toward the vehicle 10, without performing the above matching.

[0044] Fig. 8 is a flow chart showing an example of a calculation method for calculating the position and orientation of the assembled vehicle 10m represented by a captured image using the first detection model Md1 and the captured image. Fig. 9 is a schematic diagram showing examples of various images Im1 to Im6 when the calculation method shown in Fig. 8 is executed. In Fig. 9, step numbers corresponding to each step in Fig. 8 are assigned.

[0045] In the example shown in FIG. 8, the calculation unit 622 uses an optical flow method to calculate the orientation of the assembled vehicle 10m based on the orientation of the movement vector V of the assembled vehicle 10m calculated from the positional changes of the feature points of the assembled vehicle 10m between frames of captured images. The calculation unit 622 also acquires shape data Da indicating the external shape of the assembled vehicle 10m by inputting the captured images Im1 to Im4 to the first detection model Md1. The calculation unit 622 calculates the coordinates of the positioning point 10f of the assembled vehicle 10m in the image coordinate system using the shape data Da, and converts the calculated coordinates into coordinates in the global coordinate system to calculate the position of the assembled vehicle 10m. A method of generating the shape data Da and a method of calculating the position of the vehicle 10 using the shape data Da will be described below.

[0046] In an image acquisition step of step S105a, the calculation unit 622 acquires an original image Im1 output from the camera Cm. In a distortion correction step of step S105b, the calculation unit 622 corrects the distortion of the original image Im1 to generate a corrected image Im2 as a processed image. In a rotation processing step of step S105c, the calculation unit 622 rotates the corrected image Im2 so that the direction of the movement vector indicating the movement direction of the assembled vehicle 10m included in the corrected image Im2 faces a predetermined direction. As a result, the calculation unit 622 generates a rotated image Im3 as a processed image. In a crop processing step of step S105d, the calculation unit 622 deletes from the rotated image Im3 an unnecessary area A2 other than a necessary area A1 constituted by the assembled vehicle 10m and the surrounding area of ​​the assembled vehicle 10m, among the respective areas constituting the corrected image Im2. As a result, the calculation unit 622 generates a processed image Im4 as a processed image. Each of the steps shown from step S105b to step S105d is an image processing step for performing pre-processing to improve the detection accuracy when detecting the assembled vehicle 10m from the captured images Im1 to Im4. Therefore, at least one of the steps shown from step S105b to step S105d may be omitted.

[0047] In the detection process of step S105g, the calculation unit 622 inputs the processed image Im4 to the first detection model Md1. As a result, the calculation unit 622 detects the assembled vehicle 10m from the processed image Im4 and obtains a first mask image Im5 as the shape data Da. The first mask image Im5 is an image in which a mask area Ms is added to the processed image Im4 by masking the area indicating the assembled vehicle 10m among the areas constituting the processed image Im4.

[0048] In the perspective transformation process of step S105h, the calculation unit 622 performs perspective transformation on the first mask image Im5 to generate a second mask image Im6. The calculation unit 622 performs perspective transformation on the first mask image Im5 to generate a bird's-eye image seen from a viewpoint above the assembled vehicle 10m that is substantially perpendicular to the road surface Rs, for example, using a predetermined perspective transformation parameter. The perspective transformation parameter is, for example, a parameter related to position information and internal parameters of the camera Cm obtained by calibration. As a result, the calculation unit 622 generates a second mask image Im6 expressed in the image coordinate system from the first mask image Im5 expressed in the camera coordinate system. The camera coordinate system is a coordinate system that has the focal point of the camera Cm as the origin and has coordinate axes represented by the Xc axis and the Yc axis perpendicular to the Xc axis. The image coordinate system is a coordinate system having a point on an image plane projected by perspective transformation as its origin, and having coordinate axes indicated by the Xi axis and the Yi axis perpendicular to the Xi axis.

[0049] In the coordinate point calculation process of step S105i, the calculation unit 622 calculates an image coordinate point indicating the position of the assembled vehicle 10m in the image coordinate system. FIG. 10 is a diagram for explaining the details of the coordinate point calculation process. In order to calculate the image coordinate point, the calculation unit 622 first calculates a base coordinate point P0 from a first circumscribing rectangle R1 set in the mask area Ms in the first mask image Im5, which is an image before perspective transformation. FIG. 11 is a diagram for explaining a calculation method of the base coordinate point P0. In order to calculate the base coordinate point P0, the calculation unit 622 sets a base circumscribing rectangle R0 for the mask area Ms in the first mask image Im5. Next, the calculation unit 622 rotates the first mask image Im5 by a required rotation amount with the center of gravity C of the mask area Ms as the rotation center so that the direction of the movement vector V of the assembled vehicle 10m corresponding to the mask area Ms in the first mask image Im5 faces a predetermined direction. The predetermined direction is, for example, the upward direction on the screen of the display device that displays the first mask image Im5. Next, the calculation unit 622 sets a first circumscribing rectangle R1 for the mask area Ms of the rotated first mask image Im5 so that the long side is parallel to the movement vector V. Next, the calculation unit 622 rotates the first mask image Im5 to which the first circumscribing rectangle R1 has been added by the above-mentioned rotation amount, with the center of gravity C of the mask area Ms as the rotation center. As a result, the calculation unit 622 sets the coordinate point of one of the four vertices of the first circumscribing rectangle R1, which has coordinates that are closest to the positioning point 10f of the assembled vehicle 10m, as the base coordinate point P0.

[0050] Next, the calculation unit 622 performs perspective transformation on the first mask image Im5 after reverse rotation, i.e., the first mask image Im5 after calculating the base coordinate point P0, as shown in Fig. 10. As a result, the calculation unit 622 sets the coordinate point corresponding to the base coordinate point P0 as the first coordinate point P1 in the first circumscribed rectangle R1 deformed by the perspective transformation.

[0051] Next, the calculation unit 622 sets a second circumscribing rectangle R2 for the mask region Ms in the second mask image Im6 obtained by performing perspective transformation on the first mask image Im5. Then, the calculation unit 622 sets, among the vertices of the second circumscribing rectangle R2, a vertex indicating the same position as the first coordinate point P1 as the second coordinate point P2. That is, the first coordinate point P1 and the second coordinate point P2 are coordinate points indicating the same position, and therefore have a correlation with each other.

[0052] Next, the calculation unit 622 performs correction by replacing the coordinates (Xi1, Yi1) of the first coordinate point P1 with the coordinates (Xi2, Yi2) of the second coordinate point P2 depending on the magnitude relationship between the coordinate values ​​of the first coordinate point P1 and the second coordinate point P2. If the coordinate value Xi1 in the Xi direction of the first coordinate point P1 is greater than the coordinate value Xi2 in the Xi direction of the second coordinate point P2 (Xi1>Xi2), the calculation unit 622 replaces the coordinate value Xi1 in the Xi direction of the first coordinate point P1 with the coordinate value Xi2 in the Xi direction of the second coordinate point P2. When the coordinate value Yi1 of the first coordinate point P1 in the Yi direction is greater than the coordinate value Yi2 of the second coordinate point P2 in the Yi direction (Yi1>Yi2), the calculation unit 622 replaces the coordinate value Yi1 of the first coordinate point P1 in the Yi direction with the coordinate value Yi2 of the second coordinate point P2 in the Yi direction. In the example shown in FIG. 10, the coordinate value Xi1 of the first coordinate point P1 in the Xi direction is greater than the coordinate value Xi2 of the second coordinate point P2 in the Xi direction. Also, the coordinate value Yi1 of the first coordinate point P1 in the Yi direction is smaller than the coordinate value Yi2 of the second coordinate point P2 in the Yi direction. Therefore, the image coordinate point P3 has coordinates (Xi2, Yi1). In this way, the calculation unit 622 calculates the image coordinate point P3 indicating the position of the assembled vehicle 10m in the image coordinate system by correcting the first coordinate point P1 using the second coordinate point P2.

[0053] In the position conversion process of step S105j, the calculation unit 622 converts the image coordinate point P3 into a vehicle coordinate point to calculate a vehicle coordinate point indicating the position of the positioning point 10f of the assembled vehicle 10m in the global coordinate system. The calculation unit 622 converts the image coordinate point P3 into a vehicle coordinate point using relational expressions (1) to (3) described later, which use the vehicle coordinate point as a target variable and include the image coordinate point P3, the imaging parameters, and the vehicle parameters as explanatory variables. The imaging parameters are parameters related to the distance of the camera Cm from a predetermined reference point. In this embodiment, the imaging parameters are height H (FIG. 12 described later) of the camera Cm from the road surface Rs. The vehicle parameters are parameters related to the distance of the positioning point 10f of the assembled vehicle 10m from the reference point. In this embodiment, the vehicle parameters are height h (FIG. 12 described later) of the positioning point 10f of the assembled vehicle 10m from the road surface Rs.

[0054] FIG. 12 is a first diagram for explaining the details of the position conversion process. FIG. 12 illustrates the state of the assembled vehicle 10m as viewed from the left side. FIG. 13 is a second diagram for explaining the details of the position conversion process. FIG. 13 illustrates the state of the assembled vehicle 10m as viewed from the roof side. The global coordinate system shown in FIG. 12 and FIG. 13 is a coordinate system having a fixed coordinate point Pf indicating an arbitrary reference position of the road surface Rs as the origin, and coordinate axes indicated by the Xg axis and the Yg axis perpendicular to the Xg axis. The imaging coordinate point Pc is a coordinate point indicating the position in the global coordinate system of the camera Cm that output the original image Im1 used to calculate the image coordinate point P3. The fixed coordinate point Pf and the imaging coordinate point Pc are stored in advance in, for example, the storage unit 63 of the calculation device 6.

[0055] As shown in Fig. 12, the observation distance on the XgYg plane between the position of camera Cm and the position of assembled vehicle 10m (image coordinate point P3) is Do. The observation error is ΔD. The height [m] of camera Cm from road surface Rs as an imaging parameter is H. The height [m] of positioning point 10f of assembled vehicle 10m from road surface Rs as a vehicle parameter is h. In this case, the observation error ΔD can be expressed by the following equation (1). ΔD=h / H×Do Equation (1) That is, the larger the observation distance Do, the larger the observation error ΔD.

[0056] When the actual distance between the position of the camera Cm and the position of the positioning point 10f of the assembled vehicle 10m is defined as a first distance D, the first distance D can be expressed by the following formula (2). D=Do×(1-h / H) Equation (2) That is, the first distance D is determined by the observation distance Do, the height H of the camera Cm as an imaging parameter, and the height h of the positioning point 10f of the assembled vehicle 10m as a vehicle parameter.

[0057] As shown in Figure 13, if the distance between the reference position and the estimated position of the assembled vehicle 10m is the estimated distance Dp, and the actual distance between the reference position and the assembled vehicle 10m is the second distance Dt, the second distance Dt can be expressed by the following equation (3). Dt = Dp × (1-h / H) Equation (3)

[0058] Here, the estimated distance Dp can be calculated using the third distance Dc, which is the actual distance calculated from the fixed coordinate point Pf and the imaged coordinate point Pc, the image coordinate point P3, and the fixed coordinate point Pf. Therefore, the calculation unit 622 calculates the vehicle coordinate point Pv using the second distance Dt and the fixed coordinate point Pf, which are calculated by correcting the estimated distance Dp using the above formula (3). At this time, the calculated vehicle coordinate point Pv is a coordinate point indicating the position of the positioning point 10f of the assembled vehicle 10m in the global coordinate system, and therefore corresponds to the position of the assembled vehicle 10m in the real space.

[0059] The method of calculating the position and orientation of the vehicle 10 represented by the captured images Im1 to Im4 is not limited to the above. The calculation unit 622 may calculate the orientation of the vehicle 10 using, for example, the output result of a yaw rate sensor or the like mounted on the vehicle 10.

[0060] According to the first embodiment, in order to run the vehicle 10 by remote control, the calculation device 6 acquires sensor information from the external sensor Se and detects the external shape of the vehicle 10 represented by the sensor information to calculate the position of the vehicle 10. At this time, the vehicle 10 to be calculated has different appearances depending on the manufacturing processes Pr1 to Pr7 since it runs in the factory Fc1. When detecting the external shape of the vehicle 10 using the detection model Md as a machine learning model trained by executing a learning process, the feature amount extracted during the training of the first detection model Md1 differs depending on the states St1 and St2 of the vehicle 10. Therefore, in order to suppress a decrease in detection accuracy when detecting the external shape of the vehicle 10 from the sensor information due to differences in the appearance of the vehicle 10, it is preferable to perform the following when detecting the external shape of the vehicle 10 using the detection model Md. In this case, it is preferable to train the detection model Md for detecting the external shape of the vehicle 10 represented by the sensor information for each state St1 and St2 of the vehicle 10 according to the appearance of the vehicle 10. However, when the detection model Md is trained for each state St1, St2 of the vehicle 10, it may be necessary to prepare learning data for each state St1, St2 of the vehicle 10, or the learning time may be longer by the number of states St1, St2 of the vehicle 10. In contrast, according to the first embodiment, in the factory Fc1, the first sensor Se1 is installed in the first area Ar11, and the second sensor Se2 of a different type from the first sensor Se1 is installed in the second area Ar21. Thereby, when calculating the position of the first vehicle 101, the calculation device 6 can execute the following process. In this case, the calculation device 6 can obtain the first sensor information representing the first vehicle 101 output from the first sensor Se1 by detecting the first vehicle 101 from the outside by the first sensor Se1. Then, the calculation device 6 can calculate the position of the first vehicle 101 represented by the first sensor information by using the first sensor information without executing the learning process. In addition, when calculating the position of the second vehicle 102, the calculation device 6 can execute the following process.In this case, the calculation device 6 can obtain second sensor information representing the second vehicle 102 output from the second sensor Se2 by detecting the second vehicle 102 from the outside using the second sensor Se2. Then, the calculation device 6 can calculate the position of the second vehicle 102 represented by the second sensor information, etc., using the first detection model Md1, which is the detection model Md, and the captured image. In this way, it is not necessary to make the first detection model Md1 learn the feature amount of the first vehicle 101. Therefore, it is possible to save the effort of preparing learning data for the first vehicle 101. Furthermore, it is possible to reduce the time required to learn the feature amount of the first vehicle 101 out of the time required to learn the first detection model Md1. This makes it possible to shorten the learning time of the first detection model Md1. As a result, it is possible to reduce the load required for the learning process while reducing the possibility of a decrease in detection accuracy when detecting the external shape of the vehicle 10 from the first sensor information.

[0061] Here, in order to reduce the load required for the learning process, a method of calculating the position, etc. of the vehicle 10 without executing the learning process for the vehicle 10 in either state St1, St2 is considered. However, depending on the type of external sensor Se, even if the position, etc. of the vehicle 10 can be calculated without executing the learning process, the cost required to calculate the position, etc. of the vehicle 10 may be higher than when the position, etc. is calculated by executing the learning process. In contrast, according to the first embodiment, different types of external sensors Se can be installed in the first area Ar11 and the second area Ar21 depending on the price of the external sensor Se. This can reduce the possibility that the cost required to calculate the position, etc. of the vehicle 10 will be high.

[0062] According to the first embodiment, the first sensor Se1 is a lidar Li that outputs measurement point cloud data as the first sensor information. The calculation device 6 can calculate the position of the first vehicle 101 represented by the measurement point cloud data by matching the measurement point cloud data with the reference point cloud data without using the detection model Md.

[0063] According to the first embodiment, the second sensor Se2 is a camera Cm that outputs an original image Im1 as second sensor information. The calculation device 6 acquires shape data Da by inputting the captured images Im1 to Im4 to the first detection model Md1, calculates an image coordinate point P3 using the shape data Da, and converts the image coordinate point P3 into a vehicle coordinate point Pv, thereby calculating the position of the second vehicle 102, etc.

[0064] According to the first embodiment, the calculation device 6 can calculate the position and the like of the first vehicle 101 by using the measurement point cloud data output from the lidar Li without executing a learning process. However, the lidar Li is generally more expensive than the camera Cm. Therefore, when the vehicle 10 in any of the states St1, St2 is detected by the lidar Li, the cost required to calculate the position and the like of the vehicle 10 is higher than when any of the states St1, St2 is detected by the camera Cm. In contrast, according to the first embodiment, the calculation device 6 can calculate the position and the like of the second vehicle 102 by using the first detection model Md1 learned by executing a learning process and the captured images Im1 to Im4. This can reduce the cost required to calculate the position and the like of the second vehicle 102. Therefore, the calculation device 6 can calculate the position and the like of the vehicle 10 at low cost while reducing the load required for the learning process executed to calculate the position and the like of the vehicle 10.

[0065] According to the first embodiment, the traveling system 1 can remotely control the vehicle 10 to travel within the factory Fc1. Therefore, the traveling system 1 can move the vehicle 10 within the factory Fc1 without using transportation equipment such as a crane or a conveyor.

[0066] B. Second embodiment: 14 is a diagram showing a state in which a vehicle 10 travels within a factory Fc2 in the second embodiment. In this embodiment, the vehicle 10 is manufactured as a finished product by performing a platform manufacturing process Pr1, a first assembly process Pr3a, a second assembly process Pr3b, and an inspection process Pr5 in this order as the multiple manufacturing processes Pr1 to Pr8. Then, the finished vehicle 10 is stored in a storage location PL7 in a storage process Pr7 after passing through a transport process Pr8 until it is shipped.

[0067] The platform manufacturing process Pr1 and the inspection process Pr5 are the same as those in the first embodiment. The first assembly process Pr3a is a process of assembling interior parts such as a seat 19 to the platform vehicle 10p. The first assembly process Pr3a is performed at the first assembly location PL3a. The second assembly process Pr3b is a process of assembling exterior parts such as a body shell 18 to the vehicle 103 to which the interior parts are assembled. The second assembly process Pr3b is performed at the second assembly location PL3b. In this embodiment, the manufacturing location PL1, the first assembly location PL3a, the second assembly location PL3b, and the inspection location PL5 are present in the same building. The storage location PL7 is present in a building different from the building in which the manufacturing location PL1, the first assembly location PL3a, the second assembly location PL3b, and the inspection location PL5 are present. Therefore, the inspection location PL5 and the storage location PL7 are connected by a runway Ro4. The transport process Pr8 is a process of moving the vehicle 10 from the inspection site PL5 to the storage site PL7. In this embodiment, the vehicle 10 is also remotely controlled by the remote control device 7 to travel in an unmanned manner in at least a part of the section from the manufacturing site PL1 to the storage site PL7.

[0068] The platform vehicle 10p, the vehicle being assembled 10n, and the assembled vehicle 10m have different external shapes, which results in different appearances of the vehicle 10. The vehicle being assembled 10n is a vehicle 10 in which at least a seat 19 has been assembled to the platform vehicle 10p. In this embodiment, the vehicle being assembled 10n is the vehicle 10 after the first assembly process Pr3a has started and before the second assembly process Pr3b has started. The vehicle 10 after the second assembly process Pr3b has started is the assembled vehicle 10m.

[0069] In this embodiment, the vehicle 10 is classified into three states St1 to St3, a first state St1, a second state St2, and a third state St3, depending on whether or not the seat 19 is installed and whether or not the body shell 18 is installed. In this embodiment, the first state St1 refers to a case where the states St1 to St3 of the vehicle 10 are a platform vehicle 10p. The second state St2 refers to a case where the states St1 to St3 of the vehicle 10 are an assembled vehicle 10m. The third state St3 refers to a case where the states St1 to St3 of the vehicle 10 are an in-assembly vehicle 10n.

[0070] Hereinafter, the vehicle 103 classified into the third state St3 will be referred to as the "third vehicle 103." In this embodiment, the first vehicle 101 is the platform vehicle 10p. The second vehicle 102 is the assembled vehicle 10m. The third vehicle 103 is the in-assembly vehicle 10n. When it is not necessary to distinguish between the vehicles 101-103 in the multiple states St1-St3, they will simply be referred to as the "vehicle 10."

[0071] 15 is a diagram showing the configuration of a calculation system 5a in the second embodiment. The calculation system 5a includes a first sensor Se1, a second sensor Se2, a third sensor Se3, and a calculation device 6a. In this embodiment, the first sensor Se1 and the third sensor Se3 are each a lidar Li as an external sensor Se. The second sensor Se2 is a camera Cm as an external sensor Se.

[0072] As shown in FIG. 14, the first sensor Se1 is installed in a first area Ar12 in a factory Fc2. In this embodiment, the first area Ar12 includes a manufacturing location PL1. The second sensor Se2 is installed in a second area Ar22 in the factory Fc2. In this embodiment, the second area Ar22 includes a second assembly location PL3b, an inspection location PL5, a runway Ro4, and a storage location PL7. The third sensor Se3 is installed in a third area Ar31 in the factory Fc2. In this embodiment, the third area Ar31 includes a first assembly location PL3a.

[0073] Thus, the lidar Li1 as the first sensor Se1 detects the platform vehicle 10p from the outside and outputs measurement point cloud data representing the external shape of the platform vehicle 10p as the first sensor information. The cameras Cm1-Cm4 as the second sensor Se2 capture an image of the assembled vehicle 10m from the outside and output an original image Im1 including the assembled vehicle 10m as the second sensor information. The lidar Li2 as the third sensor Se3 detects the vehicle in the process of assembly 10n from the outside and outputs measurement point cloud data representing the external shape of the vehicle in the process of assembly 10n in three dimensions as the third sensor information.

[0074] 15, a calculation device 6a calculates at least one of the position and the orientation of a vehicle 10 using sensor information output from an external sensor Se installed in a factory Fc2. The calculation device 6b includes a communication unit 61, a storage unit 63a, and a CPU 62a. Note that the same components as those in the first embodiment are denoted by the same reference numerals and will not be described.

[0075] The storage unit 63a of the calculation device 6a stores various types of information including various programs P63a for controlling the operation of the calculation device 6b, management information In, and a first detection model Md1.

[0076] The CPU 62a of the calculation device 6a functions as a sensor information acquisition unit 621 and a calculation unit 622a by developing various programs P63a stored in the storage unit 63a. When the sensor information acquisition unit 621 acquires measurement point cloud data from the lidars Li1 and Li2, the calculation unit 622a calculates the position and orientation of the vehicles 10p and 10n by matching the measurement point cloud data with the reference point cloud data. When the sensor information acquisition unit 621 acquires the original image Im1, the calculation unit 622a inputs the captured images Im1 to Im4 to the first detection model Md1. As a result, the calculation unit 622a acquires the shape data Da and calculates the position and orientation of the assembled vehicle 10m represented by the captured images Im1 to Im4.

[0077] According to the second embodiment, in the factory Fc2, of the three areas Ar12 to Ar31, two areas Ar22 and Sr32 are provided with a lidar Li that outputs measurement point cloud data that allows the position, etc. of the vehicle 10 to be calculated without executing a learning process as a pre-processing. As a result, for the first vehicle 101 and the third vehicle 103, the positions, etc. of the vehicles 101 and 103 can be calculated without using the detection model Md that has been learned by executing a learning process. In this way, when the vehicle 10 is classified into three or more states St1 to St3 according to the appearance of the vehicle 10, the position, etc. of the vehicle 10 can be calculated for at least some of the three or more states St1 to St3 without using the detection model Md. Therefore, even when the vehicle 10 is classified into three or more states St1 to St3, the load required for the learning process can be reduced.

[0078] C. Third embodiment: FIG. 16 is a diagram showing a state in which the vehicle 10 travels in the factory Fc3 in the third embodiment. In this embodiment, the same manufacturing processes Pr1 to Pr8 as those in the second embodiment are executed in the same order. Thus, similar to the second embodiment, the vehicle 10 is classified into three states St1 to St3, a first state St1, a second state St2, and a third state St3, depending on whether the seat 19 is installed and whether the body shell 18 is installed. In this embodiment, the first state St1 refers to a case in which the states St1 to St3 of the vehicle 10 are installed in the installed state Sm. The second state St2 refers to a case in which the states St1 to St3 of the vehicle 10 are installed in the platform state Sp. The third state St3 refers to a case in which the states St1 to St3 of the vehicle 10 are installed in the installed state Sn. Note that, in this embodiment as well, the vehicle 10 travels in an unmanned manner by being remotely controlled by the remote control device 7 in at least a part of the section from the manufacturing site PL1 to the storage site PL7. The same components as those in the above-described embodiments are given the same reference numerals and the description thereof will be omitted.

[0079] In this embodiment, the manufacturing location PL1, the first assembly location PL3a, the second assembly location PL3b, and the inspection location PL5 are each located indoors. The inspection location PL5 and the storage location PL7 are connected by a path Ro5. The path Ro5 and the storage location PL7 are each located outdoors.

[0080] 17 is a diagram showing the configuration of a calculation system 5b in the third embodiment. The calculation system 5b includes a first sensor Se1, a second sensor Se2, a third sensor Se3, and a calculation device 6b. In this embodiment, the first sensor Se1 is a lidar Li as an external sensor Se. The second sensor Se2 and the third sensor Se3 are each a camera Cm as an external sensor Se.

[0081] As shown in FIG. 16, the first sensor Se1 is installed in a first area Ar13 in a factory Fc3. In this embodiment, the first area Ar13 includes a second assembly location PL3b, an inspection location PL5, a road Ro5, and a storage location PL7. The second sensor Se2 is installed in a second area Ar23 in the factory Fc3. In this embodiment, the second area Ar23 includes a manufacturing location PL1. The third sensor Se3 is installed in a third area Ar32 in the factory Fc3. The third area Ar32 is an area in which a third vehicle 103 travels. In this embodiment, the third area Ar32 includes the first assembly location PL3a.

[0082] Thus, the lidars Li1-Li4 serving as the first sensor Se1 detect the assembled vehicle 10m from the outside and output, as the first sensor information, measurement point cloud data that three-dimensionally represents the external shape of the assembled vehicle 10m. The camera Cm1 serving as the second sensor Se2 detects the platform vehicle 10p from the outside and outputs, as the second sensor information, an original image Im1 including the platform vehicle 10p. The camera Cm2 serving as the third sensor Se3 detects the vehicle in the process of being assembled 10n from the outside and outputs, as the third sensor information, an original image Im1 including the vehicle in the process of being assembled 10n.

[0083] 17 uses sensor information output from an external sensor Se installed in a factory Fc3 to calculate at least one of the position and the orientation of a vehicle 10. The calculation device 6b includes a communication unit 61, a storage unit 63b, and a CPU 62b.

[0084] The storage unit 63b of the calculation device 6b stores various information including various programs P63b for controlling the operation of the calculation device 6b, management information In, and a plurality of second detection models Md2 prepared as detection models Md for each state St2, St3 of the vehicle 10. When the captured images Im1 to Im4 are input, the second detection model Md2 detects the vehicle 10 from the captured images Im1 to Im4 and outputs shape data Da.

[0085] The second detection models Md2 are second learning data sets prepared for each of the states St2 and St3 of the vehicle 10, and are previously learned by supervised learning using the second learning data sets corresponding to the states St2 and St3 of the vehicle 10. The second learning data set has a plurality of training images including the vehicle 10 and a region correct answer label. The second learning data set includes, as the plurality of training images, M (M is an integer of 2 or more) first training images and N (N is an integer of 0 or more and less than M) second training images. The first training images are images including the vehicle 10 classified into one state. The second training images are images including the vehicle 10 classified into the remaining states other than the one state among the plurality of states St1 to St3 of the vehicle 10. The number M of first training images included in the second learning data set may be, for example, 1.5 times or more, or may be 2 times or more, the number N of second training images. Furthermore, when there are a plurality of other states different from the one state, the number M of first training images included in the second learning data set is, for example, greater than the total number of second training images each including a vehicle 10 classified into one of the remaining states other than the one state. In other words, the second detection model Md2 is a machine learning model that preferentially learns the feature amounts of the vehicle 10 classified into the one state by learning more first training images than second training images.

[0086] As shown in FIG. 16, in this embodiment, the platform vehicle 10p and the vehicle being assembled 10n are detected from the captured images Im1 to Im4, and the position and orientation are calculated. Therefore, the storage unit 63b of the calculation device 6b stores various information including two second detection models Md2, as shown in FIG. 17. The first second detection model Md2a is a second detection model Md2 that has been trained with more first training images including the platform vehicle 10p than with more second training images including either the vehicle being assembled 10n or the assembled vehicle 10m. The first second detection model Md2a is used when detecting the platform vehicle 10p from the captured images Im1 to Im4. The second second detection model Md2b is a second detection model Md2 that has been trained with more first training images including the vehicle being assembled 10n than with more second training images including either the platform vehicle 10p or the assembled vehicle 10m. The second detection model Md2b is used when detecting the in-assembly vehicle 10n from the captured images Im1 to Im4.

[0087] The CPU 62b of the calculation device 6b functions as a sensor information acquisition unit 621 and a calculation unit 622b by developing various programs P63b stored in the storage unit 63b. When the sensor information acquisition unit 621 acquires measurement point cloud data from the lidars Li1 to Li4, the calculation unit 622b calculates the position and orientation of the assembled vehicle 10m by matching the measurement point cloud data with the reference point cloud data. When the sensor information acquisition unit 621 acquires the original image Im1, the calculation unit 622b executes the following process. In this case, the calculation unit 622b inputs the captured images Im1 to Im4 to the second detection models Md2a and Md2b selected according to the states St2 and St3 of the vehicle 10 from the two second detection models Md2a and Md2b prepared for each state St2 and St3 of the vehicle 10. Specifically, when the sensor information acquisition unit 621 acquires an original image Im1 from a camera Cm1 installed in the second area Ar23, the calculation unit 622b inputs the captured images Im1 to Im4 to the second detection model Md2a corresponding to the platform state Sp. When the sensor information acquisition unit 621 acquires an original image Im1 from a camera Cm2 installed in the third area Ar32, the calculation unit 622b inputs the captured images Im1 to Im4 to the second detection model Md2b corresponding to the installation state Sn. In this way, the calculation unit 622b acquires shape data Da and calculates the positions and orientations of the vehicles 10p, 10n represented by the captured images Im1 to Im4.

[0088] Fig. 18 is a flowchart showing a method for calculating the position of the vehicle 10 using the second detection model Md2. The same steps as those in the calculation method shown in Fig. 8 are given the same step numbers and descriptions thereof will be omitted.

[0089] In a state acquisition step of step S105e, the calculation unit 622b acquires state information. The state information is information indicating which of the states St1 to St3 of the vehicle 10 is in. Note that the state acquisition step may be executed at any time after the image acquisition step is completed and before the detection step is started.

[0090] In a model acquisition step of step S105f, the calculation unit 622b selects and acquires the second detection models Md2a, Md2b corresponding to the states St2, St3 of the vehicle 10 specified by the state information from the two second detection models Md2a, Md2b pre-stored in the storage unit 63b of the calculation device 6b. Note that the model acquisition step may be executed at any time after the state acquisition step is completed and before the detection step is started.

[0091] According to the third embodiment, the second detection model Md2 suitable for detecting the vehicle 10 classified into one state is prepared in advance by learning more first training images including the vehicle 10 classified into one state than second training images including the vehicle 10 classified into the remaining other states. As a result, the calculation device 6b can acquire state information and select the second detection model Md2 corresponding to the one state specified by the acquired state information from the multiple second detection models Md2 prepared for each state. Then, the calculation device 6b can accurately detect the vehicles 102, 103 from the captured images Im1 to Im4 by inputting the captured images Im1 to Im4 to the second detection model Md2 corresponding to the states St2, St3 of the vehicles 102, 103 included in the captured images Im1 to Im4. Therefore, the detection accuracy when detecting the vehicles 102, 103 from the captured images Im1 to Im4 can be improved.

[0092] Here, when the vehicle 10 is captured using the camera Cm, the capturing conditions may change depending on the external environment at the time of capturing. For example, due to differences in weather such as sunny or rainy, and differences in time of day or night, differences in the illumination of sunlight in the detection range RG2 of the camera Cm may occur, and the brightness of the captured images Im1 to Im4 may change. In this way, when the capturing conditions change depending on the external environment at the time of capturing, there is a risk that the detection accuracy when detecting the vehicle 10 from the captured images Im1 to Im4 may decrease. On the other hand, when detecting the vehicle 10 using the lidar Li, it is difficult to be affected by disturbances according to the external environment such as sunlight. Therefore, in the above third embodiment, the camera Cm is installed in the second area Ar23 and the third area Ar32 that are located indoors, and the lidar Li is installed in the first area Ar13 that is partially located outdoors. In this way, it is possible to suppress a decrease in the detection accuracy when detecting the vehicle 10 from the sensor information.

[0093] D. Fourth embodiment: FIG. 19 is a diagram showing the configuration of a calculation system 5c in the fourth embodiment. In this embodiment, the type of the detection model Md is different from each of the above embodiments. The configuration of the factory Fc3 is the same as that of the third embodiment shown in FIG. 16. Thus, similarly to the third embodiment, the vehicle 10 is classified into three states St1 to St3, namely, a state St1, a second state St2, and a third state St3, depending on whether the seat 19 is installed and whether the body shell 18 is installed. In this embodiment, the first state St1 refers to a case where the states St1 to St3 of the vehicle 10 are the installed state Sm. The second state St2 refers to a case where the states St1 to St3 of the vehicle 10 are the platform state Sp. The third state St3 refers to a case where the states St1 to St3 of the vehicle 10 are the installing state Sn. In this embodiment as well, the vehicle 10 travels in an unmanned manner by being remotely controlled by the remote control device 7 in at least a part of the section from the manufacturing site PL1 to the storage site PL7. The same components as those in the above-described embodiments are given the same reference numerals and the description thereof will be omitted.

[0094] 19, the calculation system 5c includes a first sensor Se1, a second sensor Se2, a third sensor Se3, and a calculation device 6c. In this embodiment, as in the third embodiment, the first sensor Se1 is a lidar Li as an external sensor Se. The second sensor Se2 and the third sensor Se3 are cameras Cm as external sensors Se.

[0095] The calculation device 6c uses sensor information output from an external sensor Se installed in a factory Fc3 to calculate at least one of the position and the orientation of the vehicle 10. The calculation device 6c includes a communication unit 61, a storage unit 63c, and a CPU 62c.

[0096] The storage unit 63c of the calculation device 6c stores various information including various programs P63c for controlling the operation of the calculation device 6c, management information In, and a third detection model Md3 as a detection model Md. When the captured images Im1 to Im4 and the states St2 and St3 of the vehicle 10 included in the captured images Im1 to Im4 are input, the third detection model Md3 detects the vehicle 10 from the captured images Im1 to Im4 and outputs shape data Da. The third detection model Md3 is trained in advance by supervised learning using a third training data set. The third training data set has a plurality of training images including the vehicle 10, a region correct answer label, and a state correct answer label. The state correct answer label is a correct answer label indicating the states St1 to St3 of the vehicle 10 included in the training image. The state correct answer label is associated with each of the plurality of training images.

[0097] The CPU 62c of the calculation device 6c functions as a sensor information acquisition unit 621 and a calculation unit 622c by developing various programs P63c stored in the storage unit 63c. When the sensor information acquisition unit 621 acquires the measurement point cloud data from the lidars Li1 to Li4, the calculation unit 622c matches the measurement point cloud data with the reference point cloud data to calculate the position and orientation of the assembled vehicle 10m. When the sensor information acquisition unit 621 acquires the original image Im1, the calculation unit 622 inputs the captured images Im1 to Im4 and the states St2 and St3 of the vehicle 10 included in the captured images Im1 to Im4 to the third detection model Md3. Specifically, when the sensor information acquisition unit 621 acquires the original image Im1 from the camera Cm1 installed in the second area Ar23, the calculation unit 622c inputs the captured images Im1 to Im4 and information indicating that the platform state is Sp to the third detection model Md3. When the sensor information acquisition unit 621 acquires the original image Im1 from the camera Cm2 installed in the third area Ar32, the calculation unit 622c inputs the captured images Im1 to Im4 and information indicating that the vehicle 10n is being assembled to the third detection model Md3. As a result, the calculation unit 622c acquires the shape data Da and calculates the positions and orientations of the vehicles 10p, 10n represented by the captured images Im1 to Im4.

[0098] According to the fourth embodiment, a third detection model Md3 is prepared in advance, which is trained by associating a plurality of training images including the vehicle 10 with state answer labels indicating the state of the vehicle 10 included in each training image. As a result, by inputting the captured images Im1 to Im4 associated with the states St2 and St3 of the vehicle 10 to the third detection model Md3, the vehicles 102 and 103 can be detected with high accuracy from the captured images Im1 to Im4. Therefore, the detection accuracy when detecting the vehicles 102 and 103 from the captured images Im1 to Im4 can be improved.

[0099] E. Fifth embodiment: FIG. 20 is a diagram showing a state in which the vehicle 10 travels through the factory Fc4 in the fifth embodiment. In this embodiment, the same manufacturing processes Pr1 to Pr7 as those in the first embodiment are executed in the same order. Thus, similar to the first embodiment, the vehicle 10 is classified into two states St1 and St2, a first state St1 and a second state St2, depending on whether or not the body shell 18 is assembled. In this embodiment, the first state St1 refers to a case in which the states St1 and St2 of the vehicle 10 are the platform state Sp. The second state St2 refers to a case in which the states St1 and St2 of the vehicle 10 are the assembled state Sm. In this embodiment, the vehicle 10 travels in an unmanned manner by being remotely controlled by the remote control device 7 in at least a part of the section from the manufacturing location PL1 to the storage location PL7. The same reference numerals are given to the same configurations as those in the above embodiments, and the description thereof will be omitted.

[0100] 21 is a diagram showing the configuration of a calculation system 5d in the fifth embodiment. The calculation system 5d includes a first sensor Se1, a second sensor Se2, an information processing device 8, and a calculation device 6d. In this embodiment, the first sensor Se1 is a lidar Li as an external sensor Se. The second sensor Se2 is a camera Cm as an external sensor Se.

[0101] 20, in this embodiment, both the first sensor Se1 and the second sensor Se2 are installed in each of the areas Ar14 and Ar24 in the factory Fc4. In this embodiment, the first area Ar14 includes the manufacturing location PL1 and the first path Ro1. The second area Ar24 includes the assembly location PL3, the second path Ro2, the inspection location PL5, the third path Ro3, and the storage location PL7.

[0102] 21 determines the type of the external sensor Se detecting the vehicle 10 in order to calculate at least one of the position and the orientation of the vehicle 10 using sensor information output from the external sensor Se of a type corresponding to states St1, St2 of the vehicle 10. The information processing device 8 includes a communication unit 81, a storage unit 83, and a CPU 82.

[0103] The communication unit 81 of the information processing device 8 communicably connects the information processing device 8 to devices other than the information processing device 8. The communication unit 81 of the information processing device 8 is, for example, a wireless communication device.

[0104] The storage unit 83 of the information processing device 8 stores various information including various programs P83 for controlling the operation of the information processing device 8 and a database Db. The database Db is used to determine which type of external sensor Se is to be used to detect the vehicle 10 among the multiple types of external sensors Se installed in each area Ar14, Ar24. The database Db is associated with the state St1, St2 of the vehicle 10 specified by the state information, at least one of the manufacturing processes Pr1 to Pr7 specified by the process information, and the type of the external sensor Se used to detect the vehicle 10. The process information is information indicating which of the multiple manufacturing processes Pr1 to Pr7 the manufacturing process Pr1 to Pr7 being executed on the vehicle 10 is. In this embodiment, the database Db is used to determine which external sensor Se is to be used to detect the vehicle 10 among the first sensor Se1 and the second sensor Se2 in each of the first area Ar14 and the second area Ar24.

[0105] The CPU 82 of the information processing device 8 functions as a judgment information acquisition unit 821 and a judgment unit 822 by developing various programs P83 stored in the storage unit 83. The judgment information acquisition unit 821 acquires at least one of state information and process information. In this embodiment, the judgment information acquisition unit 821 acquires the process information. The judgment unit 822 executes the following process to calculate at least one of the position and the orientation of the vehicle 10 using the sensor information output from the external sensor Se of the type corresponding to the state St1, St2 of the vehicle 10. In this case, the judgment unit 822 judges which type of external sensor Se is used to detect the vehicle 10 among the multiple types of external sensors Se installed in each area Ar14, Ar24, using at least one of the state information and the process information. In this embodiment, the judgment unit 822 judges which type of external sensor Se is used to detect the vehicle 10 among the first sensor Se1 and the second sensor Se2, using the process information and the database Db. Then, the judgment unit 822 transmits the judgment result to the calculation device 6d. At least a part of the functions of the CPU 82 of the information processing device 8 may be realized by a hardware circuit.

[0106] The calculation device 6d calculates at least one of the position and the orientation of the vehicle 10, using sensor information output from an external sensor Se of a type identified by a determination result by the information processing device 8. The calculation device 6d includes a communication unit 61, a storage unit 63d, and a CPU 62d.

[0107] The storage unit 63d of the calculation device 6d stores various types of information including various programs P63d for controlling the operation of the calculation device 6d, a first detection model Md1, and management information In.

[0108] The CPU 62d of the calculation device 6d functions as a sensor information acquisition unit 621d and a calculation unit 622 by developing various programs P63d stored in the storage unit 63d. When the determination unit 822 of the information processing device 8 determines that the vehicle 10 is detected using the first sensor Se1, the sensor information acquisition unit 621d of the calculation device 6d acquires first sensor information output from the first sensor Se1 that includes the vehicle 10 to be calculated in the detection range RG1. When the determination unit 822 of the information processing device 8 determines that the vehicle 10 is detected using the second sensor Se2, the sensor information acquisition unit 621d of the calculation device 6d acquires second sensor information output from the second sensor Se2 that includes the vehicle 10 to be calculated in the detection range RG2. When identifying the external sensor Se that includes the vehicle 10 to be calculated in the detection ranges RG1 and RG2, the sensor information acquisition unit 621d may use the management information In, or may use at least one of the state information and the process information.

[0109] Fig. 22 is a first flowchart showing a processing flow in the fifth embodiment. Fig. 23 is a second flowchart showing a processing flow in the fifth embodiment. The processing flows shown in Fig. 22 and Fig. 23 are repeatedly executed at predetermined time intervals from the time when the vehicle 10 starts traveling by unmanned driving, for example.

[0110] 22, in step S501, the determination information acquisition unit 821 of the information processing device 8 acquires process information. In step S502, the determination unit 822 uses the process information and the database Db to determine which external sensor Se, the first sensor Se1 or the second sensor Se2, is to be used to detect the vehicle 10. In step S503, the determination unit 822 transmits the determination result to the calculation device 6d.

[0111] 23, in step S504, the sensor information acquisition unit 621d of the calculation device 6d identifies the external sensor Se whose detection ranges RG1, RG2 include the vehicle 10 that is the calculation target, among the external sensors Se of the types determined by the determination unit 822 of the information processing device 8. Each step from step S505 to step S513 is similar to each step from step S101 to step S110 shown in FIG.

[0112] According to the fifth embodiment, a plurality of types of external sensors Se are installed in each of the areas Ar14 and Ar24 in the factory Fc4. The information processing device 8 can use at least one of the state information and the process information to determine which type of the external sensor Se among the plurality of types of external sensors Se is used to detect the vehicle 10, and transmit the determination result to the calculation device 6d. As a result, the calculation device 6d can calculate the position and orientation of the vehicle 10 using the sensor information output from the external sensor Se of the type determined by the information processing device 8. That is, the calculation system 5d can select the type of external sensor Se according to the states St1 and St2 of the vehicle 10, and calculate the position and orientation of the vehicle 10 using the sensor information output from the selected external sensor Se.

[0113] F. Sixth embodiment: FIG. 24 is a diagram showing the configuration of a traveling system 1e in the sixth embodiment. In this embodiment, the traveling system 1e includes one or more vehicles 10e equipped with a vehicle control device 15e having both the functions of the calculation device 6 and the remote control device 7, and a first sensor Se1 and a second sensor Se2 as external sensors Se. In this embodiment, the vehicle control device 15e calculates the position and orientation of the vehicle 10e, determines a target position to which the vehicle should next travel, and generates a traveling control signal for traveling toward the determined target position. The other configurations of the traveling system 1e are the same as those of the first embodiment unless otherwise described.

[0114] The vehicle control device 15e includes an input / output interface 159, a storage unit 157e, and a CPU 150e. The storage unit 157e of the vehicle control device 15e stores various information including various programs P15e for controlling the operation of the vehicle control device 15e, a first detection model Md1 as a detection model Md, and an ideal route Ip. The CPU 150e of the vehicle control device 15e functions as a sensor information acquisition unit 151, a calculation unit 152, a signal generation unit 153, and an operation control unit 154 by developing various programs stored in the storage unit 157e. The sensor information acquisition unit 151 acquires sensor information output from an external sensor Se that includes the vehicle 10e to be calculated in the detection ranges RG1 and RG2. The sensor information acquisition unit 151 of the vehicle control device 15e has the same function as the sensor information acquisition unit 621 of the calculation device 6 described in the first embodiment. The calculation unit 152 generates position and direction information using the sensor information. The calculation unit 152 has the same function as the calculation unit 622 of the calculation device 6 described in the first embodiment. The signal generation unit 153 uses the position / direction information and the ideal route Ip to determine a target position to which the vehicle 10e should next head. Then, the signal generation unit 153 generates a driving control signal for driving the vehicle 10e toward the determined target position. The signal generation unit 153 has the same function as the remote control unit 721 of the remote control device 7 described in the first embodiment. The vehicle control device 15e repeats, at a predetermined cycle, acquisition of position / direction information of the vehicle 10e, determination of a target position, generation of a driving control signal, and control of an actuator.

[0115] Fig. 25 is a flowchart showing a process flow in the sixth embodiment. The process flow shown in Fig. 25 is repeatedly executed at predetermined time intervals from the time when the vehicle 10e starts traveling by unmanned driving, for example.

[0116] In step S601, the sensor information acquisition unit 151 of the vehicle control device 15e mounted on the vehicle 10e identifies an external sensor Se that includes the vehicle 10e to be calculated in its detection range RG1, RG2 among a plurality of external sensors Se installed in the factory Fc1. In step S602, the sensor information acquisition unit 151 transmits a request signal for acquiring sensor information to the identified external sensor Se. In step S603, the external sensor Se that has received the request signal transmits the sensor information to the vehicle 10e. When the sensor information acquisition unit 621 of the vehicle control device 15e acquires the measurement point cloud data, in step S604, the calculation unit 152 uses the measurement point cloud data to calculate the position of the vehicle 10e represented by the measurement point cloud data, and generates position direction information. When the sensor information acquisition unit 621 acquires the original image Im1, in step S605, the calculation unit 152 calculates the position of the vehicle 10e represented by the captured images Im1 to Im4 using the first detection model Md1 learned by executing the learning process and the captured images Im1 to Im4, and generates position and direction information. In step S606, the signal generation unit 153 determines a target position to which the vehicle 10e should next move, using the position and direction information and the ideal route Ip. In step S607, the signal generation unit 153 generates a driving control signal for driving toward the determined target position. In step S608, the operation control unit 154 controls the actuator using the driving control signal to drive the vehicle 10e at the acceleration and steering angle represented in the driving control signal.

[0117] According to the sixth embodiment, the traveling system 1e can make the vehicle 10e travel by autonomous control without remotely controlling the vehicle 10e by the remote control device 7.

[0118] G. Other embodiments: G-1. Other embodiment 1: At least one of the calculation devices 6, 6a to 6d, the remote control device 7, and the information processing device 8 may be configured integrally. Furthermore, each unit of the calculation devices 6, 6a to 6d, the remote control device 7, and the information processing device 8 may be realized by cloud computing configured by one or more computers. Furthermore, at least some of the functions of the calculation devices 6, 6a to 6d, the remote control device 7, and the information processing device 8 may be a function of the vehicle control device 15, 15e, or a function of the external sensor Se. In such a form, the configurations of the calculation devices 6, 6a to 6d, the remote control device 7, and the information processing device 8 can be appropriately changed.

[0119] G-2. Other embodiment 2: The calculation unit 152, 622, 622a to 622c may perform pre-processing and use the measurement point cloud data to calculate at least one of the position and orientation of the vehicle 10, 10e represented by the measurement point cloud data. That is, the first sensor Se1 that acquires the first sensor information may be the lidar Li. The calculation unit 152, 622, 622a to 622c may calculate at least one of the position and orientation of the vehicle 10, 10e represented by the captured images Im1 to Im4 using the captured images Im1 to Im4 without performing pre-processing. That is, the second sensor Se2 that acquires the second sensor information may be the camera Cm. In this form, the types of the first sensor Se1 and the second sensor Se2 can be appropriately changed in consideration of the states St1 to St3 classified according to the appearance of the vehicle 10, 10e, the prices of the lidar Li and the camera Cm, the load required for pre-processing, and the like.

[0120] G-3. Other embodiment 3: The types of the first sensor Se1, the second sensor Se2, and the third sensor Se3 are not limited to the lidar Li and the visible light camera Cm. For example, either the first sensor Se1 or the second sensor Se2 may be a thermography camera as the external sensor Se. The thermography camera is a camera having an image pickup element and an optical system that are sensitive to infrared rays. The thermography camera detects the surface temperature of an object present in the detection range including the vehicle 10, and outputs a thermography image showing the heat distribution in the detection range including the vehicle 10 as sensor information. In this form, the calculation units 152, 622, 622a to 622c can calculate at least one of the position and the orientation of the vehicle 10, 10e using the sensor information output from the external sensor Se of a type other than the lidar Li and the visible light camera Cm. When the first sensor Se1 is a thermography camera, the calculation units 152, 622, 622a to 622c, for example, match the thermography image with the reference heat distribution data to calculate the position and the like of the vehicle 10, 10e represented by the thermography image. The reference heat distribution data is, for example, three-dimensional CAD data of the vehicle 10, 10e. In this form, the calculation units 152, 622, 622a to 622c can calculate at least one of the position and the orientation of the vehicle 10, 10e represented by the thermography image using the thermography image without performing preprocessing. Furthermore, when the second sensor Se2 is a thermography camera, the calculation units 152, 622, for example, perform the following processing. In this case, when a thermographic image is input, the calculation units 152, 622, 622a to 622c input the thermographic image to a machine learning model that detects the external shape of the vehicle 10, 10e from the thermographic image and outputs the shape data Da, thereby acquiring the shape data Da. Then, the calculation units 152, 622 use the shape data Da to convert the image coordinate point P3 into a vehicle coordinate point Pv, thereby calculating the position of the vehicle 10, 10e represented by the thermographic image, etc.In this embodiment, the calculation unit 152, 622 can calculate at least one of the position and orientation of the vehicle 10, 10e represented by the thermographic image using a machine learning model that has been trained by executing a learning process as a preprocessing and the thermographic image.

[0121] G-4. Other embodiment 4: The vehicle 10, 10e may be classified into a plurality of states according to the difference in the exterior color of the vehicle 10 before and after painting. In this manner, the vehicle 10, 10e can be classified into a plurality of states according to the exterior appearance of the vehicle 10, 10e determined by at least one of the exterior shape and exterior color of the vehicle 10, 10e.

[0122] G-5. Other embodiment 5: In each of the first to fifth embodiments, the calculation devices 6, 6a to 6d and the remote control device 7 execute the processes from obtaining the position information of the vehicle 10 to generating the driving control signal. In contrast, at least a part of the processes from obtaining the position information of the vehicle 10 to generating the driving control signal may be executed by the vehicle 10. For example, the following forms (1) to (3) may be used.

[0123] (1) The remote control device 7 may acquire position information of the vehicle 10, determine a target position to which the vehicle 10 should next head, and generate a route from the current location of the vehicle 10 indicated in the acquired position information to the target position. The remote control device 7 may generate a route to a target position between the current location and the destination, or may generate a route to the destination. The remote control device 7 may transmit the generated route to the vehicle 10. The vehicle 10 may generate a travel control signal so that the vehicle 10 travels on the route received from the remote control device 7, and control an actuator using the generated travel control signal.

[0124] (2) The remote control device 7 may acquire position information of the vehicle 10 and transmit the acquired position information to the vehicles 10, 10e. The vehicle 10 may determine a target position to which the vehicle 10 should next head, generate a route from the current position of the vehicle 10 represented in the received position information to the target position, generate a travel control signal so that the vehicle 10 travels on the generated route, and control the actuator using the generated travel control signal.

[0125] (3) In the above embodiments (1) and (2), the vehicle 10 may be equipped with an internal sensor, and a detection result output from the internal sensor may be used for at least one of generating a route and generating a driving control signal. The internal sensor may include, for example, an on-board camera, an on-board lidar, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, a gyro sensor, and the like. For example, in the above embodiment (1), the remote control device 7 may acquire the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating a route. In the above embodiment (1), the vehicle 10 may acquire the detection result of the internal sensor, and reflect the detection result of the internal sensor in the driving control signal when generating a driving control signal. In the above embodiment (2), the vehicle 10 may acquire the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating a route. In the above embodiment (2), the vehicle 10 may acquire the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating a route.

[0126] (4) In the sixth embodiment, the vehicle 10e may be equipped with an internal sensor, and a detection result output from the internal sensor may be used for at least one of generating a route and generating a driving control signal. For example, the vehicle 10e may acquire the detection result of the internal sensor, and when generating a route, may reflect the detection result of the internal sensor in the route. The vehicle 10e may acquire the detection result of the internal sensor, and when generating a driving control signal, may reflect the detection result of the internal sensor in the driving control signal.

[0127] (5) In the sixth embodiment, the vehicle 10e acquires the position information of the vehicle 10e using the detection result of the external sensor Se. In contrast, the vehicle 10e may be equipped with an internal sensor, and the vehicle 10e may acquire position information using the detection result of the internal sensor, determine a target position to which the vehicle 10e should next move, generate a route from the current location of the vehicle 10e represented in the acquired position information to the target position, generate a travel control signal for traveling along the generated route, and control the actuator using the generated travel control signal. In this case, the vehicle 10e can travel without using any detection result of the external sensor Se. The vehicle 10e may acquire a target arrival time or traffic congestion information from outside the vehicle 10e, and reflect the target arrival time or traffic congestion information in at least one of the route and the travel control signal.

[0128] (6) In each of the first to fifth embodiments, the remote control device 7 automatically generates a driving control signal to be transmitted to the vehicle 10. In contrast, the remote control device 7 may generate a driving control signal to be transmitted to the vehicle 10 in accordance with the operation of an operator located outside the vehicle 10. For example, the operator may operate a control device including a display for displaying an image output from the external sensor Se, a steering wheel for remotely operating the vehicle 10, an accelerator pedal, a brake pedal, and a communication device for communicating with the remote control device 7 by wired or wireless communication, and the remote control device 7 may generate a driving control signal in accordance with the operation applied to the control device.

[0129] G-6. Other embodiment 6: In each of the above embodiments, the vehicle 10, 10e may have a configuration capable of moving by unmanned driving, and may be in the form of a platform having the configuration described below, for example. Specifically, the vehicle 10, 10e may have at least the vehicle control device 15, 15e, the drive device 11, the steering device 12, and the braking device 13 in order to perform the three functions of "running", "turning", and "stopping" by unmanned driving. When the vehicle 10, 10e acquires information from the outside for unmanned driving, the vehicle 10, 10e may further have the vehicle communication device 14. That is, the vehicle 10, 10e capable of moving by unmanned driving may not be equipped with at least a part of interior parts such as a driver's seat and a dashboard, may not be equipped with at least a part of exterior parts such as a bumper and a fender, and may not be equipped with a body shell 18. In this case, the remaining parts such as the body shell 18 may be attached to the vehicle 10, 10e before the vehicle 10, 10e is shipped from the factory Fc1-Fc4, or the remaining parts such as the body shell 18 may be attached to the vehicle 10, 10e after the vehicle 10, 10e is shipped from the factory Fc1-Fc4 without the remaining parts such as the body shell 18 being attached to the vehicle 10, 10e. Each part may be attached from any direction such as the upper side, lower side, front side, rear side, right side, or left side of the vehicle 10, 10e, and may be attached from the same direction or from different directions. The position of the vehicle 10p in the form of a platform may be determined in the same manner as the vehicle 10 in the first embodiment.

[0130] G-7. Other embodiment 7: The vehicle 10, 10e may be manufactured by combining a plurality of modules. The module means a unit composed of a plurality of parts grouped according to the location or function of the vehicle 10, 10e. For example, the platform of the vehicle 10, 10e may be manufactured by combining a front module constituting the front part of the platform, a central module constituting the central part of the platform, and a rear module constituting the rear part of the platform. The number of modules constituting the platform is not limited to three, and may be two or less or four or more. In addition to or instead of the parts constituting the platform, parts constituting parts of the vehicle 10, 10e other than the platform may be modularized. The various modules may include any exterior parts such as a bumper or a grille, or any interior parts such as a seat 19 or a console. In addition, not limited to the vehicle 10, 10e, any mobile body may be manufactured by combining a plurality of modules. Such a module may be manufactured, for example, by joining a plurality of parts by welding or a fastener, or may be manufactured by integrally molding at least a part of the parts constituting the module as one part by casting. The molding method for integrally molding a single component, particularly a relatively large component, is also called gigacast or megacast. For example, the front module, the center module, and the rear module may be manufactured using gigacast.

[0131] G-8. Other embodiment 8: Transporting the vehicles 10, 10e by using the unmanned driving of the vehicles 10, 10e is also called "self-propelled transport." Also, a configuration for realizing self-propelled transport is also called a "vehicle remote-controlled autonomous traveling transport system." Also, a production method for producing the vehicles 10, 10e by using self-propelled transport is also called "self-propelled production." In self-propelled production, for example, at least a part of the transport of the vehicles 10, 10e is realized by self-propelled transport in the factories Fc1 to Fc4 where the vehicles 10, 10e are manufactured.

[0132] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0133] 1, 1e...driving system, 5, 5a to 5d...calculation system, 6, 6a to 6d...calculation device, 7...remote control device, 8...information processing device, 10, 10e...vehicle, 10f...positioning point, 10m...assembled vehicle, 10n...vehicle under assembly, 10p...platform vehicle, 11...driving device, 12...steering device, 13...braking device, 14...vehicle communication device, 15, 15e...vehicle control device, 18...body shell, 19...seat, 61...communication unit of calculation device, 62, 62a to 62d...CPU of calculation device, 63, 63a to 63d...storage unit of calculation device , 71... communication unit of remote control device, 72... CPU of remote control device, 73... memory unit of remote control device, 81... communication unit of information processing device, 82... CPU of information processing device, 83... memory unit of information processing device, 101... first vehicle, 102... second vehicle, 103... third vehicle, 150, 150e... CPU of vehicle control device, 151, 621, 621d... sensor information acquisition unit, 152, 622, 622a to 622c... calculation unit, 153... signal generation unit, 154... operation control unit, 157, 157e... memory unit of vehicle control device, 159... input / output interface chair, 721... remote control unit, 821... judgment information acquisition unit, 822... judgment unit, A1... necessary area, A2... unnecessary area, Ar11 to Ar14... first area, Ar21 to Ar24... second area, Ar31, Ar32... third area, C... center of gravity, Cm, Cm1 to Cm7... visible light camera, ΔD... observation error, D... first distance, Da... shape data, Db... database, Dc... third distance, Do... observation distance, Dp... estimated distance, Dt... second distance, Fc1 to Fc4... factory, h... height of vehicle positioning point from road surface, H... height of camera from road surface , Im1... original image, Im2... corrected image, Im3... rotated image, Im4... processed image, Im1 to Im4... captured image, Im2 to Im4... processed image, Im5... first mask image, Im6... second mask image, In... management information, Ip... ideal path, Li, Li1 to Li7... lidar, Md... detection model, Md1... first detection model, Md2, Md2a, Md2b... second detection model, Md3... third detection model, Ms... mask area, P0... base coordinate point, P1... first coordinate point, P15, P15e, P63, P63a to P63d, P73,P83...program, P2...second coordinate point, P3...image coordinate point, PL1...manufacturing location, PL3...assembly location, PL3a...first assembly location, PL3b...second assembly location, PL5...inspection location, PL7...storage location, Pc...image coordinate point, Pf...fixed coordinate point, Pr1...platform manufacturing process, Pr2...first transfer process, Pr3...assembly process, Pr3a...first assembly process, Pr3b...second assembly process, Pr4...second transfer process, Pr5...inspection process, Pr6...third transfer process, Pr7...storage process, Pr8 ...Transportation process, Pv...Vehicle coordinate point, R0...Base circumscribing rectangle, R1...First circumscribing rectangle, R2...Second circumscribing rectangle, RG1, RG2...Detection range, Ro1...First running path, Ro2...Second running path, Ro3...Third running path, Ro4, Ro5...Running path, Rs...Road surface, Se...External sensor, Se1...First sensor, Se2...Second sensor, Se3...Third sensor, Sm...Assembled state, Sn...Assembled state, Sp...Platform state, St1...First state, St2...Second state, St3...Third state, V...Movement vector,

Claims

1. A calculation device for calculating at least one of a position and an orientation of a moving object, the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The calculation device is a sensor information acquisition unit that acquires first sensor information representing the moving body classified into a first state among the plurality of states, the first sensor outputting the first sensor information, and that acquires second sensor information representing the moving body classified into the second state, the second sensor outputting the second sensor information, by externally detecting the moving body classified into a second state different from the first state among the plurality of states by a second sensor of a different type from the first sensor information, the second sensor outputting the second sensor information; a calculation unit that calculates at least one of the position and the orientation of the moving body using the first sensor information without performing preprocessing, and that calculates at least one of the position and the orientation of the moving body using the second sensor information by performing the preprocessing.

2. A calculation system for calculating at least one of a position and an orientation of a moving object, comprising: the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The calculation system comprises: a first sensor that detects from an outside the moving object that is classified into a first state among the plurality of states, and outputs first sensor information that represents the moving object that is classified into the first state; a second sensor of a different type from the first sensor, the second sensor detecting from an outside the moving object classified into a second state different from the first state among the plurality of states, and outputting second sensor information representing the moving object classified into the second state; a calculation unit that calculates at least one of the position and the orientation of the moving body using the first sensor information without performing preprocessing, and that calculates at least one of the position and the orientation of the moving body using the second sensor information by performing the preprocessing.

3. A calculation device for calculating at least one of a position and an orientation of a moving object, the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The calculation device is a sensor information acquisition unit that acquires first sensor information representing the moving object that is classified into a first state among the plurality of states, and acquires second sensor information representing the moving object that is classified into a second state among the plurality of states that is different from the first state; a calculation unit that calculates at least one of the position and the orientation of the moving object represented by the first sensor information using the first sensor information, and calculates at least one of the position and the orientation of the moving object represented by the second sensor information using the second sensor information, The first sensor information is information output from one of a camera and a LiDAR sensor by detecting the moving object classified into the first state from the outside using the one of the sensors, A calculation device, wherein the second sensor information is information output from the other sensor, either the camera or the LiDAR, by detecting the moving body classified into the second state from the outside using the other sensor.

4. A calculation system for calculating at least one of a position and an orientation of a moving object, comprising: the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The calculation system comprises: a first sensor that detects from an outside the moving object that is classified into a first state among the plurality of states, and outputs first sensor information that represents the moving object that is classified into the first state; a second sensor that detects from an outside the moving object that is classified into a second state different from the first state among the plurality of states, and outputs second sensor information that represents the moving object that is classified into the second state; a calculation unit that calculates at least one of the position and the orientation of the moving object represented by the first sensor information using the first sensor information, and calculates at least one of the position and the orientation of the moving object represented by the second sensor information using the second sensor information, The first sensor is one of a camera and a LiDAR; When the first sensor is the camera, the second sensor is the LiDAR; A calculation system, wherein when the first sensor is the LiDAR, the second sensor is the camera.

5. A factory for manufacturing moving objects, In the factory, a plurality of manufacturing processes are carried out to manufacture the moving body, the moving body is capable of moving within the factory by unmanned operation, The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The factory is a first sensor installed in a first area in which the moving object moves when the moving object is classified into a first state among the plurality of states; a second sensor installed in a second area in which the moving object moves and which is classified into a second state different from the first state among the plurality of states; The first sensor is one of a camera and a LiDAR; When the first sensor is the camera, the second sensor is the LiDAR; A factory, wherein when the first sensor is the LiDAR, the second sensor is the camera.

6. The factory according to claim 5, the first sensor detects the moving object classified into the first state from an outside, and outputs first sensor information representing the moving object classified into the first state; the second sensor detects the moving object classified into the second state from an outside, and outputs second sensor information representing the moving object classified into the second state; the first sensor information is used to calculate at least one of a position and an orientation of the moving object represented by the first sensor information; A factory, wherein the second sensor information is used to calculate at least one of the position and the orientation of the moving object represented by the second sensor information.

7. An information processing device used to move a moving object, the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The information processing device includes: a determination information acquisition unit that acquires at least one of status information indicating which of the plurality of statuses the status of the moving body is, and process information indicating which of the plurality of manufacturing processes the manufacturing process being performed on the moving body is; An information processing device comprising: a judgment unit that judges which of a first sensor and a second sensor of a different type from the first sensor to use to detect the moving body, using at least one of the status information and the process information, in order to calculate at least one of the position and orientation of the moving body using sensor information representing the moving body output from a sensor of a type corresponding to the status of the moving body.

8. A calculation system for calculating at least one of a position and an orientation of a moving object, comprising: the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The calculation system comprises: a first sensor that detects the moving object from outside and outputs first sensor information representing the moving object; a second sensor as the sensor of a different type from the first sensor, the second sensor detecting the moving object from an outside and outputting second sensor information representing the moving object; a determination information acquisition unit that acquires at least one of status information indicating which of the plurality of statuses the status of the moving body is, and process information indicating which of the plurality of manufacturing processes the manufacturing process being performed on the moving body is; a determination unit that determines which of the first sensor and the second sensor is to be used to detect the moving body, using at least one of the state information and the process information, in order to calculate at least one of the position and the orientation of the moving body using sensor information representing the moving body output from the sensor of a type corresponding to the state of the moving body; a calculation unit that, when the determination unit determines that the moving body will be detected using the first sensor, calculates at least one of the position and the orientation of the moving body using the first sensor information without performing preprocessing, and, when the determination unit determines that the moving body will be detected using the second sensor, performs the preprocessing and calculates at least one of the position and the orientation of the moving body using the second sensor information.

9. A calculation system for calculating at least one of a position and an orientation of a moving object, comprising: the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The calculation system comprises: a first sensor that detects the moving object from outside and outputs first sensor information representing the moving object; a second sensor as the sensor of a different type from the first sensor, the second sensor detecting the moving object from an outside and outputting second sensor information representing the moving object; a determination information acquisition unit that acquires at least one of status information indicating which of the plurality of statuses the status of the moving body is, and process information indicating which of the plurality of manufacturing processes the manufacturing process being performed on the moving body is; a determination unit that determines which of the first sensor and the second sensor is to be used to detect the moving body, using at least one of the state information and the process information, in order to calculate at least one of the position and the orientation of the moving body using sensor information representing the moving body output from the sensor of a type corresponding to the state of the moving body; a calculation unit that calculates at least one of the position and the orientation of the moving object by using the sensor information output from the sensor that the determination unit has determined to be used for detecting the moving object, The first sensor is one of a camera and a LiDAR; When the first sensor is the camera, the second sensor is the LiDAR; A calculation system, wherein when the first sensor is the LiDAR, the second sensor is the camera.

10. A calculation system according to claim 2 or claim 8, A calculation system, wherein the preprocessing includes a learning process for training a machine learning model.

11. 11. The calculation system of claim 10, The calculation unit is The machine learning model is trained by executing the learning process, and the machine learning model is trained to output shape data indicating an external shape of the moving body when the second sensor information is input, thereby acquiring the shape data by inputting the second sensor information; A calculation system that uses the shape data to calculate at least one of the position and the orientation of the moving object.

12. 12. The calculation system of claim 11, The machine learning model is prepared for each of the states of the moving object; Each of the multiple machine learning models prepared for each of the states has been pre-trained to output the shape data by inputting multiple training images including M (M is an integer of 2 or more) first training images each including the moving objects classified into one of the multiple states, and N (N is an integer of 0 or more and less than M) second training images each including the moving objects classified into the remaining other states; The calculation unit is acquiring the shape data by inputting the second sensor information into the machine learning model selected according to the state of the moving object from the plurality of machine learning models prepared for each state; Using the shape data, a coordinate in an image coordinate system of a positioning point previously set for the moving object is calculated; A calculation system that calculates at least one of the position and the orientation of the moving object represented by the second sensor information by converting the calculated coordinates into coordinates in a global coordinate system.

13. 12. The calculation system of claim 11, the machine learning model is pre-trained to output the shape data by inputting a plurality of training images including the moving object and a correct answer label associated with each of the plurality of training images, the correct answer label indicating the state of the moving object included in the training image; The calculation unit is acquiring the shape data by inputting the second sensor information and the state of the moving object into the machine learning model; Using the shape data, a coordinate in an image coordinate system of a positioning point previously set for the moving object is calculated; A calculation system that calculates at least one of the position and the orientation of the moving object represented by the second sensor information by converting the calculated coordinates into coordinates in a global coordinate system.

14. A calculation system according to claim 4 or claim 9, The calculation unit is Detecting the moving object from the captured image output from the camera, and outputting shape data indicating an external shape of the moving object; Using the shape data, a coordinate in an image coordinate system of a positioning point previously set for the moving object is calculated; A calculation system that calculates at least one of the position and the orientation of the moving object represented by the captured image by converting the calculated coordinates into coordinates in a global coordinate system.

15. A calculation system according to claim 4 or claim 9, A calculation system in which the calculation unit calculates at least one of the position and orientation of the moving body represented by the measurement point cloud data by matching the measurement point cloud data output from the LiDAR with reference point cloud data prepared in advance.

16. A method for calculating at least one of a position and an orientation of a moving object, comprising the steps of: the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The calculation method is a sensor information acquisition process for acquiring first sensor information representing the moving body classified into a first state among the plurality of states, the first sensor detecting the moving body classified into the first state from the outside by the first sensor, and acquiring second sensor information representing the moving body classified into the second state, the second sensor detecting the moving body classified into a second state different from the first state among the plurality of states from the outside by a second sensor of a different type from the first sensor; a calculation step of calculating at least one of the position and the orientation of the moving body using the first sensor information without performing preprocessing, and calculating at least one of the position and the orientation of the moving body using the second sensor information by performing the preprocessing.

17. A method for calculating at least one of a position and an orientation of a moving object, comprising the steps of: the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The calculation method is a sensor information acquiring step of acquiring first sensor information representing the moving object classified into a first state among the plurality of states, and acquiring second sensor information representing the moving object classified into a second state, different from the first state, among the plurality of states; a calculation step of calculating at least one of the position and the orientation of the moving body represented by the first sensor information using the first sensor information, and calculating at least one of the position and the orientation of the moving body represented by the second sensor information using the second sensor information, The first sensor information is information output from one of a camera and a LiDAR sensor by detecting the moving object classified into the first state from the outside using the one of the sensors, A calculation method in which the second sensor information is information output from the other sensor, either the camera or the LiDAR, by detecting the moving body classified into the second state from the outside using the other sensor.

18. A method for calculating at least one of a position and an orientation of a moving object, comprising the steps of: the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The calculation method is a determination information acquisition step of acquiring at least one of status information indicating which of the plurality of statuses the status of the moving body is, and process information indicating which of the plurality of manufacturing processes the manufacturing process being performed on the moving body is; a determination process for determining, using at least one of the state information and the process information, which of a first sensor serving as the sensor that detects the moving body from the outside and outputs first sensor information representing the moving body, and a second sensor serving as the sensor of a type different from the first sensor and that outputs second sensor information representing the moving body by detecting the moving body from the outside, in order to calculate at least one of the position and the orientation of the moving body using sensor information representing the moving body output from the sensor of a type corresponding to the state of the moving body; a calculation step of calculating at least one of the position and the orientation of the moving body using the first sensor information without performing pre-processing when it is determined in the determination step that the moving body will be detected using the first sensor, and performing the pre-processing and calculating at least one of the position and the orientation of the moving body using the second sensor information when it is determined in the determination step that the moving body will be detected using the second sensor.

19. A method for calculating at least one of a position and an orientation of a moving object, comprising the steps of: the moving body is capable of moving in an unmanned manner within a factory in which a plurality of manufacturing processes are performed to manufacture the moving body; The moving body is classified into a plurality of states by having different appearances according to the plurality of manufacturing processes; The calculation method is a determination information acquisition step of acquiring at least one of status information indicating which of the plurality of statuses the status of the moving body is, and process information indicating which of the plurality of manufacturing processes the manufacturing process being performed on the moving body is; a determination process for determining, using at least one of the state information and the process information, which of a first sensor that detects the moving body from the outside and outputs first sensor information representing the moving body, and a second sensor that is a different type of sensor from the first sensor and that detects the moving body from the outside and outputs second sensor information representing the moving body, in order to calculate at least one of the position and the orientation of the moving body using sensor information representing the moving body output from a sensor of a type corresponding to the state of the moving body; a calculation step of calculating at least one of the position and the orientation of the moving object by using the sensor information output from the sensor determined to be used for detecting the moving object in the determination step, The first sensor is one of a camera and a LiDAR; When the first sensor is the camera, the second sensor is the LiDAR; A calculation method in which when the first sensor is the LiDAR, the second sensor is the camera.

Images