Calculation device, calculation system, information processing device, plant, and calculation method

Abstract

A computing device includes: a sensor information acquisition unit configured to acquire first sensor information acquired by a first sensor that detects a moving body classified as a first state from outside the moving body, and acquire second sensor information acquired by a second sensor that detects a moving body classified as a second state from outside; and a calculation unit configured to calculate at least one of a position and an orientation of the moving body by using the first sensor information without performing preprocessing, and perform preprocessing to calculate at least one of the position and the orientation of the moving body by using the second sensor information.

Description

Technical Field

[0001] The present disclosure relates to a computing device, a computing system, an information processing device, a factory, and a computing method. Background Art

[0002] In the related art, a vehicle that performs unmanned driving by remote control is known (Japanese Unexamined Patent Application Publication (Translation of PCT Application) No. 2017-538619 (JP 2017-538619A)). Summary of the invention

[0003] At least one of the position and orientation of the moving body can be calculated by using sensor information output from the sensor that detects the moving body from the outside via the sensor. In order to calculate the position of the moving body, etc. by using the sensor information, the appearance shape of the moving body can be detected from the sensor information. For example, in the case where the appearance shape of the moving body contained in the captured image output from the camera is detected, it is conceivable to train a machine learning model as preprocessing and detect the appearance shape of the moving body by using the trained machine learning model. However, in the case where the moving body moves in a factory that manufactures the moving body, the appearance of the moving body may be different depending on the manufacturing steps. For example, in the case where the moving body is a vehicle, the appearance of a vehicle in the form of a platform in which various devices such as wheels and vehicle control devices are assembled in a chassis is significantly different from the appearance of a vehicle in the form of a body in which a body such as a body shell is assembled in the chassis. Due to the difference in appearance of the moving body depending on the manufacturing steps, the detection accuracy when the appearance shape of the moving body is detected from the sensor information is likely to be reduced. When the detection accuracy when the appearance shape of the moving body is detected from the sensor information is reduced, the calculation accuracy of the position of the moving body, etc. is likely to be reduced. Therefore, in the case of detecting the appearance shape of a moving object by using a trained machine learning model, it is desirable to train the machine learning model for each appearance state of the moving object. Therefore, in the case of using the trained machine learning model to detect the appearance shape of the moving object contained in the sensor information when the moving object moves in the factory, it may be necessary to prepare data for learning for each appearance state of the moving object, or the learning time may be long depending on the number of appearance states of the moving object. On the other hand, for example, in the case of detecting the appearance shape of a moving object contained in the measurement point group data output from Light Detection And Ranging (LiDAR), a method of performing matching between the measurement point group data and the reference point group data prepared in advance to calculate the position of the moving object, etc., can be conceived. In this case, the position of the moving object, etc. can be calculated without performing preprocessing such as training a machine learning model, so that the load required for preprocessing performed to calculate the position of the moving object, etc. can be reduced. However, in general, LiDAR is more expensive than a camera. Thus, in the case of calculating the position of the moving object, etc. without performing preprocessing, the cost required for calculating the position of the moving object, etc., may be higher than the cost required for preprocessing performed to calculate the position of the moving object, etc., depending on the type of sensor. As described above, there is a need for a technology that can calculate the position of a moving object, etc. at low cost while reducing the load required for preprocessing performed to calculate the position of the moving object, etc.

[0004] The present disclosure can be implemented in the following forms.

[0005] A first aspect of the present disclosure provides a computing device. In the computing device, a mobile body is configured to move by unmanned driving in a factory that performs a plurality of manufacturing steps to manufacture the mobile body. The mobile body is classified into a plurality of states by having different appearances depending on the manufacturing steps. The computing device includes a sensor information acquisition unit and a computing unit. The sensor information acquisition unit is configured to acquire first sensor information output from a first sensor and representing a mobile body classified into the first state by detecting a mobile body classified into a first state among a plurality of states from the outside of the mobile body via a first sensor, and acquire second sensor information output from a second sensor and representing a mobile body classified into the second state by detecting a mobile body classified into a second state different from the first state from the outside via a second sensor of a different kind from the first sensor. The computing unit is configured to calculate at least one of the position and orientation of the mobile body by using the first sensor information without performing preprocessing, and to perform preprocessing to calculate at least one of the position and orientation of the mobile body by using the second sensor information. According to this aspect, the computing device is able to calculate at least one of the position and orientation of the mobile body classified into the first state by using the first sensor information without performing preprocessing. The computing device can perform preprocessing to calculate at least one of the position and orientation of the moving body classified as the second state by using the second sensor information. In this way, when calculating at least one of the position and orientation of the moving body classified as the first state, it is not necessary to perform preprocessing. Therefore, it is possible to reduce the load required for preprocessing performed to calculate the position of the moving body, etc. Further, according to this aspect, it is possible to detect the moving body classified as the first state and the moving body classified as the second state by different kinds of sensors depending on the price of the sensor. As a result, it is possible to reduce the possibility of an increase in the cost required to calculate the position of the moving body, etc.

[0006] In the second aspect of the present disclosure, the first sensor may be any one of a camera and a LiDAR, and the second sensor may be the other of the camera and the LiDAR.

[0007] A second aspect of the present disclosure provides a computing system. In the computing system, a mobile body is configured to move by unmanned driving in a factory that performs multiple manufacturing steps to manufacture the mobile body. The mobile body is classified into multiple states by having different appearances depending on the manufacturing steps. The computing system includes a first sensor, a second sensor, and a computing unit. The first sensor is configured to output first sensor information representing the mobile body classified into the first state by detecting the mobile body classified into the first state among the multiple states from the outside of the mobile body. The second sensor is configured to output second sensor information representing the mobile body classified into the second state by detecting the mobile body classified into the second state different from the first state among the multiple states from the outside. The second sensor is of a different kind from the first sensor. The computing unit is configured to calculate at least one of the position and orientation of the mobile body by using the first sensor information without performing preprocessing, and perform preprocessing to calculate at least one of the position and orientation of the mobile body by using the second sensor information. According to this aspect, the computing system is able to calculate at least one of the position and orientation of the mobile body classified into the first state by using the first sensor information without performing preprocessing. The computing system can perform preprocessing to calculate at least one of the position and orientation of the moving body classified as the second state by using the second sensor information. In this way, when calculating at least one of the position and orientation of the moving body classified as the first state, there is no need to perform preprocessing. Therefore, it is possible to reduce the load required for preprocessing performed to calculate the position of the moving body, etc. Further, according to this aspect, it is possible to detect the moving body classified as the first state and the moving body classified as the second state by different kinds of sensors depending on the price of the sensor. As a result, it is possible to reduce the possibility of an increase in the cost required to calculate the position of the moving body, etc.

[0008] In a second aspect of the present disclosure, the computing system may further include a determination information acquisition unit and a determination unit. The determination information acquisition unit may be configured to acquire at least one of state information indicating which state of a moving body is in a plurality of states and step information indicating which manufacturing step of a plurality of manufacturing steps is being performed on the moving body. The determination unit may be configured to determine which of the first sensor and the second sensor is used to detect the moving body by using at least one of the state information and the step information for calculating at least one of the position and orientation of the moving body. The computing unit may be configured to calculate at least one of the position and orientation of the moving body by using the first sensor information or the second sensor information output from the first sensor or the second sensor determined by the determination unit to be used to detect the moving body. According to this aspect, the computing system is able to calculate at least one of the position and orientation of the moving body by using the sensor information acquired by detecting the moving body via a sensor of a type corresponding to the state of the moving body.

[0009] In a second aspect of the present disclosure, the computing system may further include a determination information acquisition unit and a determination unit. The determination information acquisition unit may be configured to acquire at least one of state information indicating which state of a moving body is in a plurality of states and step information indicating which manufacturing step of a plurality of manufacturing steps is being performed on the moving body. The determination unit may be configured to determine which of the first sensor and the second sensor is used to detect the moving body by using at least one of the state information and the step information for calculating at least one of the position and orientation of the moving body. The computing unit may be configured to calculate at least one of the position and orientation of the moving body by using the first sensor information or the second sensor information output from the first sensor or the second sensor determined by the determination unit to be used to detect the moving body.

[0010] In the second aspect of the present disclosure, the first sensor may be one of a camera and a LiDAR, and the second sensor may be the other of the camera and the LiDAR.

[0011] In a second aspect of the present disclosure, the calculation unit may be configured to detect a moving body from a captured image output by a camera, and output shape data indicating an appearance shape of the moving body. The calculation unit may be configured to calculate the coordinates of a positioning point preset for the moving body in an image coordinate system by using the shape data. The calculation unit may be configured to calculate at least one of the position and orientation of the moving body represented by the captured image by transforming the calculated coordinates into coordinates in a global coordinate system. According to this aspect, the calculation unit is able to calculate at least one of the position and orientation of the moving body represented by the captured image by transforming the coordinates in the image coordinate system into coordinates in the global coordinate system by using the shape data.

[0012] In a second aspect of the present disclosure, the calculation unit may be configured to perform matching between the measurement point group data output from the LiDAR and the pre-prepared reference point group data to calculate at least one of the position and orientation of the moving body represented by the measurement point group data. According to this aspect, the calculation unit may perform matching between the measurement point group data output from the LiDAR and the pre-prepared reference point group data to calculate at least one of the position and orientation of the moving body represented by the measurement point group data.

[0013] In a second aspect of the present disclosure, preprocessing may include a learning process for training a machine learning model. According to this aspect, it is not necessary to learn the characteristic values ​​of a moving body classified as a first state in the learning process. Therefore, it is possible to save time and effort for preparing learning data for a moving body classified as a first state. Further, it is possible to reduce the time required for machine learning to learn the characteristic values ​​of a moving body classified as a first state. As a result, the learning time can be shortened. Therefore, in a state where the possibility of a decrease in detection accuracy when detecting the appearance shape of the moving body from the first sensor information is reduced, it is possible to reduce the load required for preprocessing performed to calculate the position of the moving body, etc.

[0014] In a second aspect of the present disclosure, the computing unit may be configured to acquire shape data by inputting the second sensor information into a machine learning model that is trained by performing a learning process and is trained to output shape data indicating the appearance shape of the moving body when the second sensor information is input. The computing unit may be configured to calculate at least one of the position and orientation of the moving body by using the shape data. According to this aspect, the computing unit may acquire the shape data by inputting the second sensor information into the trained machine learning model. The computing unit may calculate at least one of the position and orientation of the moving body by using the shape data.

[0015] In a second aspect of the present disclosure, a machine learning model may be prepared for each of a plurality of states of a moving body. By inputting a plurality of training images including M first training images each containing a moving body classified as a first state among a plurality of states and N second training images each containing a moving body classified as the remaining other states, a plurality of machine learning models prepared for a plurality of states may be pre-trained to output shape data, M being an integer greater than 2, and N being an integer greater than 0 and less than M. The calculation unit may be configured to acquire shape data by inputting the second sensor information into a machine learning model selected depending on the state of the moving body among a plurality of machine learning models prepared for a plurality of states. The calculation unit may be configured to calculate the coordinates of a positioning point pre-set for the moving body in an image coordinate system by using the shape data. The calculation unit may be configured to calculate at least one of the position and orientation of the moving body represented by the second sensor information by transforming the calculated coordinates into coordinates in a global coordinate system. According to this aspect, a machine learning model suitable for detecting a moving body classified into each state is prepared in advance by learning that a first training image containing a moving body classified into one state is more than a second training image containing a moving body classified into the remaining other states. As a result, the computing unit can accurately detect the moving body from the second sensor information by inputting the second sensor information into a machine learning model selected depending on the state of the moving body from among a plurality of machine learning models prepared respectively for a plurality of states of the moving body.

[0016] In a second aspect of the present disclosure, a machine learning model can be pre-trained to output shape data by inputting a plurality of training images containing a moving body and a correct answer label associated with each of the plurality of training images and indicating the state of the moving body contained in the training image. The computing unit can be configured to acquire the shape data by inputting the second sensor information and the state of the moving body into the machine learning model. The computing unit can calculate the coordinates of a predetermined positioning point pre-set for the moving body in the image coordinate system by using the shape data. The computing unit can calculate at least one of the position and orientation of the moving body represented by the second sensor information by transforming the calculated coordinates into coordinates in the global coordinate system. According to this aspect, a machine learning model trained by associating training images containing a moving body with a correct answer label indicating the state of the moving body contained in each training image is prepared in advance. As a result, the moving body can be accurately detected from the second sensor information by inputting the second sensor information associated with the state of the moving body into the machine learning model.

[0017] A third aspect of the present disclosure relates to an information processing device for moving a mobile body. In the information processing device, the mobile body is configured to move by unmanned driving in a factory that performs multiple manufacturing steps to manufacture the mobile body. The mobile body is classified into multiple states by having different appearances depending on the manufacturing steps. The information processing device includes a determination information acquisition unit and a determination unit. The determination information acquisition unit is configured to acquire at least one of state information indicating which state of the mobile body is in multiple states and step information indicating which manufacturing step of multiple manufacturing steps is being performed on the mobile body. The determination unit is configured to determine which of a first sensor and a second sensor of a different type from the first sensor is used to detect the mobile body by using at least one of the state information and the step information, so as to calculate at least one of the position and orientation of the mobile body. According to this aspect, the information processing device can determine which of the first sensor and the second sensor is used to detect the mobile body by using at least one of the state information and the step information.

[0018] A fourth aspect of the present disclosure provides a factory for manufacturing a mobile body. In the factory, multiple manufacturing steps are performed to manufacture the mobile body. The mobile body is configured to move in the factory by unmanned driving. The mobile body is classified into multiple states due to different appearances in the manufacturing steps. The factory includes a first sensor and a second sensor. The first sensor is installed in a first area where the mobile body moves in a first state classified as one of multiple states. The second sensor is installed in a second area where the mobile body moves in a second state classified as one of multiple states different from the first state. The first sensor is any one of a camera and a LiDAR. In the case where the first sensor is a camera, the second sensor is a LiDAR. In the case where the first sensor is a LiDAR, the second sensor is a camera. According to this aspect, different types of sensors can be installed in the first area and the second area.

[0019] In a fourth aspect of the present disclosure, the first sensor may be configured to output first sensor information representing the mobile body classified as the first state by detecting the mobile body classified as the first state from outside the mobile body. The second sensor may be configured to output second sensor information representing the mobile body classified as the second state by detecting the mobile body classified as the second state from outside. The first sensor information may be used to calculate at least one of the position and orientation of the mobile body represented by the first sensor information. The second sensor information may be used to calculate at least one of the position and orientation of the mobile body represented by the second sensor information. According to this aspect, at least one of the position and orientation of the mobile body classified as the first state may be calculated by detecting the mobile body classified as the first state using any one of the sensors of the camera and the LiDAR. At least one of the position and orientation of the mobile body classified as the second state may be calculated by detecting the mobile body classified as the second state using a sensor of a different type from the sensor that detects the mobile body classified as the first state among the camera and the LiDAR. As a result, the cost required for calculating at least one of the position and orientation of the mobile body can be reduced.

[0020] A fifth aspect of the present disclosure provides a calculation method. In the calculation method for calculating at least one of the position and orientation of a moving body, the moving body is configured to move by unmanned driving in a factory that performs multiple manufacturing steps to manufacture the moving body. The moving body is classified into multiple states by having different appearances depending on the manufacturing steps. The calculation method includes: a sensor information acquisition step, acquiring a first sensor information output from a first sensor and indicating a moving body classified into the first state by detecting a moving body classified into a first state among multiple states from the outside of the moving body via a first sensor, and acquiring a second sensor information output from a second sensor and indicating a moving body classified into the second state by detecting a moving body classified into a second state different from the first state from the outside via a second sensor of a different kind from the first sensor; and a calculation step, calculating at least one of the position and orientation of the moving body by using the first sensor information without performing preprocessing, and performing preprocessing to calculate at least one of the position and orientation of the moving body by using the second sensor information. According to this aspect, at least one of the position and orientation of the moving body classified into the first state can be calculated by using the first sensor information without performing preprocessing. It is possible to perform preprocessing to calculate at least one of the position and orientation of a moving body classified as the second state by using the second sensor information. In this way, when calculating at least one of the position and orientation of a moving body classified as the first state, it is not necessary to perform preprocessing. Therefore, it is possible to reduce the load required for preprocessing performed to calculate the position of the moving body, etc. Further, according to this aspect, it is possible to detect a moving body classified as the first state and a moving body classified as the second state by different kinds of sensors depending on the price of the sensor. As a result, it is possible to reduce the possibility of an increase in the cost required to calculate the position of the moving body, etc.

[0021] In a fifth aspect of the present disclosure, the first sensor information may be information output from one of the camera and the LiDAR by detecting a moving body classified as a first state from the outside via one of the camera and the LiDAR. The second sensor information may be information output from the other of the camera and the LiDAR by detecting a moving body classified as a second state from the outside via the other of the camera and the LiDAR. According to this aspect, at least one of the position and orientation of the moving body classified as the first state can be calculated by detecting the moving body classified as the first state using one of the camera and the LiDAR. At least one of the position and orientation of the moving body classified as the second state can be calculated by detecting the moving body classified as the second state using a sensor of a different type from the sensor that detects the moving body classified as the first state among the camera and the LiDAR. As a result, the cost required to calculate at least one of the position and orientation of the moving body can be reduced.

[0022] In a fifth aspect of the present disclosure, the calculation method may further include: a determination information acquisition step of acquiring at least one of state information indicating which state of a moving body is one of a plurality of states and step information indicating which manufacturing step of a plurality of manufacturing steps is being performed on the moving body; and a determination step of determining which of the first sensor and the second sensor is used for detecting the moving body by using at least one of the state information and the step information for calculating at least one of the position and orientation of the moving body. In the calculation step, in the case where it is determined in the determination step that the moving body is detected by using the first sensor, at least one of the position and orientation of the moving body may be calculated by using the first sensor information without performing preprocessing, and in the case where it is determined in the determination step that the moving body is detected by using the second sensor, at least one of the position and orientation of the moving body may be calculated by using the second sensor information by performing preprocessing. According to this aspect, at least one of the position and orientation of the moving body can be calculated by using the sensor information acquired by detecting the moving body via a sensor of a type corresponding to the state of the moving body. In a fifth aspect of the present disclosure, the first sensor may be one of a camera and a LiDAR, and the second sensor may be the other of the camera and the LiDAR. According to this aspect, at least one of the position and orientation of the moving body can be calculated by using sensor information acquired by detecting the moving body via a sensor of a type corresponding to the state of the moving body.

[0023] The present disclosure may also be implemented in various forms other than the above-mentioned computing device, computing system, information processing device, factory, and computing method. For example, the present disclosure may be implemented in the following forms: a manufacturing method of a computing device, an information processing device, a computing system, and a mobile body; a control method of a computing device, an information processing device, a computing system, and a mobile body; a computer program for implementing the control method; and a non-transitory recording medium having the computer program recorded thereon. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, in which like numerals represent like elements, and in which: Figure 1 is a diagram showing the configuration of a traveling system in a first embodiment; Figure 2 is a diagram showing the configuration of a vehicle control device in a first embodiment; Figure 3 is a diagram showing a state in which the vehicle in the first embodiment is traveling in a factory; Figure 4 is a diagram showing a configuration of a computing device; Figure 5is a diagram showing a configuration of a remote control device; Figure 6 is a flowchart showing the flow of processing in the first embodiment; Figure 7 is a flowchart showing an example of a method for calculating the position of a vehicle represented by measurement point group data, etc.; Figure 8 is a flowchart showing an example of a method for calculating the position of a vehicle represented by a captured image, etc.; Fig. 9 It shows that when executing Figure 8 A schematic diagram of an example of each image when the calculation method is shown in FIG. Fig.10 is a diagram showing details of the coordinate point calculation steps; Fig.11 is a diagram showing a method of calculating basic coordinate points; Fig.12 is a first diagram showing details of the position transformation step; Fig.13 is a second diagram showing details of the position transformation step; Fig.14 is a diagram showing a state in which a vehicle in the second embodiment is traveling in a factory; Fig.15 is a diagram showing a configuration of a computing system in a second embodiment; Fig.16 is a diagram showing a state where a vehicle in the third embodiment is traveling in a factory; Fig.17 is a diagram showing a configuration of a computing system in a third embodiment; Fig.18 is a flow chart showing a method for calculating a position of a vehicle using a second detection model; Fig.19 is a diagram showing the configuration of a computing system in a fourth embodiment; Fig. 20 is a diagram showing a state where a vehicle in the fifth embodiment is traveling in a factory; Fig.21 is a diagram showing the configuration of a computing system in a fifth embodiment; Fig. 22 is a first flowchart showing the flow of processing in the fifth embodiment; Fig.23 is a second flowchart showing the flow of processing in the fifth embodiment; Fig.24 is a diagram showing a configuration of a traveling system in a sixth embodiment; and Fig.25 is a flowchart showing the flow of processing in the sixth embodiment. DETAILED DESCRIPTION First embodiment

[0025] Figure 1 1 is a diagram showing the configuration of a travel system 1 in the first embodiment. The travel system 1 is a system for moving a moving body in a factory where moving bodies are manufactured without relying on driving operations of passengers riding on the moving body. The travel system 1 includes: one or more moving bodies, a computing system 5 including a first sensor Se1, a second sensor Se2 and a computing device 6, and a remote control device 7.

[0026] "Mobile body" refers to an object that can move, for example, a vehicle 10 or an electric vertical lift (a so-called flying car). In the present embodiment, the mobile body is a vehicle 10. The vehicle 10 may be a vehicle that travels on wheels or a vehicle that travels on tracks, such as a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, or a construction vehicle. The vehicle 10 includes a pure electric vehicle (Battery Electric Vehicle, BEV), a gasoline vehicle, a hybrid vehicle, and a fuel cell electric vehicle. In addition, in the case where the mobile body is not the vehicle 10, the expressions "vehicle" and "car" in the present disclosure may be replaced with "mobile body" as appropriate, and the expression "travel" may be replaced with "moving" as appropriate.

[0027] The vehicle 10 includes a driving device 11, a steering device 12, a braking device 13, a vehicle communication device 14, and a vehicle control device 15. The driving device 11 accelerates the vehicle 10. The driving device 11 includes wheels, a driving force source for rotating the wheels, and a power supply device for supplying power to the driving force source. For example, in the case where the vehicle 10 is a battery electric vehicle, the driving force source is a motor for traveling, and the power supply device is a battery, such as a lithium-ion battery. The steering device 12 changes the traveling direction of the vehicle 10. The braking device 13 decelerates the vehicle 10. The vehicle communication device 14 communicates with an external device through wireless communication or the like. The external device is a device arranged at a location different from the vehicle 10 as the main vehicle, such as a remote control device 7, or another vehicle 10. The vehicle communication device 14 is, for example, a wireless communication device.

[0028] Figure 21 is a diagram showing the configuration of the vehicle control device 15 in the first embodiment. The vehicle control device 15 controls the operation of the vehicle 10. The vehicle control device 15 includes an input and output interface 159, a storage unit 157, and a central processing unit (CPU) 150. In the vehicle control device 15, the input and output interface 159, the storage unit 157, and the CPU 150 are connected to each other via a bus or the like in a manner that allows bidirectional communication with each other. The input and output interface 159 is used to communicate with various devices mounted on the vehicle 10 as the host vehicle. The storage unit 157 of the vehicle control device 15 stores various information, including various programs P15 for controlling the operation of the vehicle control device 15. The storage unit 157 of the vehicle control device 15 includes, for example, a read-only memory (Random-Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), and a hard disk drive (Hard Disk Drive, HDD). The CPU 150 of the vehicle control device 15 operates as the operation controller 154 by loading various programs P15 stored in the storage unit 157. The operation controller 154 performs driving control of the vehicle 10 by controlling the operation of the actuators that change the acceleration and deceleration and the steering angle of the vehicle 10. "Driving control" refers to, for example, various controls for driving the actuators that exhibit the three functions of "driving", "turning" and "stopping" of the vehicle 10, such as the adjustment of the acceleration, speed, steering angle, etc. of the vehicle 10. In the present embodiment, the actuators include the actuators of the drive device 11, the actuators of the steering device 12, and the actuators of the brake device 13. In addition, at least a part of the functions of the CPU 150 of the vehicle control device 15 can be implemented by hardware circuits.

[0029] The vehicle 10 is capable of traveling by unmanned driving. "Unmanned driving" refers to driving that does not rely on the driving operation of the occupants. The driving operation refers to an operation related to at least one of "driving", "turning" and "stopping" of the vehicle 10. Unmanned driving is implemented by automatic or manual remote control using a device provided outside the vehicle 10 or by autonomous control of the vehicle 10. The vehicle 10 that travels by unmanned driving may carry occupants who do not perform driving operations. Examples of occupants who do not perform driving operations include persons who simply sit in a seat of the vehicle 10, and persons who perform work different from driving operations (such as assembly, inspection, or operating switches) while riding in the vehicle 10. In addition, driving performed using the driving operation of the occupants may be referred to as "manned driving".

[0030] In the present embodiment, the vehicle 10 is remotely controlled by the remote control device 7 to travel by unmanned driving in at least a partial section of the factory Fc1 . Figure 31 is a diagram showing a state in which the vehicle 10 in the first embodiment is traveling in the factory Fc1. The factory Fc1 is a location of a manufacturing facility in which the vehicle 10 is installed. The factory Fc1 is also referred to as a production line of the vehicle 10. In the factory Fc1, a plurality of manufacturing steps Pr1 to Pr7 are performed to manufacture the vehicle 10. In the present embodiment, the platform manufacturing step Pr1, the first transport step Pr2, the assembly step Pr3, the second transport step Pr4, and the inspection step Pr5 are sequentially performed as the manufacturing steps Pr1 to Pr7, and as a result, the vehicle 10 as a finished product is manufactured. Then, through the third transport step Pr6, the vehicle 10 as a finished product is stored in the storage location PL7 in the storage step Pr7 until the vehicle 10 is shipped.

[0031] The platform manufacturing step Pr1 is a step of manufacturing a platform vehicle 10p. The platform vehicle 10p is a vehicle 10 having a platform form. The platform vehicle 10p is a vehicle 10 in the process of being manufactured and has a configuration capable of traveling by unmanned driving. Figure 1 As shown in , the platform vehicle 10p includes at least a driving device 11, a steering device 12, a braking device 13 and a vehicle control device 15, so as to exhibit the three functions of "driving", "turning" and "stopping" by unmanned driving. In this embodiment, the vehicle 10 obtains information from a remote control device 7 set at a location different from the vehicle 10, so as to travel by remote control. Therefore, the platform vehicle 10p also includes a vehicle communication device 14. The platform manufacturing step Pr1 is performed at the manufacturing location PL1.

[0032] Figure 3 The assembly step Pr3 shown in FIG. 1 is a step of assembling the interior components such as seats and the exterior components such as the painted vehicle body shell 18 into the platform vehicle 10p. The assembly step Pr3 is performed at the assembly site PL3.

[0033] The inspection step Pr5 is a step of inspecting the manufactured vehicle 10. The inspection step Pr5 is performed at the inspection location PL5.

[0034] In this embodiment, the manufacturing site PL1, the assembly site PL3, the inspection site PL5, and the storage site PL7 exist in different buildings. Therefore, the manufacturing site PL1 and the assembly site PL3 are connected via the first route Ro1. The assembly site PL3 and the inspection site PL5 are connected via the second route Ro2. The inspection site PL5 and the storage site PL7 are connected via the third route Ro3. Each of the first route Ro1, the second route Ro2, and the third route Ro3 can be a private road, or it can also be a public road. The places PL1 to PL4 where the manufacturing steps Pr1, Pr3, Pr5, and Pr7 are respectively performed can exist on one site or on multiple sites. In addition, the places PL1 to PL4 where the manufacturing steps Pr1, Pr3, Pr5, and Pr7 are respectively performed can exist at one address or at multiple addresses.

[0035] The first transport step Pr2 is a step of moving the vehicle 10 from the manufacturing site PL1 to the assembly site PL3. The second transport step Pr4 is a step of moving the vehicle 10 from the assembly site PL3 to the inspection site PL5. The third transport step Pr6 is a step of moving the vehicle 10 from the inspection site PL5 to the storage site PL7.

[0036] like Figure 1 As shown in FIG. 1 , the appearance of the vehicle 10 is different between the platform vehicle 10p and the assembled vehicle 10m due to the difference in the appearance shape of the vehicle 10. The assembled vehicle 10m is a vehicle 10 in which at least the body shell 18 is assembled into the platform vehicle 10p. In this embodiment, as shown in FIG. Figure 3 As shown in , the vehicle 10 after the assembly step Pr3 starts is the assembled vehicle 10m. As described above, since the vehicle 10 has different appearances depending on the manufacturing steps Pr1 to Pr7, the vehicle 10 is classified into a plurality of states St1, St2 depending on the appearance of the vehicle 10. In the present embodiment, the vehicle 10 is classified into two states St1, St2 including a first state St1 and a second state St2 before and after the vehicle body shell 18 is assembled. In the present embodiment, the first state St1 refers to a case where the states St1, St2 of the vehicle 10 are the platform state Sp. The platform state Sp refers to a state where the vehicle 10 is in the platform form. The second state St2 refers to a case where the states St1, St2 of the vehicle 10 are the assembled state Sm. The assembled state Sm refers to a state where at least the vehicle body shell 18 is assembled to the platform vehicle 10p. In addition, the number and kind of the manufacturing steps Pr1 to Pr7 performed in the factory Fc1, and the number and kind of the states St1, St2 of the vehicle 10 classified depending on the appearance of the vehicle 10 are not limited to the above.

[0037] Hereinafter, the vehicle 101 classified into the first state St1 will be referred to as the "first vehicle 101". The vehicle 102 classified into the second state St2 will be referred to as the "second vehicle 102". In the present embodiment, the first vehicle 101 is a platform vehicle 10p. The second vehicle 102 is an assembled vehicle 10m. In the case where there is no need to distinguish between the vehicles 101 and 102 in the states St1 and St2, the vehicles 101 and 102 will be simply referred to as "vehicle 10".

[0038] like Figure 1 As shown in , the computing system 5 includes a first sensor Se1 and a second sensor Se2 as external sensors Se provided at a location different from the vehicle 10 , and a computing device 6 .

[0039] like Figure 3 As shown in , the first sensor Se1 is installed in the first area Ar11 in the factory Fc1. The first area Ar11 is an area in which the first vehicle 101 travels. In the present embodiment, the first area Ar11 includes the manufacturing location PL1 and the first distance Ro1. The installation position, orientation and installation number of the first sensor Se1 in the first area Ar11 are determined in consideration of the detection range RG1 of the first sensor Se1, etc. The first sensor Se1 detects the first vehicle 101 from the outside to output first sensor information representing the first vehicle 101 as a detection result. In the present embodiment, the first sensor Se1 is a light detection and ranging (LiDAR) Li. Therefore, in the present embodiment, the LiDAR Li1 and Li2 as the first sensor Se1 detect the platform vehicle 10p from the outside to output measurement point group data that three-dimensionally represents the appearance shape of the platform vehicle 10p as the first sensor information.

[0040] The second sensor Se2 is installed in the second area Ar21 in the factory Fc1. The second area Ar21 is an area in which the second vehicle 102 travels. In the present embodiment, the second area Ar21 includes the assembly site PL3, the second route Ro2, the inspection site PL5, the third route Ro3, and the storage site PL7. The installation position, orientation, and installation number of the second sensor Se2 in the second area Ar21 are determined in consideration of the detection range RG2 of the second sensor Se2, etc. The second sensor Se2 detects the second vehicle 102 from the outside to output second sensor information representing the second vehicle 102 as a detection result. In the present embodiment, the second sensor Se2 is a visible light camera Cm including an imaging element and an optical system that are sensitive to visible light. Hereinafter, the visible light camera Cm is also referred to as "camera Cm". The camera Cm is, for example, any one of a CCD image sensor and a CMOS image sensor. Therefore, in the present embodiment, the cameras Cm1 to Cm5 as the second sensor Se2 image the assembled vehicle 10m from the outside to output an original image containing the assembled vehicle 10m as the second sensor information.

[0041] Figure 4 is a diagram showing the configuration of the calculation device 6. The calculation device 6 calculates at least one of the position and orientation of the vehicle 10 using sensor information output from the external sensor Se installed in the factory Fc1 as a detection result of the vehicle 10. The position of the vehicle 10 is the position of a positioning point 10f set in advance for a specific part of the vehicle 10. The position of the vehicle 10 is represented by X, Y, and Z coordinates in the global coordinate system of the factory Fc1. The orientation of the vehicle 10 is a direction represented by a vector from the rear side to the front side of the vehicle 10 along the front-rear axis of the vehicle 10 passing through the center of gravity of the vehicle 10.

[0042] The computing device 6 includes a communication unit 61, a storage unit 63, and a CPU 62. In the computing device 6, the communication unit 61, the storage unit 63, and the CPU 62 are connected to each other via a bus or the like in a bidirectionally communicable manner with each other.

[0043] The communication unit 61 of the computing device 6 connects the computing device 6 and other devices outside the computing device 6 in a manner that the computing device 6 can communicate with each other. The communication unit 61 of the computing device 6 is, for example, a wireless communication device.

[0044] The storage unit 63 of the computing device 6 stores various kinds of information including various programs P63 for controlling the operation of the computing device 6, management information In, and a first detection model Md1 as a detection model Md. The storage unit 63 of the computing device 6 includes, for example, RAM, ROM, and a hard disk drive (HDD).

[0045] The management information In is information indicating the driving order of a plurality of vehicles 10 driving in the detection ranges RG1, RG2 of the external sensor Se. In the management information In, the vehicle identifier and the sensor identifiers Li1, Li2, Cm1 to Cm5 are associated with each other. The vehicle identifier is a non-repeating unique identifier assigned between the vehicles 10 for identifying the vehicle 10. The sensor identifiers Li1, Li2, Cm1 to Cm5 are non-repeating unique identifiers assigned between the external sensors Se installed in the factory Fc1 for identifying the external sensor Se. The management information In is created by using, for example, the position and orientation of the vehicle 10, the transmission history of the driving control signal to the vehicle 10, and the installation place of the external sensor Se.

[0046] The detection model Md is a trained machine learning model that detects the appearance shape of the vehicle 10 represented by the captured image using artificial intelligence. When the captured image is input, the detection model Md detects the appearance shape of the vehicle 10 from the captured image to output shape data indicating the appearance shape of the vehicle 10. For example, a convolutional neural network (CNN) that implements either semantic segmentation or instance segmentation can be used for the detection model Md.

[0047] In the present embodiment, the storage unit 63 of the computing device 6 stores a first detection model Md1 as the detection model Md. When a captured image is input, the first detection model Md1 detects the vehicle 10 from the captured image to output shape data. The captured image is any one of an original image output from the camera Cm as original data and a processed image obtained by processing the original image. The first detection model Md1 is pre-trained by supervised learning using a first learning data set. The first learning data set includes a plurality of training images containing the vehicle 10 and a regional correct answer label indicating whether each area in the training image is an area indicating the vehicle 10 or an area indicating a vehicle other than the vehicle 10. When the CNN is trained, the parameters of the CNN are preferably updated by back propagation (error back propagation method) to reduce the error between the output result of the first detection model Md1 and the regional correct answer label.

[0048] like Figure 4 As shown in , the CPU 62 of the calculation device 6 operates as a sensor information acquisition unit 621 and a calculation unit 622 by loading various programs P63 stored in the storage unit 63 .

[0049] The sensor information acquisition unit 621 acquires sensor information output from the external sensor Se including the vehicle 10 as the calculation target in the detection range RG1, RG2. In the present embodiment, the sensor information acquisition unit 621 specifies the external sensor Se including the vehicle 10 as the calculation target in the detection range RG1, RG2 by using the management information In stored in the storage unit 63 of the calculation device 6. The sensor information acquisition unit 621 acquires sensor information output from the specified external sensor Se.

[0050] The calculation unit 622 calculates at least one of the position and orientation of the vehicle 10 by using the sensor information acquired by the sensor information acquisition unit 621. Specifically, in the case where the first sensor information is acquired by the sensor information acquisition unit 621, the calculation unit 622 calculates at least one of the position and orientation of the first vehicle 101 represented by the first sensor information by using the first sensor information without performing preprocessing. In the case where the second sensor information is acquired by the sensor information acquisition unit 621, the calculation unit 622 performs preprocessing to calculate at least one of the position and orientation of the second vehicle 102 represented by the second sensor information by using the second sensor information. In the present embodiment, the preprocessing is a learning process of training the first detection model Md1 as a machine learning model. Therefore, in the case where the sensor information acquisition unit 621 acquires the measurement point group data, the calculation unit 622 calculates the position and orientation of the platform vehicle 10p represented by the measurement point group data by using the measurement point group data without performing learning processing. In the case where the sensor information acquisition unit 621 acquires the original image, the calculation unit 622 calculates the position and orientation of the assembled vehicle 10m represented by the captured image by using the first detection model Md1 trained by performing the learning process and the captured image. Then, the calculation unit 622 transmits positional direction information indicating the calculation result of the position and orientation of the vehicle 10 to the remote control device 7. In addition, at least a part of the functions of the CPU 62 of the calculation device 6 can be realized by a hardware circuit.

[0051] Figure 5 is a diagram showing a configuration of the remote control device 7. The remote control device 7 generates a travel control signal that defines a travel operation of the vehicle 10, and transmits the generated travel control signal to the vehicle 10. The remote control device 7 includes a communication unit 71, a storage unit 73, and a CPU 72. In the remote control device 7, the communication unit 71, the storage unit 73, and the CPU 72 are connected to each other via a bus or the like in a manner that can bidirectionally communicate with each other.

[0052] The communication unit 71 of the remote control device 7 connects the remote control device 7 and other devices other than the remote control device 7 in a manner that the remote control device 7 can communicate with each other. The communication unit 71 of the remote control device 7 is, for example, a wireless communication device.

[0053] The storage unit 73 of the remote control device 7 stores various information including various programs P73 for controlling the operation of the remote control device 7 and an ideal path Ip. The ideal path Ip is a path on which the vehicle 10 should travel. The ideal path Ip is represented by a node indicating a departure point, a node indicating a passing point, a node indicating a destination, and a link connecting each node. The storage unit 73 of the remote control device 7 includes, for example, a RAM, a ROM, and a hard disk drive (HDD).

[0054] The CPU 72 of the remote control device 7 operates as a remote controller 721 by loading various programs P73 stored in the storage unit 73. The remote controller 721 generates a travel control signal and transmits the generated travel control signal to the vehicle 10 to remotely control the travel operation of the vehicle 10. First, the remote controller 721 determines the target position to which the vehicle 10 should go next to generate the travel control signal. In the present embodiment, the target position is represented by X, Y, and Z coordinates in the global coordinate system of the plant Fc1. The remote controller 721 determines the target position to which the vehicle 10 should go next by using the position direction information and the ideal path Ip. The remote controller 721 determines the target position on the ideal path Ip ahead of the current position of the vehicle 10. The remote controller 721 generates a travel control signal for causing the vehicle 10 to travel toward the determined target position. In the present embodiment, the travel control signal includes the acceleration and steering angle of the vehicle 10 as parameters. The remote controller 721 calculates the travel speed of the vehicle 10 according to the transition of the position of the vehicle 10, and compares the calculated travel speed with the predetermined target speed of the vehicle 10. The remote controller 721 determines the acceleration so that the vehicle 10 accelerates when the driving speed is lower than the target speed, and determines the acceleration so that the vehicle 10 decelerates when the driving speed is higher than the target speed. The remote controller 721 determines the steering angle so that the vehicle 10 does not deviate from the ideal path Ip when the vehicle 10 is located on the ideal path Ip, and determines the steering angle so that the vehicle 10 returns to the ideal path Ip when the vehicle 10 is not located on the ideal path Ip (in other words, when the vehicle 10 deviates from the ideal path Ip). In this way, the remote controller 721 generates a driving control signal including the acceleration and steering angle of the vehicle 10 as parameters. The remote controller 721 transmits the generated driving control signal to the vehicle 10. The remote controller 721 repeatedly performs the acquisition of the position direction information, the determination of the target position, the generation of the driving control signal, and the transmission of the driving control signal at a predetermined cycle. In addition, at least a part of the functions of the CPU 72 of the remote control device 7 can be realized by a hardware circuit.

[0055] Figure 6 is a flowchart showing the flow of processing in the first embodiment. Figure 6 The processing flow shown in , for example, is repeatedly executed at predetermined time intervals starting from the time when the vehicle 10 starts to travel by unmanned driving.

[0056] In step S101, the sensor information acquisition unit 621 of the computing device 6 specifies the external sensor Se whose detection range RG1, RG2 includes the vehicle 10 as the calculation target among the external sensors Se installed in the factory Fc1 by using the management information In. In step S102, the sensor information acquisition unit 621 transmits a request signal for acquiring sensor information to the specified external sensor Se. In step S103, the external sensor Se that has received the request signal transmits the sensor information to the computing device 6. In the case where the sensor information acquisition unit 621 of the computing device 6 acquires the measurement point group data, in step S104, the computing unit 622 calculates the position and orientation of the platform vehicle 10p represented by the measurement point group data by using the measurement point group data without using the first detection model Md1. In the case where the sensor information acquisition unit 621 of the computing device 6 acquires the captured image as the original image, in step S105, the computing unit 622 calculates the position and orientation of the assembled vehicle 10m represented by the captured image by using the first detection model Md1 and the captured image. In step S106 , the calculation unit 622 transmits the position direction information to the remote control device 7 .

[0057] In step S107, the remote controller 721 of the remote control device 7 determines the target position to which the vehicle 10 should go next by using the position direction information and the ideal path Ip. In step S108, the remote controller 721 generates a travel control signal for causing the vehicle 10 to travel toward the determined target position. In step S109, the remote controller 721 transmits the travel control signal to the vehicle 10. In step S110, the operation controller 154 of the vehicle control device 15 mounted on the vehicle 10 controls the actuator by using the received travel control signal so that the vehicle 10 travels at the acceleration and steering angle indicated by the travel control signal. The vehicle control device 15 repeatedly performs the reception of the travel control signal and the control of the actuator at a predetermined period.

[0058] Figure 7 1 is a flowchart showing an example of a calculation method for calculating the position and orientation of the platform vehicle 10p represented by the measurement point group data by using the measurement point group data without using the first detection model Md1. Figure 7In the example shown in , the calculation unit 622 performs matching between the measurement point group data and the reference point group data prepared in advance to calculate the position and orientation of the platform vehicle 10p represented by the measurement point group data. The reference point group data is, for example, the three-dimensional CAD data of the platform vehicle 10p. For matching between the measurement point group data and the reference point group data, for example, an iterative closest point (ICP) or normal distribution transform (NDT) algorithm is used.

[0059] In step S104a, the calculation unit 622 acquires the measurement point group data. In step S104b, the calculation unit 622 performs matching between the measurement point group data and the reference point group data to calculate the position and orientation of the platform vehicle 10p represented by the measurement point group data.

[0060] In addition, the calculation method of the position and orientation of the vehicle 10 represented by the measurement point group data is not limited to the above. The calculation unit 622 can calculate the position and orientation of the vehicle 10 by using, for example, the distance between the LiDAR Li and the vehicle 10, and the angle difference between the optical axis reference of the LiDAR Li and the vector from the LiDAR Li to the vehicle 10, without performing matching.

[0061] Figure 8 : is a flowchart showing an example of a calculation method of calculating the position and orientation of the assembled vehicle 10 m represented by a captured image by using the first detection model Md1 and the captured image. Fig. 9 It shows that when executing Figure 8 Schematic diagram of an example of each of the various images Im1 to Im6 when the calculation method shown in Fig. 9 In the Figure 8 The step number corresponding to the step.

[0062] exist Figure 8 In the example shown in , the calculation unit 622 uses the optical flow method to calculate the orientation of the assembled vehicle 10m based on the orientation of the motion vector V of the assembled vehicle 10m calculated according to the position change of the feature points of the assembled vehicle 10m between each frame of the captured image. The calculation unit 622 obtains the shape data Da indicating the appearance shape of the assembled vehicle 10m by inputting the captured images Im1 to Im4 into the first detection model Md1. The calculation unit 622 calculates the coordinates of the positioning point 10f of the assembled vehicle 10m in the image coordinate system by using the shape data Da, and transforms the calculated coordinates into coordinates in the global coordinate system to calculate the position of the assembled vehicle 10m. Hereinafter, a method of generating the shape data Da and a method of calculating the position of the vehicle 10 using the shape data Da will be described.

[0063] In the image acquisition step of step S105a, the calculation unit 622 acquires the original image Im1 output from the camera Cm. In the distortion correction step of step S105b, the calculation unit 622 corrects the distortion of the original image Im1 to generate a correction image Im2 as a processing image. In the rotation processing step of step S105c, the calculation unit 622 rotates the correction image Im2 so that the direction of the motion vector indicating the motion direction of the assembled vehicle 10m contained in the correction image Im2 is oriented toward a predetermined direction. As a result, the calculation unit 622 generates a rotated image Im3 as a processing image. In the cropping processing step of step S105d, the calculation unit 622 deletes the unnecessary area A2, except for the necessary area A1 including the assembled vehicle 10m and the surrounding area of ​​the assembled vehicle 10m, from the rotated image Im3 among the areas forming the correction image Im2. As a result, the calculation unit 622 generates a post-processing image Im4 as a processing image. Each of steps S105b to S105d is an image processing step of performing preprocessing for improving the detection accuracy when detecting the assembled vehicle 10m from the captured images Im1 to Im4. Therefore, at least one of steps S105b to S105d may be omitted.

[0064] In the detection step of step S105g, the calculation unit 622 inputs the post-processed image Im4 to the first detection model Md1. As a result, the calculation unit 622 detects the assembled vehicle 10m from the post-processed image Im4 to obtain the first mask image Im5 as the shape data Da. The first mask image Im5 is an image obtained by masking the area indicating the assembled vehicle 10m among the areas forming the post-processed image Im4 to add the mask area Ms in the post-processed image Im4.

[0065] In the perspective transformation step of step S105h, the calculation unit 622 generates a second mask image Im6 by performing perspective transformation on the first mask image Im5. The calculation unit 622 performs perspective transformation on the first mask image Im5 into a bird's-eye view image viewed from an upper viewpoint of the assembled vehicle 10m substantially perpendicular to the road surface Rs by using, for example, predetermined perspective transformation parameters. The perspective transformation parameters are, for example, parameters related to the position information and internal parameters of the camera Cm obtained by calibration. As a result, the calculation unit 622 generates a second mask image Im6 represented in the image coordinate system based on the first mask image Im5 represented in the camera coordinate system. The camera coordinate system is a coordinate system having the focus of the camera Cm as the origin and having coordinate axes represented by the Xc axis and the Yc axis orthogonal to the Xc axis. The image coordinate system is a coordinate system having a point of the image plane projected by perspective transformation as the origin and having coordinate axes represented by the Xi axis and the Yi axis orthogonal to the Xi axis.

[0066] In the coordinate point calculation step of step S105i, the calculation unit 622 calculates an image coordinate point indicating the position of the assembled vehicle 10m in the image coordinate system. Fig.10 : is a diagram showing the details of the coordinate point calculation step. To calculate the image coordinate point, the calculation unit 622 first calculates the basic coordinate point P0 based on the first circumscribed rectangle R1 set for the mask region Ms in the first mask image Im5 which is the image before perspective transformation. Fig.11 is a diagram showing a method for calculating the basic coordinate point P0. In order to calculate the basic coordinate point P0, the calculation unit 622 sets a basic circumscribed rectangle R0 for the mask area Ms in the first mask image Im5. Next, the calculation unit 622 rotates the first mask image Im5 by a desired rotation amount with the center of gravity C of the mask area Ms as the rotation center, so that the direction of the motion vector V of the assembled vehicle 10m corresponding to the mask area Ms in the first mask image Im5 faces a predetermined direction. The predetermined direction is, for example, an upward direction on the screen of the display device displaying the first mask image Im5. Then, the calculation unit 622 sets a first circumscribed rectangle R1 for the mask area Ms of the rotated first mask image Im5 so that its long side is parallel to the motion vector V. Then, the calculation unit 622 rotates the first mask image Im5 to which the first circumscribed rectangle R1 is added in the reverse direction by a rotation amount with the center of gravity C of the mask area Ms as the rotation center. As a result, the calculation unit 622 sets the coordinate point of the vertex which is one of the four vertices of the first circumscribed rectangle R1 and has the coordinates closest to the anchor point 10f of the assembled vehicle 10m as the base coordinate point P0.

[0067] Next, if Fig.10 As shown in , the calculation unit 622 performs perspective transformation on the first mask image Im5 after the reverse rotation (i.e., the first mask image Im5 after the basic coordinate point P0 is calculated). As a result, the calculation unit 622 sets the coordinate point corresponding to the basic coordinate point P0 in the first circumscribed rectangle R1 deformed by the perspective transformation as the first coordinate point P1.

[0068] Next, the calculation unit 622 sets a second circumscribed rectangle R2 for the mask area Ms in the second mask image Im6 obtained by performing perspective transformation on the first mask image Im5. Then, the calculation unit 622 sets the vertex indicating the same position as the first coordinate point P1 among the vertices of the second circumscribed rectangle R2 as the second coordinate point P2. That is, the first coordinate point P1 and the second coordinate point P2 are coordinate points indicating the same position, so the first coordinate point P1 and the second coordinate point P2 have correlation.

[0069] Next, the calculation unit 622 performs a correction to replace the coordinates (Xi1, Yi1) of the first coordinate point P1 with the coordinates (Xi2, Yi2) of the second coordinate point P2 according to the size relationship between the coordinate values ​​of the first coordinate point P1 and the coordinate values ​​of the second coordinate point P2. When the coordinate value Xi1 of the first coordinate point P1 in the Xi direction is greater than the coordinate value Xi2 of the second coordinate point P2 in the Xi direction (Xi1>Xi2), the calculation unit 622 replaces the coordinate value Xi1 of the first coordinate point P1 in the Xi direction with the coordinate value Xi2 of the second coordinate point P2 in the Xi direction. When the coordinate value Yi1 of the first coordinate point P1 in the Yi direction is greater than the coordinate value Yi2 of the second coordinate point P2 in the Yi direction (Yi1>Yi2), the calculation unit 622 replaces the coordinate value Yi1 of the first coordinate point P1 in the Yi direction with the coordinate value Yi2 of the second coordinate point P2 in the Yi direction. Fig.10 In the example shown in , the coordinate value Xi1 of the first coordinate point P1 in the Xi direction is greater than the coordinate value Xi2 of the second coordinate point P2 in the Xi direction. The coordinate value Xi1 of the first coordinate point P1 in the Yi direction is less than the coordinate value Yi2 of the second coordinate point P2 in the Yi direction. Therefore, the image coordinate point P3 has the coordinates (Xi2, Yi1). As described above, the calculation unit 622 calculates the image coordinate point P3 indicating the position of the assembled vehicle 10m in the image coordinate system by correcting the first coordinate point P1 using the second coordinate point P2.

[0070] In the position transformation step of step S105j, the calculation unit 622 calculates the vehicle coordinate point indicating the position of the positioning point 10f of the assembled vehicle 10m in the global coordinate system by transforming the image coordinate point P3 into the vehicle coordinate point. The calculation unit 622 uses the relationship formulas from the following equations (1) to (3) including the vehicle coordinate point as the target variable and including the image coordinate point P3, the imaging parameter and the vehicle parameter as the explanatory variables to transform the image coordinate point P3 into the vehicle coordinate point. The imaging parameter is a parameter related to the distance between the camera Cm and the predetermined reference point. In the present embodiment, the imaging parameter is the height H (described below) of the camera Cm from the road surface Rs. Fig.12 The vehicle parameter is a parameter related to the distance between the positioning point 10f of the assembled vehicle 10m and the reference point. In this embodiment, the vehicle parameter is the height h of the positioning point 10f of the assembled vehicle 10m from the road surface Rs (see the following description). Fig.12 ).

[0071] Fig.12 1 is a first diagram showing details of the position conversion step. Fig.12 The state of the assembled vehicle 10m is shown when viewed from the left side. Fig.13 is a second diagram showing details of the position conversion step. Fig.13The state of the assembled vehicle 10m is shown when viewed from the top side. Fig.12 and Fig.13 The global coordinate system shown in is a coordinate system having a fixed coordinate point Pf indicating an arbitrary reference position on the road surface Rs as an origin and having coordinate axes represented by an Xg axis and a Yg axis orthogonal to the Xg axis. The imaging coordinate point Pc is a coordinate point indicating the position of the camera Cm that outputs the original image Im1 used to calculate the image coordinate point P3 in the global coordinate system. The fixed coordinate point Pf and the imaging coordinate point Pc are pre-stored in, for example, the storage unit 63 of the computing device 6.

[0072] like Fig.12 As shown in , the observation distance between the position of the camera Cm and the position of the assembled vehicle 10m (image coordinate point P3) on the XgYg plane is represented as Do. The observation error is represented as ΔD. The height [m] of the camera Cm from the road surface Rs as an imaging parameter is represented as H. The height [m] of the positioning point 10f of the assembled vehicle 10m from the road surface Rs as a vehicle parameter is represented as h. In this case, the observation error ΔD can be represented by the following formula (1). ΔD = h / H × Do Formula (1) That is, the larger the observation distance Do is, the larger the observation error ΔD is.

[0073] When the actual distance between the position of the camera Cm and the position of the positioning point 10f of the assembled vehicle 10m is defined as the first distance D, the first distance D can be expressed by the following equation (2). D = Do × (1 - h / H) Formula (2) That is, the first distance D is determined based on the observation distance Do, the height H of the camera Cm as an imaging parameter, and the height h of the positioning point 10f of the assembled vehicle 10m as a vehicle parameter.

[0074] like Fig.13 As shown in , when the distance between the reference position and the estimated position of the assembled vehicle 10m is defined as the estimated distance Dp, and the actual distance between the reference position and the assembled vehicle 10m is defined as the second distance Dt, the second distance Dt can be expressed by the following formula (3). Dt = Dp × (1 - h / H) Formula (3)

[0075] Here, the estimated distance Dp can be calculated by using the third distance Dc, the image coordinate point P3, and the fixed coordinate point Pf, the third distance Dc being the actual distance obtained from the fixed coordinate point Pf and the imaging coordinate point Pc. Therefore, the calculation unit 622 calculates the vehicle coordinate point Pv by using the second distance Dt obtained by correcting the estimated distance Dp using the above formula (3), and the fixed coordinate point Pf. In this case, since the calculated vehicle coordinate point Pv is a coordinate point indicating the position of the positioning point 10f of the assembled vehicle 10m in the global coordinate system, the calculated vehicle coordinate point Pv corresponds to the position of the assembled vehicle 10m in the real space.

[0076] The calculation method of the position and orientation of the vehicle 10 represented by the captured images Im1 to Im4 is not limited to the above. The calculation unit 622 can calculate the orientation of the vehicle 10 by using the output of a yaw rate sensor or the like installed on the vehicle 10, for example.

[0077] According to the first embodiment, in order to make the vehicle 10 travel by remote control, the computing device 6 acquires sensor information from the external sensor Se and detects the appearance shape of the vehicle 10 represented by the sensor information to calculate the position of the vehicle 10, etc. In this case, since the vehicle 10 as the calculation target travels in the factory Fc1, the vehicle 10 has different appearances depending on the manufacturing steps Pr1 to Pr7. In the case of detecting the appearance shape of the vehicle 10 by using the detection model Md which is a machine learning model trained by performing the learning process, the feature value extracted when the first detection model Md1 is trained is different depending on the states St1, St2 of the vehicle 10. Therefore, in order to suppress the decrease in detection accuracy when the appearance shape of the vehicle 10 is detected from the sensor information due to the difference in the appearance of the vehicle 10, it is desirable to detect the appearance shape of the vehicle 10 by using the detection model Md as shown below. In this case, it is desirable to train the detection model Md for detecting the appearance shape of the vehicle 10 represented by the sensor information for each of the states St1, St2 of the vehicle 10 corresponding to the appearance of the vehicle 10. However, in the case of training the detection model Md for each of the states St1, St2 of the vehicle 10, it may be necessary to prepare data for learning for each of the states St1, St2 of the vehicle 10, or the learning time may become longer depending on the number of states St1, St2 of the vehicle 10. Meanwhile, according to the first embodiment, in the factory Fc1, the first sensor Se1 is installed in the first area Ar11, and the second sensor Se2 of a different kind from the first sensor Se1 is installed in the second area Ar21. As a result, when calculating the position of the first vehicle 101, etc., the computing device 6 can perform the following processing. In this case, the computing device 6 can acquire the first sensor information indicating the first vehicle 101 output from the first sensor Se1 by detecting the first vehicle 101 from the outside via the first sensor Se1. The computing device 6 can calculate the position of the first vehicle 101, etc. indicated by the first sensor information, by using the first sensor information without performing the learning process. When calculating the position of the second vehicle 102, etc., the computing device 6 can perform the following processing. In this case, the computing device 6 can acquire the second sensor information indicating the second vehicle 102 output from the second sensor Se2 by detecting the second vehicle 102 from the outside via the second sensor Se2. The computing device 6 is able to calculate the position of the second vehicle 102 indicated by the second sensor information, etc., by using the first detection model Md1 as the detection model Md and the captured image. In this way, the first detection model Md1 does not need to learn the feature values ​​of the first vehicle 101. Therefore, time and effort for preparing data for learning for the first vehicle 101 can be saved.Furthermore, the time required for learning the feature value of the first vehicle 101 can be reduced from the time required for training the first detection model Md1. As a result, the learning time of the first detection model Md1 can be shortened. As can be seen from the above, in a state where the possibility of a decrease in the detection accuracy when detecting the appearance shape of the vehicle 10 from the first sensor information is reduced, the load required for the learning process can be reduced.

[0078] Here, in order to reduce the load required for the learning process, a method can be conceived in which the position of the vehicle 10, etc., is calculated without performing the learning process for the vehicle 10 in any state St1, St2. However, depending on the type of the external sensor Se, even in the case where the position of the vehicle 10, etc. can be calculated without performing the learning process, the cost required to calculate the position of the vehicle 10, etc. may be higher than the cost required to perform the learning process to calculate the position of the vehicle 10, etc. According to the first embodiment, different types of external sensors Se may be installed in the first area Ar11 and the second area Ar21, depending on the price of the external sensor Se. As a result, the possibility of an increase in the cost required to calculate the position of the vehicle 10, etc. can be reduced.

[0079] According to the first embodiment, the first sensor Se1 is LiDAR Li that outputs measurement point group data as first sensor information. The calculation device 6 can perform matching between the measurement point group data and the reference point group data without using the detection model Md to calculate the position of the first vehicle 101 represented by the measurement point group data.

[0080] According to the first embodiment, the second sensor Se2 is a camera Cm that outputs an original image Im1 as second sensor information. The computing device 6 can obtain shape data Da by inputting the captured images Im1 to Im4 into the first detection model Md1, calculate the image coordinate point P3 by using the shape data Da, and calculate the position of the second vehicle 102 by transforming the image coordinate point P3 into the vehicle coordinate point Pv, etc.

[0081] According to the first embodiment, the computing device 6 can calculate the position of the first vehicle 101, etc. by using the measurement point group data output from the LiDARLi without performing a learning process. However, the LiDAR Li is generally more expensive than the camera Cm. Therefore, in the case where the vehicle 10 in any state St1, St2 is detected by the LiDAR Li, the cost required to calculate the position of the vehicle 10, etc. is higher than the case where any state St1, St2 is detected by the camera Cm. At the same time, according to the first embodiment, the computing device 6 can calculate the position, etc. by using the first detection model Md1 trained by performing a learning process and the captured images Im1 to Im4 for the second vehicle 102. As a result, the possibility of an increase in the cost required to calculate the position of the second vehicle 102, etc. can be reduced. Therefore, the computing device 6 is able to calculate the position of the vehicle 10, etc. at a low cost, while reducing the load required for the learning process performed to calculate the position of the vehicle 10, etc.

[0082] According to the first embodiment, the travel system 1 can make the vehicle 10 travel in the factory Fc1 by remote control. Therefore, the travel system 1 can move the vehicle 10 in the factory Fc1 without using a transportation facility such as a crane or a conveyor. B. Second Embodiment

[0083] Fig.14 1 is a diagram showing a state where the vehicle 10 in the second embodiment is traveling in the factory Fc2. In this embodiment, the platform manufacturing step Pr1, the first assembly step Pr3a, the second assembly step Pr3b, and the inspection step Pr5 are sequentially performed as a plurality of manufacturing steps Pr1 to Pr8, and as a result, the vehicle 10 as a finished product is manufactured. Then, through the transportation step Pr8, the vehicle 10 as a finished product is stored in the storage location PL7 in the storage step Pr7 until the vehicle 10 is shipped.

[0084] The platform manufacturing step Pr1 and the inspection step Pr5 are the same as those in the first embodiment. The first assembly step Pr3a is a step of assembling internal components such as the seat 19 into the platform vehicle 10p. The first assembly step Pr3a is performed at the first assembly location PL3a. The second assembly step Pr3b is a step of assembling external components such as the body shell 18 into the vehicle 103 assembled with the internal components. The second assembly step Pr3b is performed at the second assembly location PL3b. In the present embodiment, the manufacturing location PL1, the first assembly location PL3a, the second assembly location PL3b and the inspection location PL5 exist in the same building. The storage location PL7 exists in a building different from the building where the manufacturing location PL1, the first assembly location PL3a, the second assembly location PL3b and the inspection location PL5 exist. Therefore, the inspection location PL5 and the storage location PL7 are connected via the route Ro4. The transportation step Pr8 is a step of moving the vehicle 10 from the inspection location PL5 to the storage location PL7. Furthermore, in the present embodiment, the vehicle 10 is remotely controlled by the remote control device 7 to travel by unmanned driving in at least a part of the section from the manufacturing site PL1 to the storage site PL7 .

[0085] The appearance of the vehicle 10 may differ between the platform vehicle 10p, the under-assembly vehicle 10n, and the assembled vehicle 10m due to the difference in the appearance shape of the vehicle 10. The under-assembly vehicle 10n is a vehicle 10 in which at least the seat 19 is assembled into the platform vehicle 10p. In the present embodiment, the vehicle 10 after the first assembly step Pr3a starts and before the second assembly step Pr3b starts is the under-assembly vehicle 10n. The vehicle 10 after the second assembly step Pr3b starts is the assembled vehicle 10m.

[0086] In the present embodiment, the vehicle 10 is classified into three states St1 to St3 including a first state St1, a second state St2, and a third state St3, depending on whether the seat 19 is assembled and whether the vehicle body shell 18 is assembled. In the present embodiment, the first state St1 refers to a case where the states St1 to St3 of the vehicle 10 are a platform state Sp. The second state St2 refers to a case where the states St1 to St3 of the vehicle 10 are an assembled state Sm. The third state St3 refers to a case where the states St1 to St3 of the vehicle 10 are an assembling state Sn.

[0087] Hereinafter, the vehicle 103 classified as the third state St3 will be referred to as the "third vehicle 103". In the present embodiment, the first vehicle 101 is a platform vehicle 10p. The second vehicle 102 is an assembled vehicle 10m. The third vehicle 103 is an assembling vehicle 10n. In the case where there is no need to distinguish between the vehicles 101 to 103 in the states St1 to St3, the vehicles 101 to 103 will be referred to simply as "vehicles 10".

[0088] Fig.15 is a diagram showing the configuration of a computing system 5a in the second embodiment. The computing system 5a includes a first sensor Se1, a second sensor Se2, a third sensor Se3, and a computing device 6a. In this embodiment, the first sensor Se1 and the third sensor Se3 are Lidar Li as an external sensor Se. The second sensor Se2 is a camera Cm as an external sensor Se.

[0089] like Fig.14 As shown in , the first sensor Se1 is installed in the first area Ar12 in the factory Fc2. In the present embodiment, the first area Ar12 includes the manufacturing location PL1. The second sensor Se2 is installed in the second area Ar22 in the factory Fc2. In the present embodiment, the second area Ar22 includes the second assembly location PL3b, the inspection location PL5, the route Ro4, and the storage location PL7. The third sensor Se3 is installed in the third area Ar31 in the factory Fc2. In the present embodiment, the third area Ar31 includes the first assembly location PL3a.

[0090] Therefore, the LiDAR Li1 as the first sensor Se1 detects the platform vehicle 10p from the outside to output the measurement point group data representing the appearance shape of the platform vehicle 10p as the first sensor information. The cameras Cm1 to Cm4 as the second sensor Se2 image the assembled vehicle 10m from the outside to output the original image Im1 containing the assembled vehicle 10m as the second sensor information. The LiDAR Li2 as the third sensor Se3 detects the assembling vehicle 10n from the outside to output the measurement point group data representing the appearance shape of the assembling vehicle 10n in three dimensions as the third sensor information.

[0091] Fig.15 The calculation device 6a shown in the figure calculates at least one of the position and orientation of the vehicle 10 by using the sensor information output from the external sensor Se installed in the factory Fc2. The calculation device 6a includes a communication unit 61, a storage unit 63a, and a CPU 62a. In addition, the same configuration as that in the first embodiment is denoted by the same reference numerals, and the description thereof will be omitted.

[0092] The storage unit 63a of the computing device 6a stores various kinds of information including various programs P63a for controlling the operation of the computing device 6a, management information In, and the first detection model Md1.

[0093] The CPU 62a of the computing device 6a operates as a sensor information acquisition unit 621 and a computing unit 622a by loading various programs P63a stored in the storage unit 63a. When the sensor information acquisition unit 621 acquires the measurement point group data from the LiDAR Li1, Li2, the computing unit 622a performs matching between the measurement point group data and the reference point group data to calculate the position and orientation of the platform vehicle 10p and the vehicle under assembly 10n. When the sensor information acquisition unit 621 acquires the original image Im1, the computing unit 622a inputs the captured images Im1 to Im4 into the first detection model Md1. As a result, the computing unit 622a acquires the shape data Da, and calculates the position and orientation of the assembled vehicle 10m represented by the captured images Im1 to Im4.

[0094] According to the second embodiment, in the factory Fc2, the LiDAR Li that outputs the measurement point group data for calculating the position of the vehicle 10 and the like without performing the learning process as a preprocessing is installed in two areas Ar22, Ar31 among the three areas Ar12 to Ar31. As a result, for the first vehicle 101 and the third vehicle 103, the positions of the vehicles 101 and 103 and the like can be calculated without using the detection model Md trained by performing the learning process. As described above, in the case where the vehicle 10 is classified into three or more states St1 to St3 depending on the appearance of the vehicle 10, the position of the vehicle 10 and the like can be calculated without using the detection model Md for at least a part of the three or more states St1 to St3. Therefore, even in the case where the vehicle 10 is classified into three or more states St1 to St3, the load required for the learning process can be reduced. C. Third Embodiment

[0095] Fig.161 is a diagram showing a state in which the vehicle 10 in the third embodiment is traveling in the factory Fc3. In the present embodiment, the manufacturing steps Pr1 to Pr8 that are the same as those in the second embodiment are performed in the same order. Therefore, as in the second embodiment, the vehicle 10 is classified into three states St1 to St3 including a first state St1, a second state St2, and a third state St3, depending on whether the seat 19 is assembled and whether the vehicle body shell 18 is assembled. In the present embodiment, the first state St1 refers to a case in which the states St1 to St3 of the vehicle 10 are an assembled state Sm. The second state St2 refers to a case in which the states St1 to St3 of the vehicle 10 are a platform state Sp. The third state St3 refers to a case in which the states St1 to St3 of the vehicle 10 are an assembling state Sn. Moreover, in the present embodiment, the vehicle 10 is remotely controlled by the remote control device 7 to travel by unmanned driving in at least a partial section from the manufacturing location PL1 to the storage location PL7. The same configurations as those in the above-mentioned respective embodiments are denoted by the same reference numerals, and descriptions thereof will be omitted.

[0096] In this embodiment, the manufacturing site PL1, the first assembly site PL3a, the second assembly site PL3b and the inspection site PL5 are located indoors. The inspection site PL5 and the storage site PL7 are connected via the route Ro5. The route Ro5 and the storage site PL7 are located outdoors.

[0097] Fig.17 is a diagram showing the configuration of a computing system 5b in the third embodiment. The computing system 5b includes a first sensor Se1, a second sensor Se2, a third sensor Se3, and a computing device 6b. In this embodiment, the first sensor Se1 is a LiDAR Li as an external sensor Se. The second sensor Se2 and the third sensor Se3 are cameras Cm as external sensors Se.

[0098] like Fig.16 As shown in, the first sensor Se1 is installed in the first area Ar13 in the factory Fc3. In the present embodiment, the first area Ar13 includes the second assembly location PL3b, the inspection location PL5, the route Ro5, and the storage location PL7. The second sensor Se2 is installed in the second area Ar23 in the factory Fc3. In the present embodiment, the second area Ar23 includes the manufacturing location PL1. The third sensor Se3 is installed in the third area Ar32 in the factory Fc3. The third area Ar32 is an area in which the third vehicle 103 travels. In the present embodiment, the third area Ar32 includes the first assembly location PL3a.

[0099] Therefore, the LiDAR Li1 to Li4 as the first sensor Se1 detect the assembled vehicle 10m from the outside to output the measurement point group data that three-dimensionally represents the appearance shape of the assembled vehicle 10m as the first sensor information. The camera Cm1 as the second sensor Se2 detects the platform vehicle 10p from the outside to output the original image Im1 containing the platform vehicle 10p as the second sensor information. The camera Cm2 as the third sensor Se3 detects the assembling vehicle 10n from the outside to output the original image Im1 containing the assembling vehicle 10n as the third sensor information.

[0100] Fig.17 The calculation device 6b shown in FIG. 6 calculates at least one of the position and orientation of the vehicle 10 by using sensor information output from the external sensor Se installed in the plant Fc3. The calculation device 6b includes a communication unit 61, a storage unit 63b, and a CPU 62b.

[0101] The storage unit 63b of the computing device 6b stores various kinds of information including various programs P63b for controlling the operation of the computing device 6b, management information In, and a plurality of second detection models Md2 prepared respectively for the states St2, St3 of the vehicle 10 as the detection models Md. When the captured images Im1 to Im4 are input, the second detection model Md2 detects the vehicle 10 from the captured images Im1 to Im4 to output shape data Da.

[0102] Each second detection model Md2 is pre-trained by supervised learning using a second learning data set corresponding to the state St2, St3 of the vehicle 10 prepared for each state St2, St3 of the vehicle 10. The second learning data set includes a plurality of training images containing the vehicle 10, and a regional correct answer label. The second learning data set includes M (M is an integer greater than two) first training images and N (N is an integer greater than zero and less than M) second training images as training images. The first training image is an image containing the vehicle 10 classified as one state. The second training image is an image containing the vehicle 10 classified as the remaining other states among the states St1 to St3 of the vehicle 10 except the one state. The number M of the first training images included in the second learning data set can be, for example, more than 1.5 times or more than 2 times the number N of the second training images. In the case where there are multiple other states different from the one state, the number M of the first training images included in the second learning data set is, for example, greater than the total number of second training images containing the vehicle 10 classified as the remaining other states except the one state. That is, the second detection model Md2 is a machine learning model that preferentially learns the feature value of the vehicle 10 classified into the one state by learning the first training image more than the second training image.

[0103] like Fig.16 As shown in FIG. 1 , in this embodiment, the platform vehicle 10 p and the vehicle under assembly 10 n are detected from the captured images Im1 to Im4 to calculate the positions and orientations of the platform vehicle 10 p and the vehicle under assembly 10 n. Fig.17 As shown in , the storage unit 63b of the computing device 6b stores various kinds of information including two second detection models Md2. The two second detection models Md2 are a second detection model Md2a and a second detection model Md2b. The second detection model Md2a is a second detection model Md2 that learns that the first training image containing the platform vehicle 10p is more than the second training image containing any one of the assembling vehicle 10n and the assembled vehicle 10m. The second detection model Md2a is used to detect the platform vehicle 10p from the captured images Im1 to Im4. The second detection model Md2b is a second detection model Md2 that learns that the first training image containing the assembling vehicle 10n is more than the second training image containing any one of the platform vehicle 10p and the assembled vehicle 10m. The second detection model Md2b is used to detect the assembling vehicle 10n from the captured images Im1 to Im4.

[0104] The CPU 62b of the computing device 6b operates as a sensor information acquisition unit 621 and a computing unit 622b by loading various programs P63b stored in the storage unit 63b. In the case where the sensor information acquisition unit 621 acquires the measurement point group data from the LiDAR Li1 to Li4, the computing unit 622b performs matching between the measurement point group data and the reference point group data to calculate the position and orientation of the assembled vehicle 10m. In the case where the sensor information acquisition unit 621 acquires the original image Im1, the computing unit 622b performs the following processing. In this case, the computing unit 622b inputs the captured images Im1 to Im4 to the second detection models Md2a, Md2b selected depending on the states St2, St3 of the vehicle 10 from the two second detection models Md2a, Md2b prepared for the states St2, St3 of the vehicle 10, respectively. Specifically, when the sensor information acquisition unit 621 acquires the original image Im1 from the camera Cm1 installed in the second area Ar23, the calculation unit 622b inputs the captured images Im1 to Im4 into the second detection model Md2a corresponding to the platform state Sp. When the sensor information acquisition unit 621 acquires the original image Im1 from the camera Cm2 installed in the third area Ar32, the calculation unit 622b inputs the captured images Im1 to Im4 into the second detection model Md2b corresponding to the assembling state Sn. As a result, the calculation unit 622b acquires the shape data Da, and calculates the positions and orientations of the platform vehicle 10p and the assembling vehicle 10n represented by the captured images Im1 to Im4.

[0105] Fig.18 is a flowchart showing a method for calculating the position of the vehicle 10 using the second detection model Md2. Figure 8 The same steps as those in the calculation method shown in FIG. 1 are denoted by the same step numbers, and description thereof will be omitted.

[0106] In the state acquisition step of step S105e, the calculation unit 622b acquires state information. The state information is information indicating which state of the state St1 to St3 the vehicle 10 is in. In addition, the state acquisition step can be performed at any time point after the image acquisition step ends and before the detection step starts.

[0107] In the model acquisition step of step S105f, the calculation unit 622b selects and acquires the second detection models Md2a, Md2b corresponding to the states St2, St3 of the vehicle 10 specified by using the state information from the two second detection models Md2a, Md2b pre-stored in the storage unit 63b of the calculation device 6b. In addition, the model acquisition step can be performed at any time point after the state acquisition step ends and before the detection step starts.

[0108] According to the third embodiment, the second detection model Md2 suitable for detecting the vehicle 10 classified into the one state is prepared in advance by learning that the first training image containing the vehicle 10 classified into the one state is more than the second training image containing the vehicle 10 classified into the remaining other states. As a result, the computing device 6b can acquire the state information and select the second detection model Md2 corresponding to the state specified by using the acquired state information from the second detection models Md2 prepared for each state. The computing device 6b can accurately detect the vehicles 102, 103 from the captured images Im1 to Im4 by inputting the captured images Im1 to Im4 into the second detection model Md2 corresponding to the states St2, St3 of the vehicles 102, 103 included in the captured images Im1 to Im4. Therefore, the detection accuracy when detecting the vehicles 102, 103 from the captured images Im1 to Im4 can be improved.

[0109] Here, when imaging the vehicle 10 by using the camera Cm, the imaging condition may change due to the external environment at the time of imaging. For example, due to differences caused by weather such as sunny or rainy days, or due to differences caused by time periods such as daytime or nighttime, differences in the irradiation state of sunlight in the detection range RG2 of the camera Cm may result, thereby possibly changing the brightness of the captured images Im1 to Im4. As described above, in the case where the imaging condition changes due to the external environment at the time of imaging, the detection accuracy when detecting the vehicle 10 from the captured images Im1 to Im4 may be reduced. On the other hand, the detection of the vehicle 10 by using the LiDAR Li is not easily affected by interference corresponding to the external environment such as sunlight. Therefore, in the third embodiment, the camera Cm is installed in the second area Ar23 and the third area Ar32 existing indoors, and the LiDAR Li is installed in the first area Ar13 partially existing outdoors. In this way, it is possible to suppress the reduction of the detection accuracy when detecting the vehicle 10 from the sensor information. D. Fourth Embodiment

[0110] Fig.19 4 is a diagram showing the configuration of a computing system 5c in the fourth embodiment. In this embodiment, the type of detection model Md is different from that in the above-mentioned embodiments. Fig.16 The configuration in the third embodiment shown in FIG. 1 is the same. Therefore, as in the third embodiment, the vehicle 10 is classified into three states St1 to St3 including a first state St1, a second state St2 and a third state St3, depending on whether the seat 19 is assembled and whether the body shell 18 is assembled. In the present embodiment, the first state St1 refers to a case where the states St1 to St3 of the vehicle 10 are an assembled state Sm. The second state St2 refers to a case where the states St1 to St3 of the vehicle 10 are a platform state Sp. The third state St3 refers to a case where the states St1 to St3 of the vehicle 10 are an assembling state Sn. Moreover, in the present embodiment, the vehicle 10 is remotely controlled by the remote control device 7 to travel by unmanned driving in at least a portion of the section from the manufacturing location PL1 to the storage location PL7. Configurations identical to those in the above-mentioned embodiments are represented by the same figure marks, and descriptions thereof will be omitted.

[0111] like Fig.19 As shown in , the computing system 5c includes a first sensor Se1, a second sensor Se2, a third sensor Se3 and a computing device 6c. As in the third embodiment, in this embodiment, the first sensor Se1 is a LiDAR Li as an external sensor Se. The second sensor Se2 and the third sensor Se3 are cameras Cm as external sensors Se.

[0112] The calculation device 6c calculates at least one of the position and orientation of the vehicle 10 by using sensor information output from the external sensor Se installed in the plant Fc3. The calculation device 6c includes a communication unit 61, a storage unit 63c, and a CPU 62c.

[0113] The storage unit 63c of the computing device 6c stores various kinds of information, including various programs P63c for controlling the operation of the computing device 6c, management information In, and a third detection model Md3 as a detection model Md. When the captured images Im1 to Im4 and the states St2, St3 of the vehicle 10 contained in the captured images Im1 to Im4 are input, the third detection model Md3 detects the vehicle 10 from the captured images Im1 to Im4 to output shape data Da. The third detection model Md3 is pre-trained by supervised learning using a third learning data set. The third learning data set includes a plurality of training images containing the vehicle 10, region correct answer labels, and state correct answer labels. The state correct answer label is a correct answer label indicating the states St1 to St3 of the vehicle 10 contained in the training image. The state correct answer label is associated with each training image.

[0114] The CPU 62c of the computing device 6c operates as a sensor information acquisition unit 621 and a computing unit 622c by loading various programs P63c stored in the storage unit 63c. In the case where the sensor information acquisition unit 621 acquires the measurement point group data from the LiDAR Li1 to Li4, the computing unit 622c performs matching between the measurement point group data and the reference point group data to calculate the position and orientation of the assembled vehicle 10m. In the case where the sensor information acquisition unit 621 acquires the original image Im1, the computing unit 622 inputs the states St2 and St3 of the vehicle 10 contained in the captured images Im1 to Im4 and the captured images Im1 to Im4z into the third detection model Md3. Specifically, in the case where the sensor information acquisition unit 621 acquires the original image Im1 from the camera Cm1 installed in the second area Ar23, the computing unit 622c inputs the captured images Im1 to Im4 and the information indicating the platform state Sp into the third detection model Md3. In the case where the sensor information acquisition unit 621 acquires the original image Im1 from the camera Cm2 installed in the third area Ar32, the calculation unit 622c inputs the captured images Im1 to Im4 and the information indicating the assembling state Sn into the third detection model Md3. As a result, the calculation unit 622c acquires the shape data Da and calculates the positions and orientations of the platform vehicle 10p and the assembling vehicle 10n represented by the captured images Im1 to Im4.

[0115] According to the fourth embodiment, a third detection model Md3 trained by associating training images containing the vehicle 10 with a state correct answer label indicating the state of the vehicle 10 contained in each training image is prepared in advance. As a result, it is possible to accurately detect the vehicles 102, 103 from the captured images Im1 to Im4 by inputting the captured images Im1 to Im4 associated with the states St2, St3 of the vehicle 10 into the third detection model Md3. Therefore, it is possible to improve the detection accuracy when detecting the vehicles 102, 103 from the captured images Im1 to Im4. E. Fifth Embodiment

[0116] Fig. 20 is a diagram showing a state in which the vehicle 10 in the fifth embodiment is traveling in the factory Fc4. In the present embodiment, the manufacturing steps Pr1 to Pr7 that are the same as those in the first embodiment are performed in the same order. Therefore, as in the first embodiment, the vehicle 10 is classified into two states St1, St2 including a first state St1 and a second state St2, depending on whether the vehicle body shell 18 is assembled. In the present embodiment, the first state St1 refers to a case in which the states St1, St2 of the vehicle 10 are the platform state Sp. The second state St2 refers to a case in which the states St1, St2 of the vehicle 10 are the assembled state Sm. Moreover, in the present embodiment, the vehicle 10 is remotely controlled by the remote control device 7 to travel by unmanned driving in at least a part of the section from the manufacturing location PL1 to the storage location PL7. The same configurations as those in the above-mentioned respective embodiments are denoted by the same reference numerals, and descriptions thereof will be omitted.

[0117] Fig.21 is a diagram showing the configuration of a computing system 5d in the fifth embodiment. The computing system 5d includes a first sensor Se1, a second sensor Se2, an information processing device 8, and a computing device 6d. In this embodiment, the first sensor Se1 is a LiDAR Li as an external sensor Se. The second sensor Se2 is a camera Cm as an external sensor Se.

[0118] like Fig. 20 As shown in FIG. 1 , in the present embodiment, both the first sensor Se1 and the second sensor Se2 are installed in each of the areas Ar14 and Ar24 in the factory Fc4. In the present embodiment, the first area Ar14 includes the manufacturing site PL1 and the first route Ro1. The second area Ar24 includes the assembly site PL3, the second route Ro2, the inspection site PL5, the third route Ro3, and the storage site PL7.

[0119] Fig.21The information processing device 8 shown in FIG. 8 determines the type of the external sensor Se detecting the vehicle 10 to calculate at least one of the position and orientation of the vehicle 10 by using sensor information output by the external sensor Se of the type corresponding to the states St1, St2 of the vehicle 10. The information processing device 8 includes a communication unit 81, a storage unit 83, and a CPU 82.

[0120] The communication unit 81 of the information processing device 8 connects the information processing device 8 and other devices outside the information processing device 8 so that the devices can communicate with each other. The communication unit 81 of the information processing device 8 is, for example, a wireless communication device.

[0121] The storage unit 83 of the information processing device 8 stores various kinds of information, including various programs P83 and a database Db for controlling the operation of the information processing device 8. The database Db is used to determine which external sensor Se among a plurality of external sensors Se installed in each of the areas Ar14, Ar24 is used to detect the vehicle 10. In the database Db, at least one of the states St1, St2 of the vehicle 10 specified by using the state information, and the manufacturing steps Pr1 to Pr7 specified by using the step information is associated with the kind of external sensor Se used to detect the vehicle 10. The step information is information indicating which manufacturing step of the manufacturing steps Pr1 to Pr7 is being performed on the vehicle 10. In the present embodiment, the database Db is used to determine which external sensor Se of the first sensor Se1 and the second sensor Se2 is used to detect the vehicle 10 in each of the first area Ar14 and the second area Ar24.

[0122] The CPU 82 of the information processing device 8 operates as a determination information acquisition unit 821 and a determination unit 822 by loading various programs P83 stored in the storage unit 83. The determination information acquisition unit 821 acquires at least one of state information and step information. In the present embodiment, the determination information acquisition unit 821 acquires step information. The determination unit 822 performs the following processing in order to calculate at least one of the position and orientation of the vehicle 10 by using sensor information output from the external sensor Se of the type corresponding to the state St1, St2 of the vehicle 10. In this case, the determination unit 822 determines which external sensor Se among various external sensors Se installed in each of the areas Ar14, Ar24z is used to detect the vehicle 10 by using at least one of the state information and the step information. In the present embodiment, the determination unit 822 determines which external sensor Se of the first sensor Se1 and the second sensor Se2 is used to detect the vehicle 10 by using the step information and the database Db. Then, the determination unit 822 transmits the determination result to the calculation device 6d. Furthermore, at least a part of the functions of the CPU 82 of the information processing device 8 may be realized by a hardware circuit.

[0123] The calculation device 6d calculates at least one of the position and orientation of the vehicle 10 by using sensor information output by the external sensor Se of the kind specified by the determination result of the information processing device 8. The calculation device 6d includes a communication unit 61, a storage unit 63d, and a CPU 62d.

[0124] The storage unit 63d of the computing device 6d stores various kinds of information including various programs P63d for controlling the operation of the computing device 6d, the first detection model Md1, and management information In.

[0125] The CPU 62d of the calculation device 6d operates as a sensor information acquisition unit 621d and a calculation unit 622 by loading various programs P63d stored in the storage unit 63d. When the determination unit 822 of the information processing device 8 determines that the first sensor Se1 is to be used to detect the vehicle 10, the sensor information acquisition unit 621d of the calculation device 6d acquires the first sensor information output from the first sensor Se1 including the vehicle 10 as the calculation target in the detection range RG1. When the determination unit 822 of the information processing device 8 determines that the second sensor Se2 is to be used to detect the vehicle 10, the sensor information acquisition unit 621d of the calculation device 6d acquires the second sensor information output from the second sensor Se2 including the vehicle 10 as the calculation target in the detection range RG2. In addition, when the external sensor Se including the vehicle 10 as the calculation target in the detection ranges RG1 and RG2 is specified, the sensor information acquisition unit 621d may use the management information In, or may use at least one of the state information and the step information.

[0126] Fig. 22 is a first flowchart showing the flow of processing in the fifth embodiment. Fig.23 is a second flowchart showing the flow of processing in the fifth embodiment. Fig. 22 and Fig.23 The processing flow shown in , for example, is repeatedly executed at predetermined time intervals starting from the time when the vehicle 10 starts to travel by unmanned driving.

[0127] like Fig. 22 As shown in FIG. 5 , in step S501, the determination information acquisition unit 821 of the information processing device 8 acquires step information. In step S502, the determination unit 822 determines which external sensor Se of the first sensor Se1 and the second sensor Se2 is used to detect the vehicle 10 by using the step information and the database Db. In step S503, the determination unit 822 transmits the determination result to the calculation device 6 d.

[0128] like Fig.23 As shown in FIG. 1 , in step S504, the sensor information acquisition unit 621d of the calculation device 6d specifies the external sensor Se of the vehicle 10 as the calculation target in the detection range RG1, RG2 among the various external sensors Se determined by the determination unit 822 of the information processing device 8. Steps S505 to S513 are respectively Figure 6 Steps S102 to S110 shown in FIG. 1 are the same.

[0129] According to the fifth embodiment, various types of external sensors Se are installed in each of the areas Ar14, Ar24 in the plant Fc4. The information processing device 8 is capable of determining which type of external sensor Se among the various types of external sensors Se is used to detect the vehicle 10 by using at least one of the state information and the step information, and transmits the determination result to the calculation device 6d. As a result, the calculation device 6d is capable of calculating the position and orientation of the vehicle 10 by using the sensor information output from the external sensor Se of the type determined by the information processing device 8. That is, the calculation system 5d can select the type of external sensor Se corresponding to the state St1, St2 of the vehicle 10, and calculate the position and orientation of the vehicle 10 by using the sensor information output from the selected external sensor Se. F. Sixth Embodiment

[0130] Fig.24 is a diagram showing the configuration of the travel system 1e in the sixth embodiment. In this embodiment, the travel system 1e includes one or more vehicles 10e on which a vehicle control device 15e having two functions of a computing device 6 and a remote control device 7 is mounted, and a first sensor Se1 and a second sensor Se2 as external sensors Se. In this embodiment, the vehicle control device 15e calculates the position and orientation of the vehicle 10e, determines the target position to which the vehicle 10e should go next, and generates a travel control signal for causing the vehicle 10e to travel toward the determined target position. Unless otherwise described, the other configurations of the travel system 1e are the same as those in the first embodiment.

[0131] The vehicle control device 15e includes an input and output interface 159, a storage unit 157e, and a CPU 150e. The storage unit 157e of the vehicle control device 15e stores various kinds of information, including various programs P15e for controlling the operation of the vehicle control device 15e, a first detection model Md1 as a detection model Md, and an ideal path Ip. The CPU 150e of the vehicle control device 15e operates as a sensor information acquisition unit 151, a calculation unit 152, a signal generation unit 153, and an operation controller 154 by loading various programs stored in the storage unit 157e. The sensor information acquisition unit 151 acquires sensor information output from an external sensor Se of the vehicle 10e included in the detection range RG1, RG2 as a calculation target. The sensor information acquisition unit 151 of the vehicle control device 15e has the same function as the sensor information acquisition unit 621 of the calculation device 6 described in the first embodiment. The calculation unit 152 generates position direction information by using the sensor information. The calculation unit 152 has the same function as the calculation unit 622 of the calculation device 6 described in the first embodiment. The signal generating unit 153 determines the target position to which the vehicle 10e should go next by using the position direction information and the ideal path Ip. Then, the signal generating unit 153 generates a travel control signal for causing the vehicle 10e to travel toward the determined target position. The signal generating unit 153 has the same function as the remote controller 721 of the remote control device 7 described in the first embodiment. The vehicle control device 15e repeatedly performs the acquisition of the position direction information of the vehicle 10e, the determination of the target position, the generation of the travel control signal, and the control of the actuator in a predetermined cycle.

[0132] Fig.25 is a flowchart showing the flow of processing in the sixth embodiment. Fig.25 The processing flow shown in , for example, is repeatedly executed at predetermined time intervals starting from the time when the vehicle 10e starts to travel by unmanned driving.

[0133] In step S601, the sensor information acquisition unit 151 of the vehicle control device 15e installed on the vehicle 10e specifies the external sensor Se of the vehicle 10e as the calculation target included in the detection range RG1, RG2 among the external sensors Se installed in the factory Fc1. In step S602, the sensor information acquisition unit 151 transmits a request signal for acquiring sensor information to the specified external sensor Se. In step S603, the external sensor Se that has received the request signal transmits the sensor information to the vehicle 10e. In the case where the sensor information acquisition unit 621 of the vehicle control device 15e acquires the measurement point group data, in step S604, the calculation unit 152 calculates the position of the vehicle 10e represented by the measurement point group data, etc., by using the measurement point group data to generate position direction information. In the case where the sensor information acquisition unit 621 acquires the original image Im1, in step S605, the calculation unit 152 calculates the position of the vehicle 10e represented by the captured images Im1 to Im4, etc., by using the first detection model Md1 trained by performing the learning process and the captured images Im1 to Im4, and generates position direction information. In step S606, the signal generation unit 153 determines the target position to which the vehicle 10e should go next by using the position direction information and the ideal path Ip. In step S607, the signal generation unit 153 generates a travel control signal for causing the vehicle 10e to travel toward the determined target position. In step S608, the operation controller 154 controls the actuator by using the travel control signal so that the vehicle 10e travels at the acceleration and steering angle indicated by the travel control signal.

[0134] According to the sixth embodiment, the travel system 1 e can cause the vehicle 10 e to travel by autonomous control without remotely controlling the vehicle 10 e via the remote control device 7 . G. Other Embodiments G-1. Other Example 1

[0135] At least one of the computing devices 6, 6a to 6d, the remote control device 7, and the information processing device 8 may be configured as one piece. In addition, each unit of the computing devices 6, 6a to 6d, the remote control device 7, and the information processing device 8 may be implemented by cloud computing configured by one or more computers. In addition, at least a part of the functions of the computing devices 6, 6a to 6d, the remote control device 7, and the information processing device 8 may be a single function of the vehicle control device 15, 15e, or may also be a single function of the external sensor Se. In such a form, the configuration of the computing devices 6, 6a to 6d, the remote control device 7, and the information processing device 8 may be changed as appropriate. G-2. Other Example 2

[0136] The calculation units 152, 622, 622a to 622c may perform preprocessing to calculate at least one of the position and orientation of the vehicle 10, 10e represented by the measurement point group data by using the measurement point group data. That is, the first sensor Se1 that acquires the first sensor information may be a LiDAR Li. In addition, the calculation units 152, 622, 622a to 622c may calculate at least one of the position and orientation of the vehicle 10, 10e represented by the captured images Im1 to Im4 by using the captured images Im1 to Im4 without performing preprocessing. That is, the second sensor Se2 that acquires the second sensor information may be a camera Cm. In such a form, the types of the first sensor Se1 and the second sensor Se2 may be changed as appropriate in consideration of the states St1 to St3 classified depending on the appearance of the vehicle 10, 10e, the prices of the LiDAR Li and the camera Cm, the load required for preprocessing, and the like. G-3. Other Example 3

[0137] The types of the first sensor Se1, the second sensor Se2, and the third sensor Se3 are not limited to the LiDAR Li and the visible light camera Cm. For example, any one of the first sensor Se1 and the second sensor Se2 may be a thermal imaging camera as an external sensor Se. A thermal imaging camera is a camera including an imaging element and an optical system having sensitivity to infrared rays. The thermal imaging camera detects the surface temperature of an object existing in a detection range including the vehicle 10 to output a thermal imaging image indicating the heat distribution in the detection range including the vehicle 10 as sensor information. In such a form, the calculation units 152, 622, 622a to 622c may calculate at least one of the position and orientation of the vehicle 10, 10e by using the sensor information output from various types of external sensors Se other than the LiDAR Li and the visible light camera Cm. In the case where the first sensor Se1 is a thermal imaging camera, the calculation units 152, 622, 622a to 622c perform matching between the thermal imaging image and the reference heat distribution data to calculate the position of the vehicle 10, 10e represented by the thermal imaging image, etc. The reference thermal distribution data is, for example, three-dimensional CAD data of the vehicle 10, 10e. In such a form, the calculation units 152, 622, 622a to 622c can calculate at least one of the position and orientation of the vehicle 10, 10e represented by the thermal imaging image by using the thermal imaging image without performing preprocessing. In the case where the second sensor Se2 is a thermal imaging camera, the calculation units 152, 622 perform the following processing, for example. In this case, the calculation units 152, 622, 622a to 622c obtain shape data Da by inputting the thermal imaging image into a machine learning model, which detects the appearance shape of the vehicle 10, 10e from the thermal imaging image when the thermal imaging image is input to output shape data Da. Then, the calculation units 152, 622 calculate the position of the vehicle 10, 10e represented by the thermal imaging image, etc. by transforming the image coordinate point P3 into the vehicle coordinate point Pv using the shape data Da. In such a form, the computing unit 152 , 622 may compute at least one of the position and orientation of the vehicle 10 , 10 e represented by the thermal imaging image by using a machine learning model trained by performing a learning process as preprocessing and the thermal imaging image. G-4. Other Example 4

[0138] The vehicle 10, 10e may be classified into a plurality of states depending on the difference in the exterior color of the vehicle 10 before and after painting. In this form, the vehicle 10, 10e may be classified into a plurality of states depending on the exterior of the vehicle 10, 10e determined by at least one of the exterior shape and exterior color of the vehicle 10, 10e. G-5. Other Embodiment 5

[0139] In each of the first to fifth embodiments, the computing devices 6, 6a to 6d and the remote control device 7 perform processing from acquisition of position information of the vehicle 10 to generation of a travel control signal. At the same time, the vehicle 10 may perform at least a portion of the processing from acquisition of position information of the vehicle 10 to generation of a travel control signal. For example, the following forms (1) to (3) may be used.

[0140] (1) The remote control device 7 can acquire the position information of the vehicle 10, determine the target position to which the vehicle 10 should go next, and generate a path from the current position of the vehicle 10 indicated by the acquired position information to the target position. The remote control device 7 can generate a path to the target position between the current position and the destination, or can also generate a path to the destination. The remote control device 7 can transmit the generated path to the vehicle 10. The vehicle 10 can generate a travel control signal that causes the vehicle 10 to travel on the path received from the remote control device 7, and control the actuator by using the generated travel control signal.

[0141] (2) The remote control device 7 can acquire the position information of the vehicle 10 and transmit the acquired position information to the vehicle 10, 10e. The vehicle 10 can determine the target position to which the vehicle 10 should go next, generate a path from the current position of the vehicle 10 indicated by the received position information to the target position, generate a travel control signal for causing the vehicle 10 to travel on the generated path, and control the actuator by using the generated travel control signal.

[0142] (3) In the above forms (1) and (2), an internal sensor may be installed on the vehicle 10, and the detection results output from the internal sensor may be used for at least one of the generation of the path and the generation of the driving control signal. The internal sensor may include, for example, a vehicle-mounted camera, a vehicle-mounted LiDAR, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, a gyroscope sensor, etc. For example, in the above form (1), the remote control device 7 may obtain the detection results of the internal sensor, and reflect the detection results of the internal sensor in the path when generating the path. In the above form (1), the vehicle 10 may obtain the detection results of the internal sensor, and reflect the detection results of the internal sensor in the driving control signal when generating the driving control signal. In the above form (2), the vehicle 10 may obtain the detection results of the internal sensor, and reflect the detection results of the internal sensor in the path when generating the path. In the above form (2), the vehicle 10 may obtain the detection results of the internal sensor, and reflect the detection results of the internal sensor in the path when generating the path.

[0143] (4) In the sixth embodiment, the vehicle 10e may be equipped with an internal sensor, and the detection result outputted from the internal sensor may be used in at least one of the generation of the route and the generation of the travel control signal. For example, the vehicle 10e may obtain the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating the route. For example, the vehicle 10e may obtain the detection result of the internal sensor, and reflect the detection result of the internal sensor in the travel control signal when generating the travel control signal.

[0144] (5) In the sixth embodiment, the vehicle 10e acquires the position information of the vehicle 10e by using the detection result of the external sensor Se. In this case, the vehicle 10e may be equipped with an internal sensor, and the vehicle 10e may acquire the position information by using the detection result of the internal sensor, determine the target position that the vehicle 10e should go to next, generate a path from the current position of the vehicle 10e represented by the acquired position information to the target position, generate a travel control signal for causing the vehicle 10e to travel on the generated path, and control the actuator by using the generated travel control signal. In this case, the vehicle 10e can travel without using the detection result of the external sensor Se at all. In addition, the vehicle 10e may acquire the target arrival time or traffic congestion information from outside the vehicle 10e, and reflect the target arrival time or traffic congestion information in at least one of the path and the travel control signal.

[0145] (F6) In each of the first to fifth embodiments, the remote control device 7 automatically generates a travel control signal to be transmitted to the vehicle 10. At this time, the remote control device 7 can generate the travel control signal to be transmitted to the vehicle 10 in response to the operation of an operator located outside the vehicle 10. For example, the operator can operate an operating device including a display that displays an image output from the external sensor Se, a steering wheel, an accelerator pedal, and a brake pedal for remotely operating the vehicle 10, and a communication device that communicates with the remote control device 7 via wired communication or wireless communication, and the remote control device 7 can generate a travel control signal corresponding to the operation applied to the operating device. G-6. Other Example 6

[0146] In each of the above-mentioned embodiments, the vehicle 10, 10e may have a configuration capable of moving by unmanned driving, and, for example, may have the form of a platform having the configuration described below. Specifically, in order to exhibit the three functions of "driving", "turning" and "stopping" by unmanned driving, the vehicle 10, 10e may include at least a vehicle control device 15, 15e, a drive device 11, a steering device 12 and a braking device 13. In the case where the vehicle 10, 10e obtains information from the outside for unmanned driving, the vehicle 10, 10e may also include a vehicle communication device 14. That is, the vehicle 10, 10e capable of moving by unmanned driving does not need to be equipped with at least a part of the internal components such as the driver's seat and the dashboard, nor does it need to be equipped with at least a part of the external components such as the bumper and the rearview mirror, and it does not need to be equipped with a body shell 18. In this case, the remaining parts (such as the body shell 18) can be assembled to the vehicle 10, 10e when the vehicle 10, 10e is shipped from the factory Fc1 to Fc4, or the remaining parts (such as the body shell 18) can be assembled to the vehicle 10, 10e after the vehicle 10, 10e is shipped from the factory Fc1 to Fc4 without being assembled thereon. Each part can be assembled from any direction such as the upper side, the lower side, the front side, the rear side, the right side or the left side of the vehicle 10, 10e, and can be assembled from the same direction or different directions. In addition, the position determination for the vehicle 10p having a platform form is the same as that of the vehicle 10 in the first embodiment. G-7. Other Examples 7

[0147] Vehicles 10 and 10e can be manufactured by combining multiple modules. A module refers to a unit composed of multiple components assembled depending on the part or function of the vehicle 10 and 10e. For example, the platform of the vehicle 10 and 10e can be manufactured by combining a front module constituting the front part of the platform, a center module constituting the center part of the platform, and a rear module constituting the rear part of the platform. In addition, the number of modules constituting the platform is not limited to 3, and can be less than 2 or more than 4. In addition to or instead of the components constituting the platform, the components constituting the parts different from the platform in the vehicle 10 and 10e can be modularized. In addition, various modules can include any external components (such as a bumper or grille) or any internal components (such as a seat 19 or a console). In addition, in addition to the vehicle 10 and 10e, any aspect of the mobile body can also be manufactured by combining modules. Such a module can be manufactured, for example, by joining the components via welding or fasteners, or can be manufactured by casting at least a part of the components constituting the module to be integrally molded into one component. The molding method of integrally molding a component, especially a relatively large component, is also called giga casting or mega casting. For example, the front module, the center module and the rear module can be manufactured by using giga casting. G-8. Other Example 8

[0148] The transportation of the vehicle 10, 10e by using the unmanned driving to make the vehicle 10, 10e travel is also referred to as "autonomous transportation". The configuration for realizing the autonomous transportation is also referred to as "vehicle remote-controlled autonomous driving transportation system". The production method of producing the vehicle 10, 10e by using the autonomous transportation is also referred to as "autonomous production". In the autonomous production, for example, at least a part of the transportation of the vehicle 10, 10e is realized by autonomous transportation in the factories Fc1 to Fc4 in which the vehicle 10, 10e is manufactured.

[0149] The present disclosure is not limited to the above-described embodiments, and can be implemented in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features of each form described in the Summary of the Invention section may be appropriately replaced or combined to solve some or all of the above-described purposes, or to achieve some or all of the above-described effects. Where technical features are not described as necessary in this specification, these features may be deleted as appropriate.

Claims

1. A computing device, wherein: A moving body is configured to move by unmanned driving in a factory that performs a plurality of manufacturing steps to manufacture the moving body, and the moving body is classified into a plurality of states by having different appearances depending on the manufacturing steps, characterized in that the calculation device includes: a sensor information acquisition unit configured to acquire first sensor information output from the first sensor by detecting the mobile body classified into a first state among the plurality of states from outside the mobile body via the first sensor and indicating the mobile body classified into the first state, and acquire second sensor information output from the second sensor by detecting the mobile body classified into a second state different from the first state among the plurality of states from the outside via a second sensor of a different kind from the first sensor and indicating the mobile body classified into the second state; and A calculation unit configured to calculate at least one of the position and orientation of the mobile body by using the first sensor information without performing preprocessing, and to perform the preprocessing to calculate at least one of the position and orientation of the mobile body by using the second sensor information.

2. The computing device according to claim 1, wherein: The first sensor is any one of a camera and light detection and ranging; and The second sensor is the other of a camera and light detection and ranging.

3. A computing system, wherein: A moving body is configured to move by unmanned driving in a factory that performs a plurality of manufacturing steps to manufacture the moving body, and the moving body is classified into a plurality of states by having different appearances depending on the manufacturing steps, characterized in that the computing system includes: a first sensor configured to output first sensor information indicating the moving body classified into a first state among the plurality of states by detecting the moving body classified into the first state from outside the moving body; a second sensor configured to output second sensor information indicating the moving body classified into a second state different from the first state among the plurality of states by detecting the moving body classified into the second state from the outside, the second sensor being of a different type from the first sensor; and A calculation unit configured to calculate at least one of the position and orientation of the mobile body by using the first sensor information without performing preprocessing, and to perform the preprocessing to calculate at least one of the position and orientation of the mobile body by using the second sensor information.

4. The computing system according to claim 3, characterized in that: Also includes: a determination information acquisition unit configured to acquire at least one of state information indicating which state of the plurality of states the state of the moving body is and step information indicating which manufacturing step of the plurality of manufacturing steps is being performed on the moving body; as well as a determination unit configured to determine which of the first sensor and the second sensor is used for detecting the moving body for calculating at least one of a position and an orientation of the moving body by using at least one of the state information and the step information; Wherein, the calculation unit is configured to, when the determination unit determines that the mobile body is detected by using the first sensor, calculate at least one of the position and orientation of the mobile body by using the first sensor information without performing preprocessing, and when the determination unit determines that the mobile body is detected by using the second sensor, perform the preprocessing to calculate at least one of the position and orientation of the mobile body by using the second sensor information.

5. The computing system according to claim 3, characterized in that: Also includes: a determination information acquisition unit configured to acquire at least one of state information indicating which state of the plurality of states the state of the moving body is and step information indicating which manufacturing step of the plurality of manufacturing steps is being performed on the moving body; as well as a determination unit configured to determine which of the first sensor and the second sensor is used for detecting the moving body for calculating at least one of a position and an orientation of the moving body by using at least one of the state information and the step information; The calculation unit is configured to calculate at least one of the position and orientation of the moving body by using the first sensor information or the second sensor information output from the first sensor or the second sensor determined by the determination unit to be used for detecting the moving body.

6. The computing system according to any one of claims 3 to 5, characterized in that: The first sensor is one of a camera and light detection and ranging; and The second sensor is the other of a camera and light detection and ranging.

7. The computing system according to claim 6, characterized in that: The computing unit is configured as detecting the moving object from a captured image output by the camera, and outputting shape data indicating an outer shape of the moving object; calculating the coordinates of a positioning point preset for the moving object in an image coordinate system by using the shape data; as well as At least one of the position and orientation of the moving object represented by the captured image is calculated by transforming the calculated coordinates into coordinates in a global coordinate system.

8. The computing system according to claim 6 or 7, characterized in that: The calculation unit is configured to perform matching between the measurement point group data output from the light detection and ranging and the reference point group data prepared in advance to calculate at least one of the position and orientation of the moving body represented by the measurement point group data.

9. The computing system according to any one of claims 3 to 8, characterized in that: The preprocessing includes a learning process for training a machine learning model.

10. The computing system according to claim 9, characterized in that The computing unit is configured as acquiring the shape data by inputting the second sensor information into the machine learning model that is trained by executing the learning process and is trained to output shape data indicating the appearance shape of the moving body when the second sensor information is input, and At least one of the position and orientation of the moving object is calculated by using the shape data.

11. The computing system according to claim 10, characterized in that: The machine learning model is prepared for each of the plurality of states of the mobile body; Pre-training a plurality of machine learning models prepared for the plurality of states respectively to output the shape data by inputting a plurality of training images including M first training images each including the moving body classified as a first state among the plurality of states and N second training images each including the moving body classified as the remaining other states, where M is an integer greater than 2 and N is an integer greater than 0 and less than M; and the calculation unit is configured to acquire the shape data by inputting the second sensor information into a machine learning model selected depending on the state of the moving body among the plurality of machine learning models prepared respectively for the plurality of states, calculating the coordinates of a positioning point preset for the moving object in an image coordinate system by using the shape data; as well as At least one of the position and orientation of the moving object represented by the second sensor information is calculated by transforming the calculated coordinates into coordinates in a global coordinate system.

12. The computing system according to claim 10, characterized in that: The machine learning model is pre-trained to output the shape data by inputting a plurality of training images containing the moving body and a correct answer label associated with each of the plurality of training images and indicating a state of the moving body contained in the training image; and The calculation unit is configured to acquire the shape data by inputting the second sensor information and the state of the moving body into the machine learning model, calculating the coordinates of a predetermined positioning point preset for the moving object in an image coordinate system by using the shape data; as well as At least one of the position and orientation of the moving object represented by the second sensor information is calculated by transforming the calculated coordinates into coordinates in a global coordinate system.

13. An information processing device for moving a moving object, wherein: The mobile body is configured to move by unmanned driving in a factory that performs a plurality of manufacturing steps to manufacture the mobile body, and the mobile body is classified into a plurality of states by having different appearances depending on the manufacturing steps, characterized in that the information processing device includes: a determination information acquisition unit configured to acquire at least one of state information indicating which state of the plurality of states the state of the moving body is and step information indicating which manufacturing step of the plurality of manufacturing steps is being performed on the moving body; and A determination unit is configured to determine which of a first sensor and a second sensor of a different type from the first sensor is used to detect the moving body for calculating at least one of a position and an orientation of the moving body by using at least one of the state information and the step information.

14. A factory for manufacturing a moving body, wherein a plurality of manufacturing steps are performed in the factory to manufacture the moving body, the moving body is configured to move in the factory by unmanned driving, and the moving body is classified into a plurality of states by having different appearances according to the manufacturing steps, characterized in that The factory includes: a first sensor installed in a first area where the moving body moves that is classified into a first state among the plurality of states; and a second sensor installed in a second area where the moving body moves and is classified into a second state different from the first state among the plurality of states, wherein: The first sensor is any one of a camera and light detection and ranging; In the case where the first sensor is the camera, the second sensor is the light detection and ranging; and In the case where the first sensor is the light detection and ranging, the second sensor is the camera.

15. The plant according to claim 14, characterized in that: the first sensor is configured to output first sensor information indicating the moving body classified into the first state by detecting the moving body classified into the first state from outside the moving body; the second sensor is configured to output the second sensor information indicating the moving body classified into the second state by detecting the moving body classified into the second state from the outside; The first sensor information is used to calculate at least one of a position and an orientation of the mobile object represented by the first sensor information; and The second sensor information is used to calculate at least one of a position and an orientation of the moving object represented by the second sensor information.

16. A method for calculating at least one of a position and an orientation of a moving object, wherein: The moving body is configured to move by unmanned driving in a factory that performs a plurality of manufacturing steps to manufacture the moving body, and the moving body is classified into a plurality of states by having different appearances depending on the manufacturing steps, characterized in that the calculation method includes: a sensor information acquisition step of acquiring first sensor information output from the first sensor by detecting the mobile body classified into a first state among the plurality of states from outside the mobile body via the first sensor and indicating the mobile body classified into the first state, and acquiring second sensor information output from the second sensor by detecting the mobile body classified into a second state different from the first state among the plurality of states from the outside via a second sensor different from the first sensor and indicating the mobile body classified into the second state; and A calculation step of calculating at least one of the position and orientation of the mobile body by using the first sensor information without performing preprocessing, and performing the preprocessing to calculate at least one of the position and orientation of the mobile body by using the second sensor information.

17. The calculation method according to claim 16, characterized in that: The first sensor information is information output from one of a camera and light detection and ranging by detecting the moving object classified as the first state from the outside via the camera and the light detection and ranging; and The second sensor information is information output from the other of the camera and the light detection and ranging by detecting the moving object classified into the second state from the outside via the other of the camera and the light detection and ranging.

18. The calculation method according to claim 16, characterized in that: Also includes: determining an information acquisition step of acquiring at least one of state information indicating which of the plurality of states the state of the moving body is and step information indicating which of the plurality of manufacturing steps is being performed on the moving body; as well as a determining step of determining which of the first sensor and the second sensor is used for detecting the moving object, for calculating at least one of a position and an orientation of the moving object, by using at least one of the state information and the step information; In the calculation step, when it is determined in the determination step that the mobile body is detected by using the first sensor, at least one of the position and orientation of the mobile body is calculated by using the first sensor information without performing the preprocessing, and when it is determined in the determination step that the mobile body is detected by using the second sensor, at least one of the position and orientation of the mobile body is calculated by using the second sensor information using the preprocessing.

19. The calculation method according to claim 18, characterized in that: The first sensor is one of a camera and light detection and ranging; and The second sensor is the other of a camera and light detection and ranging.

Images