System, device, and method

Abstract

To precisely estimate at least one of the location and orientation of a moving object.SOLUTION: A system comprises: a sensor that acquires appearance information containing the appearance of a moving object movable by unmanned driving; a step information acquisition unit that acquires step information about progress of a step of manufacturing the moving object; a comparison information acquisition unit that acquires comparison information representing the appearance of the moving object responsive to the step information; and an estimation unit that estimates at least one of the location and orientation of the moving object by making a comparison between the comparison information and the appearance information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a system, apparatus, and method for estimating at least one of the position and orientation of a moving object.

Background Art

[0002] In the manufacturing process of vehicles, a technology for driving a vehicle by remote control is known (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When moving a moving object such as a vehicle by remote control, a process of estimating the position and orientation of the moving object is executed. The position and orientation of the moving object can be estimated by comparing the appearance information of the moving object acquired using a camera, LiDAR, etc. with comparison information prepared in advance. However, since the appearance of the moving object changes as the manufacturing process progresses, there is a possibility that correct estimation results cannot be obtained when estimating the position and orientation of the moving object during manufacturing.

Means for Solving the Problems

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to a first aspect of the present disclosure, a system is provided. The system includes a sensor that acquires appearance information including the appearance of a moving body movable by autonomous driving, a process information acquisition unit that acquires process information regarding the progress of the manufacturing process of the moving body, a comparison information acquisition unit that acquires comparison information representing the appearance of the moving body according to the process information, and an estimation unit that estimates at least one of the position and orientation of the moving body by comparing the comparison information and the appearance information. According to the system of this aspect, at least one of the position and orientation of the moving body can be correctly estimated. (2) In the system of the above aspect, the sensor repeatedly acquires the appearance information, the process information acquisition unit repeatedly acquires the process information, and when the content of the process information newly acquired by the process information acquisition unit is the same as the content at the previous acquisition, the comparison information acquisition unit does not acquire the comparison information according to the newly acquired process information, and the estimation unit may compare the same comparison information as at the previous comparison with the appearance information. According to the system of this aspect, when the content of the process information newly acquired by the process information acquisition unit is the same as the content at the previous acquisition, the comparison information acquisition unit does not acquire the comparison information according to the newly acquired process information, so that the processing load for the comparison information acquisition unit to acquire the same comparison information can be reduced. (3) The system of the above aspect may further include a defect detection unit that detects a defective mounting of a component on the moving body using the degree of coincidence between the appearance of the moving body represented by the comparison information and the appearance of the moving body included in the appearance information. According to the system of this aspect, the defective mounting of a component on the moving body can be detected by the defect detection unit. (4) The system of the above aspect may further include a defect countermeasure unit that executes at least one of a process of stopping the movement of the moving body and a process of notifying the occurrence of the defective mounting when the defective mounting is detected by the defect detection unit. According to the system of this aspect, when a defective mounting of a component occurs in the moving body, it is possible to take countermeasures against the defective mounting. (5) The system according to the above aspect further includes a database in which the process information and the comparison information are associated, and the comparison information acquisition unit may acquire the comparison information associated with the process information acquired by the process information acquisition unit in the database. According to the system of this aspect, it is possible to acquire comparison information suitable for correctly estimating at least one of the position and orientation of the moving body by using a database. (6) The system according to the above aspect further includes a process management device that manages the manufacture of the moving body, and the process information acquisition unit may acquire the process information from the process management device. According to the system of this aspect, the process information can be easily acquired. (7) The system according to the above aspect further includes a plurality of the sensors and a database in which the identification information of each of the plurality of sensors is associated with the comparison information. The process information acquisition unit acquires, as the process information, the identification information of the sensor that has acquired the appearance information, and the comparison information acquisition unit may acquire the comparison information associated with the identification information of the sensor that has acquired the appearance information in the database. According to the system of this aspect, it is possible to acquire comparison information suitable for correctly estimating at least one of the position and orientation of the moving body by using a database. (8) According to the second aspect of the present disclosure, an apparatus is provided. This apparatus includes an appearance information acquisition unit that acquires appearance information including the appearance of a moving body movable by autonomous driving from a sensor, a process information acquisition unit that acquires process information regarding the progress of the manufacturing process of the moving body, a comparison information acquisition unit that acquires comparison information representing the appearance of the moving body according to the process information, and an estimation unit that estimates at least one of the position and orientation of the moving body by comparing the comparison information and the appearance information. According to the apparatus of this aspect, at least one of the position and orientation of the moving body can be correctly estimated. (9) According to a third aspect of the present disclosure, a method is provided. The method includes obtaining appearance information including the appearance of a movable body movable by autonomous driving, obtaining process information regarding the progress of the manufacturing process of the movable body, obtaining comparison information representing the appearance of the movable body according to the process information, and estimating at least one of the position and orientation of the movable body by comparing the comparison information and the appearance information. According to the method of this aspect, at least one of the position and orientation of the movable body can be correctly estimated. The present disclosure can also be realized in various forms other than systems, devices, and methods. For example, it can be realized in the form of a computer program and a recording medium on which the computer program is recorded.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Modes for Carrying Out the Invention

[0008] A. First Embodiment: FIG. 1 is an explanatory diagram showing the configuration of the system 10 in the first embodiment. The system 10 is used to move a moving body by unmanned driving in a factory that manufactures the moving body.

[0009] In the present disclosure, the “moving body” means an object that can move, for example, a vehicle or an electric vertical take-off and landing aircraft (so-called flying car). The vehicle may be a vehicle that travels on wheels or a vehicle that travels on an endless track, and examples thereof include a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, and a construction vehicle. The vehicle includes a battery electric vehicle (BEV), a gasoline vehicle, a hybrid vehicle, and a fuel cell vehicle. When the moving body is other than a vehicle, the expressions “vehicle” and “car” in the present disclosure can be appropriately replaced with “moving body”, and the expression “travel” can be appropriately replaced with “move”.

[0010] “Unmanned driving” means driving without depending on the driving operation of a passenger. The driving operation means an operation related to at least any one of “running”, “turning”, and “stopping” of the vehicle. Unmanned driving is realized by automatic or manual remote control using a device located outside the vehicle, or by autonomous control of the vehicle. A passenger who does not perform a driving operation may be on board a vehicle traveling by unmanned driving. Passengers who do not perform a driving operation include, for example, a person simply sitting on the seat of the vehicle, or a person performing work different from the driving operation, such as assembly, inspection, and operation of switches, while on board the vehicle. Note that driving by the driving operation of a passenger is sometimes called “manned driving”.

[0011] In the present disclosure, “remote control” includes “complete remote control” in which all of the operations of the vehicle are completely determined from outside the vehicle, and “partial remote control” in which a part of the operations of the vehicle is determined from outside the vehicle. Further, “autonomous control” includes “complete autonomous control” in which the vehicle autonomously controls its own operations without receiving any information from a device outside the vehicle, and “partial autonomous control” in which the vehicle autonomously controls its own operations using information received from a device outside the vehicle.

[0012] System 10 includes a remotely controllable vehicle 100, a server device 200 that remotely controls the vehicle 100, an external sensor group 300 installed in the factory, and a process management device 400 that manages the manufacturing of the vehicle 100 in the factory.

[0013] The vehicle 100 includes a vehicle control device 110 for controlling each part of the vehicle 100, an actuator group 120 that drives under the control of the vehicle control device 110, and a communication device 130 for communicating with the server device 200 by wireless communication. The actuator group 120 includes at least one actuator. In this embodiment, the actuator group 120 includes an actuator of a driving device for accelerating the vehicle 100, an actuator of a steering device for changing the traveling direction of the vehicle 100, and an actuator of a braking device for decelerating the vehicle 100. The driving device includes a battery, a traveling motor driven by the power of the battery, and driving wheels rotated by the traveling motor. The actuator of the driving device includes the traveling motor.

[0014] The vehicle control device 110 is constituted by a computer including a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input / output interface 113 are connected so as to be communicable bidirectionally via the internal bus 114. The actuator group 120 and the communication device 130 are connected to the input / output interface 113.

[0015] The processor 111 functions as a travel control unit 115 by executing a computer program PG1 pre-stored in the memory 112. The travel control unit 115 controls the actuator group 120. When a passenger is on board the vehicle 100, the travel control unit 115 can control the actuator group 120 according to the operation of the passenger to drive the vehicle 100. Regardless of whether a passenger is on board the vehicle 100 or not, the travel control unit 115 can control the actuator group 120 according to the travel control signal received from the server device 200 to drive the vehicle 100.

[0016] The server device 200 is constituted by a computer including a processor 201, a memory 202, an input / output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input / output interface 203 are connected so as to be communicable bidirectionally via the internal bus 204. A communication device 205 for communicating with the vehicle 100 by wireless communication is connected to the input / output interface 203. In the present embodiment, the communication device 205 can communicate with the external sensor group 300 and the process management device 400 by wired communication or wireless communication.

[0017] By executing the computer program PG2 stored in advance in the memory 202, the processor 201 functions as an appearance information acquisition unit 210, a process information acquisition unit 220, a comparison information acquisition unit 230, an estimation unit 240, and a remote control unit 250. The appearance information acquisition unit 210 acquires appearance information from the external sensor group 300. The appearance information includes information regarding the appearance of the vehicle 100 detected by the external sensor group 300. The process information acquisition unit 220 acquires process information regarding the progress of the manufacturing process of the vehicle 100, in other words, to which stage the manufacturing process of the vehicle 100 has progressed. In the present embodiment, the process information acquisition unit 220 acquires process information from the process management device 400. The comparison information acquisition unit 230 acquires comparison information CD representing the appearance of the vehicle 100 according to the process information. In the present embodiment, a plurality of comparison information CDs are stored in advance in the memory 202. The comparison information acquisition unit 230 acquires the comparison information CD according to the process information from among the plurality of comparison information CDs stored in the memory 202. Each comparison information CD is created, for example, using the CAD data of the vehicle 100. The estimation unit 240 estimates the position and orientation of the vehicle 100 by comparing the appearance information with the comparison information CD. The position of the vehicle 100 in the factory is represented by the position coordinates of X, Y, and Z in the global coordinate system in the factory. The orientation of the vehicle 100 in the factory is represented by the orientation with respect to the global coordinate system. The remote control unit 250 remotely controls the vehicle 100 based on the position and orientation of the vehicle 100 estimated by the estimation unit 240.

[0018] The external sensor group 300 is composed of at least one external sensor. The external sensor is a sensor installed outside the vehicle 100. In the present embodiment, the external sensor is installed in the factory. The external sensor is a distance measuring device. More specifically, the external sensor is a LiDAR. The external sensor is provided with a communication device (not shown) and can communicate with the server device 200 by wired communication or wireless communication.

[0019] The project management device 400 executes the overall management of the manufacturing process of the vehicle 100 in the factory. The project management device 400 is composed of at least one computer. The project management device 400 is equipped with a communication device (not shown) and can communicate with the server device 200 and various factory facilities through wired communication or wireless communication. The project management device 400 collects information from various factory facilities and generates process information.

[0020] FIG. 2 is an explanatory diagram showing how the vehicle 100 moves by remote control in the factory KJ. In FIG. 2, the vehicle 100 is schematically represented. FIG. 2 shows the global coordinate system GA of the factory KJ. In this embodiment, the factory KJ includes a first location PL1, a second location PL2, and a third location PL3. The first location PL1, the second location PL2, and the third location PL3 are connected by a driving route SR on which the vehicle 100 can travel. A plurality of external sensors 301 are installed around the driving route SR. Each external sensor 301 is included in the external sensor group 300.

[0021] In this embodiment, the first location PL1, the second location PL2, and the third location PL3 are locations where the vehicle 100 is assembled. When the vehicle 100 is assembled at the first location PL1, at least the vehicle control device 110, the actuator group 120, and the communication device 130 are installed. Therefore, when the vehicle 100 is assembled at the first location PL1, it is in a state where it can travel by remote control, in other words, it can perform the three functions of "running", "turning", and "stopping" by remote control.

[0022] The vehicle 100 assembled at the first location PL1 moves from the first location PL1 to the second location PL2 by being remotely controlled by the server device 200. At the second location PL2, some body parts, seats, etc. are attached to the vehicle 100. When some body parts, seats, etc. are attached to the vehicle 100, the appearance of the vehicle 100 changes. The vehicle 100 assembled at the second location PL2 moves from the second location PL2 to the third location PL3 by being remotely controlled by the server device 200. At the third location PL3, the remaining parts are attached to the vehicle 100. When the remaining parts are attached to the vehicle 100, the appearance of the vehicle 100 changes. Thereafter, the vehicle 100 is shipped from the factory KJ after going through an inspection process and the like. In the following description, the process of assembling the vehicle 100 carried out at the first location PL1 is referred to as the first assembly process, the process of assembling the vehicle 100 carried out at the second location PL2 is referred to as the second assembly process, and the process of assembling the vehicle 100 carried out at the third location PL3 is referred to as the third assembly process. Note that after the vehicle 100 is shipped from the factory KJ in a state where parts other than the vehicle control device 110, the actuator group 120, and the communication device 130 are not attached to the vehicle 100, the unattached parts may be attached to the vehicle 100.

[0023] FIG. 3 is a flowchart showing the content of the travel control process. From step S1 to step S4 of the travel control process, it is executed by the processor 201 of the server device 200, and from step S5 to step S6 of the travel control process, it is executed by the processor 111 of the vehicle control device 110. The travel control process is repeatedly executed at a predetermined cycle until the vehicle 100 to be remotely controlled reaches the destination. In the following description, the vehicle 100 to be remotely controlled may be referred to as the target vehicle 100. The travel control process is started, for example, when a predetermined start button is operated by an operator at the factory KJ. Prior to the start of the travel control process, the server device 200 acquires the identification number of the target vehicle 100. For example, a two-dimensional code in which the identification number of the target vehicle 100 is recorded is attached to the target vehicle 100, and when an operator at the factory KJ reads the two-dimensional code attached to the target vehicle 100 with a reading device, the identification number of the target vehicle 100 is transmitted from the reading device to the server device 200.

[0024] In step S1, the server device 200 acquires the vehicle position information of the vehicle 100 using the detection result output from the external sensor 301, which is a sensor located outside the vehicle 100. The vehicle position information is the position information that serves as the basis for generating the travel control signal. In the present embodiment, the vehicle position information includes the position and orientation of the vehicle 100 in the reference coordinate system of the factory KJ. In the present embodiment, the reference coordinate system of the factory KJ is a global coordinate system, and any position within the factory KJ is expressed by the coordinates of X, Y, and Z in the global coordinate system. Details of step S1 will be described later.

[0025] In step S2, the server device 200 determines the target position to which the vehicle 100 should next head. In the present embodiment, the target position is represented by the coordinates of X, Y, and Z in the global coordinate system. In the memory 202 of the server device 200, a reference route RR, which is the route along which the vehicle 100 should travel, is stored in advance. The route is represented by a node indicating the starting point, a node indicating the passing point, a node indicating the destination, and links connecting the respective nodes. The server device 200 determines the target position to which the vehicle 100 should next head using the vehicle position information and the reference route RR. The server device 200 determines the target position on the reference route RR ahead of the current position of the vehicle 100.

[0026] In step S3, the server device 200 generates a driving control signal for driving the vehicle 100 toward the determined target position. In the present embodiment, the driving control signal includes the acceleration and steering angle of the vehicle 100 as parameters. In other embodiments, the driving control signal may include the speed of the vehicle 100 as a parameter instead of, or in addition to, the acceleration of the vehicle 100. The server device 200 calculates the driving speed of the vehicle 100 from the change in the position of the vehicle 100 and compares the calculated driving speed with the target speed. Overall, when the driving speed is lower than the target speed, the server device 200 determines the acceleration so that the vehicle 100 accelerates, and when the driving speed is higher than the target speed, the server device 200 determines the acceleration so that the vehicle 100 decelerates. Further, when the vehicle 100 is located on the reference route RR, the server device 200 determines the steering angle and acceleration so that the vehicle 100 does not deviate from the reference route RR, and when the vehicle 100 is not located on the reference route RR, in other words, when the vehicle 100 has deviated from the reference route RR, the server device 200 determines the steering angle and acceleration so that the vehicle 100 returns to the reference route RR.

[0027] In step S4, the server device 200 transmits the generated driving control signal to the vehicle 100. The server device 200 repeats the acquisition of the position of the vehicle 100, the determination of the target position, the generation of the driving control signal, and the transmission of the driving control signal at a predetermined cycle.

[0028] In step S5, the vehicle control device 110 mounted on the vehicle 100 receives a driving control signal transmitted from the server device 200. In step S6, the vehicle control device 110 controls the actuator group 120 using the received driving control signal, thereby driving the vehicle 100 at the acceleration and steering angle represented by the driving control signal. The vehicle control device 110 repeats the reception of the driving control signal and the control of the actuator group 120 at a predetermined cycle.

[0029] FIG. 4 is a flowchart showing the content of the vehicle position information acquisition process. In the present embodiment, the vehicle position information acquisition process is repeatedly executed by the processor 201 of the server device 200 while the driving control process shown in FIG. 3 is being executed. When the vehicle position information acquisition process is started, first, in step S110, the appearance information acquisition unit 210 acquires appearance information from the external sensor 301. In the present embodiment, the appearance information is three-dimensional point cloud data. The appearance information includes the point cloud data of the target vehicle 100 and the point cloud data other than the target vehicle 100, such as various facilities in the factory KJ and the road surface of the driving road SR.

[0030] In step S120, the process information acquisition unit 220 acquires process information from the process management device 400. The process information represents the progress of the manufacturing process of the current target vehicle 100. The process management device 400 has a database in which the identification numbers of the vehicles 100 being manufactured in the factory KJ and the progress of the manufacturing processes of the respective vehicles 100 are associated. The process information acquisition unit 220 acquires process information regarding the progress of the manufacturing process of the target vehicle 100 from the database of the process management device 400 using the identification number of the target vehicle 100 acquired when starting the driving control process.

[0031] In step S130, the comparison information acquisition unit 230 determines whether the content of the latest process information acquired by the process information acquisition unit 220 is the same as the content of the process information at the time of the previous acquisition. For example, if the progress of the manufacturing process of the target vehicle 100 represented in the latest process information at the time of this acquisition is at a stage after the end of the first assembly process and before the start of the second assembly process, and the progress of the manufacturing process of the target vehicle 100 represented in the process information at the time of the previous acquisition is at a stage after the end of the first assembly process and before the start of the second assembly process, the comparison information acquisition unit 230 determines that the content of the latest process information is the same as the content of the process information at the time of the previous acquisition. If it is not determined in step S130 that the content of the latest process information is the same as the content of the process information at the time of the previous acquisition, the comparison information acquisition unit 230 proceeds to step S140. Note that when the acquisition of the process information of the target vehicle 100 is the first time, the comparison information acquisition unit 230 skips step S130 and proceeds to step S140.

[0032] In step S140, the comparison information acquisition unit 230 acquires a comparison information CD representing the appearance of the vehicle 100 according to the progress of the manufacturing process of the target vehicle 100 from among a plurality of comparison information CDs pre-stored in the memory 202. In the present embodiment, the plurality of comparison information CDs include a first comparison information CD1 and a second comparison information CD2. The first comparison information CD1 represents the appearance of the vehicle 100 from the first assembly process to the second assembly process. The second comparison information CD2 represents the appearance of the vehicle 100 from the second assembly process to the third assembly process. In the following description, when the first comparison information CD1 and the second comparison information CD2 are not particularly distinguished, they are simply referred to as comparison information CD. In the present embodiment, the memory 202 pre-stores a database DB in which the progress of the manufacturing process of the vehicle 100 and the comparison information CD to be used for matching at the progress of the manufacturing process are associated with each other. The comparison information acquisition unit 230 acquires a comparison information CD representing the appearance of the vehicle 100 according to the progress of the manufacturing process of the target vehicle 100 by referring to the database DB. In the present disclosure, the comparison information CD to be used for matching is a comparison information CD suitable for correctly estimating the position and orientation of the vehicle 100 by matching.

[0033] In step S150, the estimation unit 240 estimates the position and orientation of the target vehicle 100 by comparing the appearance information and the comparison information CD. The estimation unit 240 acquires vehicle position information by estimating the position and orientation of the target vehicle 100. In the present embodiment, the appearance information includes point cloud data acquired by the external sensor 301, and the comparison information CD includes point cloud data created from CAD data or the like of the target vehicle 100. The estimation unit 240 executes matching between the point cloud data of the appearance information and the point cloud data of the comparison information CD, detects the point cloud data of the target vehicle 100 from the point cloud data of the appearance information, and estimates the position and orientation of the target vehicle 100 from the matching result. More specifically, the estimation unit 240 detects the point cloud of the target vehicle 100 from the point cloud acquired by the external sensor 301 by executing matching. The estimation unit 240 can grasp the position and orientation of the target vehicle 100 in the local coordinate system of the external sensor 301 from the matching result. Since the external sensor 301 is fixed to the factory KJ, the positional relationship between the local coordinate system of the external sensor 301 and the global coordinate system GA of the factory KJ is known. Therefore, the estimation unit 240 can estimate the position and orientation of the target vehicle 100 in the global coordinate system GA of the factory KJ by executing matching. As a matching method, for example, NDT (Normal Distributions Transform), ICP (Iterative Closest Point), or the like can be used.

[0034] If it is determined in step S130 that the content of the latest process information is the same as the content of the process information at the time of the previous acquisition, the comparison information acquisition unit 230 skips step S140 and proceeds to step S150. That is, when the content of the process information currently acquired by the process information acquisition unit 220 is the same as the content at the time of the previous acquisition, the comparison information acquisition unit 230 does not acquire the comparison information CD corresponding to the currently acquired process information. In this case, the estimation unit 240 compares, in step S150, the same comparison information CD as at the previous comparison with the appearance information currently acquired. After step S150, the processor 201 ends the vehicle position information acquisition process and proceeds to step S2 shown in FIG. 3. Note that the method executed in the vehicle position information acquisition process may be referred to as the vehicle position information method.

[0035] FIG. 5 is an explanatory diagram showing a state in which the comparison information CD corresponding to the progress of the manufacturing process of the vehicle 100 is used. Since the appearance of the vehicle 100 changes as the manufacturing process of the vehicle 100 progresses, if the same comparison information CD is used for estimating the position and orientation of the vehicle 100 while ignoring the progress of the manufacturing process, a deviation may occur between the actual appearance of the vehicle 100 and the appearance of the vehicle 100 represented by the comparison information CD. When a deviation occurs between the actual appearance of the vehicle 100 and the appearance of the vehicle 100 represented by the comparison information CD, it becomes difficult to accurately estimate the position and orientation of the vehicle 100. In the present embodiment, in estimating the position and orientation of the vehicle 100, the comparison information CD representing the appearance of the vehicle 100 according to the progress of the manufacturing process of the vehicle 100 is used. For example, between the first assembly process and the second assembly process, the first comparison information CD1 representing the appearance of the vehicle 100 between the first assembly process and the second assembly process is used, and between the second assembly process and the third assembly process, the second comparison information CD2 representing the appearance of the vehicle 100 between the second assembly process and the third assembly process is used. Therefore, it is possible to suppress the occurrence of a deviation between the actual appearance of the vehicle 100 and the appearance of the vehicle 100 represented by the comparison information CD.

[0036] According to the system 10 in the present embodiment described above, since the divergence between the appearance of the actual vehicle 100 and the appearance of the vehicle 100 represented on the comparison information CD is suppressed, the position and orientation of the vehicle 100 can be accurately estimated. In particular, when the vehicle 100 during manufacturing is caused to travel by remote control, the appearance of the vehicle 100 changes as the manufacturing process of the vehicle 100 to be remotely controlled progresses. In the present embodiment, since the server device 200 switches the comparison information CD used for matching according to the progress of the manufacturing process, it is possible to suppress the situation where the position and orientation of the vehicle 100 to be remotely controlled cannot be accurately estimated.

[0037] Also, in the present embodiment, the process information acquisition unit 220 of the server device 200 acquires process information from the process management device 400. Therefore, the process information can be easily acquired.

[0038] Also, in the present embodiment, when the content of the process information acquired this time is the same as the content at the time of the previous acquisition, the comparison information acquisition unit 230 of the server device 200 does not acquire a new comparison information CD, and when a new comparison information CD is not acquired, the estimation unit 240 performs matching between the comparison information CD acquired previously and the appearance information acquired this time. Therefore, since the trouble of acquiring a new comparison information CD can be eliminated, the processing load for acquiring the comparison information CD can be reduced.

[0039] Also, in the present embodiment, the comparison information acquisition unit 230 of the server device 200 refers to the database DB stored in advance in the memory 202 and selects the comparison information CD used for matching from among a plurality of comparison information CDs. Therefore, it is possible to select a comparison information CD suitable for accurately estimating the position and orientation of the vehicle 100.

[0040] B. Second Embodiment: FIG. 6 is an explanatory diagram showing the configuration of the system 10b in the second embodiment. The system 10b in this embodiment is different from the first embodiment in that it includes a notification device 500, and a defect detection unit 260 and a defect handling unit 270 are provided in the server device 200. For other configurations, they are the same as those in the first embodiment unless otherwise specified.

[0041] In this embodiment, the processor 201 of the server device 200 functions as an appearance information acquisition unit 210, a process information acquisition unit 220, a comparison information acquisition unit 230, an estimation unit 240, and a remote control unit 250 by executing a computer program PG2 stored in advance in the memory 202, and further functions as a defect detection unit 260 and a defect handling unit 270. The defect detection unit 260 detects the occurrence of a defective mounting of parts in the vehicle 100. When the defect handling unit 270 detects a defective mounting by the defect detection unit 260, it executes a process of stopping the running of the vehicle 100 and a process of notifying the notification device 500 that a defective mounting has occurred in the vehicle 100.

[0042] The notification device 500 is a device for notifying the administrator of the system 10b or the worker of the factory KJ that a defective mounting has occurred in the vehicle 100. The notification device 500 is, for example, a warning buzzer provided in the factory KJ or a warning lamp provided in the factory KJ. The notification device 500 may be a tablet terminal carried by the administrator of the system 10 or the worker of the factory. In the following description, the administrator of the system 10b or the worker of the factory KJ is referred to as the administrator or the like. The notification device 500 includes a communication device (not shown) and can communicate with the server device 200 by wired communication or wireless communication.

[0043] FIG. 7 is a flowchart showing the content of the position information acquisition process in the present embodiment. From step S210 to step S250 is the same as from step S110 to step S150 in the position information acquisition process of the first embodiment shown in FIG. 4. After step S250, in step S255, the defect detection unit 260 determines whether or not the degree of coincidence between the appearance of the vehicle 100 included in the appearance information and the appearance of the vehicle 100 included in the comparison information CD is equal to or less than a predetermined value. The shorter the total distance obtained by summing the distances between corresponding points in the point cloud data of the appearance information and the point cloud data of the comparison information CD, the higher the degree of coincidence of the appearance. Therefore, when the total distance of the distances between corresponding points is equal to or greater than a predetermined distance, the defect detection unit 260 determines that the degree of coincidence between the appearance of the vehicle 100 included in the appearance information and the appearance of the vehicle 100 included in the comparison information CD is equal to or less than a predetermined value.

[0044] If it is not determined in step S255 that the degree of coincidence between the appearance of the vehicle 100 included in the appearance information and the appearance of the vehicle 100 included in the comparison information CD is equal to or less than a predetermined value, the server device 200 ends the position information acquisition process and proceeds to step S2 of the travel control process shown in FIG. 3.

[0045] In step S255, when it is determined that the degree of coincidence between the appearance of the vehicle 100 included in the appearance information and the appearance of the vehicle 100 included in the comparison information CD is equal to or less than a predetermined value, in step S270, the defect handling unit 270 stops the running of the vehicle 100, and in step S280, notifies the occurrence of a defective part mounting in the vehicle 100. In the present embodiment, the defect handling unit 270 generates a running control signal for stopping the running of the vehicle 100, and transmits the running control signal to the vehicle 100 via the communication device 205, thereby stopping the running of the vehicle 100. The defect handling unit 270 generates a notification signal for notifying the defective mounting, and transmits the notification signal to the notification device 500 via the communication device 205, thereby notifying the occurrence of the defective mounting. For example, when the notification device 500 is an alarm buzzer, the notification signal is a signal for sounding the alarm buzzer, and when the notification device 500 is an alarm lamp, the notification signal is a signal for lighting the alarm lamp. When the notification device 500 is a tablet terminal, the notification signal is a signal for causing the tablet terminal to display a message notifying that a defective part mounting has occurred in the vehicle 100. Thereafter, the server device 200 ends the position information acquisition process and aborts the running control process shown in FIG. 3.

[0046] According to the system 10b in the present embodiment described above, the defect detection unit 260 can detect that a defective part mounting has occurred in the vehicle 100. Further, in the present embodiment, when the defect detection unit 260 detects the occurrence of a defective part mounting in the vehicle 100, the defect handling unit 270 executes a process of stopping the running of the vehicle 100. Therefore, it is possible to suppress the running of the vehicle 100 from continuing with the defective mounting remaining. Furthermore, in the present embodiment, when the defect detection unit 260 detects the occurrence of a defective part mounting in the vehicle 100, the defect handling unit 270 executes a process of notifying the occurrence of a defective part mounting in the vehicle 100. Therefore, an administrator or the like can recognize the occurrence of the defective mounting at an early stage and take countermeasures.

[0047] C. Third Embodiment: FIG. 8 is an explanatory diagram schematically showing the configuration of the system 10c in the third embodiment. In the third embodiment, it is different from the first embodiment in that the system 10c does not include the server device 200 and the vehicle 100 travels by autonomous control instead of remote control. For other configurations, they are the same as those in the first embodiment unless otherwise specified.

[0048] In this embodiment, the vehicle 100 is configured to be capable of traveling by autonomous control. The vehicle 100 can communicate with the external sensor group 300 and the process management device 400 by wireless communication using the communication device 130. In this embodiment, the processor 111 of the vehicle control device 110 functions as a travel control unit 155c, an appearance information acquisition unit 191, a process information acquisition unit 192, a comparison information acquisition unit 193, and an estimation unit 194 by executing a computer program PG1 stored in advance in the memory 112. In this embodiment, the travel control unit 155c generates a travel control signal by itself and controls the actuator group 120 using the generated travel control signal to make the host vehicle travel. The appearance information acquisition unit 191 acquires the appearance information of the host vehicle from the external sensor group 300. The process information acquisition unit 192 acquires process information regarding the progress of the manufacturing process of the host vehicle. The comparison information acquisition unit 193 acquires comparison information CD representing the appearance of the host vehicle according to the process information. In this embodiment, a plurality of comparison information CDs are stored in advance in the memory 112, and the comparison information acquisition unit 193 acquires the comparison information CD corresponding to the process information from among the plurality of comparison information CDs stored in the memory 112. The estimation unit 194 estimates the position and orientation of the host vehicle by comparing the appearance information with the comparison information CD. A database DB and a reference route RR are stored in advance in the memory 112.

[0049] FIG. 9 is a flowchart showing the details of the travel control process in the third embodiment. In this embodiment, the travel control process is executed by the processor 111 of the vehicle control device 110. In step S11, the vehicle control device 110 acquires the vehicle position information of the host vehicle by executing the vehicle position information acquisition process shown in FIG. 4. Specifically, in step S110, the appearance information acquisition unit 191 acquires appearance information from the external sensor 301. In step S120, the process information acquisition unit 192 acquires process information regarding the progress of the manufacturing process of the host vehicle from the process management device 400. In step S130, the comparison information acquisition unit 193 determines whether the content of the latest process information acquired by the process information acquisition unit 192 is the same as the content of the process information at the time of the previous acquisition. If it is not determined in step S130 that the content of the latest process information is the same as the content of the process information at the time of the previous acquisition, the comparison information acquisition unit 193 proceeds to step S140. If it is determined in step S130 that the content of the latest process information is the same as the content of the process information at the time of the previous acquisition, the comparison information acquisition unit 193 skips step S140 and proceeds to step S150. In step S140, the comparison information acquisition unit 193 acquires a comparison information CD representing the appearance of the vehicle 100 according to the progress of the manufacturing process of the host vehicle from among a plurality of comparison information CDs stored in advance in the memory 112. In step S150, the estimation unit 194 acquires the vehicle position information by estimating the position and orientation of the host vehicle by comparing the appearance information with the comparison information CD.

[0050] In step S21, the travel control unit 155c determines the target position to which the host vehicle should next head. In step S31, the travel control unit 155c generates a travel control signal for causing the host vehicle to travel toward the determined target position. In step S41, the travel control unit 155c controls the actuator group 120 using the generated travel control signal, so that the host vehicle travels according to the parameters represented by the travel control signal. The travel control unit 155c repeats the acquisition of vehicle position information, determination of the target position, generation of the travel control signal, and control of the actuator group 120 at a predetermined cycle.

[0051] According to the system 10c in the present embodiment described above, the vehicle 100 can be made to travel by autonomous control of the vehicle 100 without the server device 200 remotely controlling the vehicle 100.

[0052] D. Other Embodiments: (D1) In each of the above embodiments, the external sensor 301 is a LiDAR, and the estimation units 240 and 194 estimate the position and orientation of the vehicle 100 by point cloud matching using the point cloud data output from the LiDAR. In contrast, the external sensor 301 may be a camera. In this case, the estimation units 240 and 194 may estimate the position and orientation of the vehicle 100 by image matching using the image output from the camera.

[0053] (D2) In each of the above embodiments, the estimation units 240 and 194 estimate the position and orientation of the vehicle 100. In contrast, the estimation units 240 and 194 may estimate either the position or the orientation of the vehicle 100 and not the other. For example, when the vehicle 100 is equipped with a GPS receiver, the position information of the vehicle 100 can be acquired by the GPS receiver, so the estimation units 240 and 194 may estimate the orientation of the vehicle 100 without estimating the position of the vehicle 100. For example, when the reference route RR is a straight line, the estimation units 240 and 194 may estimate the position of the vehicle 100 without estimating the orientation of the vehicle 100.

[0054] (D3) In each of the above embodiments, the process information acquisition units 220 and 192 acquire process information from the process management device 400. In contrast, when the appearance information acquired from the external sensor 301 includes the identification information of the external sensor 301, and a database DB in which the identification information of the external sensor 301 is associated with the progress of the manufacturing process of the vehicle 100 is pre-stored in the memories 202 and 112, the process information acquisition units 220 and 192 may acquire the progress of the manufacturing process of the vehicle 100 by using the identification information of the external sensor 301 from which the appearance information acquisition units 210 and 191 have acquired the appearance information and the database DB. Alternatively, when the vehicle 100 is equipped with a GPS receiver and a database DB in which the position information is associated with the progress of the manufacturing process of the vehicle 100 is pre-stored in the memories 202 and 112, the process information acquisition units 220 and 192 may acquire the progress of the manufacturing process of the vehicle 100 by using the position information acquired by the GPS receiver and the database DB.

[0055] (D4) In each of the above embodiments, a database DB in which the progress of the manufacturing process of the vehicle 100 is associated with the comparison information CD to be used for matching at the progress of the manufacturing process is pre-stored in the memory 202 of the server device 200 and the memory 112 of the vehicle control device 110. The comparison information acquisition units 230 and 193 acquire the comparison information CD representing the appearance of the vehicle 100 corresponding to the progress of the manufacturing process of the target vehicle 100 by referring to the database DB. In contrast, a database DB in which the identification information of the external sensor 301 is associated with the comparison information CD to be used for matching with the appearance information acquired by the external sensor 301 may be pre-stored in the memory 202 of the server device 200 and the memory 112 of the vehicle control device 110. The process information acquisition units 220 and 192 acquire the identification information of the external sensor 301 that has acquired the appearance information as process information, and the comparison information acquisition units 230 and 193 may acquire the comparison information CD corresponding to the external sensor 301 from which the appearance information acquisition units 210 and 191 have acquired the appearance information by referring to the database DB. Also in this case, the comparison information CD suitable for correctly estimating the position and orientation of the vehicle 100 can be selected.

[0056] (D5) In each of the above embodiments, the comparison information acquisition units 230 and 193 determine whether the content of the process information at the current acquisition is the same as that at the previous acquisition in steps S130 and S230 of the vehicle position information acquisition process. If it is determined that they are the same, steps S140 and S240 are skipped, and thus the comparison information CD is not acquired in steps S140 and S240. In contrast, the comparison information acquisition units 230 and 193 may acquire the comparison information CD in steps S140 and S240 without determining whether the content of the process information at the current acquisition is the same as that at the previous acquisition in steps S130 and S230 of the vehicle position information acquisition process.

[0057] (D6) In the third embodiment described above, the defect detection unit 260 and the defect countermeasure unit 270 shown in FIG. 6 may be provided in the vehicle control device 110. In step S11 of the traveling control process shown in FIG. 9, the vehicle control device 110 may execute the vehicle position information acquisition process shown in FIG. 7 instead of the vehicle position information acquisition process shown in FIG. 4. In this case, the defect detection unit 260 can detect that a component mounting defect has occurred in the host vehicle. Further, when the defect detection unit 260 detects the occurrence of a component mounting defect in the host vehicle, the defect countermeasure unit 270 can execute a process of stopping the traveling of the host vehicle.

[0058] (D7) In the first and second embodiments described above, the server device 200 executes the process from the acquisition of vehicle position information to the generation of a traveling control signal. In contrast, at least a part of the process from the acquisition of vehicle position information to the generation of a traveling control signal may be executed by the vehicle 100. For example, the following forms (1) to (3) may be adopted.

[0059] (1) The server device 200 may acquire vehicle position information, determine a target position to which the vehicle 100 should next head, and generate a route from the current position of the vehicle 100 represented in the acquired vehicle position information to the target position. The server device 200 may generate a route to a target position between the current position and the destination, or may generate a route to the destination. The server device 200 may transmit the generated route to the vehicle 100. The vehicle 100 may generate a travel control signal so that the vehicle 100 travels on the route received from the server device 200, and control the actuator group 120 using the generated travel control signal.

[0060] (2) The server device 200 may acquire vehicle position information and transmit the acquired vehicle position information to the vehicle 100. The vehicle 100 may determine a target position to which the vehicle 100 should next head, generate a route from the current position of the vehicle 100 represented in the received vehicle position information to the target position, generate a travel control signal so that the vehicle 100 travels on the generated route, and control the actuator group 120 using the generated travel control signal.

[0061] (3) In the forms (1) and (2) above, an internal sensor is mounted on the vehicle 100, and the detection result output from the internal sensor may be used for at least one of the generation of the route and the generation of the driving control signal. The internal sensor is a sensor mounted on the vehicle 100. The internal sensor may include, for example, a sensor for detecting the motion state of the vehicle 100, a sensor for detecting the operating state of each part of the vehicle 100, and a sensor for detecting the environment around the vehicle 100. Specifically, the internal sensor may include, for example, a camera, LiDAR, millimeter-wave radar, ultrasonic sensor, GPS sensor, acceleration sensor, gyro sensor, etc. For example, in the form (1) above, the server device 200 may acquire the detection result of the internal sensor and reflect the detection result of the internal sensor in the route when generating the route. In the form (1) above, the vehicle 100 may acquire the detection result of the internal sensor and reflect the detection result of the internal sensor in the driving control signal when generating the driving control signal. In the form (2) above, the vehicle 100 may acquire the detection result of the internal sensor and reflect the detection result of the internal sensor in the route when generating the route. In the form (2) above, the vehicle 100 may acquire the detection result of the internal sensor and reflect the detection result of the internal sensor in the driving control signal when generating the driving control signal.

[0062] (D8) In the above-described first to second embodiments, the server device 200 automatically generates the driving control signal transmitted to the vehicle 100. In contrast, the server device 200 may generate the driving control signal transmitted to the vehicle 100 according to the operation of an external operator located outside the vehicle 100. For example, an external operator operates a control device including a display for displaying point cloud data output from LiDAR, which is an external sensor 301, or an imaging image output from a camera, a steering wheel for remotely operating the vehicle 100, an accelerator pedal, a brake pedal, and a communication device for communicating with the server device 200 by wired or wireless communication, and the server device 200 may generate a driving control signal corresponding to the operation applied to the control device.

[0063] (D9) In the above-described first embodiment, the vehicle 100 only needs to be configured to be movable by autonomous driving. For example, it may be in the form of a platform having the configuration described below. Specifically, the vehicle 100 only needs to include at least a vehicle control device 110 and an actuator group 120 in order to exhibit the three functions of "running", "turning", and "stopping" by autonomous driving. When the vehicle 100 acquires information from the outside for autonomous driving, the vehicle 100 may further include a communication device 130. That is, for the vehicle 100 that can be moved by autonomous driving, at least a part of the interior parts such as the driver's seat and the dashboard may not be installed, at least a part of the exterior parts such as the bumper and the fender may not be installed, and the body shell may not be installed. In this case, until the vehicle 100 is shipped from the factory KJ, the remaining parts such as the body shell may be installed on the vehicle 100, or after the vehicle 100 is shipped from the factory KJ in a state where the remaining parts such as the body shell are not installed on the vehicle 100, the remaining parts such as the body shell may be installed on the vehicle 100. Each part may be installed from any direction such as the upper side, the lower side, the front side, the rear side, the right side, or the left side of the vehicle 100, and they may be installed from the same direction or from different directions respectively. Note that the positioning of the platform form can also be performed in the same manner as the vehicle 100 in the first embodiment.

[0064] (D10) The vehicle 100 may be manufactured by combining a plurality of modules. A module means a unit composed of a plurality of parts grouped according to the parts and functions of the vehicle 100. For example, the platform of the vehicle 100 may be manufactured by combining a front module that constitutes the front part of the platform, a center module that constitutes the central part of the platform, and a rear module that constitutes the rear part of the platform. Note that the number of modules constituting the platform is not limited to three, and may be two or less or four or more. Further, in addition to the parts constituting the platform, or instead of these, the parts constituting a portion of the vehicle 100 different from the platform may be modularized. Further, each type of module may include any exterior parts such as bumpers and grills, and any interior parts such as seats and consoles. Further, not limited to the vehicle 100, any type of moving body may be manufactured by combining a plurality of modules. Such modules may be manufactured, for example, by joining a plurality of parts by welding or fixtures, or by integrally molding at least a part of the parts constituting the module by casting as a single part. The molding method of integrally molding a single part, particularly a relatively large part, is also called gigacasting or megacasting. For example, the above-mentioned front module, center module, and rear module may be manufactured using gigacasting.

[0065] (D11) Using the running of the vehicle 100 by autonomous driving to transport the vehicle 100 is also called "self-propelled transport". Further, the configuration for realizing self-propelled transport is also called a "vehicle remote control autonomous driving transport system". Further, the production method of producing the vehicle 100 using self-propelled transport is also called "self-propelled production". In self-propelled production, for example, in the factory KJ that manufactures the vehicle 100, at least a part of the transport of the vehicle 100 is realized by self-propelled transport.

[0066] (D12)In the above-described first embodiment, some or all of the functions and processes realized software-wise may be realized hardware-wise. Also, some or all of the functions and processes realized hardware-wise may be realized software-wise. As the hardware for realizing the various functions in the above-described embodiments, for example, various circuits such as integrated circuits and discrete circuits may be used.

[0067] The present disclosure is not limited to the above-described embodiments, and can be realized in various configurations without departing from the gist thereof. For example, the technical features in the embodiments corresponding to the technical features in each of the forms described in the summary of the invention can be appropriately replaced or combined in order to solve some or all of the above-described problems, or to achieve some or all of the above-described effects. Also, if the technical feature is not described as essential in this specification, it can be appropriately deleted.

Explanation of Reference Numerals

[0068] 10 to 10c... system, 100... vehicle, 110... vehicle control device, 111... processor, 112... memory, 113... input / output interface, 114... internal bus, 115, 115c... travel control unit, 120... actuator group, 130... communication device, 191... appearance information acquisition unit, 192... process information acquisition unit, 193... comparison information acquisition unit, 194... estimation unit, 200... server device, 201... processor, 202... memory, 203... input / output interface, 204... internal bus, 205... communication device, 210... appearance information acquisition unit, 220... process information acquisition unit, 230... comparison information acquisition unit, 240... estimation unit, 250... remote control unit, 260... defect detection unit, 270... defect countermeasure unit, 300... external sensor group, 301... external sensor, 400... process management device, 500... notification device

Claims

1. A system comprising: a sensor configured to obtain appearance information including an appearance of a mobile object capable of moving by autonomous driving; a process information acquisition unit configured to obtain process information regarding a progress of a manufacturing process of the mobile object; a comparison information acquisition unit configured to obtain comparison information representing the appearance of the mobile object according to the process information; an estimation unit configured to estimate at least one of a position and an orientation of the mobile object by comparing the comparison information with the appearance information; The system.

2. The system according to claim 1, wherein the sensor repeatedly obtains the appearance information, the process information acquisition unit repeatedly obtains the process information, the comparison information acquisition unit does not obtain the comparison information according to the process information obtained this time when the content of the process information obtained this time by the process information acquisition unit is the same as the content at the previous acquisition, and the estimation unit compares the same comparison information as that at the previous comparison with the appearance information.

3. The system according to claim 1, further comprising: a defect detection unit configured to detect a defective mounting of a component in the mobile object using a degree of coincidence between the appearance of the mobile object represented by the comparison information and the appearance of the mobile object included in the appearance information.

4. The system according to claim 3, further comprising: a defect handling unit configured to execute at least one of a process of stopping the movement of the mobile object and a process of notifying the occurrence of the defective mounting when the defective mounting is detected by the defect detection unit.

5. The system according to claim 1, further comprising: a database in which the process information and the comparison information are associated with each other, wherein the comparison information acquisition unit obtains the comparison information associated with the process information obtained by the process information acquisition unit in the database.

6. The system according to claim 1, further comprising: a process management device configured to manage the manufacturing of the mobile object, wherein the process information acquisition unit obtains the process information from the process management device.

7. The system according to claim 1, further comprising: a plurality of the sensors; a database in which identification information of each of the plurality of sensors and the comparison information are associated with each other; and the process information acquisition unit obtains, as the process information, the identification information of the sensor that has obtained the appearance information. A system in which the comparison information acquisition unit acquires the comparison information associated with the identification information of the sensor that has acquired the appearance information in the database.

8. An apparatus comprising: An appearance information acquisition unit that acquires the appearance information from a sensor that acquires appearance information including the appearance of a moving body movable by autonomous driving; A process information acquisition unit that acquires process information regarding the progress of the manufacturing process of the moving body; A comparison information acquisition unit that acquires comparison information representing the appearance of the moving body according to the process information; An estimation unit that estimates at least one of the position and orientation of the moving body by comparing the comparison information and the appearance information; An apparatus comprising the same.

9. A method comprising: Acquiring appearance information including the appearance of a moving body movable by autonomous driving; Acquiring process information regarding the progress of the manufacturing process of the moving body; Acquiring comparison information representing the appearance of the moving body according to the process information; Estimating at least one of the position and orientation of the moving body by comparing the comparison information and the appearance information. A method.

Images