CONTROL DEVICE, METHOD, AND SYSTEM
The control device and system improve the accuracy of vehicle control by using three-dimensional point cloud data and reference data to match and adapt point cloud density, addressing the challenge of precise vehicle positioning and orientation in manufacturing environments.
Patent Information
- Application Number
- JP2023218821
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2023-12-26
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-11-02
AI Technical Summary
Existing technologies for controlling the movement of moving objects, such as vehicles, through autonomous or remote control, face challenges in accurately determining the position and orientation of these objects using 3D point cloud data, necessitating improved matching techniques for precise control.
A control device and system that utilizes three-dimensional point cloud data from a distance measuring device, combined with reference point cloud data, to perform matching and acquire moving object information, including external features and manufacturing information, to generate accurate control commands for unmanned vehicle movement.
Enhances the accuracy of matching processes, enabling precise control of vehicles in manufacturing environments without the need for external transport devices, by adapting point cloud density and type based on vehicle characteristics and distance.
Smart Images

Figure 0007593467000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a control device, a method, and a system. [Background technology]
[0002] Patent Document 1 discloses a technique for running a vehicle autonomously or by remote control during the vehicle manufacturing process. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Special Publication No. 2017-538619 Summary of the Invention [Problem to be solved by the invention]
[0004] In order to autonomously or remotely control the movement of a moving object such as a vehicle, a technique is known that estimates the position and orientation of the moving object by matching with 3D point cloud data of the moving object acquired by a distance measuring device. In order to appropriately control the movement of the moving object, a technique for improving the accuracy of such matching is desired. [Means for solving the problem]
[0005] The present disclosure can be realized in the following forms. According to one embodiment of the present disclosure, there is provided a control device used to generate a control command for moving a moving body configured to be movable by unmanned operation in a factory that manufactures the moving body. In order to generate the control command, the control device includes a matching unit that performs matching using measurement point cloud data, which is 3D point cloud data of the moving body acquired using a distance measuring device, and reference point cloud data representing reference 3D point cloud data, and a moving body information acquisition unit that acquires moving body information capable of identifying external features of the moving body. The moving body information acquisition unit acquires the moving body information from a process management device that manages the manufacturing process of the moving body in the factory, and the matching unit performs a matching process according to the moving body information. Using the reference point cloud data The matching is performed. The mobile object information is information that can identify at least one of the paint condition of the mobile object at the time the matching is performed and whether or not an exterior part is attached to the mobile object. According to one embodiment of the present disclosure, there is provided a method used for generating a control command for moving a moving body in a factory that manufactures moving bodies configured to be movable by unmanned operation. This method includes a step of performing matching using measurement point cloud data, which is 3D point cloud data of the moving body acquired using a distance measuring device, and reference point cloud data representing reference 3D point cloud data, in order to generate the control command, and a step of acquiring moving body information capable of identifying external features of the moving body. In the step of acquiring the moving body information, the moving body information is acquired from a process management device that manages the manufacturing process of the moving body in the factory, and in the step of performing matching, a step of acquiring a control command corresponding to the moving body information. Using the reference point cloud data The matching is performed. The mobile object information is information that can identify at least one of the paint condition of the mobile object at the time the matching is performed and whether or not an exterior part is attached to the mobile object. According to one embodiment of the present disclosure, a system is provided. The system includes a mobile body, a matching unit that performs matching using measurement point cloud data, which is 3D point cloud data of the mobile body acquired using a distance measuring device, and reference point cloud data representing reference 3D point cloud data in order to generate a control command for moving the mobile body in a factory that manufactures the mobile body configured to be movable by unmanned operation, and a mobile body information acquisition unit that acquires mobile body information that can identify external features of the mobile body. The mobile body information acquisition unit acquires the mobile body information from a process management device that manages the manufacturing process of the mobile body in the factory, and the matching unit performs matching according to the mobile body information. Using the reference point cloud data The matching is performed. The mobile object information is information that can identify at least one of the paint condition of the mobile object at the time the matching is performed and whether or not an exterior part is attached to the mobile object. The present disclosure can also be realized in the following forms.
[0006] (1) According to one embodiment of the present disclosure, there is provided a control device used to generate a control command for moving a moving body by unmanned driving. The control device includes a matching unit that performs matching using measurement point cloud data, which is 3D point cloud data of the moving body acquired by a distance measuring device, and reference point cloud data representing reference 3D point cloud data, in order to generate the control command, and a moving body information acquisition unit that acquires moving body information capable of identifying external features of the moving body. The matching unit performs the matching according to the moving body information. According to this aspect, matching using the measurement point cloud data of a moving object can be performed according to the external appearance characteristics of the moving object, thereby increasing the possibility of performing matching with high accuracy. (2) In the control device of the above embodiment, the moving object information may include model information capable of identifying a model of the moving object, the reference point cloud data may include model point cloud data corresponding to the model, and the matching unit may execute the matching using the model point cloud data corresponding to the model information. According to this embodiment, the model point cloud data corresponding to the model of the moving object can be used for matching, which further increases the possibility of executing matching with high accuracy. (3) In the control device of the above aspect, the mobile object information may include, as the model information, manufacturing information used for managing a manufacturing process of the mobile object. According to this aspect, the manufacturing information can be used as the model information. (4) In the control device of the above aspect, the matching unit may change at least one of the density of the point cloud in the measurement point cloud data and the density of the point cloud in the reference point cloud data in accordance with the moving body information. According to this aspect, by changing the density of the point cloud in matching in accordance with the moving body information, it is possible to increase the possibility that matching can be performed with high accuracy. (5) In the control device of the above aspect, the matching unit may change at least one of the density of the points in the measurement point cloud data and the density of the points in the reference point cloud data in accordance with the distance between the ranging device and the moving body. According to this aspect, the density of the points in the matching is changed in accordance with the distance between the ranging device and the moving body in addition to the moving body information, so that it is possible to further increase the possibility of performing matching with high accuracy. In addition to the form of the control device described above, the present disclosure can be realized in the form of, for example, a system, a control method, a computer program for realizing the control method, and a non-transitory recording medium on which the computer program is recorded. [Brief description of the drawings]
[0007] [Figure 1] A conceptual diagram showing the system configuration. [Diagram 2] FIG. 2 is a block diagram showing the configuration of a vehicle and a control device. [Diagram 3] FIG. 4 is an explanatory diagram showing an example of a matching condition in the first embodiment. [Figure 4A] 4 is a flowchart showing a processing procedure for vehicle travel control in the first embodiment. [Figure 4B] 13 is a flowchart of a command generation process. [Diagram 5] FIG. 11 is an explanatory diagram showing an example of a matching condition in the second embodiment. [Figure 6]FIG. 13 is a conceptual diagram showing the configuration of a system according to a third embodiment. [Figure 7] 11 is a flowchart showing a processing procedure for vehicle travel control in a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] A. First embodiment: 1 is a conceptual diagram showing the configuration of a system 10 in this embodiment. The system 10 includes one or more vehicles 100 as moving objects, a server 200 used to generate control commands for remotely controlling the vehicles 100, a plurality of distance measuring devices 300 that measure three-dimensional point cloud data of the vehicles 100, and a process control device 400 that controls the manufacturing process of the vehicles 100. In the first embodiment, the server 200 corresponds to the "control device" of the present disclosure.
[0009] In this disclosure, the term "mobile body" refers to an object that can move, such as a vehicle or an electric vertical take-off and landing aircraft (a so-called flying car). The vehicle 100 may be a vehicle that runs on wheels or a vehicle that runs on caterpillar tracks, such as a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, or a construction vehicle. The vehicle 100 includes an electric vehicle (BEV: Battery Electric Vehicle), a gasoline-powered vehicle, a hybrid vehicle, and a fuel cell vehicle. When the mobile body is other than the vehicle 100, the expressions "vehicle" and "car" in this disclosure can be appropriately replaced with "mobile body", and the expression "running" can be appropriately replaced with "movement".
[0010] The vehicle 100 is configured to be capable of traveling by unmanned driving. "Unmanned driving" means driving without the driving operation of a passenger. Driving operation means at least one of the operations of "running", "turning" and "stopping" of the vehicle 100. Unmanned driving is realized by automatic or manual remote control using a device located outside the vehicle 100, or by autonomous control of the vehicle 100. The vehicle 100 traveling by unmanned driving may have a passenger who does not perform driving operation on board. The passenger who does not perform driving operation includes, for example, a person who simply sits in the seat of the vehicle 100, and a person who performs work other than driving operation, such as assembly, inspection, and operation of switches, while riding on the vehicle 100. Note that driving by a passenger who performs driving operation is sometimes called "manned driving".
[0011] In this specification, "remote control" includes "full remote control" in which all of the operations of the vehicle 100 are completely determined from outside the vehicle 100, and "partial remote control" in which some of the operations of the vehicle 100 are determined from outside the vehicle 100. In addition, "autonomous control" includes "full autonomous control" in which the vehicle 100 autonomously controls its own operations without receiving any information from devices external to the vehicle 100, and "partial autonomous control" in which the vehicle 100 autonomously controls its own operations using information received from devices external to the vehicle 100.
[0012] The vehicle 100 is preferably a Battery Electric Vehicle (BEV).
[0013] The system 10 in this embodiment is configured as a remote control system that runs the vehicle 100 by remote control. In this embodiment, the remote control of the vehicle 100 is executed in a factory that manufactures the vehicle 100. The factory includes a first location PL1 and a second location PL2. The first location PL1 is, for example, a location where the vehicle 100 is assembled, and the second location PL2 is, for example, a location where the vehicle 100 is inspected. The first location PL1 and the second location PL2 are connected by a travel path SR on which the vehicle 100 can run. Any position in the factory is expressed by xyz coordinate values of a reference coordinate system Σr.
[0014] A plurality of distance measuring devices 300 are installed around the travel path SR, with the vehicle 100 as the measurement target. The server 200 can obtain the relative position and orientation of the vehicle 100 with respect to the target route TR and the traveling direction of the vehicle 100 in real time using the three-dimensional point cloud data measured by each distance measuring device 300. A camera or LiDAR (Light Detection And Ranging) can be used as the distance measuring device 300. In particular, LiDAR is preferable because it can obtain high-precision three-dimensional point cloud data. The distance measuring device 300 in this embodiment is configured by LiDAR. In this embodiment, the position of each distance measuring device 300 is fixed, and the relative relationship between the reference coordinate system Σr and the device coordinate system of each distance measuring device 300 is known. A coordinate transformation matrix for mutually transforming the coordinate values of the reference coordinate system Σr and the coordinate values of the device coordinate system of each distance measuring device 300 is stored in advance in the server 200.
[0015] The server 200 in this embodiment is configured as a remote control device that generates a control command for remotely controlling the vehicle 100 and transmits the control command to the vehicle 100. More specifically, the server 200 generates a control command for causing the vehicle 100 to travel along a target route TR and transmits the control command to the vehicle 100. The vehicle 100 travels according to the received control command. Therefore, the system 10 can move the vehicle 100 from a first location PL1 to a second location PL2 by remote control without using a transport device such as a crane or a conveyor. The target route TR in this embodiment corresponds to a reference route described later. The control command will be described in detail later.
[0016] FIG. 2 is a block diagram showing the configuration of the vehicle 100 and the server 200. The vehicle 100 includes a vehicle control device 110 for controlling each part of the vehicle 100, an actuator group 120 that operates under the control of the vehicle control device 110, a communication device 130 for communicating with the server 200 by wireless communication, and a GNSS (Global Navigation Satellite System) receiver 140 for acquiring position information of the vehicle 100. In this embodiment, the actuator group 120 includes an actuator of a drive device for accelerating the vehicle 100, an actuator of a steering device for changing the traveling direction of the vehicle 100, and an actuator of a braking device for decelerating the vehicle 100. The drive device includes a battery, a driving motor driven by the power of the battery, and a driving wheel rotated by the driving motor. The actuator of the drive device includes a driving motor. The actuator group 120 may further include an actuator for swinging the wipers of the vehicle 100, an actuator for opening and closing the power windows of the vehicle 100, and the like.
[0017] The vehicle control device 110 is configured by a computer including a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input / output interface 113 are connected to each other via the internal bus 114 so as to be able to communicate bidirectionally. The actuator group 120, the communication device 130, and the GNSS receiver 140 are connected to the input / output interface 113.
[0018] In this embodiment, the processor 111 executes a program PG1 stored in advance in the memory 112, thereby functioning as the vehicle control unit 115 and the position information acquisition unit 116. The vehicle control unit 115 controls the actuator group 120. When a driver is on board the vehicle 100, the vehicle control unit 115 can drive the vehicle 100 by controlling the actuator group 120 in response to an operation by the driver. The vehicle control unit 115 can also drive the vehicle 100 by controlling the actuator group 120 in response to a control command transmitted from the server 200, regardless of whether a driver is on board the vehicle 100 or not. The position information acquisition unit 116 acquires position information indicating the current location of the vehicle 100 using the GNSS receiver 140. However, the position information acquisition unit 116 and the GNSS receiver 140 can be omitted.
[0019] The server 200 is configured by a computer including a processor 201, a memory 202, an input / output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input / output interface 203 are connected via the internal bus 204 so as to be able to communicate bidirectionally. A communication device 205 is connected to the input / output interface 203 for communicating with the vehicle 100, the distance measuring device 300, and the process control device 400 by wireless communication.
[0020] In this embodiment, the processor 201 executes a program PG2 pre-stored in the memory 202, thereby functioning as a three-dimensional point cloud data acquisition unit 210, a vehicle information acquisition unit 215, a matching unit 220, a matching condition determination unit 225, an estimation unit 230, and a command generation unit 240. In addition to the program PG2, the memory 202 stores reference point cloud data 250, a target route TR, and correction data 260, which will be described later.
[0021] The three-dimensional point cloud data acquisition unit 210 acquires three-dimensional point cloud data of the vehicle 100 measured by the distance measuring device 300. This three-dimensional point cloud data is data indicating the three-dimensional positions of the point cloud detected by the distance measuring device 300. The three-dimensional point cloud data of the moving body measured by the distance measuring device 300 is also called measured point cloud data.
[0022] The vehicle information acquisition unit 215 acquires vehicle information capable of identifying the external characteristics of the vehicle 100. The vehicle information acquisition unit 215 corresponds to a mobile body information acquisition unit that acquires mobile body information capable of identifying the external characteristics of the mobile body. The vehicle information is information capable of identifying, for example, the shape, color, pattern, gloss, etc., of the external characteristics of the vehicle 100. More specifically, the vehicle information may be information capable of identifying, for example, the body shape of the vehicle 100, the presence or absence and type of exterior parts, the body color, the material of the body, and the paint condition of the body. The paint condition of the body is, for example, whether the body has been painted and the number of times, and the type of paint.
[0023] The vehicle information in this embodiment includes identification information Vi of the vehicle 100. This identification information Vi corresponds to manufacturing information used to manage the manufacturing process of the vehicle 100. The identification information Vi includes model information Ti capable of identifying the model of the vehicle 100. Furthermore, the identification information Vi in this embodiment includes body color information Ci capable of identifying the body color of the vehicle 100. In this specification, the "model" refers to information that specifies at least the body shape of the vehicle 100 among the external features of the vehicle 100. In other embodiments, the "model" may be information that specifies various external features of the vehicle 100, such as the body color of the vehicle 100, the presence or absence and type of exterior parts, in addition to the body shape.
[0024] The matching unit 220 performs template matching (hereinafter, simply referred to as matching) using the measurement point cloud data and the reference point cloud data 250. This matching is performed to generate a control command for the vehicle 100. As a matching algorithm, various algorithms such as ICP (Iterative Closest Point) and NDT (Normal Distributions Transform) can be used.
[0025] The reference point cloud data 250 in this embodiment includes model point cloud data 251 corresponding to the model of the vehicle 100. More specifically, the reference point cloud data 250 includes a plurality of model point cloud data 251 corresponding to a plurality of models and associated with each model.
[0026] The matching condition determination unit 225 determines the matching conditions according to the vehicle information acquired by the vehicle information acquisition unit 215. The above-mentioned matching unit 220 executes matching according to the matching conditions determined by the matching condition determination unit 225. That is, the matching unit 220 executes matching according to the vehicle information. Details of the matching conditions will be described later.
[0027] Fig. 3 is an explanatory diagram showing an example of matching conditions determined in this embodiment. Fig. 3 shows the correspondence between identification information Vi and matching conditions. More specifically, Fig. 3 shows identification information Vi, appearance features (appearance specifications) specified by the identification information Vi, and matching conditions determined according to the identification information Vi.
[0028] The identification information Vi1 shown in FIG. 3 includes model information Ti1 and vehicle body color information Ci1. The identification information Vi2 includes model information Ti2 and vehicle body color information Ci2. The identification information Vi3 includes model information Ti1 and vehicle body color information Ci3. The model information Ti1 is model information Ti capable of identifying a model that specifies a vehicle body shape BS1. The model information Ti2 is model information Ti capable of identifying a model that specifies a vehicle body shape BS2. The vehicle body color information Ci1 and the vehicle body color information Ci2 are information that respectively specify white as the vehicle body color. The vehicle body color information Ci3 is information that specifies black as the vehicle body color. That is, for example, the vehicle 100 identified by the identification information Vi1 has a vehicle body shape BS1 and a white vehicle body color.
[0029] As shown in FIG. 3, matching condition MC1 is applied in response to identification information Vi1. Matching condition MC1 specifies that model point cloud data TP1 is used in matching, and that the density of the point cloud in model point cloud data TP1 is specified to density ds1. Matching condition MC2 is applied in response to identification information Vi2. Matching condition MC2 specifies that model point cloud data TP2 is used in matching, and that the density of the point cloud in model point cloud data TP2 is specified to density ds1. Matching condition MC3 is applied in response to identification information Vi3. Matching condition MC3 specifies that model point cloud data TP1 is used in matching, and that the density of the point cloud in model point cloud data TP1 is specified to density ds2. Density ds2 is a density lower than density ds1.
[0030] The estimation unit 230 estimates the position and orientation of the vehicle 100 using the result of matching performed by the matching unit 220. The position and orientation estimated by the estimation unit 230 in this manner correspond to vehicle position information, which will be described later. In this embodiment, the position of the vehicle 100 is estimated as xyz coordinates of a predetermined detection point of the vehicle 100. The orientation of the vehicle 100 is calculated as the direction of a vector along the central axis of the vehicle 100 and directed from the rear side to the front side of the vehicle 100. This vector is specified, for example, using the coordinates of a center position in the vehicle width direction on the front side of the vehicle 100 and the coordinates of a center position in the vehicle width direction on the rear side of the vehicle 100. Note that the position and orientation of the vehicle 100 may be estimated so as to be usable for generating a control command for unmanned driving, and the manner of estimating the position and orientation is not limited to the above. Furthermore, when three-dimensional point cloud data is not available, the estimation unit 230 can estimate the position and orientation of the vehicle 100 using the travel history of the vehicle 100 and position information detected by the GNSS receiver 140 mounted on the vehicle 100. The estimation unit 230 may estimate only one of the position and orientation of the vehicle 100. In this case, the other of the position and orientation of the vehicle 100 is determined, for example, using the travel history of the vehicle 100. Note that the travel direction of the vehicle 100 can be estimated, for example, using any one or two or more of the orientation, travel history, and control command history of the vehicle 100.
[0031] The command generating unit 240 uses the estimated position and orientation of the vehicle 100 to generate a control command for driving the vehicle 100 in an unmanned driving manner and transmits the control command to the vehicle 100. In this embodiment, the control command is a command for driving the vehicle 100 according to the target route TR stored in the memory 202. Specifically, the control command in this embodiment is a driving control signal, which will be described later.
[0032] The control command for driving the vehicle 100 in an unmanned driving mode may include at least one of a driving control signal and generation information for generating the driving control signal. Therefore, in other embodiments, the control command may include generation information instead of or in addition to the driving control signal. As the generation information, for example, vehicle position information, a route, or a target position, which will be described later, may be used.
[0033] The process management device 400 in this embodiment is configured by a computer. The process management device 400 manages the entire manufacturing process of the vehicle 100 in a factory according to various manufacturing information included in the manufacturing information data Pi. The manufacturing information data Pi includes the above-mentioned identification information Vi. For example, the identification information Vi is acquired by the process management device 400 from an RF-ID (Radio Frequency-Identification: Radio Mobile Identification) tag attached to the vehicle 100 via short-range wireless communication. For example, when one vehicle 100 starts traveling along a target route TR, the identification information Vi for identifying the vehicle 100 is transmitted from the process management device 400 to the server 200. The position of the vehicle 100 detected by the server 200 is also transmitted to the process management device 400. The function of the process management device 400 may be implemented in the same device as the server 200.
[0034] FIG. 4A is a flowchart showing a process procedure of the driving control of the vehicle 100 in the first embodiment. In S1, the server 200 acquires vehicle position information of the vehicle 100 using a detection result output from an external sensor that is a sensor located outside the vehicle 100. The vehicle position information is position information that is the basis for generating a driving control signal. In this embodiment, the vehicle position information includes the position and orientation of the vehicle 100 in the reference coordinate system Σr of the factory. In this embodiment, the reference coordinate system Σr of the factory is a global coordinate system, and any position in the factory is expressed by X, Y, and Z coordinates in the global coordinate system. In this embodiment, the external sensor is a distance measuring device 300 installed in the factory, and the distance measuring device 300 outputs distance measuring point cloud data as a detection result. That is, in S1, the server 200 acquires vehicle position information using distance measuring point cloud data acquired from the distance measuring device 300 that is an external sensor. Specifically, the server 200 acquires vehicle position information by template matching using the distance measuring point cloud data and the reference point cloud data 250.
[0035] At S2, the server 200 determines a target position to which the vehicle 100 should next head. In this embodiment, the target position is represented by X, Y, and Z coordinates in a global coordinate system. A reference route, which is a route along which the vehicle 100 should travel, is stored in advance in the memory of the server 200. The route is represented by nodes indicating a departure point, nodes indicating passing points, nodes indicating a destination, and links connecting the nodes. The server 200 uses the vehicle position information and the reference route to determine a target position to which the vehicle 100 should next head. The server 200 determines a target position on the reference route that is ahead of the current location of the vehicle 100.
[0036] In S3, the server 200 generates a driving control signal for driving the vehicle 100 toward the determined target position. In this embodiment, the driving control signal includes the acceleration and steering angle of the vehicle 100 as parameters. The server 200 calculates the driving speed of the vehicle 100 from the transition of the position of the vehicle 100, and compares the calculated driving speed with the target speed. When the driving speed is lower than the target speed as a whole, the server 200 determines the acceleration so that the vehicle 100 accelerates, and when the driving speed is higher than the target speed, the server 200 determines the acceleration so that the vehicle 100 decelerates. In addition, when the vehicle 100 is located on the reference route, the server 200 determines the steering angle and acceleration so that the vehicle 100 does not deviate from the reference route, and when the vehicle 100 is not located on the reference route, in other words, when the vehicle 100 deviates from the reference route, the server 200 determines the steering angle and acceleration so that the vehicle 100 returns to the reference route. In other embodiments, the cruise control signal may include the speed of the vehicle 100 as a parameter instead of or in addition to the acceleration of the vehicle 100.
[0037] In S4, server 200 transmits the generated driving control signal to vehicle 100. Server 200 repeats, at a predetermined cycle, acquisition of vehicle position information, determination of a target position, generation of a driving control signal, and transmission of the driving control signal.
[0038] In this embodiment, specifically, a command generation process, which will be described later, is executed in steps S1 to S4.
[0039] At S5, vehicle 100 receives a driving control signal transmitted from server 200. At S6, vehicle 100 controls an actuator of vehicle 100 using the received driving control signal, thereby causing vehicle 100 to drive at an acceleration and steering angle represented by the driving control signal. Vehicle 100 repeats receiving the driving control signal and controlling the actuator of vehicle 100 at a predetermined cycle. According to system 10 in this embodiment, vehicle 100 can be driven by remote control, and vehicle 100 can be moved without using transportation equipment such as a crane or a conveyor.
[0040] 4B is a flowchart of a command generation process for implementing the control method in this embodiment. This command generation process is executed every time the three-dimensional point cloud data acquisition unit 210 acquires new measurement point cloud data from the distance measuring device 300 that is responsible for measuring the vehicle 100 that is the target of unmanned driving. For example, the three-dimensional point cloud data acquisition unit 210 may execute a pre-processing to remove background point cloud data representing stationary objects from the newly acquired measurement point cloud data prior to the start of the command generation process. Hereinafter, the vehicle 100 that is the target of unmanned driving is also referred to as the target vehicle 100.
[0041] At S100, the vehicle information acquisition unit 215 acquires vehicle information of the target vehicle 100. At S100 in this embodiment, the vehicle information acquisition unit 215 acquires, as the vehicle information, the identification information Vi transmitted from the process management device 400. At S110, the matching condition determination unit 225 determines matching conditions according to the vehicle information acquired at S100. At S120, the matching unit 220 performs matching according to the matching conditions determined at S110.
[0042] In this embodiment, the matching unit 220 performs matching in S120 using the model point cloud data TP1 according to the model information Ti. For example, when the body shape of the target vehicle 100 is the body shape BS1 and the body color is white, the identification information Vi1 shown in FIG. 3 is acquired as the vehicle information of the vehicle 100 in S100. In this case, in S120, the model point cloud data TP1 is used for matching in accordance with the matching condition MC1 determined in S110. On the other hand, when the body shape of the target vehicle 100 is the body shape BS2 and the body color is white, the identification information Vi2 is acquired as the vehicle information of the target vehicle 100 in S100. In this case, in S120, the model point cloud data TP2 is used for matching in accordance with the matching condition MC2 determined in S110.
[0043] In this embodiment, the matching unit 220 changes the density of the point cloud in matching in S120 according to the vehicle information. More specifically, the matching unit 220 changes the density of the point cloud of the reference point cloud data 250 according to the vehicle body color information Ci. For example, when the vehicle body shape of the target vehicle 100 is the vehicle body shape BS1 and the vehicle body color is black, in S100, the identification information Vi3 is acquired as the vehicle information of the target vehicle 100. In this case, in S120, matching is performed in a state where the density of the point cloud in the model point cloud data TP1 is set to the density ds2 according to the matching condition MC3 determined in S110. On the other hand, when the identification information Vi is acquired in S100 as described above, in S120, matching is performed in a state where the density of the point cloud in the model point cloud data TP1 is set to the density ds1 according to the matching condition MC1.
[0044] The matching unit 220 in this embodiment realizes the change in density of the point cloud according to the vehicle information by the correction data 260. The correction data 260 is data for correcting the density of the point cloud of the reference point cloud data 250, and is, for example, data in which a reduction rate for reducing the density of the point cloud in the reference point cloud data 250 is recorded for each vehicle body color. Note that, in another embodiment, for example, the reference point cloud data 250 having different densities may be stored in the memory 202 for each vehicle body color, and the matching unit 220 may use the reference point cloud data 250 according to the vehicle information for matching, thereby realizing the change in density of the point cloud according to the vehicle information.
[0045] The characteristics of the exterior of the vehicle 100, particularly the body color and the paint state of the body, affect the density of the point cloud in the measured point cloud data for the vehicle 100. For example, when the body color of the vehicle 100 is white, the reflectance of light on the body surface of the vehicle 100 is high compared to when the body color is black, so the density of the point cloud in the measured point cloud data is higher. Therefore, as in this embodiment, for the vehicle 100 whose body color is white, the density of the point cloud of the reference point cloud data 250 is set to density ds1 and matching is performed, and for the vehicle 100 whose body color is black, the density of the point cloud of the reference point cloud data 250 is set to density ds2 lower than density ds1 and matching is performed, thereby increasing the possibility of improving the accuracy of matching for each vehicle 100. The above-mentioned correction data 260 is defined based on the results of experiments and simulations, for example, so that the accuracy of matching for the vehicle 100 of each body color is equal to or higher than a predetermined accuracy. In another embodiment, the density of the points in the measurement point cloud data may be changed in accordance with the vehicle information during matching. Also, both the density of the points in the measurement point cloud data and the density of the points in the reference point cloud data 250 may be changed in accordance with the vehicle information during matching.
[0046] At S130, the estimation unit 230 estimates the position and orientation of the target vehicle 100 using the result of the matching performed at S120. At S140, the command generation unit 240 generates a control command using the position and orientation estimated at S130. The control command thus generated is transmitted to the target vehicle 100. Then, the vehicle control unit 115 of the target vehicle 100 controls the actuator group 120 using the received control command, thereby causing the target vehicle 100 to travel.
[0047] According to the server 200 in the present embodiment described above, the matching unit 220 executes matching according to vehicle information capable of identifying the external characteristics of the vehicle 100 to be controlled. In this manner, matching can be executed according to the external characteristics of the vehicle 100. Therefore, even when various vehicles 100 each having different external characteristics are to be remotely controlled, matching can be executed according to the external characteristics of each vehicle 100. This increases the possibility that matching can be executed accurately in the system 10.
[0048] In the present embodiment, the matching unit 220 performs matching using the model point cloud data 251 corresponding to the model information Ti. In this way, the model point cloud data 251 corresponding to the model of the vehicle 100 can be used for matching, so that the possibility of performing matching with high accuracy is further increased.
[0049] Furthermore, in this embodiment, the vehicle information includes manufacturing information of the vehicle 100 as the model information Ti. In this manner, the manufacturing information of the vehicle 100 can be used as the model information Ti. Therefore, for example, by using the manufacturing information of the vehicle 100 as the model information Ti at a manufacturing plant of the vehicle 100, matching using the above-described model point cloud data 251 can be easily performed.
[0050] In the present embodiment, the matching unit 220 changes at least one of the density of the point cloud in the measurement point cloud data and the density of the point cloud in the reference point cloud data 250 according to the vehicle information. Therefore, by changing the density of the point cloud in matching according to the vehicle information, it is possible to increase the possibility that matching can be performed with high accuracy.
[0051] B. Second embodiment: FIG. 5 is a schematic diagram for explaining an example of a matching condition determined in the second embodiment. FIG. 5 shows the correspondence between a combination of vehicle information and a separation distance and a matching condition. The "separation distance" represents the distance between the target vehicle 100 and the distance measuring device 300 that is responsible for measuring the target vehicle 100. In this embodiment, the distance measured by the distance measuring device 300 is used as the separation distance for a certain vehicle 100. In the second embodiment, unlike the first embodiment, the density of the point cloud in matching is changed according to the separation distance in addition to the vehicle information. Parts of the configuration of the server 200 and the system 10 in the second embodiment that are not particularly described are the same as those in the first embodiment.
[0052] In the example of FIG. 5, the matching condition MC1 is applied in response to a combination of the identification information Vi1 and the separation distance d1. The matching condition MC2 is applied in response to a combination of the identification information Vi2 and the separation distance d1. The matching condition MC2b is applied in response to a combination of the identification information Vi2 and the separation distance d2. The separation distance d2 is longer than the separation distance d1. The matching condition MC2b specifies the density of the point cloud in matching to a density ds1b lower than the density ds1 specified by the matching condition MC2. The matching condition MC3 is applied in response to a combination of the identification information Vi3 and the separation distance d1. The matching condition MC3b is applied in response to a combination of the identification information Vi3 and the separation distance d2. The matching condition MC3b specifies the density of the point cloud in matching to a density ds2b lower than the density ds2 specified by the matching condition MC3. Generally, the density of the point cloud in the measurement point cloud data for the vehicle 100 increases as the separation distance for the vehicle 100 decreases, and decreases as the separation distance for the vehicle 100 increases. Therefore, in matching, as in this embodiment, by lowering the density of the point cloud in the reference point cloud data 250 when the separation distance is longer, the possibility of improving the matching accuracy is increased.
[0053] The change in density of the point cloud according to the above-mentioned separation distance can be realized, for example, by defining the correction data 260 so that the density of the point cloud in the reference point cloud data 250 is corrected according to the separation distance in addition to the vehicle body information. In this case, the correction data 260 is defined based on the results of experiments and simulations, for example, so that the accuracy of each matching in the case where the separation distance is different is equal to or higher than a predetermined accuracy. Note that, in other embodiments, the density of the measurement point cloud data may be changed according to the separation distance in the matching. Also, in the matching, both the density of the point cloud in the measurement point cloud data and the density of the point cloud in the reference point cloud data 250 may be changed according to the separation distance.
[0054] According to the second embodiment described above, the matching unit 220 changes at least one of the density of the points in the measurement point cloud data and the density of the points in the reference point cloud data 250 according to the separation distance between the distance measuring device 300 and the vehicle 100. This can further increase the possibility of performing matching with high accuracy.
[0055] C. Third embodiment: 6 is a conceptual diagram showing the configuration of a system 10v in the third embodiment. Unlike the first embodiment, the system 10v in this embodiment does not include a server 200. Also, the vehicle 100v in this embodiment can run by the autonomous control of the vehicle 100v. Parts of the configuration of the system 10v and the vehicle 100v in the third embodiment that are not particularly described are similar to those in the first embodiment.
[0056] In this embodiment, the communication device 130 of the vehicle 100v can communicate with the distance measuring device 300 and the process management device 400. The processor 111v of the vehicle control device 110v executes the program PG2 stored in the memory 112v to function as the vehicle control unit 115v, the position information acquisition unit 116, the three-dimensional point cloud data acquisition unit 210, the vehicle information acquisition unit 215, the matching unit 220, the matching condition determination unit 225, the estimation unit 230, and the command generation unit 240. The vehicle control unit 115v acquires the detection results by the sensors, generates a driving control signal using the detection results, and outputs the generated driving control signal to operate the actuator group 120, thereby making it possible to drive the vehicle 100v by autonomous control. In this embodiment, the vehicle control unit 115v functions as the command generation unit 240. In addition to the program PG1, the memory 112v stores reference point cloud data 250 including model point cloud data 251, a target route TR, and correction data 260. The vehicle control device 110v in the third embodiment corresponds to the "control device" in the present disclosure.
[0057] FIG. 7 is a flowchart showing a process procedure of the travel control of the vehicle 100v in the third embodiment. In S11, the vehicle 100v acquires vehicle position information using the detection result output from the distance measuring device 300, which is an external sensor. In S21, the vehicle 100v determines a target position to which the vehicle 100v should next head. In S31, the vehicle 100v generates a travel control signal for driving the vehicle 100v toward the determined target position. In S41, the vehicle 100v controls the actuator of the vehicle 100v using the generated travel control signal, thereby driving the vehicle 100v according to the parameters represented in the travel control signal. The vehicle 100v repeats the acquisition of vehicle position information, the determination of the target position, the generation of the travel control signal, and the control of the actuator at a predetermined cycle. According to the system 10v in this embodiment, the vehicle 100v can be driven by the autonomous control of the vehicle 100v without remotely controlling the vehicle 100v by the server 200.
[0058] In this embodiment, from S11 to S41, a command generation process similar to that in Fig. 4B is executed. This command generation process is executed every time the 3D point cloud data acquisition unit 210 of the vehicle 100v acquires new measurement point cloud data from the distance measuring device 300 in charge of measuring the target vehicle 100v. In this embodiment, the target vehicle 100v means the subject vehicle.
[0059] In this embodiment, each step of FIG. 4B is executed by the processor 111v of the vehicle 100v. In S130 in this embodiment, the vehicle information acquisition unit 215 of the vehicle 100v may acquire, for example, the identification information Vi transmitted from the process management device 400, or may acquire the identification information Vi previously stored in the memory 112v. In addition, in S140, the command generation unit 240 of the vehicle 100v generates and outputs a control command using the position and orientation estimated in S130. Specifically, the command generation unit 240 in this embodiment generates and outputs a traveling control signal using vehicle position information including the position and orientation. The vehicle control unit 115v controls the actuator group 120 using the traveling control signal to cause the vehicle 100 to travel.
[0060] The vehicle control device 110v in the present embodiment described above can also perform matching according to the external characteristics of the vehicle 100. Therefore, even when various vehicles 100 having different external characteristics are targeted for autonomous control, matching can be performed according to the external characteristics of each vehicle 100. This increases the possibility that matching can be performed accurately in the system 10.
[0061] In another embodiment in which the vehicle 100v runs under autonomous control like this embodiment, the matching unit 220 of the vehicle 100v may change at least one of the density of the point cloud in the measurement point cloud data and the density of the point cloud in the reference point cloud data 250 according to the separation distance like the second embodiment. Also, in another embodiment in which the vehicle 100v runs under autonomous control, for example, the system 10 may be provided with a server 200.
[0062] D. Other embodiments: (D1) In the above embodiment, the moving body information includes the model information Ti, but it does not have to include the model information Ti. For example, the moving body information may include only the body color information Ci.
[0063] (D2) In the above embodiment, the manufacturing information of the moving body is used as the model information Ti of the moving body. However, the manufacturing information does not have to be used as the model information Ti.
[0064] (D3) In the above embodiment, the matching unit 220 changes the density of the point cloud in matching in accordance with the moving body information, but it is not necessary to change the density of the point cloud in accordance with the moving body information.
[0065] (D4) In each of the above embodiments, the vehicle 100 may have a configuration capable of moving by unmanned driving, and may be in the form of a platform having the configuration described below, for example. Specifically, the vehicle 100 may have at least a control device that controls the running of the vehicle 100 and an actuator of the vehicle 100v in order to perform the three functions of "running", "turning", and "stopping" by unmanned driving. When the vehicle 100 acquires information from the outside for unmanned driving, the vehicle 100 may further have a communication device. That is, the vehicle 100 that can move by unmanned driving may not have at least a part of the interior parts such as a driver's seat and a dashboard, may not have at least a part of the exterior parts such as a bumper and a fender, and may not have a body shell. In this case, the remaining parts such as a body shell may be attached to the vehicle 100 before the vehicle 100 is shipped from the factory, or the remaining parts such as a body shell may be attached to the vehicle 100 after the vehicle 100 is shipped from the factory in a state in which the remaining parts such as the body shell are not attached to the vehicle 100. Each part may be attached from any direction, such as the upper side, lower side, front side, rear side, right side, or left side of the vehicle 100, and may be attached from the same direction or from different directions. Note that the position of the platform configuration may be determined in the same manner as the vehicle 100 in the first embodiment. When performing matching in the above embodiment for the platform configuration, the mobile body information may be information that can identify the external characteristics of the platform configuration, and may be information that can identify, for example, whether or not each of the exterior parts described above is attached. For example, manufacturing information of the mobile body may be used as such mobile body information.
[0066] (D5) In each of the above embodiments, in the system 10, various functional units such as the three-dimensional point cloud data acquisition unit 210, the vehicle information acquisition unit 215, the matching unit 220, the matching condition determination unit 225, the estimation unit 230, and the command generation unit 240 may be provided in the vehicle 100. In this case, as described in the third embodiment, all of the three-dimensional point cloud data acquisition unit 210, the vehicle information acquisition unit 215, the matching unit 220, the matching condition determination unit 225, the estimation unit 230, and the command generation unit 240 may be provided in the vehicle 100, or some of these functional units may be provided in the vehicle 100. In addition, in the system 10, some or all of these functional units may be provided in devices external to the server 200 and the vehicle 100.
[0067] (D6) In the first embodiment, the processes from obtaining vehicle position information to generating a driving control signal are executed by the server 200. In contrast, at least a part of the processes from obtaining vehicle position information to generating a driving control signal may be executed by the vehicle 100. For example, the following forms (1) to (3) may be used.
[0068] (1) The server 200 may acquire vehicle position information, determine a target position to which the vehicle 100 should next head, and generate a route from the current location of the vehicle 100 represented in the acquired vehicle position information to the target position. The server 200 may generate a route to a target position between the current location and the destination, or may generate a route to the destination. The server 200 may transmit the generated route to the vehicle 100. The vehicle 100 may generate a driving control signal so that the vehicle 100 drives on the route received from the server 200, and may control an actuator of the vehicle 100v using the generated driving control signal.
[0069] (2) The server 200 may acquire vehicle position information and transmit the acquired vehicle position information to the vehicle 100. The vehicle 100 may determine a target position to which the vehicle 100 should next head, generate a route from the current location of the vehicle 100 represented in the received vehicle position information to the target position, generate a driving control signal so that the vehicle 100 travels on the generated route, and control an actuator of the vehicle 100v using the generated driving control signal.
[0070] (3) In the above-mentioned (1) and (2) modes, the vehicle 100 may be equipped with an internal sensor, and a detection result output from the internal sensor may be used for at least one of generating a route and generating a driving control signal. The internal sensor is a sensor equipped in the vehicle 100. Specifically, the internal sensor may include, for example, a camera, a LiDAR, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, a gyro sensor, and the like. For example, in the above-mentioned (1) mode, the server 200 may acquire the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating a route. In the above-mentioned (1) mode, the vehicle 100 may acquire the detection result of the internal sensor, and reflect the detection result of the internal sensor in the driving control signal when generating a driving control signal. In the above-mentioned (2) mode, the vehicle 100 may acquire the detection result of the internal sensor, and reflect the detection result of the internal sensor in the route when generating a route ... driving control signal when generating a driving control signal.
[0071] (D7) In the above third embodiment, an internal sensor may be mounted on the vehicle 100v, and a detection result output from the internal sensor may be used for at least one of generating a route and generating a driving control signal. For example, the vehicle 100v may acquire the detection result of the internal sensor, and when generating a route, may reflect the detection result of the internal sensor in the route. The vehicle 100v may acquire the detection result of the internal sensor, and when generating a driving control signal, may reflect the detection result of the internal sensor in the driving control signal.
[0072] (D8) In the first embodiment, the server 200 automatically generates a driving control signal to be transmitted to the vehicle 100. In contrast, the server 200 may generate a driving control signal to be transmitted to the vehicle 100 according to the operation of an external operator located outside the vehicle 100. For example, an external operator may operate a control device including a display for displaying an image output from an external sensor, a steering wheel for remotely controlling the vehicle 100, an accelerator pedal, a brake pedal, and a communication device for communicating with the server 200 by wired communication or wireless communication, and the server 200 may generate a driving control signal according to the operation applied to the control device. Hereinafter, driving the vehicle 100 by such control is also referred to as "remote manual driving". In a form in which remote manual driving is performed, for example, the estimation results of the position and orientation of the vehicle 100 estimated by the estimation unit 230 may be displayed on a display included in the control device. In this case, the estimation results of the position and orientation may be displayed on the display by, for example, characters or symbols, or on a map.
[0073] (D9) The vehicle 100 may be manufactured by combining a plurality of modules. A module means a unit composed of a plurality of parts grouped according to the location or function of the vehicle 100. For example, the platform of the vehicle 100 may be manufactured by combining a front module that constitutes the front part of the platform, a central module that constitutes the central part of the platform, and a rear module that constitutes the rear part of the platform. The number of modules that constitute the platform is not limited to three, and may be two or less, or four or more. In addition to or instead of the parts that constitute the platform, parts that constitute parts of the vehicle 100 that are different from the platform may be modularized. The various modules may include any exterior parts such as a bumper or a grille, or any interior parts such as a seat or a console. In addition, not limited to the vehicle 100, any type of moving body may be manufactured by combining a plurality of modules. Such a module may be manufactured, for example, by joining a plurality of parts by welding or a fastener, or may be manufactured by integrally molding at least a part of the parts that constitute the module as one part by casting. The molding method for integrally molding a single component, particularly a relatively large component, is also called gigacast or megacast. For example, the front module, the center module, and the rear module may be manufactured using gigacast.
[0074] (D10) Transporting the vehicle 100 by using the driving of the vehicle 100 through unmanned driving is also called "self-propelled transport." Also, a configuration for realizing self-propelled transport is also called a "vehicle remote-controlled autonomous driving transport system." Also, a production method for producing the vehicle 100 using self-propelled transport is also called "self-propelled production." In self-propelled production, for example, at a factory where the vehicle 100 is manufactured, at least a part of the transportation of the vehicle 100 is realized by self-propelled transport.
[0075] (D11) In each of the above embodiments, some or all of the functions and processes implemented by software may be implemented by hardware. Also, some or all of the functions and processes implemented by hardware may be implemented by software. As hardware for implementing the various functions in each of the above embodiments, various circuits such as integrated circuits and discrete circuits may be used.
[0076] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention column can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]
[0077] 10,10v...system, 100,100v...vehicle, 110,110v...vehicle control device, 111,111v...processor, 112,112v...memory, 113...input / output interface, 114...internal bus, 115,115v...vehicle control unit, 116...position information acquisition unit, 120...actuator group, 130...communication device, 140...GNSS receiver, 200...server, 201...processor 202...memory, 203...input / output interface, 204...internal bus, 205...communication device, 210...3D point cloud data acquisition unit, 215...vehicle information acquisition unit, 220...matching unit, 225...matching condition determination unit, 230...estimation unit, 240...command generation unit, 250...reference point cloud data, 251...model point cloud data, 260...correction data, 300...distance measurement device, 400...process management device
Claims
1. A control device used to generate a control command for moving a moving body configured to be movable by unmanned operation in a factory that manufactures the moving body, the control device comprising: a matching unit that performs matching using measurement point cloud data, which is three-dimensional point cloud data of the moving object acquired using a distance measuring device, and reference point cloud data, which represents reference three-dimensional point cloud data, in order to generate the control command; a mobile object information acquisition unit that acquires mobile object information capable of identifying an external appearance feature of the mobile object, the mobile object information acquisition unit acquires the mobile object information from a process management device that executes management of a manufacturing process of the mobile object in the factory; The matching unit performs the matching using the reference point cloud data corresponding to the moving object information, The moving object information is information that can identify at least one of a paint condition of the moving object at the time when the matching is performed and whether or not an exterior part is attached to the moving object. Control device.
2. The control device according to claim 1 , The matching unit changes at least one of the density of the points in the measurement point cloud data and the density of the points in the reference point cloud data in accordance with the moving body information.
3. The control device according to claim 2, A control device in which the matching unit changes at least one of the density of the point cloud in the measurement point cloud data and the density of the point cloud in the reference point cloud data depending on the distance between the ranging device and the moving body.
4. A method for generating a control command for moving a moving object configured to be movable by unmanned operation in a factory for manufacturing the moving object, comprising: A step of performing matching using measurement point cloud data, which is three-dimensional point cloud data of the moving object acquired by a distance measuring device, and reference point cloud data representing reference three-dimensional point cloud data in order to generate the control command; acquiring moving body information capable of identifying an external characteristic of the moving body; In the step of acquiring the mobile object information, the mobile object information is acquired from a process management device that executes management of a manufacturing process of the mobile object in the factory; In the step of performing matching, the matching is performed using the reference point cloud data corresponding to the moving body information; The moving object information is information that can identify at least one of a paint condition of the moving object at the time when the matching is performed and whether or not an exterior part is attached to the moving object. method.
5. A moving body, a matching unit that performs matching using measurement point cloud data, which is three-dimensional point cloud data of the moving body acquired using a distance measuring device, and reference point cloud data representing reference three-dimensional point cloud data, in order to generate a control command for moving the moving body in a factory that manufactures the moving body configured to be movable by unmanned operation; a mobile object information acquisition unit that acquires mobile object information capable of identifying an external appearance feature of the mobile object, the mobile object information acquisition unit acquires the mobile object information from a process management device that executes management of a manufacturing process of the mobile object in the factory; The matching unit performs the matching using the reference point cloud data corresponding to the moving object information, The moving object information is information that can identify at least one of a paint condition of the moving object at the time when the matching is performed and whether or not an exterior part is attached to the moving object. system.
Citation Information
Patent Citations
Method and system for tracking vehicle
JP2001229486A
Vehicle type discrimination device and vehicle type discrimination method
JP2017220076A
Method for operating a vehicle and method for operating a manufacturing system
JP2017538619A
Method, apparatus, and device for calibrating intelligent roadside unit
JP2020034554A
