Control device

By determining the scope of use in vehicle self-harvesting, and using only part of the three-dimensional point group data to speculate on the position and direction, the control cycle delay problem caused by long processing time is solved, and faster and more accurate control of the moving body is achieved.

CN119065278APending Publication Date: 2024-12-03TOYOTA JIDOSHA KK
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410467963.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-10-30
Filing Date
2024-04-18
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

During the self-handling process of the vehicle, when using three-dimensional point group data to estimate the position and direction of the moving body, a long processing time may lead to a delay in the control cycle.

Method used

By determining the usage range, only a portion of the three-dimensional point cluster data is used to infer the position and direction of the moving body, reduce the processing load, and limit the usage range under high load conditions to reduce period delay.

Benefits of technology

It effectively reduces the cycle delay in the control of the mobile body, improves the inferred speed and accuracy, and ensures the stable driving of the vehicle.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065278A_ABST
    Figure CN119065278A_ABST
Patent Text Reader

Abstract

Provided is a control device configured to generate a control command for controlling a moving body using three-dimensional point group data of the moving body measured by a distance measurement device, the control device being provided with: a range determination unit configured to determine a use range, the use range is a range including only a part of the three-dimensional point group data; and an estimation unit configured to estimate at least one of a position and a direction of the moving body using the three-dimensional point group data of the use range.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to a control device for a mobile object. Background Art

[0002] In the manufacturing process of a vehicle, there is known a technology for allowing the vehicle to be transported by itself (for example, Japanese Patent Publication No. 2017-538619). Summary of the invention

[0003] When a mobile object such as a vehicle is moved by self-transportation, a process of estimating the position and direction of the mobile object is performed. The position and direction of the mobile object can be estimated using three-dimensional point group data obtained by a distance measuring device such as a camera or radar. In order to stabilize the driving control of the mobile object, this estimation process needs to be calculated at a faster cycle. However, since a long processing time is required to process a large amount of three-dimensional point group data, there is a possibility that the control of the mobile object will have a cycle delay.

[0004] The present disclosure can be implemented as the following aspects.

[0005] According to one embodiment of the present disclosure, a control device is provided, which is configured to generate a control instruction for controlling the moving body using three-dimensional point group data of the moving body measured by a distance measuring device. The control device comprises: a range determination unit, which is configured to determine a use range, which is a range that only includes a part of the three-dimensional point group data; and an estimation unit, which is configured to estimate at least one of the position and direction of the moving body using the three-dimensional point group data of the use range.

[0006] According to this control device, since at least one of the position and the direction of the moving object is estimated using only a part of the three-dimensional point group data, the possibility of a cycle delay occurring in the control of the moving object can be reduced.

[0007] In the control device, the use range may be determined to include a characteristic portion, which is a portion in an observation area of ​​the moving body observable by the distance measuring device and in which a density of characteristic quantities is higher than an average value in the observation area.

[0008] According to this control device, an appropriate usage range can be set.

[0009] In the control device, the characteristic portion may be determined based on the model of the moving body, the detection direction of the moving body by the distance measuring device, and the distance between the distance measuring device and the moving body.

[0010] According to this control device, it is possible to accurately determine an effective use range for estimating at least one of the position and direction of the moving object.

[0011] In the above-mentioned control device, the above-mentioned range determination unit may also be configured to execute the processing of determining the above-mentioned usage range when the processing load of the above-mentioned control device is greater than a preset reference value, and may be configured not to execute the processing of determining the above-mentioned usage range when the above-mentioned processing load is below the above-mentioned reference value.

[0012] According to this control device, when the processing load of the control device is high, the possibility of a cycle delay occurring in the control of the moving body can be reduced.

[0013] In the control device, at least one of a detection direction of the moving object and a distance between the distance measuring device and the moving object may be determined using the three-dimensional point group data.

[0014] In the control device, the estimation unit may be configured to estimate at least one of a position and a direction of the moving object by matching the three-dimensional point group data of the use range with template data. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Hereinafter, features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the accompanying drawings, in which like reference numerals represent like elements, and in which:

[0016] Figure 1 It is a conceptual diagram showing the structure of the remote control system in the embodiment.

[0017] Figure 2 It is a block diagram showing the configuration of a vehicle and a remote control device in the first embodiment.

[0018] Figure 3 This is a flowchart showing the processing procedure of the remote control in the first embodiment.

[0019] Figure 4 It is an explanatory diagram showing a process of determining a use range of three-dimensional point group data.

[0020] Figure 5 It is a block diagram showing the configuration of a vehicle and a remote control device in the second embodiment.

[0021] Figure 6 This is a flowchart showing a process procedure of vehicle control in the second embodiment. DETAILED DESCRIPTION

[0022] First embodiment

[0023] Figure 11 is a conceptual diagram showing the structure of the remote control system 10 in the embodiment. The remote control system 10 includes: one or more vehicles 100 as mobile bodies; a remote control device 200 that generates a control command for remotely controlling the vehicle 100 and sends it to the vehicle 100; a plurality of distance measuring devices 300 that measure the three-dimensional point group data of the vehicle 100; and a process management device 400 that manages the manufacturing process of the vehicle 100. In the first embodiment, the remote control device 200 is equivalent to the "control device" of the present disclosure.

[0024] The vehicle 100 is preferably a pure electric vehicle (BEV). In addition, the mobile object is not limited to an electric vehicle, and may be, for example, a gasoline vehicle, a hybrid vehicle, or a fuel cell vehicle. The mobile object is not limited to the vehicle 100, and may be, for example, an electric vertical take-off and landing vehicle (so-called flying vehicle).

[0025] In the present disclosure, a "mobile body" refers to an object that can move. A vehicle may be a vehicle that travels on wheels or a vehicle that travels on tracks, such as a passenger car, a truck, a bus, a two-wheeled vehicle, a four-wheeled vehicle, a tank, a construction vehicle, etc. In the case where the mobile body is other than a vehicle, the expressions "vehicle" and "car" in the present disclosure can be appropriately replaced with "mobile body", and the expression "travel" can be appropriately replaced with "move".

[0026] The vehicle 100 is configured to be able to travel by unmanned driving. "Unmanned driving" refers to driving that does not rely on the driving operation of the passengers. The driving operation refers to an operation related to at least any one of "driving", "turning" and "stopping" of the vehicle 100. Unmanned driving is achieved by automatic or manual remote control using a device located outside the vehicle 100, or autonomous control of the vehicle 100. In the vehicle 100 that travels by unmanned driving, there may also be passengers who do not perform driving operations. Passengers who do not perform driving operations include, for example, people who only sit in the seats of the vehicle 100, and people who perform operations different from driving operations such as assembly, inspection, and switch operations while riding in the vehicle 100. In addition, driving based on the driving operations of the passengers is sometimes referred to as "manned driving".

[0027] In this specification, “remote control” includes “full remote control” in which all actions of the vehicle 100 are completely determined from outside the vehicle 100, and “partial remote control” in which part of the actions of the vehicle 100 are determined from outside the vehicle 100. In addition, “autonomous control” includes “full autonomous control” in which the vehicle 100 autonomously controls its own actions without receiving any information from a device outside the vehicle 100, and “partial autonomous control” in which the vehicle 100 autonomously controls its own actions using information received from a device outside the vehicle 100.

[0028] In the present embodiment, remote control of the vehicle 100 is performed in a factory that manufactures the vehicle 100. The factory includes a first location PL1 and a second location PL2. The first location PL1 is, for example, a location where the vehicle 100 is assembled, and the second location PL2 is, for example, a location where the vehicle 100 is inspected. The first location PL1 and the second location PL2 are connected by a travel road SR on which the vehicle 100 can travel. Any position in the factory is represented by an xyz coordinate value of a reference coordinate system Σr.

[0029] A plurality of distance measuring devices 300 are provided around the travel road SR, with the vehicle 100 as the measurement object. The remote control device 200 can use the three-dimensional point group data measured by each distance measuring device 300 to obtain the relative position and direction of the vehicle 100 with respect to the target route TR in real time. As the distance measuring device 300, a camera or a laser radar (LiDAR: Light Detection And Ranging) can be used. In particular, LiDAR is preferred in terms of obtaining high-precision three-dimensional point group data. The position of each distance measuring device 300 is fixed, and the relative relationship between the reference coordinate system Σr and the device coordinate system of each distance measuring device 300 is known. The coordinate conversion matrix for converting the coordinate values ​​of the reference coordinate system Σr and the coordinate values ​​of the device coordinate system of each distance measuring device 300 is pre-stored in the remote control device 200.

[0030] The remote control device 200 generates a control command for causing the vehicle 100 to travel along the target route TR, and sends the control command to the vehicle 100. The vehicle 100 travels according to the received control command. Therefore, the remote control system 10 can move the vehicle 100 from the first place PL1 to the second place PL2 by remote control without using a transport device such as a crane or a conveyor.

[0031] Figure 21 is a block diagram showing the structure of the vehicle 100 and the remote control device 200. The vehicle 100 is provided with a vehicle control device 110 for controlling each part of the vehicle 100, an actuator group 120 driven under the control of the vehicle control device 110, a communication device 130 for communicating with the remote control device 200 through wireless communication, and a GPS receiver 140 for obtaining the position information of the vehicle 100. In the present embodiment, the actuator group 120 includes an actuator of a driving device for accelerating the vehicle 100, an actuator of a steering device for changing the traveling direction of the vehicle 100, and an actuator of a braking device for decelerating the vehicle 100. The driving device includes a battery, a driving motor driven by the power of the battery, and a driving wheel rotated by the driving motor. The actuator of the driving device includes a driving motor. In addition, the actuator group 120 may also include an actuator for swinging the wiper of the vehicle 100, an actuator for opening and closing the electric window of the vehicle 100, and the like.

[0032] The vehicle control device 110 is composed of a computer including a processor 111, a memory 112, an input / output interface 113, and an internal bus 114. The processor 111, the memory 112, and the input / output interface 113 are connected in a bidirectional communication manner via the internal bus 114. The input / output interface 113 is connected to an actuator group 120, a communication device 130, and a GPS receiver 140.

[0033] In the present embodiment, the processor 111 functions as a vehicle control unit 115 and a position information acquisition unit 116 by executing a program PG1 pre-stored in the memory 112. The vehicle control unit 115 controls the actuator group 120. When a driver is riding in the vehicle 100, the vehicle control unit 115 controls the actuator group 120 according to the operation of the driver, thereby enabling the vehicle 100 to travel. The vehicle control unit 115 may also control the actuator group 120 according to a control command sent from the remote control device 200 to enable the vehicle 100 to travel, regardless of whether the driver is riding in the vehicle 100. The position information acquisition unit 116 uses the GPS receiver 140 to acquire position information indicating the current location of the vehicle 100. However, the position information acquisition unit 116 and the GPS receiver 140 may be omitted.

[0034] The remote control device 200 is composed of a computer including a processor 201, a memory 202, an input / output interface 203, and an internal bus 204. The processor 201, the memory 202, and the input / output interface 203 are connected in a bidirectional communication manner via the internal bus 204. The input / output interface 203 is connected to a communication device 205 for communicating with the vehicle 100, the distance measuring device 300, and the process management device 400 by wireless communication.

[0035] In the present embodiment, the processor 201 functions as a three-dimensional point group data acquisition unit 210 , a range determination unit 220 , an estimation unit 230 , and a remote control command generation unit 240 by executing a program PG2 pre-stored in the memory 202 .

[0036] The three-dimensional point group data acquisition unit 210 acquires the three-dimensional point group data measured by the distance measuring device 300 . The three-dimensional point group data is data indicating the three-dimensional position of the point group measured by the distance measuring device 300 .

[0037] The range determination unit 220 determines a range including only a part of the three-dimensional point group data, that is, a use range. This determination method will be described later.

[0038] The estimation unit 230 estimates the position and direction of the vehicle 100 using the three-dimensional point group data of the use range determined by the range determination unit 220. In the present embodiment, the estimation unit 230 estimates the position and direction of the vehicle 100 by performing template matching using a template point group TP stored in the memory 202. In addition, when the three-dimensional point group data cannot be used, the estimation unit 230 can estimate the position and direction of the vehicle 100 using the driving history of the vehicle 100 and the position information measured by the GPS receiver 140 mounted on the vehicle 100. The estimation unit 230 can also estimate only one of the position and direction of the vehicle 100. In this case, the other of the position and direction of the vehicle 100 is determined using the driving history of the vehicle 100, etc.

[0039] The position and direction of the vehicle are also referred to as “vehicle position information.” In the present embodiment, the vehicle position information includes the position and direction of the vehicle 100 in the reference coordinate system of the factory.

[0040] The remote control command generation unit 240 generates a control command for remote control using the estimated position and direction of the vehicle 100 and transmits the control command to the vehicle 100. The control command is a command for causing the vehicle 100 to travel along the target route TR stored in the memory 202. The control command can be generated as a command including a driving force or a braking force and a steering angle. Alternatively, the control command can also be generated as a command including at least one of the position and direction of the vehicle 100 and the future travel route.

[0041] In this embodiment, the control command includes the acceleration and steering angle of the vehicle 100 as parameters. In other embodiments, the control command may include the speed of the vehicle 100 as a parameter instead of the acceleration of the vehicle 100 or in addition to the acceleration of the vehicle 100.

[0042] The process management device 400 manages the entire manufacturing process of the vehicle 100 in the factory. For example, when a vehicle 100 starts traveling along the target route TR, individual information such as the identification number and model of the vehicle 100 is sent from the process management device 400 to the remote control device 200. The position of the vehicle 100 measured by the remote control device 200 is also sent to the process management device 400. In addition, the function of the process management device 400 can also be installed in the same device as the remote control device 200.

[0043] The remote control device 200 is also referred to as a “server”, and the distance measuring device 300 is also referred to as an “external sensor”. In addition, the control command is also referred to as a “driving control signal”, the target route TR is also referred to as a “reference path”, and the reference coordinate system is also referred to as a “global coordinate system”.

[0044] Figure 3 1 is a flowchart showing the processing steps of the remote control in the first embodiment. The processing of the remote control device 200 is executed at a fixed cycle. Alternatively, the remote control device 200 may be executed every time the 3D point group data acquisition unit 210 acquires new 3D point group data from the distance measuring device 300 responsible for measuring the vehicle 100 to be controlled. Figure 3 The three-dimensional point cloud data acquisition unit 210 may perform a pre-processing of removing background point cloud data representing a stationary object from the newly acquired three-dimensional point cloud data.

[0045] In step S10, the range determination unit 220 determines whether the processing load of the remote control device 200 is greater than a reference value. The processing load of the remote control device 200 is, for example, the utilization rate of the processor 201. The reference value of the processing load is set to a value that can determine and realize a desired processing speed for stable driving control in consideration of the speed of the vehicle 100. Alternatively, the reference value of the processing load may be set to a predetermined fixed value.

[0046] When the processing load is below the reference value, the process proceeds to step S50. In step S50, the estimation unit 230 estimates the position and direction of the vehicle 100 by performing matching of all the three-dimensional point group data with the template point group TP. In addition, the phrase "all the three-dimensional point group data" refers to the three-dimensional point group data for which the use range is not determined. As described above, the three-dimensional point group data may also be pre-processed to remove the background point group data.

[0047] On the other hand, when the processing load is greater than the reference value, the process proceeds to step S20. In step S20, the range determination unit 220 determines a use range that is a range including only a part of the three-dimensional point group data.

[0048] Figure 4 FIG. 2 is an explanatory diagram showing a process of determining a usage range of three-dimensional point group data. Figure 4 As shown above, when the processing load is below the reference value, the entire range Ra of the three-dimensional point group data measured by the distance measuring device 300 is used directly. Figure 4 As shown below, when the processing load is greater than the reference value, only the three-dimensional point group data within the use range Ru that is smaller than the entire range Ra of the three-dimensional point group data measured by the ranging device 300 is used. The use range Ru refers to a range narrower than the entire range Ra of the field of view of the ranging device 300. The number of point groups measured in this use range Ru is less than the number of point groups measured in the entire range Ra. The reason for making the use range Ru narrower than the entire range Ra is that if only a part of the three-dimensional point group data is used, the estimation speed of the position and direction of the vehicle 100 can be increased, and the possibility of generating a periodic delay in remote control can be reduced.

[0049] The use range Ru is preferably determined to include a characteristic portion CP in an observation area of ​​the vehicle 100 that can be observed by the distance measuring device 300 and in which the density of characteristic quantities is higher than the average value in the observation area. Figure 4 In the example of , a prominent part of the back side of the vehicle 100 is represented as a characteristic part CP. The characteristic amount is, for example, an edge amount, which is the amount of characteristic points extracted by applying a characteristic point extraction operator to the image of the vehicle 100. The observation area of ​​the vehicle 100 is, for example, the entire back side of the vehicle 100. If the use range Ru is determined in such a way as to include the characteristic part CP with a large characteristic amount, a point group suitable for estimating the position and direction of the vehicle 100 can be used.

[0050] The range of the characteristic portion CP for determining the use range Ru can be determined by, for example, any of the following methods.

[0051] Method M1 for determining characteristic part CP

[0052] The characteristic portion CP is determined based on the model of the vehicle 100 , the detection direction of the vehicle 100 by the distance measuring device 300 , and the distance between the distance measuring device 300 and the vehicle 100 .

[0053] The model of the vehicle 100 can be determined based on the individual information sent from the process management device 400 to the remote control device 200. Alternatively, it can also be set that the remote control device 200 obtains the model of the vehicle 100 through wireless communication with the vehicle 100. In addition, the model of the vehicle 100 can be information indicating the appearance shape of the vehicle 100, and can also be a serial number within the company. The detection direction of the vehicle 100 by the ranging device 300 can be determined based on the position of the part where the point group is dense in the three-dimensional point group data. For example, a bounding box surrounding the part where the point group is dense can also be determined, and the direction from the ranging device 300 to the reference position of the bounding box is determined as the "detection direction". The distance between the ranging device 300 and the vehicle 100 can be determined, for example, as the distance from the ranging device 300 to the reference position of the bounding box. The shape and position of the characteristic part CP are determined according to the model of the vehicle, and the positional relationship between the bounding box of the point group and the characteristic part CP is also determined according to the model of the vehicle, so this information is stored as vehicle information VI for each model. Therefore, if the model of the vehicle 100 , the detection direction of the vehicle 100 by the distance measuring device 300 , and the distance between the distance measuring device 300 and the vehicle 100 are known, the position and shape of the characteristic portion CP can be determined based on these.

[0054] Method M2 for determining characteristic part CP

[0055] The characteristic portion CP is determined based on the density of the point group of the three-dimensional point group data.

[0056] The high-density portion of the point group in the three-dimensional point group data also has many feature quantities, so the high-density portion of the point group can be determined as the characteristic portion CP. For example, a bounding box is determined to surround the dense portion of the point group, and the bounding box is divided into a plurality of voxels (micro boxes), and the density of the point group is calculated for each voxel. The portion of the voxel set whose density of the point group is greater than a predetermined reference value can be determined as the characteristic portion CP.

[0057] The usage range Ru is set to include the characteristic portion CP. Figure 4 In the example of , the use range Ru is set to a range larger than the range circumscribed with the characteristic portion CP. Thus, even when the accuracy of the determination process of the position and shape of the characteristic portion CP is low, the use range Ru can be set in a manner that reliably includes the characteristic portion CP.

[0058] In step S30 , the estimation unit 230 extracts the three-dimensional point group data of the use range Ru. In step S40 , the estimation unit 230 estimates the position and direction of the vehicle 100 by matching the three-dimensional point group data of the use range Ru with the template point group TP.

[0059] In step S60, the remote control command generation unit 240 generates a control command value using the position and direction of the vehicle 100 estimated in step S40 or step S50, and transmits the control command value to the vehicle 100. The details of step S60 are as follows.

[0060] In step S60, the remote control command generation unit 240 first uses the vehicle position information including the position and direction of the vehicle 100 and the target route TR to determine the target position to which the vehicle 100 should go next. The remote control command generation unit 240 determines the target position on the target route TR ahead of the current position of the vehicle 100, and generates a control command value for driving the vehicle 100 toward the target position. In the present embodiment, the control command value includes the acceleration and steering angle of the vehicle 100 as parameters. The remote control command generation unit 240 calculates the driving speed of the vehicle 100 according to the change of the position of the vehicle 100, and compares the calculated driving speed with the target speed. As a whole, the remote control command generation unit 240 determines the acceleration to accelerate the vehicle 100 when the driving speed is lower than the target speed, and determines the acceleration to decelerate the vehicle 100 when the driving speed is higher than the target speed. In addition, when the vehicle 100 is on the target route TR, the remote control command generation unit 240 determines the steering angle and acceleration in a manner that the vehicle 100 does not deviate from the target route TR. When the vehicle 100 is not on the target route TR, in other words, when the vehicle 100 deviates from the target route TR, the remote control command generation unit 240 determines the steering angle and the acceleration so as to restore the vehicle 100 to the target route TR. In other embodiments, the control command value may include the speed of the vehicle 100 as a parameter instead of the acceleration of the vehicle 100 or in addition to the acceleration of the vehicle 100. The control command value generated in this way is transmitted from the remote control device 200 to the vehicle 100.

[0061] The control process performed by the processor 111 of the vehicle 100 includes steps S70 and S80. In step S70, the vehicle control unit 115 waits until the control command value is obtained from the remote control device 200. When the control command value is obtained, the process enters step S80, and the vehicle control unit 115 controls the actuator group 120 according to the obtained control command value. According to the remote control system 10 of this embodiment, the vehicle 100 can be driven by remote control, and the vehicle 100 can be moved along the target route TR without using a transport device such as a crane or a conveyor.

[0062] In the above Figure 3In the processing steps of , the use range Ru of the three-dimensional point group data is limited only when the processing load of the remote control device 200 is large, but the use range Ru may be always limited regardless of the processing load. In this case, step S10 and step S50 are omitted.

[0063] As described above, in the first embodiment, a use range Ru that includes only a portion of the three-dimensional point group data is determined, and the three-dimensional point group data of the use range Ru is used to estimate at least one of the position and direction of the moving object, thereby reducing the possibility of periodic delays in remote control.

[0064] In the above-mentioned embodiment, the vehicle 100 only needs to have a structure that can be moved by remote control, for example, it can also be a platform with the structure described below. Specifically, in order to perform the three functions of "driving", "turning" and "stopping" by remote control, the vehicle 100 only needs to have at least a vehicle control unit 115 and a communication device 150. That is, the vehicle 100 that can be moved by remote control may not be equipped with at least a part of the interior parts such as the driver's seat and the instrument panel, may not be equipped with at least a part of the exterior parts such as the bumper and the fender, and may not be equipped with a body shell. In this case, the remaining parts such as the body shell can be installed on the vehicle 100 before the vehicle 100 is shipped from the factory, or the remaining parts such as the body shell can be installed on the vehicle 100 after the vehicle 100 is shipped from the factory in a state where the remaining parts such as the body shell are not installed on the vehicle 100. In addition, for the platform, the position can be determined in the same way as the vehicle 100 in each embodiment.

[0065] Second embodiment

[0066] Figure 5 2 is a block diagram showing the configuration of the vehicle 100 and the remote control device 200 in the second embodiment. Figure 2 The differences between the structures of the first embodiment shown are mainly the following three points.

[0067] In addition to the functions of the processor 111 of the vehicle 100 , the functions of a three-dimensional point group data acquisition unit 121 , a range identification unit 122 , an estimation unit 123 , and a control command generation unit 124 are provided.

[0068] The memory 112 of the vehicle 100 stores the template point group TP, the vehicle information VI, and the target route TR.

[0069] The functions of the three-dimensional point group data acquisition unit 210 , the range determination unit 220 , the estimation unit 230 , and the remote control command generation unit 240 are omitted from the functions of the processor 201 of the remote control device 200 .

[0070] The functions of the three-dimensional point group data acquisition unit 121, the range determination unit 122, the estimation unit 123 and the control instruction generation unit 124 are substantially the same as those of the three-dimensional point group data acquisition unit 210, the range determination unit 220, the estimation unit 230 and the remote control instruction generation unit 240, respectively, and therefore their descriptions are omitted.

[0071] In the second embodiment, the vehicle 100 performs the following processing: determining a range that includes only a portion of the three-dimensional point group data, namely, a use range Ru, and using the three-dimensional point group data of the use range Ru to estimate at least one of the position and direction of the vehicle 100. That is, in the second embodiment, the vehicle control device 110 of the vehicle 100 is equivalent to the "control device" of the present disclosure.

[0072] Figure 6 This is a flowchart showing a process procedure of vehicle control in the second embodiment. Figure 6 Steps S110 to S160 correspond to Figure 3 However, in step S160, since the control command value is generated by the control command generator 124 of the vehicle 100, the control of the actuator group 120 of the vehicle 100 is executed without transmitting and receiving the control command value between the remote control device 200 and the vehicle 100.

[0073] The template point group TP, vehicle information VI, and target route TR are stored in the memory 112 of the vehicle 100 before the vehicle 100 starts traveling along the target route TR. These data may be supplied from the remote control device 200 or the process management device 400, or may be written into the memory 112 of the vehicle 100 by other means.

[0074] As described above, in the second embodiment, as in the first embodiment, a range including only a portion of the three-dimensional point group data, namely, a usage range Ru, is determined, and the three-dimensional point group data of the usage range Ru is used to infer at least one of the position and direction of the moving body, thereby reducing the possibility of periodic delays in the control of the vehicle.

[0075] Other Implementations

[0076] In various embodiments described below, "server 200" means remote control device 200, and "external sensor" means distance measuring device 300. In addition, "driving control signal" means control command, "reference path" means target route TR, and "global coordinate system" means reference coordinate system Σr.

[0077] In each of the above-mentioned embodiments, the external sensor is a laser radar (LiDAR: Light Detection And Ranging). In contrast, the external sensor may not be a LiDAR, and may be a camera, for example. In the case where the external sensor is a camera, the server 200, for example, measures the shape of the vehicle 100 from the captured image, calculates the coordinates of the positioning points of the vehicle 100 in the coordinate system of the captured image, that is, the local coordinate system, and converts the calculated coordinates into coordinates in the global coordinate system, thereby obtaining the position of the vehicle 100. The shape of the vehicle 100 contained in the captured image can be measured, for example, by inputting the captured image into a measurement model that utilizes artificial intelligence. The measurement model is prepared, for example, inside or outside the system 10 and pre-stored in the memory of the server 200. As a measurement model, for example, a machine learning model that has been trained to achieve either semantic segmentation or instance segmentation can be cited. As the machine learning model, for example, a convolutional neural network (hereinafter referred to as CNN) trained by supervised learning using a learning data set can be used. The learning data set includes, for example, a plurality of training images including the vehicle 100 and labels indicating whether each region in the training image is a region representing the vehicle 100 or a region other than the vehicle 100. When learning the CNN, it is preferred to update the parameters of the CNN by backpropagation (error backpropagation method) to reduce the error between the output result of the measurement model and the label. The server 200 can obtain the direction of the vehicle 100 by, for example, using the optical flow method to infer the direction of the movement vector of the vehicle 100 calculated based on the position change of the feature point of the vehicle 100 between the frames of the captured image.

[0078] In the first embodiment, the server 200 performs the processing from the acquisition of the vehicle position information including the position and direction of the vehicle 100 to the generation of the travel control signal. In contrast, at least a part of the processing from the acquisition of the vehicle position information to the generation of the travel control signal may be performed by the vehicle 100. For example, the following methods (1) to (3) may be used.

[0079] (1) The server 200 may also obtain vehicle position information, determine the target position to which the vehicle 100 should go next, and generate a path from the current position of the vehicle 100 indicated by the obtained vehicle position information to the target position. The server 200 may generate a path to the target position between the current position and the destination, or may generate a path to the destination. The server 200 may also send the generated path to the vehicle 100. The vehicle 100 may also generate a travel control signal in such a way that the vehicle 100 travels on the path received from the server 200, and use the generated travel control signal to control the actuator of the vehicle 100.

[0080] (2) The server 200 may also obtain vehicle position information and transmit the obtained vehicle position information to the vehicle 100. The vehicle 100 may also determine a target position to which the vehicle 100 should go next, generate a path from the current position of the vehicle 100 indicated by the received vehicle position information to the target position, generate a travel control signal to make the vehicle 100 travel on the generated path, and control the actuator of the vehicle 100 using the generated travel control signal.

[0081] (3) In the above-mentioned methods (1) and (2), the vehicle 100 may be equipped with an internal sensor, and at least one of the generation of the route and the generation of the driving control signal may use the measurement results outputted from the internal sensor. The internal sensor is a sensor mounted on the vehicle 100. Specifically, the internal sensor may include, for example, a camera, a LiDAR, a millimeter wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, a gyroscope sensor, etc. For example, in the above-mentioned method (1), the server 200 may also obtain the measurement results of the internal sensor, and reflect the measurement results of the internal sensor in the route when generating the route. In the above-mentioned method (1), the vehicle 100 may also obtain the measurement results of the internal sensor, and reflect the measurement results of the internal sensor in the driving control signal when generating the driving control signal. In the above-mentioned method (2), the vehicle 100 may also obtain the measurement results of the internal sensor, and reflect the measurement results of the internal sensor in the route when generating the route. In the above-mentioned method (2), the vehicle 100 may also obtain the measurement results of the internal sensor, and reflect the measurement results of the internal sensor in the route when generating the driving control signal.

[0082] In the above-mentioned embodiments, the vehicle 100 may be equipped with an internal sensor, and the measurement result outputted from the internal sensor may be used in at least one of the generation of the route and the generation of the travel control signal. For example, the vehicle 100 may obtain the measurement result of the internal sensor, and reflect the measurement result of the internal sensor on the route when generating the route. The vehicle 100 may obtain the measurement result of the internal sensor, and reflect the measurement result of the internal sensor on the travel control signal when generating the travel control signal.

[0083] In the first embodiment described above, the server 200 automatically generates a travel control signal to be transmitted to the vehicle 100. Alternatively, the server 200 may generate a travel control signal to be transmitted to the vehicle 100 in accordance with an operation of an external operator located outside the vehicle 100. For example, the external operator may operate an operating device including a display for displaying a captured image output from an external sensor, a steering wheel for remotely operating the vehicle 100, an accelerator pedal, a brake pedal, and a communication device for communicating with the server 200 via wired communication or wireless communication, and the server 200 may generate a travel control signal in accordance with the operation applied to the operating device.

[0084] In the above-mentioned embodiments, the vehicle 100 only needs to have a structure that can be moved by unmanned driving, for example, it can also be a platform with the structure described below. Specifically, in order to play the three functions of "driving", "turning" and "stopping" by unmanned driving, the vehicle 100 at least has a control device that controls the driving of the vehicle 100 and an actuator of the vehicle 100. In the case where the vehicle 100 obtains information from the outside for unmanned driving, the vehicle 100 also has a communication device. That is, the vehicle 100 that can move by unmanned driving may not be installed with at least a part of the interior parts such as the driver's seat and the instrument panel, may not be installed with at least a part of the exterior parts such as the bumper and the fender, and may not be installed with the body shell. In this case, the remaining parts such as the body shell can be installed on the vehicle 100 before the vehicle 100 is shipped from the factory, or the remaining parts such as the body shell can be installed on the vehicle 100 after the vehicle 100 is shipped from the factory in a state where the remaining parts such as the body shell are not installed on the vehicle 100. Each component can be installed from any direction such as the upper side, lower side, front side, rear side, right side or left side of the vehicle 100, and can be installed from the same direction or from different directions. In addition, the platform can also be positioned in the same manner as the vehicle 100 in the first embodiment.

[0085] The vehicle 100 can also be manufactured by combining a plurality of modules. A module refers to a unit composed of a plurality of components that are aggregated according to the location and function of the vehicle 100. For example, the platform of the vehicle 100 can be manufactured by combining a front module that constitutes the front part of the platform, a central module that constitutes the central part of the platform, and a rear module that constitutes the rear part of the platform. In addition, the number of modules that constitute the platform is not limited to 3, and may be less than 2 or more than 4. In addition, the components that constitute the parts of the vehicle 100 that are different from the platform may also be modularized on the basis of the components that constitute the platform, or instead of this. In addition, various modules may also include any exterior components such as bumpers and grilles, and any interior components such as seats and consoles. In addition, not limited to the vehicle 100, any type of mobile body may also be manufactured by combining a plurality of modules. Such a module may be manufactured, for example, by joining a plurality of components by welding, fixings, etc., or by integrally molding at least a portion of the components that constitute the module into one component by casting. The molding method of molding a component, especially a relatively large component, in an integral manner is also called giga casting or mega casting. For example, the front module, central module, and rear module mentioned above can also be manufactured using Giga Foundry.

[0086] Transporting the vehicle 100 by driving the unmanned vehicle 100 is also referred to as "autonomous transport". In addition, a structure for realizing autonomous transport is also referred to as "vehicle remote control autonomous driving transport system". In addition, a production method of producing the vehicle 100 by autonomous transport is also referred to as "autonomous production". In autonomous production, for example, in a factory that manufactures the vehicle 100, at least a part of the transport of the vehicle 100 is realized by autonomous transport.

[0087] The present disclosure is not limited to the above-mentioned embodiments, and can be implemented in various structures within the scope of its main purpose. For example, the technical features in the embodiments corresponding to the technical features in each method recorded in the invention content section can be appropriately replaced or combined in order to solve part or all of the above-mentioned problems, or to achieve part or all of the above-mentioned effects. In addition, if the technical feature is not described as a necessary feature in this specification, it can be appropriately deleted.

Claims

1. A control device configured to generate a control command for controlling a moving object using three-dimensional point group data of the moving object measured by a distance measuring device, the control device comprising: a range determination unit configured to determine a use range, the use range being a range including only a portion of the three-dimensional point group data; and The estimation unit is configured to estimate at least one of a position and a direction of the moving object using the three-dimensional point group data in the use range.

2. The control device according to claim 1, in, The use range is determined to include a characteristic portion, which is a portion in an observation area of ​​the moving body that can be observed by the distance measuring device and in which a density of characteristic quantities is higher than an average value in the observation area.

3. The control device according to claim 2, in, The characteristic portion is determined according to the model of the moving object, the detection direction of the moving object by the distance measuring device, and the distance between the distance measuring device and the moving object.

4. The control device according to claim 1, in, The range determination unit is configured to execute a process of determining the use range when a processing load of the control device is greater than a preset reference value, and is configured not to execute a process of determining the use range when the processing load is less than the reference value.

5. The control device according to claim 3, in, At least one of a detection direction of the moving object and a distance between the distance measuring device and the moving object is determined using the three-dimensional point group data.

6. The control device according to claim 1, in, The estimation unit is configured to estimate at least one of a position and a direction of the moving object by matching the three-dimensional point group data of the use range with template data.