SENSOR DETECTION INTEGRATION DEVICE

Abstract

Sensor detection integration device (006) comprising the following: a sensor object information unit that generates integration object information (205A, 205B) by integrating sensor object information (207A, 207B, 207C) with respect to a sensor object obtained by multiple detection sensors for an external field that detect an object in an external field, the sensor object information integration unit (010) comprises: an integration object information storage unit (106) that stores the integration object information (205A, 205B), a prediction update unit (100) that extracts a predictable prediction object from the integration object information (205A, 205B) previously stored in the integration object information storage unit (106), and predicts a position of the prediction object after a predetermined time without using the sensor object information (207A, 207B, 207C), an allocation unit (101) that estimates the prediction object to be assigned to the sensor object information (207A, 207B, 207C), an offset parameter update unit (102) that updates an offset parameter required for calculating an offset to be applied to the sensor object information (207A, 207B, 207C), an integration target information generation unit (104) that applies the offset to the sensor object based on the offset parameter and generates integration target information (204) in which the prediction object and the sensor object with applied offset are linked, based on an estimation result of the assignment unit (101), and an integration update unit (105) that generates the integration object information (205A, 205B) with the estimated position of the object based on the integration target information (204).

Description

Technical field

[0001] The present invention relates to a sensor detection integration device and, in particular, to a sensor detection integration device having a function for estimating the state of an object using information obtained from multiple sensors, including different types of sensors that detect the object. State of the art

[0002] In autonomous driving of a motor vehicle or similar, an object around the vehicle (external field) is detected, and driving is planned and determined according to the detection result. Various sensors exist for detecting an object, such as radar, cameras, sonar, and laser radar. Since the sensors have different properties, such as detection range, detectable object, detection accuracy, and cost, it is necessary to combine several sensors according to the purpose. However, to use the data output by the combined sensors for operational planning and determination, it is necessary to examine the specifications according to the sensor combination. To simplify the complexity, a function for integrating information from combined sensors is required, and PTL 1 discloses an example of such a function.

[0003] PTL 2 discloses an object recognition device comprising: a data receiving unit that generates observation data from corresponding sensors in accordance with sensor detection data of an object in the vicinity of a host vehicle; an association processing unit that generates association data based on an association reference value, which denotes a correspondence between the observation data and object data of a previous process cycle;and an update processing unit which, based on the association data, updates a state vector contained in the object data of the previous process cycle and updates the object data by including recent association data, which are observation data that most recently corresponded to the object data, wherein the association processing unit generates the association reference value using preferably the most recent association data of the same sensor as that of the observation data of a current process cycle.

[0004] PTL 3 discloses a sensor integration system comprising a processor, a storage device connected to the processor, and an output device connected to the processor, wherein the storage device contains observation data observed by a plurality of sensors, the observation data containing selectively described elements among predefined elements.

[0005] PTL 4 discloses the following: a vehicle positioning device connected to a first sensor that outputs satellite positioning data, and to a second sensor that detects a state value of the vehicle and outputs the state value as state value data, and connected to at least one of the following: a third sensor that detects a terrestrial object and outputs data of a relative relation between the terrestrial object and the vehicle, and a fourth sensor that detects a road line shape or road marking shape and outputs road line shape data. List of prior art patent literature PTL 1: JP 2012 - 163 495 A PTL 2: DE 10 2017 210 108 A1 PTL 3: JP 2012 - 163 495 A PTL 4: DE 11 2019 007 155 T5 Summary of the invention: Technical problem

[0006] Even when the same object is detected, its coordinates can vary depending on the sensor. This is because the detected portion of the object differs depending on the sensor. For example, if sensor information is integrated according to the conventional technique disclosed in PTL 1 above, it is consequently not possible to correct coordinates that vary for each sensor. Therefore, the coordinates of an integrated object generated by integrating sensor information can change rapidly each time the combination of sensors performing the detection changes. Consequently, a problem arises, such as that an autonomous driving planning device, which plans and determines the driving process, will perform an erroneous determination.

[0007] The present invention was carried out in light of the above circumstances and its object is to create a sensor detection integration device that is capable of preventing a rapid change in the coordinates of an integrated object and, for example, preventing an erroneous determination in a plan determination device for autonomous driving, even if the combination of sensors performing the detection changes. Solution to the problem

[0008] To solve the above problem, according to the present invention, a sensor detection integration device comprises a sensor object information unit that generates integration object information by integrating sensor object information with respect to a sensor object obtained from multiple external field detection sensors that detect an object in an external field. The sensor object information integration unit comprises an integration object information storage unit that stores the integration object information, a prediction update unit that extracts a predicted predicted object from the integration object information previously stored in the integration object information storage unit and predicts a position of the predicted object after a predetermined time without using the sensor object information, and an assignment unit that estimates the predicted object.that is to be assigned to the sensor object information, an offset parameter update unit that updates an offset parameter required to calculate an offset to be applied to the sensor object information, an integration target information generation unit that applies the offset to the sensor object based on the offset parameter and generates integration target information in which the predicted object and the sensor object with the applied offset are linked, based on an estimation result of the assignment unit, and an integration update unit that generates the integration object information with the estimated position of the object based on the integration target information.

[0009] According to the present invention, a sensor detection integration device also receives vehicle behavior information, sensor object information, sensor road information, positioning information, and map information acquired by an information acquisition device comprising a vehicle behavior detection sensor, multiple external field detection sensors, a positioning system, and a map unit. The device integrates the received information and transmits the result of this integration to a planning device for autonomous driving. The sensor detection integration device includes a sensor object information integration unit that integrates the sensor object information regarding a sensor object, obtained by the multiple external field detection sensors that detect an object in an external field, as integration object information.and a vehicle environment information integration unit that integrates the integration object information, the vehicle behavior information, the sensor road information, the positioning information, and the map information as vehicle environment information and outputs the vehicle environment information to the autonomous driving planning device. The sensor object information integration unit comprises an integration object information storage unit that stores the integration object information, a prediction update unit that extracts a predicted predicted object from the integration object information previously stored in the integration object information storage unit and predicts the position of the predicted object after a predetermined time without using the sensor object information, and an assignment unit that estimates the predicted object.that is to be assigned to the sensor object information, and an offset parameter update unit that updates an offset parameter required for calculating an offset to be applied to the sensor object information, an integration target information generation unit that calculates the offset to be applied to the sensor object information based on the offset parameter in order to apply the calculated offset to the sensor object, and integration target information in which the prediction object and the sensor object with the applied offset are linked, based on an estimation result of the assignment unit, and an integration update unit that estimates the position of the object based on the integration target information in order to generate integration object information. Advantageous effects of the invention

[0010] Since, according to the present invention, an object position is estimated in a state in which information about a position detected by an external field detection sensor that recognizes an object within an external object is corrected or changed, it is possible to prevent the rapid change of coordinates of an integrated object and, for example, to prevent an erroneous determination in a plan determination device for autonomous driving, even if the combination of external field detection sensors performing the detection changes.

[0011] Other tasks, configurations and advantageous effects than those described above are illustrated by the descriptions of the following embodiments. Brief description of the drawings [ Fig. 1] Fig. Figure 1 is a system configuration diagram representing a system 1000 for autonomous driving with a sensor detection integration device 006 according to a first embodiment of the present invention. [ Fig. 2] Fig. Figure 2 is a functional block diagram representing a sensor object information integration unit 010 of the sensor detection integration device 006 according to the first embodiment of the present invention. [ Fig. 3] Fig. 3 is a flow chart that is executed by an assignment unit 101 of the sensor object information integration unit 010 in the first embodiment of the present invention. [ Fig. 4] Fig. Figure 4 is a flowchart that represents an assignment determination (S512) in a process of the assignment unit 101 of the sensor object information integration unit 010 in the first embodiment of the present invention. [ Fig. 5] Fig. 5 is a sequence plan that is executed by an offset parameter update unit 102 of the sensor object information integration unit 010 in the first embodiment of the present invention. [ Fig. 6] Fig. 6 is a flow chart that is executed by an integration target information generation unit 104 of the sensor object information integration unit 010 in the first embodiment of the present invention. [ Fig. 7] Fig. 7 is a sequence plan that is executed by a prediction update unit 100 of the sensor object information integration unit 010 in the first embodiment of the present invention. [ Fig. 8] Fig. Figure 8 is a sequence plan that is executed by an integration update unit 105 of the sensor object information integration unit 010 in the first embodiment of the present invention. [ Fig. 9] Fig. Figure 9 is a schematic diagram representing a processing result example in which rapid movement of an integrated object occurs due to a difference in detection coordinates of sensors in a conventional technique. [ Fig. 10] Fig. Figure 10 is a schematic diagram illustrating a processing result example in which rapid movement of the integrated object is suppressed even when there is a difference in the detection coordinates of the sensors, according to the first embodiment. [ Fig. 11] Fig. Figure 11 is a schematic diagram illustrating a processing result example in which an additional delay time is provided until the integrated object follows an input value when an object actually moves, in the conventional technique for taking measures different from that in the present invention for the problem of the fast movement of the integrated object. [ Fig. 12] Fig. Figure 12 is a schematic diagram illustrating a processing result example of suppressing the additional delay time until the integrated object follows the input value, even if the object is actually moving, according to the first embodiment. [ Fig. 13] Fig. Figure 13 is a schematic diagram illustrating a processing result example in which a velocity value of the integrated object changes rapidly when an offset is applied, in a second embodiment of the present invention. [ Fig. 14] Fig. Figure 14 is a schematic diagram illustrating a processing result example in which a rapid change in the velocity value of the integrated object is suppressed when the offset is applied, in a third embodiment of the present invention. Description of embodiments

[0012] The embodiments of the present invention are described in detail below with reference to the drawings. [First embodiment] <Systemkonfiguration eines Systems für das autonome Fahren mit einer Sensorerkennungsintegrationsvorrichtung>

[0013] Fig. Figure 1 is a system configuration diagram representing a system 1000 for autonomous driving with a sensor detection integration device 006 according to a first embodiment of the present invention.

[0014] The system 1000 for autonomous driving of the illustrated embodiment basically comprises an information acquisition device 009, an input communication network 005, a sensor detection integration device 006, a planning device 007 for autonomous driving and an actuator group 008.

[0015] The information acquisition device 009 comprises a vehicle behavior detection sensor 001, an external field detection sensor group 002, a positioning system 003, and a mapping unit 004. The vehicle behavior detection sensor 001 includes a gyroscope, a wheel speed sensor, a steering angle sensor, an accelerometer, and the like, mounted in a vehicle. The sensors detect yaw rate, wheel speed, steering angle, acceleration, and the like, each representing the vehicle's behavior. The vehicle behavior detection sensor then outputs this information (vehicle behavior information) to the input communication network 005.The external field detection sensor group 002 detects and identifies an object in the vehicle's external field, such as a white line on the road, a sign, or similar, and outputs the information (sensor object information and sensor road information) to the input communication network 005. The external field detection sensor group 002 uses a combination of several different external field detection sensors (hereinafter referred to simply as sensors), such as radar, a camera (a monocular camera, a stereo camera, or similar), and sonar. There is no specific limitation on the sensor configuration. The positioning system 003 estimates the vehicle's position and outputs positioning information to the input communication network 005.As an example of the positioning system 003, a satellite positioning system (also referred to as a global positioning system) is illustrated. The map unit 004 outputs map information about the vehicle to the input communication network 005.

[0016] The input communication network 005 receives information from the information acquisition device 009 and transmits the received information to the sensor detection integration device 006. The controller area network (CAN), Ethernet (registered trademark), wireless communication, and similar technologies are used as the input communication network 005. CAN is a network generally used in an in-vehicle system.

[0017] The sensor detection integration device 006 receives data from the input communication network 005, including vehicle behavior information, sensor object information, sensor road information, position determination information, and map information, and integrates the received information as vehicle environment information (details are described later). The sensor detection integration device then outputs (transmits) the vehicle environment information to the planning device 007 for autonomous driving.

[0018] The autonomous driving planning device 007 receives information from the input communication network 005 and the vehicle's environment information from the sensor detection integration device 006. The autonomous driving planning device plans and determines how the vehicle is to move and outputs command information to the actuator group 008. The actuator group 008 comprises various actuators for steering the vehicle and operates according to the command information from the autonomous driving planning device 007. <Interne Konfiguration der Sensorerkennungsintegrationsvorrichtung>

[0019] The sensor detection integration device 006 in the present embodiment comprises an information storage unit 011, a sensor object information integration unit 010 and a vehicle environment information integration unit 012.

[0020] The information storage unit 011 stores information from the input communication network 005 and outputs information in response to requests from the sensor object information integration unit 010 and the vehicle environment information integration unit 012. The sensor object information integration unit 010 acquires the sensor object information from the information storage unit 011 and integrates the information of the same object, detected by multiple sensors, into a single, unified set of data. The sensor object information integration unit then outputs the integration result to the vehicle environment information integration unit 012 as integration object information (details are described later).The vehicle's own environment information integration unit 012 acquires the integration object information from the sensor object information integration unit 010 and the vehicle's own behavior information, sensor road information, position determination information, and map information from the information storage unit 011. The vehicle's own environment information integration unit then integrates the acquired information as vehicle environment information and outputs this information to the planning device 007 for autonomous driving. A conventionally known processing method can be applied to the integration processing in the vehicle's own environment information integration unit 012. Consequently, a detailed description of this method is omitted here, and the integration processing in the sensor object information integration unit 010 is described in detail below. <Interne Konfiguration der Sensorobjektinformationsintegrationseinheit in der Sensorerkennungsintegrationsvorrichtung>

[0021] Fig. Figure 2 is a functional block diagram representing the sensor object information integration unit 010 in the sensor detection integration device 006 according to the first embodiment of the present invention.

[0022] The sensor object information integration unit 010 of the illustrated embodiment mainly comprises a prediction update unit 100, an assignment unit 101, an offset parameter update unit 102, an assignment information storage unit 103, an integration target information generation unit 104, an integration update unit 105, and an integration object information storage unit 106. The process of the sensor object information integration unit 010 is executed continuously and repeatedly many times. At each execution, the time at which the purpose is to estimate the information is defined. For the purposes of the following description, it is assumed that the estimation of information is performed at time t_1 and then at time t_2, which is the time after a time Δt.

[0023] Before describing the function of each unit in the Sensor Object Information Integration Unit 010, the information sent or received in the Sensor Object Information Integration Unit 010 is broadly described. Sensor object information 207A, 207B, and 207C includes a sensor object ID assigned by a tracking process within the sensor, a relative position for the vehicle, and a relative velocity for the vehicle. Information such as the object type, the detection time, and the reliability of the information may also be retained. Predictive object information 200 and integration object information 205A and 205B include information about the time of an estimation target, an object ID, a relative position of an object, the relative velocity of the object, or the equivalent thereof. Additionally, information such as...The object type, error covariance, and the reliability of the information are additionally maintained. In the mapping information 201A, 201B, and 201C with respect to an object ID of each object in the prediction object information 200, information indicating whether the object ID is excluded from the mapping at time t_2 of the current estimation target, and information indicating an associated sensor object, are recorded. The sensor object information with respect to the associated sensor object includes the sensor type, the sensor object ID, information indicating whether the object is a mapping target at time t_1, an initial offset, and an offset duration time, which is the time during which the offset application continues.The integration target information 204 includes information about a sensor object that is a target of mapping (in other words, linking) to the object ID of each object in the prediction object information 200. The position and velocity of the sensor object information in the integration target information 204 do not necessarily correspond to the original sensor object information 207A, and an offset obtained from the initial offset, offset duration time, and the like is applied.

[0024] The prediction update unit 100, which forms the sensor object information integration unit 010, receives as input the integration object information 205B at time t_1 (for example, integration object information 205B before a processing step) from the integration object information storage unit 106 and outputs the prediction object information 200 at time t_2 to the assignment unit 101 and the integration update unit 105.The mapping unit 101 receives as input the sensor object information 207A from the information storage unit 011, the mapping information 201C at time t_1 (for example, mapping information 201C before a processing step) from the mapping information storage unit 103, the prediction object information 200 at time t_2, and outputs the mapping information 201A, which specifies which prediction object information 200 corresponds to which sensor object information, to the offset parameter update unit 102 at time t_2. At this time, information indicating whether a separation exists or not, and information indicating whether a mapping target exists at time t_1, are added. Additionally, the sensor object information 207A is output to the offset parameter update unit 102 as sensor object information 207B without being modified.The offset parameter update unit 102 receives the mapping information 201A at time t_2 and the sensor object information 207B as input and outputs the mapping information 201B at time t_2 to the integration target information generation unit 104 and the mapping information storage unit 103. At time t_2, offset parameter information about the initial offset and the offset duration is updated. Additionally, the sensor object information 207B is entered into the integration target information generation unit 104 as sensor object information 207C without being modified. The mapping information storage unit 103 stores the mapping information 201B and outputs the mapping information 201B to the mapping unit 101 as mapping information 201C.The integration target information generation unit 104 receives as input the mapping information 201B at time t_2 and the sensor object information 207C and links a result (sensor object on which the offset application is completed), obtained by applying an offset to the coordinates and velocity of the corresponding object information, with each predicted object at time t_2. The integration target information generation unit then outputs the result as integration target information 204 to the integration update unit 105. The integration update unit 105 receives as input the integration target information 204 and the predicted object information 200 at time t_2 from the predicted update unit 100. The integration update unit estimates the state (position, velocity, etc.) of each object at time t_2.The integration update unit outputs a result of the estimation as integration object information 205A to the integration object information storage unit 106 and transfers the integration object information to the vehicle environment information integration unit 012. The integration object information storage unit 106 stores the integration object information 205A and outputs the integration object information as integration object information 205B to the prediction update unit 100. (Assignment process of the assignment unit)

[0025] Fig. 3 is a flow chart that is executed by the assignment unit 101 of the sensor object information integration unit 010 in the first embodiment of the present invention.

[0026] In S500, the process of assignment unit 101 is started. In S502, an unprocessed prediction object is extracted from prediction update unit 100 based on prediction object information 200. In S504, it is determined whether an unprocessed prediction object exists. If the unprocessed prediction object exists, the process continues to S506. In S506, the ID list of sensor object information for the assignment target at time t_1 is extracted from assignment information storage unit 103 based on assignment information 201C at time t_1. In S508, all sensor objects in the sensor object information that are assignment target candidates at time t_2 are extracted. Alternatively, it is possible to extract a set containing at least all assignment targets using a method such as indexing.Next, in S510, it is determined whether an unprocessed sensor object exists among the sensor objects that are the assignment target candidates. If the unprocessed sensor object exists, the assignment determination (details are described later) is performed on the unprocessed sensor object in S512. The process then returns to S508. If no unprocessed sensor object exists in S510, the process continues to S532. In S532, it is checked whether a sensor object exists that is not the assignment target at time t_1 among the sensor object IDs that are the assignment target at time t_2. If the sensor object that is not the assignment target exists in S532, the process continues to S534. In S534, information indicating that a separation exists is added to the prediction object, and then the process returns to S502.The information added in S534 is used for offset parameter initialization (from S559 to S583 in . Fig. 5) is used, which is described later. If there is no sensor object in S532 that is not the assignment target, the process returns to S502. If there is no unprocessed prediction object in S504, the process ends (S536).

[0027] Fig. Figure 4 is a flow chart of the assignment determination (S512) in the process of the assignment unit 101 of the sensor object information integration unit 010 in the first embodiment of the present invention.

[0028] The process starts in S514. In S516, it is determined whether the sensor object is an assignment target of the prediction object at time t_2. If the sensor object is the assignment target, the process continues to S518. In S518, it is checked whether the sensor object ID is contained in the assignment target at time t_1. If the sensor object ID is contained, the process continues to S520. If the sensor object ID is not contained, the process continues to S522. In S520, information indicating that the sensor object is the assignment target itself at time t_1 is added to the sensor object ID. Then the process continues to S524. In S522, information indicating that the sensor object is not the assignment target at time t_1 is added to the sensor object ID. Then the process continues to S524. The information added in S520 and S522 is used for the offset parameter update (from S568 to S574 in Fig. 5) is used, which is described later. In S524, the prediction object and the sensor object are assigned to each other in the assignment information 201C at time t_2 and the assignment determination process (S512) ends (S526). (Update process of the offset parameter update unit).

[0029] Fig. 5 is a sequence plan that is executed by the offset parameter update unit 102 of the sensor object information integration unit 010 in the first embodiment of the present invention.

[0030] In S550, the process of offset parameter update unit 102 is started. The process then proceeds to S553. In S553, an unprocessed forecast object is extracted from the mapping information 201A of mapping unit 101. In S556, it is determined whether an unprocessed forecast object exists. If the unprocessed forecast object exists, the process proceeds to S559. If no unprocessed forecast object exists, the process proceeds to S586. In S559, it is determined whether any separation exists for the forecast object ID (see S534 in...). Fig. 3) If there is no separation for the prediction object ID, the process continues to S562. If there is any separation, the process continues to S577. In S562, the sensor object to be mapped is extracted. Then the process continues to S565. In S565, it is determined whether an unprocessed sensor object is present. If the unprocessed sensor object is present, the process continues to S568. If no unprocessed sensor object is present, the process returns to S553. In S568, it is examined whether the sensor object is itself a mapping target at time t_1 (see S520 and S522 in). Fig. 4) If the sensor object is the mapping target, the process continues to S571. If the sensor object is not the mapping target, the process continues to S574. In S571, the offset parameter for the sensor object (specifically, the offset parameter required to calculate the offset to be applied to the sensor object information) is updated. The process then returns to S562. The offset parameter update adds the difference (t_2 - t_1) between the estimation target time and the offset duration time t_offset of the mapping information 201A from the mapping unit 101. In S574, the offset parameter for the sensor object is initialized. The process then returns to S562. The contents of the offset parameter initialization process are described.The initial offset is represented by Δx, the predicted object position by x_prediction, and the sensor object position by x_sensor. The position is assumed to be a multidimensional quantity with an X and a Y component. The position can be three-dimensional. During the initialization of the offset parameter, Δx is set to x_prediction - x_sensor, and the offset duration is set to 0.

[0031] When the prediction object ID is disconnected in S559, the corresponding sensor object is extracted in S577, and then the process continues to S580. In S580, it is checked whether or not there is any unprocessed sensor object information. If unprocessed sensor object information is present, the process continues to S583. If no unprocessed sensor object information is present, the process returns to S553. In S583, the offset parameter for the sensor object is initialized. The initialization of the offset parameter is the same process as S574 described above. If no unprocessed prediction object is present in S556, the process of offset parameter update unit 102 is terminated in S586. (Generation process of the integration target information generation unit)

[0032] Fig. 6 is a flow chart that is executed by the integration target information generation unit 104 of the sensor object information integration unit 010 according to the first embodiment of the present invention.

[0033] In S600, the process of the integration target information generation unit 104 is started, and the process continues to S603. In S603, the unprocessed prediction object is extracted from the assignment information 201B by the offset parameter update unit 102, and the process continues to S606. In S606, it is determined whether an unprocessed prediction object exists. If the unprocessed prediction object exists, the process continues to S609. If no unprocessed prediction object exists, the process continues to S624. In S609, the sensor object associated with the extracted prediction object is extracted. Then the process continues to S612. In S612, it is determined whether an unprocessed sensor object exists. If the unprocessed sensor object exists, the process continues to S618. If no unprocessed sensor object exists, the process returns to S603.In section S618, data with the applied offset is generated for the sensor object. The process then continues to section S621, which describes the generation of data to which the offset is applied. Here, the offset functions c_x(t,Δx,T) and c_v(t,Δx,T) are introduced with respect to position and (relative) velocity. This represents the offset to be applied to the sensor object. c_x(t,Δx,T) is set to c_x(t,Δx,T) = (1-t / T)Δx if 0 ≤ t < T, and is set to 0 in other cases. c_v(t,Δx,T) is normally set to 0. Here, T is a positive number representing the time (offset application completion time) from when the offset is added to the sensor object until the offset is not added.When the offset duration is set to t_offset, the coordinate of the sensor object is defined as x_sensor, the velocity is defined as v_sensor, the initial offset is defined as Δx, the position of the sensor object with the offset is defined as x_offset, and the velocity is defined as v_offset. After the offset is applied, the sensor object has a position of x_offset = x_sensor + c_x(t,Δx,T) and a velocity of v_offset = v_sensor + c_v(t,Δx,T). Regarding the offset function, if 0 ≤ t < T, c_x(0,Δx,T) = Δx, c_x(t,Δx,T) · Δx > 0, and c_x(t,Δx,T) · Δx decreases monotonically. If c_v(t,Δx,T) · Δx ≤ 0, the same effect can be obtained even if c_x(t,Δx,T) = 0 and c_v(t,Δx,T) = 0. Here, “·” represents the dot product of the vector.This means that the offset function, in which the amount of the offset applied to the position of the sensor object is gradually reduced (damped) over time, and the amount becomes 0 for a predetermined period (offset application completion time), is applied here. Then, in S621, the data of the sensor object to which the offset was applied are linked (grouped) as the assignment target of the prediction object and registered as integration target information 204. The process then returns to S609. If there is no unprocessed prediction object in S606, the process of integration target information generation unit 104 is terminated in S624. (Prediction update process)

[0034] Fig. 7 is a sequence plan that is executed by the prediction update unit 100 of the sensor object information integration unit 010 in the first embodiment of the present invention.

[0035] In S650, the process of prediction update unit 100 is started. The process then proceeds to S653. In S653, an unprocessed integrated object is extracted from the integration object information that is already stored (previously) in integration object information storage unit 106. The process then proceeds to S656. In S656, it is determined whether an unprocessed integrated object exists. If the unprocessed integrated object exists, the process proceeds to S659. If no unprocessed integrated object exists, the process proceeds to S662. In S659, the state (position or similar) of the object (prediction object) at time t_2 is predicted without using the sensor object information from information storage unit 011. The process then returns to S653.A conventionally known processing method can be applied to the object state prediction process in S659. If no unprocessed integrated object exists in S656, the prediction update unit 100 process in S662 terminates. (Update process of the integration update unit)

[0036] Fig. Figure 8 is a flow chart that is executed by the integration update unit 105 of the sensor object information integration unit 010 in the first embodiment of the present invention.

[0037] In S700, processing of the integration update unit 105 is started. The process then proceeds to S703. In S703, the unprocessed prediction object is extracted from the prediction update unit 100 based on the prediction object information 200. The process then proceeds to S706. In S706, it is determined whether an unprocessed prediction object exists. If the unprocessed prediction object exists, the process proceeds to S712. If no unprocessed prediction object exists, the process proceeds to S718. In S712, several sensor objects, which are assignment targets of the prediction object, are extracted from the integration target information 204 by the integration target information generation unit 104. The process then proceeds to S715.In S715, the object position is estimated from the prediction object and the multiple sensor objects, and the integration object information 205A, containing the estimated object position, is transferred to the vehicle's own environment information integration unit 012. The process then returns to S703. If no unprocessed prediction object is present in S706, the process of the integration update unit 105 is terminated in S718. <Betrieb und vorteilhafte Effekte der Sensorerkennungsintegrationsvorrichtung>

[0038] The advantageous effects of preventing a situation in which the change in the position of the integrated object within a predetermined time due to the difference in the detection points of the sensors exceeds the threshold of the change set by the planning device 007 for autonomous driving, and of avoiding unnecessary operational determination according to the present embodiment, are described below.

[0039] Fig. Figure 9 is a schematic diagram representing a processing result example in which rapid movement of the integrated object occurs due to the difference in the detection coordinates of the sensors in the sensor object information integration unit of a conventional sensor detection integration device that does not apply offset.

[0040] It is assumed that the actual object is continuously located in a specific relative position. Sensor A detects the object at predetermined coordinates until time t_2. Sensor B also detects the object at predetermined coordinates after time t_3, but these coordinates differ from those detected by Sensor A. Until time t_2, the coordinates of the integrated object are estimated to be in a position corresponding to the position of Sensor A. After time t_3, the position of the integrated object moves in the direction of the coordinates detected by Sensor B due to integration processing in the conventional technique.If the time required for the time series filter used in the prediction update and integrated update of the present embodiment to follow the original data is set to T_F1, the movement of the integrated object will also be completed at T_F1.

[0041] Fig. Figure 10 is a schematic diagram illustrating a processing result example in which the rapid movement of the integrated object is suppressed even when there is a difference in the detection coordinates of the sensors, according to the present embodiment.

[0042] After time t_3, when detection by sensor B is started, the integration target information generation unit 104 generates the data of the sensor object to which the offset was applied through sensor B. The generated data is then input into the integration update unit 105. Since the data of the sensor object to which the offset was applied moves through sensor B over time T, during which the offset application continues, the time when the integrated object completes the movement is T_F1+T.

[0043] To summarize the above description according to the present embodiment, the time until the movement of the integrated object is completed due to the difference in the sensor's detection points can be extended from T_F1 to T_F1+T, and the position of the integrated object can be changed within a fixed time. It is possible to prevent the amount of change from exceeding the threshold set by the plan determination device 007 for autonomous driving and to avoid unnecessary determination.

[0044] Next, the advantageous effects that, if the object actually moves, the movement of the integrated object corresponding to the movement is carried out without causing an additional delay, and no additional delay occurs in the determination of the plan determination device 007 for autonomous driving, are described below according to the present embodiment.

[0045] Fig. Figure 11 is a schematic diagram illustrating a processing result example in which the additional delay time occurs until the integrated object follows the input value when the object is actually moving, in the conventional technique for taking countermeasures for the rapid movement of the integrated object, such that when the different sensor object is the assignment target, the reliability of the object is reduced and the integration update is performed, which differs from the present invention.

[0046] The actual object begins to move at time t_2, as described by the real object in Fig. Figure 11 illustrates this. The movement is completed at time t5. However, the integrated object exhibits a slower lag due to the integration processing by the filter, whose reliability is reduced. If the filter with reduced reliability follows the original data at time T_F2, it takes time T_F2 for the integrated object to follow the original data.

[0047] Fig. Figure 12 is a schematic diagram illustrating a processing result example of suppressing the additional delay time until the integrated object follows the input value, even if the object is actually moving, according to the present embodiment.

[0048] At time t_2, sensor B starts detection, and the integration target information generation unit 104 generates the information to which the offset is applied. The sensor object to which the offset was applied moves essentially parallel to the coordinates of the sensor object through the original sensor B in a time shorter than T, which is the time scale of the offset application. The integrated object also follows the data, and the following time is completed at T_F1.

[0049] This means that, according to the present embodiment, no additional delay is caused to the actual movement of the object beyond the delay caused by the filter. Consequently, no additional delay is caused during the determination by the plan determination device 007 for autonomous driving.

[0050] In further summary of the effect description based on Fig. 10 and Fig. 12, which are described above, move in Fig. 10. The data of the sensor object to which the offset was applied are passed through sensor B over time T while the offset application continues, and the amount of the offset is applied in such a way that it gradually decreases over time. Consequently, the data progresses from sensor A to time t_2. When the detection of sensor B is started, the data of the sensor object to which the offset was applied are passed through sensor B to the integration update unit 105. Therefore, it is possible to obtain integrated results that are continuously connected. Furthermore, in Fig. 12. The offset remains almost unchanged. Consequently, the sensor object with the applied offset moves through sensor B essentially parallel to the sensor object through sensor B and gradually approaches the original sensor object through sensor B as time passes. Even if the relative position of the vehicle changes at t_3, the relative change is appropriately reflected by sensor B as the sensor object to which the offset was applied, and the sensor object is input into the integration update unit 105. Therefore, the integration result can be obtained without causing the additional delay.

[0051] As described above, since in the present embodiment the object position is estimated in a state in which information about the position detected by the sensor that detects an object in an external field is corrected or changed, it is possible to prevent the rapid change of coordinates of the integrated object and, for example, to prevent an erroneous determination in the plan determination device 007 for autonomous driving, even if the combination of sensors that perform the detection changes. [Second embodiment]

[0052] The system configuration of the autonomous driving system and the internal configuration and processing flow of the sensor detection integration device in a second embodiment are the same as those described above with reference to Fig. 1, Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6, Fig. 7 to Fig. 8 first embodiment described.

[0053] c_v(t,Δx,T) of the offset functions used in the generation of data with applied offset (S618) in the integration target information generation unit 104 in Fig. The behavior of the second embodiment, as described in section 6, differs from that in the first embodiment. Specifically, in the second embodiment, c_v(t,Δx,T) is set to -Δx / T when 0 ≤ t < T, and is set to 0 in all other cases.

[0054] Alternatively, the offset function can be as described below. The condition of the first embodiment, c_x(0,Δx,T) = Δx, 0 ≤ t < T, c_x(t,Δx,T)·Δx > 0, c_x(t,Δx,T)·Δx decreases monotonically, c_v(t,Δx,T) · Δx ≤ 0, and if T ≤ t, c_x(t,Δx,T) = 0, c_v(t,Δx,T) = 0 and 0 ≤ t < T, is the integral relation of cx(t,Δx,T)=∫Ttcv(t,Δx,T)dt This is fulfilled. That is, an offset function is adopted here, in which the offset to be applied to the velocity of the sensor object is the derivative of the offset to be applied to the position of the sensor object.

[0055] Since the offset function is adopted in the present embodiment, in addition to the effect of the first embodiment when the applied amount of the offset is changed, the consistency of the speed offset (relative speed) is also ensured and applied. Consequently, the effect of preventing a situation in which the sensor object with the applied offset is excluded from the assignment when the assignment is performed with respect to speed is maintained. [Third embodiment]

[0056] The system configuration of the autonomous driving system and the internal configuration and processing flow of the sensor detection integration device in the third embodiment are the same as those in the first and second embodiments described above with reference to Fig. 1, Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6, Fig. 7 to Fig. 8 are described.

[0057] In the third embodiment, the offset function, which is used in the generation of data with applied offset (S618) in the integration target information generation unit 10 in Fig. The third embodiment, when 0 ≤ t < T, has different displacement functions than the second embodiment. That is, in the third embodiment, when 0 ≤ t < T, the displacement functions c_x(t,Δx,T) and c_v(t,Δx,T) are... cx(t,Δx,T)=12(cos(πtT)+1)Δx cv(t,Δx,T)=−π2Tsin(πtT)Δx and in other cases c_x(t,Δx,T) = 0 and c_v(t,Δx,T) = 0. The other processes are the same.

[0058] Alternatively, the offset function can be as described below. The condition of the second embodiment, c_x(0,Δx,T) = Δx, 0 ≤ t < T, c_x(t,Δx,T)-Δx > 0, c_x(t,Δx,T)·Δx decreases monotonically, c_v(t,Δx,T) · Δx ≤ 0, T ≤ t c_x(t,Δx,T) = 0, c_v(t,Δx,T) = 0, 0 ≤ t < T, is the integral relation of cx(t,Δx,T)=∫Ttcv(t,Δx,T)dt This condition is fulfilled. In addition to the condition c_v(0,Δx,T) = c_v(T,Δx,T) = 0, c_v(t,Δx,T) · Δx > c_v(T / 2,Δx,T), this also fulfills the condition that Δx and c_v(t,Δx,T) are continuous functions with respect to t. That is, the offset function is used here, in which the magnitude of the offset applied to the velocity of the sensor object is gradually reduced (damped) over time, and the magnitude becomes 0 for a predetermined period (offset application completion time).

[0059] Since the offset function is adopted in the present embodiment, the following effect is obtained in addition to the effect of the second embodiment. That is, if the speed (relative speed) of the integrated object changes more than the predetermined threshold within the predetermined time, the autonomous driving plan determination device 007 is determined to rapidly decrease or rapidly increase the object's speed, thus preventing a situation in which unnecessary acceleration or deceleration of the vehicle occurs. This is further described below.

[0060] Fig. Figure 13 is a schematic diagram illustrating a processing result example in which, for example, the velocity value of the integrated object changes rapidly when the offset is applied, in the second embodiment of the present invention.

[0061] It is assumed that the object actually has a constant velocity. Sensor A can accurately detect the object's velocity and detects it up to time t_1. Sensor B starts detection after time t_2 and can also accurately detect the object's velocity. It is assumed that Sensor A and Sensor B detect different positions. According to the process of the second embodiment, an offset is applied to the velocity due to the difference in position, and the information from the sensor object to which the offset was applied is generated by Sensor B. Since the velocity of the integrated object is also updated by Sensor B using the velocity information from the sensor object to which the offset was applied, the velocity of the integrated object changes.The time until the change in speed is complete is T_F1, which is the time scale of the filter.

[0062] Fig. Figure 14 is a schematic diagram illustrating a processing result example in which a rapid change in the velocity value of the integrated object is suppressed when the offset is applied, in the third embodiment of the present invention.

[0063] According to the third embodiment, the offset is applied to the velocity in the same way, but it takes T_2 for the offset applied to the velocity to reach its maximum, which differs from the second embodiment. It also takes T_2 from the state in which the offset applied to the velocity is at its maximum to the state in which no offset is applied. Therefore, the time until the velocity change is complete is extended to T_2.

[0064] As described above, the effects are therefore obtained as follows. That is, it is possible to prevent the change in speed (relative speed) of the integrated object beyond the predetermined threshold within the predetermined time, in order to avoid the determination that the autonomous driving planning device 007 rapidly slows down or speeds up the object, and to avoid unnecessary acceleration or deceleration of the vehicle itself. [Fourth embodiment]

[0065] The system configuration of the autonomous driving system and the internal configuration and processing flow of the sensor detection integration device in a fourth embodiment are the same as those in the first, second, and third embodiments described above with reference to Fig. 1, Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6, Fig. 7 to Fig. 8 are described.

[0066] In the fourth embodiment, the offset function, which is taken over in the data generation (S618), in which the offset is in the integration target information generation unit 104 in Fig. The fourth embodiment, as applied in the fourth embodiment, differs from that in the third embodiment. That is, in the fourth embodiment, r_4, which satisfies 0 < r_4 < 1 / 2, is predetermined, and T4=r4T a4=1T4(T−T4) is introduced in advance. If 0 ≤ t < T_4, the offset functions c_x(t,Δx,T) and c_v(t,Δx,T) are as follows. cx(t,Δx,T)=(1−t22a4)Δx cx(t,Δx,T)=−a4tΔx

[0067] If T_4 ≤ t <T-T_4, cx(t,Δx,T)=(−a4T4(t−0.5t)+0.5)Δx cv(t,Δx,T)=−a4T4Δx

[0068] If T-T_4 ≤ t < T_4, cx(t,Δx,T)=0.5a4(T−t)2Δx cv(t,Δx,T)=−a4(T−t)Δx

[0069] In other words, this can compute a function similar to the offset function of the third embodiment using only the product-sum operation without using the trigonometric function. The other processes are the same.

[0070] Since the offset function is adopted in the present embodiment, in addition to the effect of the third embodiment, the effect of reducing the processing of the trigonometric function, which requires a longer processing time than the product-sum operation, and the processing time is obtained. [Fifth embodiment]

[0071] The system configuration of the autonomous driving system and the internal configuration and processing flow of the sensor detection integration device in a fifth embodiment are the same as those in the first, second, third, and fourth embodiments, which are described with reference to Fig. 1, Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6, Fig. 7 to Fig. 8 are described. Furthermore, regarding the offset function, which is used in data generation with offset application (S618) in the integration target information generation unit 104 in Fig. 6 is adopted, any of those depicted in the first to fourth embodiments may be adopted.

[0072] In the present fifth embodiment, the following applies to the offset function if the magnitude of the initial offset Δx is less than or equal to a certain threshold x_5, that is to say |Δx|2≤x5

[0073] In this case (that is, if the predictor object and the sensor object are sufficiently close to each other), c_x(t,Δx,T) = 0 and c_v(t,Δx,T) = 0. Here, x_5 is a positive number. This means that if it is determined that the initial offset is small and the offset should not be applied depending on the sensor object's condition, then the offset is not applied here.

[0074] According to the present embodiment, in addition to the effects adopted in the first to fourth embodiments, the following effects are obtained: That is, if the initial offset is small, the offset application is not performed, thus avoiding the adverse effect on the plan determination device 007 for autonomous driving due to the offset application, and reducing the processing time. [Sixth embodiment]

[0075] The system configuration of the autonomous driving system and the internal configuration and processing flow of the sensor detection integration device in a sixth embodiment are the same as those in the first, second, third, and fourth embodiments, which are described with reference to Fig. 1, Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6, Fig. 7 to Fig. 8 are described.

[0076] Any one of the offset functions shown in the first to fourth embodiments is adopted. c˜x(t,Δx,T) c˜v(t,Δx,T) In the sixth embodiment, the offset function is used as the following expressions. c˜v(t,Δx,T) c˜v(t,Δx,k6|Δx|)

[0077] Here, k_6 is a positive number. The other processes are the same. That is, here, the offset function is used to determine the offset application completion time for each prediction object, or the offset function is used to ensure that the maximum value (temporal maximum) of the magnitude of the difference vector between the offset applied to the position or velocity of the sensor object and the offset applied to the position or velocity of the sensor object before the predetermined time is equal to any prediction object.

[0078] According to the present embodiment, in addition to the effects adopted in the first to fourth embodiments, the maximum value of the amount of the offset change is set for any prediction object. The time to resume the offset application changes (for example, time T is extended) according to the amount of the initial offset. If the initial offset is small, the time to resume the offset application is reduced. Consequently, it is possible to achieve the effect of avoiding the adverse effect of the offset application on the planning device 007 for autonomous driving and the effect of reducing the processing time.

[0079] The present invention is not limited to the embodiments described above, and various modifications are possible. The embodiment described above is provided in detail, by way of example, to illustrate the present invention in an easily understandable manner, and the embodiment described above is not necessarily limited to a case with all the described configurations. Furthermore, some components in one embodiment can be replaced by components in another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. With respect to some components in the embodiments, other components can be added, omitted, or replaced.

[0080] Some or all of the configurations, functions, functional units, processing means, and the like may be implemented in hardware, for example, by being designed with an integrated circuit. Furthermore, the aforementioned components, functions, and the like may be implemented in software by the processor, which interprets and executes a program to perform the respective functions. Information such as a program, a table, and a file that implements each function may be stored in working memory, a storage device such as a hard disk drive (HDD) or solid-state drive (SSD), or a recording medium such as an integrated circuit (IC) card, an SD card, or a DVD.

[0081] Control lines and information lines deemed necessary for the descriptions are shown; however, not all control lines and information lines in the product are necessarily depicted. In practice, it can be assumed that almost all components are interconnected. Reference symbol list 001 Self-vehicle behavior detection sensor 002 Detection sensor group for the external field 003 Position determination system 004 Card Unit 005 Inbound communication network 006 Sensor detection integration device 007 Planning device for autonomous driving 008 Actuator group 009 Information acquisition device 010 Sensor Object Information Integration Unit 011 Information storage unit 012 Own Vehicle Environment Information Integration Unit 100 forecast update units 101 Assignment unit 102 Offset parameter update unit 103 Allocation Information Storage Unit 104 Integration target information generation unit 105 Integration Update Unit 106 Integration object information storage unit 200 prediction object information 204 Integration target information 201A, 201B, 201C Assignment Information 205A, 205B Integration object information 207A, 207B, 207C Sensor object information 1000 systems for autonomous driving

Claims

[1] Sensor detection integration device (006) comprising the following: a sensor object information unit that generates integration object information (205A, 205B) by integrating sensor object information (207A, 207B, 207C) with respect to a sensor object obtained by multiple detection sensors for an external field that detect an object in an external field, the sensor object information integration unit (010) comprises: an integration object information storage unit (106) that stores the integration object information (205A, 205B), a prediction update unit (100) that extracts a predictable prediction object from the integration object information (205A, 205B) previously stored in the integration object information storage unit (106), and predicts a position of the prediction object after a predetermined time without using the sensor object information (207A, 207B, 207C), an allocation unit (101) that estimates the prediction object to be assigned to the sensor object information (207A, 207B, 207C), an offset parameter update unit (102) that updates an offset parameter required for calculating an offset to be applied to the sensor object information (207A, 207B, 207C), an integration target information generation unit (104) that applies the offset to the sensor object based on the offset parameter and generates integration target information (204) in which the prediction object and the sensor object with applied offset are linked, based on an estimation result of the assignment unit (101), and an integration update unit (105) that generates the integration object information (205A, 205B) with the estimated position of the object based on the integration target information (204). [2] Sensor detection integration device (006) according to claim 1, wherein the offset parameter update unit (102) updates the offset parameter such that the offset to be applied to the sensor object information (207A, 207B, 207C) has an amount depending on an external situation. [3] Sensor detection integration device (006) according to claim 1, wherein the offset parameter update unit (102) updates the offset parameter by adding a difference of an estimation target time to an offset duration time obtained by the assignment unit (101). [4] Sensor detection integration device (006) according to claim 1, wherein the integration target information generation unit (104) performs an offset function to decrease an amount of the offset to be applied to a position or velocity of the sensor object over time and to set the amount to 0 for a predetermined offset application completion time. [5] Sensor detection integration device (006) according to claim 1, wherein the integration target information generation unit (104) performs an offset function for determining the offset to be applied to a velocity of the sensor object as a derivative of the offset to be applied to a position of the sensor object. [6] Sensor detection integration device (006) according to claim 1, wherein the integration target information generation unit (104) performs an offset function to determine an offset application completion time for each prediction object. [7] Sensor detection integration device (006) according to claim 1, wherein the integration target information generation unit (104) assumes an offset function to cause a maximum value of an magnitude of a difference vector to be the same for all prediction objects, wherein the difference vector is between the offset to be applied to a position or velocity of the sensor object and the offset to be applied to the position or velocity of the sensor object before a predetermined time. [8] Sensor detection integration device (006) according to claim 1, wherein the integration target information generation unit (104) is configured to not apply the offset when a situation of the sensor object determines that the offset does not need to be applied. [9] Sensor detection integration device (006) according to claim 1, wherein the integration target information generation unit (104) performs an offset function which is able to calculate the offset to be applied to the sensor object only by a product sum operation. [10] Sensor detection integration device (006) receiving self-vehicle behavior information, sensor object information (207A, 207B, 207C), sensor road information, positioning information and map information acquired by an information acquisition device (009) comprising a self-vehicle behavior detection sensor (001), multiple external field detection sensors, a positioning system (003) and a map unit (004), integrating the received information and transmitting a result of the integration to a planning device (007) for autonomous driving, wherein the sensor detection integration device (006) comprises: a sensor object information integration unit (010) that integrates the sensor object information (207A, 207B, 207C) relating to a sensor object, obtained by the multiple external field detection sensors that detect an object in an external field, as integration object information (205A, 205B); and a vehicle environment information integration unit (012) that integrates the integration object information (205A, 205B), the vehicle behavior information, the sensor road information, the positioning information and the map information as vehicle environment information and outputs the vehicle environment information to the planning device (007) for autonomous driving, wherein the sensor object information integration unit (010) comprises an integration object information storage unit (106) that stores the integration object information (205A, 205B), a prediction update unit (100) that extracts a predictable prediction object from the integration object information (205A, 205B) previously stored in the integration object information storage unit (106), and predicts a position of the prediction object after a predetermined time without using the sensor object information (207A, 207B, 207C), an assignment unit (101) that estimates the prediction object to be assigned to the sensor object information (207A, 207B, 207C), an offset parameter update unit (102) that updates an offset parameter required for calculating an offset to be applied to the sensor object information (207A, 207B, 207C), an integration target information generation unit (104) that calculates the offset to be applied to the sensor object information (207A, 207B, 207C) based on the offset parameter in order to apply the calculated offset to the sensor object, and generates integration target information (204) in which the prediction object and the sensor object are linked with the applied offset, based on an estimation result of the assignment unit (101), and an integration update unit (105) that estimates the position of the object based on the integration target information (204) in order to generate integration object information (205A, 205B).

Images