In-vehicle devices, identification methods, computer programs, and in-vehicle systems

Abstract

This vehicle-mounted device is mounted in a vehicle and is provided with an identifying unit which, on the basis of first information indicating the position of a first object detected by means of a first sensor mounted in the vehicle, second information indicating the position of a second object detected by means of a second sensor outside the vehicle, and a tracking result of the position of the first object or the second object, identifies the first object and the second object.

Description

Technical Field

[0001] The present disclosure relates to an in-vehicle device, an identification method, a computer program, and an in-vehicle system. This application claims priority based on Japanese Patent Application No. 2021-109867 filed on July 1, 2021, and incorporates all the descriptions described in the above Japanese application.

Background Art

[0002] Systems have been developed that assist in the driving control of a vehicle using sensors mounted on the vehicle such as an automobile. In such a system, by communicating with others (external devices such as other vehicles or roadside devices), the detection result of an object in the blind spot area of the sensor is obtained, and an attempt is made to detect an object without a blind spot (see, for example, Patent Document 1 and Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

[0004] An in-vehicle device according to an aspect of the present disclosure is an in-vehicle device mounted on a vehicle, and includes an identification unit that identifies a first object and a second object based on first information indicating the position of a first object detected by a first sensor mounted on the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and a tracking result of the position of the first object or the second object.

[0005] An identification method relating to another aspect of the present disclosure is a method for identifying an object using an in-vehicle device mounted on a vehicle, comprising the steps of identifying the first object and the second object based on first information indicating the position of a first object detected by a first sensor mounted on the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and the result of tracking the position of the first object or the second object.

[0006] A computer program according to another aspect of the present disclosure causes a computer mounted in a vehicle to function as an identification unit that identifies the first object and the second object based on first information indicating the position of a first object detected by a first sensor mounted in the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and the result of tracking the position of the first object or the second object.

[0007] An in-vehicle system according to another aspect of the present disclosure is an in-vehicle system mounted on a vehicle, comprising: a receiving unit that receives information indicating the position of an object detected by a sensor outside the vehicle; an object tracking unit that tracks an object around the vehicle; and an identification unit that identifies an object detected by a sensor mounted on the vehicle and an object detected by a sensor outside the vehicle based on information indicating the position of an object detected by a sensor mounted on the vehicle, the information received by the receiving unit, and the tracking results from the object tracking unit.

[0008] Furthermore, this disclosure can also be implemented as a computer program for causing a computer to execute characteristic steps included in the identification method. Needless to say, such a computer program can be distributed via computer-readable non-temporary recording media such as CD-ROMs (Compact Disc-Read Only Memory) or communication networks such as the Internet. Additionally, this disclosure can be implemented as a semiconductor integrated circuit that implements part or all of an in-vehicle device. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows an example of a driver assistance system according to the present disclosure. [Figure 2] Figure 2 shows an example of the hardware configuration of an in-vehicle system installed in a vehicle according to the embodiment of this disclosure. [Figure 3] Figure 3 shows the functional configuration of the in-vehicle system shown in Figure 2. [Figure 4] Figure 4 shows an example of a grid map that includes blind spots managed by the blind spot management unit. [Figure 5] Figure 5 shows an example of a list of targets received by the target receiving unit. [Figure 6] Figure 6 shows an example of the relative positions of the vehicles. [Figure 7] Figure 7 shows an example of the position of an object observed by a vehicle. [Figure 8] Figure 8 is a diagram illustrating an example of the first selection process for effective targets. [Figure 9] Figure 9 is a diagram illustrating an example of the first selection process for effective targets. [Figure 10] Figure 10 shows an example of the object tracking results by the object tracking unit. [Figure 11] Figure 11 is a diagram illustrating an example of the second selection process for effective targets. [Figure 12] Figure 12 is a diagram illustrating an example of the second selection process for effective targets. [Figure 13] Figure 13 is a conceptual diagram illustrating a validation gate. [Figure 14] Figure 14 is a diagram illustrating an example of object tracking processing. [Figure 15] Figure 15 is a diagram illustrating an example of the target selection process performed by the target selection unit. [Figure 16] Figure 16 is a diagram illustrating an example of the target selection process performed by the target selection unit. [Figure 17] Figure 17 is a diagram illustrating an example of the transmission process by the target transmission unit. [Figure 18] FIG. 18 is a flowchart showing an example of a processing procedure of an in-vehicle system. [Figure 19] FIG. 19 is a flowchart showing an example of a processing procedure of an active object first selection process (step S6 in FIG. 18). [Figure 20] FIG. 20 is a flowchart showing an example of a processing procedure of an active object second selection process (step S7 in FIG. 18). [Figure 21] FIG. 21 is a flowchart showing an example of a processing procedure of a transmission object selection process (step S8 in FIG. 18).

Mode for Carrying Out the Invention

[0010] [Problems to be Solved by the Present Disclosure] For an object within the overlapping detection ranges of the sensors of the host vehicle and other vehicles, multiple detection results may be obtained. Therefore, there is a possibility that a single vehicle may be misrecognized as two or more vehicles, resulting in unnecessary vehicle driving control. Thus, it is necessary to accurately identify whether these objects are the same object. For example, when another person detects a second object in the blind spot area behind the first object detected by the sensor mounted on the host vehicle, it is necessary to identify whether the first object and the second object are the same object.

[0011] However, Patent Document 1 and Patent Document 2 do not disclose a specific method for identifying an object.

[0012] The present invention has been made in view of such circumstances, and an object thereof is to provide an in-vehicle device, an identification method, a computer program, and an in-vehicle system capable of efficiently performing vehicle driving control.

[0013] [Effects of the Present Disclosure] According to the present disclosure, vehicle driving control can be efficiently performed.

[0014] [Outline of Embodiments of the Present Disclosure] First, an overview of the embodiments of this disclosure will be listed and described. (1) An in-vehicle device according to one embodiment of the present disclosure is an in-vehicle device mounted on a vehicle, and includes an identification unit that identifies the first object and the second object based on first information indicating the position of a first object detected by a first sensor mounted on the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and the result of tracking the position of the first object or the second object.

[0015] This configuration uses the tracking results of object positions to identify the first and second objects. Therefore, by considering the continuity of object positions from the past, the first and second objects can be accurately identified. This prevents misidentification of one object as multiple objects, thus preventing unnecessary vehicle control and enabling efficient vehicle control.

[0016] (2) In (1), the in-vehicle device further includes an information selection unit that selects second information from the second information excluding the position of the second object that is within the detectable area of ​​the object by the first sensor, and the identification unit may identify the first object and the second object based on the tracking result, the first information, and the second information selected by the information selection unit.

[0017] With this configuration, if a second object detected by the second sensor is located within the detection range of the first sensor (excluding blind spots), it can be determined that the second object was falsely detected by the second sensor and excluded. This allows for the identification of both the first and second objects, prioritizing the detection result of the first sensor.

[0018] (3)(2) The second information further indicates the area occupied by the second object, and the information selection unit may exclude the second object in which at least a portion of the area occupied by the second object overlaps with the detectable area.

[0019] With this configuration, if a portion of the area occupied by the second object overlaps with the detectable area, the second object can be excluded. This allows for the identification of both the first and second objects by prioritizing the object detection results from the first sensor.

[0020] (4) In any of (1) to (3), the identification unit selects the first object preferentially from the first and second objects which have been identified as the same object, and the in-vehicle device may further include an object tracking unit which corrects the tracking result based on the position of the object selected by the identification unit.

[0021] This configuration allows for the continuous tracking of objects, prioritizing the first object, and then identifying both the first and second objects.

[0022] (5)(4) The identification unit may select the object that is closest to the position indicated by the tracking result from among the first object and the second object that are included in the setting area set based on the tracking result.

[0023] In this configuration, the object closest to the position indicated by the tracking result is selected from among the objects included in the set area. This enables highly continuous object tracking with smoothly connected object positions.

[0024] (6)(5) The object tracking unit may update the tracking result by calculating the positions of the first object and the second object included in the set area, which are weighted probabilistically.

[0025] With this configuration, even if multiple objects exist within the defined area, the tracking results can be updated based on the probabilistically weighted positions of the objects, enabling highly continuous object tracking with smoothly connected object positions.

[0026] (7) In any of (1) to (6), the in-vehicle device may further include a transmission target determination unit that determines the location information indicated by the tracking result as a target for transmission to a device outside the vehicle if the second object is not included in the setting area set based on the tracking result.

[0027] With this configuration, if a second object is present within the designated area, the location information indicated by the tracking result can be prevented from being transmitted to a device outside the vehicle. Since this information is already being transmitted by other vehicles as second-hand information, there is no need to transmit it redundantly. This suppresses the transmission of unnecessary information and reduces the amount of communication traffic.

[0028] (8) In any of (1) to (7), the first information further indicates the detection time, speed, and direction of movement of the first object, and the second information further indicates the detection time, speed, and direction of movement of the second object, and the in-vehicle device may further include a correction unit that corrects the position of the second object based on the first information and the second information.

[0029] This configuration allows for correction of the second object's position by considering the difference in detection times between the first and second objects. This enables identification of the first and second objects based on their detection positions at the same time. As a result, the first and second objects can be accurately identified.

[0030] (9) An identification method according to another embodiment of the present disclosure is a method for identifying an object by an in-vehicle device mounted on a vehicle, which includes the steps of identifying the first object and the second object based on first information indicating the position of a first object detected by a first sensor mounted on the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and a result of tracking the position of the first object or the second object.

[0031] This configuration includes the characteristic processing steps of the above-described in-vehicle device. Therefore, this configuration can achieve the same operation and effects as the above-described in-vehicle device.

[0032] (10) A computer program according to another embodiment of the present disclosure causes a computer mounted in a vehicle to function as an identification unit that identifies the first object and the second object based on first information indicating the position of a first object detected by a first sensor mounted in the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and the result of tracking the position of the first object or the second object.

[0033] This configuration allows the computer to function as the aforementioned in-vehicle device. Therefore, it can achieve the same functions and effects as the aforementioned in-vehicle device.

[0034] (11) Other embodiment of the in-vehicle system of the present disclosure is an in-vehicle system mounted on a vehicle, comprising: a receiving unit that receives information indicating the position of an object detected by a sensor outside the vehicle; an object tracking unit that tracks an object around the vehicle; and an identification unit that identifies an object detected by a sensor mounted on the vehicle and an object detected by a sensor outside the vehicle based on information indicating the position of an object detected by a sensor mounted on the vehicle, the information received by the receiving unit, and the tracking results by the object tracking unit.

[0035] This configuration uses the tracking results of object positions to identify objects detected by sensors mounted on the vehicle and objects detected by sensors outside the vehicle. Therefore, by considering the continuity of object positions from the past, the two types of objects can be accurately identified. This prevents misidentification of one object as multiple objects, thus preventing unnecessary vehicle control and enabling efficient vehicle control.

[0036] [Details of the embodiments of this disclosure] The embodiments of this disclosure will be described below with reference to the drawings. The embodiments described below are all specific examples of this disclosure. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples and do not limit this disclosure. Furthermore, components in the following embodiments that are not described in the independent claims are components that can be added at will. Also, the figures are schematic diagrams and are not necessarily strictly accurate representations.

[0037] Furthermore, identical components will be assigned the same symbols. Since their functions and names are also identical, their explanations will be omitted as appropriate.

[0038] [Overall configuration of the driver assistance system] Figure 1 shows an example of the configuration of a driver assistance system according to the present disclosure. Referring to Figure 1, the driver assistance system includes vehicles 1, 2, 3, and 6 traveling on road 5, and a pedestrian 4. Vehicle 1 is the vehicle itself, equipped with the onboard system described later, and traveling in the driving lane of Road 5.

[0039] Vehicle 2 is a vehicle traveling in the opposite lane of Road 5. Vehicle 2 is equipped with an object detection function and a function to transmit the detection results, and is considered another vehicle from the perspective of Vehicle 1. Vehicle 3 is a vehicle traveling in the driving lane, diagonally to the left and in front of Vehicle 1. Vehicle 6 is a vehicle traveling in the driving lane, following behind Vehicle 1. Pedestrian 4 is located diagonally to the left and in front of vehicle 3, in an area that is not directly visible from vehicle 1 (blind spot).

[0040] [Hardware configuration of the in-vehicle system] Figure 2 shows an example of the hardware configuration of an in-vehicle system mounted on a vehicle 1 according to the embodiment of this disclosure.

[0041] Referring to Figure 2, the in-vehicle system 10 mounted on vehicle 1 includes in-vehicle sensors 101A and 101B, a control device 120, a relay device 130, and a communication device 140.

[0042] The on-board sensors 101A and 101B are, for example, on-board cameras, LiDAR (Light Detection and Ranging) devices, and millimeter-wave radar devices, and observe the positions of objects present around the vehicle 1. Alternatively, the on-board sensors 101A and 101B may observe the region where objects are present.

[0043] The relay device 130 relays data transmitted and received between devices that make up the in-vehicle system 10. The relay device 130 is composed of, for example, an ECU (Electronic Control Unit). An ECU is a type of computer that has a processor such as a CPU (Central Processing Unit) and memory.

[0044] The control device 120 controls the operation of vehicle 1. Vehicle 1 is composed of, for example, an ECU for autonomous driving. The control device 120 controls, for example, the accelerator opening, brake amount, steering angle, etc. of vehicle 1.

[0045] The communication device 140 transmits and receives data to and from external devices of vehicle 1 (for example, vehicle 2 or roadside communication devices). The communication device 140 is composed of, for example, a TCU (Telematics Control Unit).

[0046] The in-vehicle sensors 101A and 101B, the control device 120, the relay device 130, and the communication device 140 are connected by an in-vehicle network such as CAN (Controller Area Network) or Ethernet (registered trademark).

[0047] [Functional configuration of the in-vehicle system] Figure 3 shows the functional configuration of the in-vehicle system 10 shown in Figure 2. The in-vehicle system 10 includes an in-vehicle sensor 101, an object detection unit 102, a blind spot area management unit 103, a first effective target selection unit 104, an object tracking unit 105, a second effective target selection unit 106, a target receiving unit 107, a time shift correction unit 108, a transmitted target selection unit 109, a target transmission unit 110, and a driving control unit 111.

[0048] The on-board sensor 101 includes on-board sensors 101A and 101B shown in Figure 2. Furthermore, the on-board sensor 101 includes a sensor for detecting the position of vehicle 1 (e.g., a GPS (Global Positioning System) receiver).

[0049] The object detection unit 102, the blind spot area management unit 103, the object tracking unit 105, and the operation control unit 111 are functional processing units that are realized by executing a predetermined computer program stored in the memory of the control device 120 on the processor of the control device 120 shown in Figure 2.

[0050] The effective target first selection unit 104, the effective target second selection unit 106, the time shift correction unit 108, and the transmitted target selection unit 109 are functional processing units that are realized by executing a predetermined computer program stored in the memory of the relay device 130 on the processor of the relay device 130 shown in Figure 2.

[0051] The target receiving unit 107 and the target transmitting unit 110 are communication modules included in the communication device 140 shown in Figure 2.

[0052] [Regarding the object detection unit 102] The object detection unit 102 acquires the region of an object observed by the on-board sensor 101 (hereinafter referred to as the "object observation region") and detects objects present around the vehicle 1. The object detection unit 102 may, for example, detect the center position of the object region as the object's position, or it may detect the position within the object region closest to the vehicle 1 as the object's position. The object detection unit 102 also detects the object's speed and direction of movement. The object's speed and direction of movement may be obtained directly from the on-board sensor 101, such as a speed sensor and a gyro sensor, or they may be calculated from the time change of the object's position.

[0053] [Regarding the Blind Spot Management Unit 103] The blind spot management unit 103 detects the blind spot area of ​​the vehicle 1 based on the object observation area observed by the on-board sensor 101 and the position of the vehicle 1, and manages the detected blind spot area.

[0054] Figure 4 shows an example of a grid map that includes blind spots managed by the blind spot management unit 103. Referring to Figure 4, the blind spot management unit 103 acquires map information of a predetermined range around the vehicle 1, including the vehicle 1, as a grid map 30, based on the position of the vehicle 1 detected by the on-board sensor 101. The grid map 30 is defined, for example, to include the detection range of objects by the on-board sensors 101A and 101B. The grid map 30 divides the predetermined range into a plurality of grids 31 of a predetermined size. The size of each grid 31 may be uniform or may vary from place to place. For example, the size of the grid 31 may vary depending on the observation accuracy of the physical quantities of the on-board sensors 101A and 101B, and the size of the grid 31 may be smaller in areas with higher observation accuracy near the vehicle 1.

[0055] The blind spot area management unit 103 displays object observation areas 32A, 32B, and 32C, which are grids 31 containing object observation areas observed by the on-board sensor 101, on the grid map 30. Object observation areas 32A, 32B, and 32C correspond to the areas of vehicle 3, vehicle 2, and vehicle 6 shown in Figure 1, respectively.

[0056] Furthermore, the blind spot management unit 103 calculates blind spot areas that are out of sight of the vehicle 1 due to the presence of object observation areas 32A, 32B, and 32C, based on the positional relationship between the vehicle 1 and object observation areas 32A, 32B, and 32C. The blind spot management unit 103 displays the blind spot areas 33A, 33B, and 33C, which are grids 31 containing the calculated blind spot areas, on the grid map 30.

[0057] [Regarding the target receiving unit 107] The target receiving unit 107 receives targets broadcast by vehicle 2. A target refers to information about an object, including its position. In other words, the target receiving unit 107 receives information about the object detected by vehicle 2 and information about vehicle 2 as targets. Here, the target received from vehicle 2 includes information about the object (the object detected by vehicle 2 and vehicle 2 itself), the detection time, the speed of movement, and the direction of movement.

[0058] [Regarding the time shift correction unit 108] The time shift correction unit 108 corrects the difference between the detection time of the object indicated by the target received by the target receiving unit 107 and the detection time of the object detected by the object detection unit 102. In other words, the time shift correction unit 108 corrects the difference in detection time by estimating the position of the object that the vehicle 2 would have detected at the detection time of the object detected by the object detection unit 102. The method of correcting the difference in detection time by the time shift correction unit 108 will be described below with reference to the drawings.

[0059] Figure 5 shows an example of a list of targets received by the target receiving unit 107. The list of targets received by the target receiving unit 107 (hereinafter referred to as the "received target list") contains X targets. Each target includes the time of detection, the detected position (position coordinates), the speed of movement, and the direction of movement. The detected position is indicated, for example, by latitude and longitude. The direction of movement is indicated, for example, by the azimuth angle with north as 0 degrees. For example, the detection time of target number 1 is T1, the detected position is (N1, E1), the speed of movement is V1, and the direction of movement is D1.

[0060] Figure 6 shows an example of the relative positions of the vehicles. Referring to Figure 6, assume that both Vehicle 1 and Vehicle 2 are observing Vehicle 7. Vehicle 2 broadcasts the target of Vehicle 7, and Vehicle 1 receives the target.

[0061] Figure 7 shows an example of the position of an object observed by a vehicle. Detection position 12D indicates the detection position of vehicle 7 by vehicle 1. Detection position 22D indicates the detection position of vehicle 7 by vehicle 2. Here, the detection time of vehicle 7 by vehicle 1 is assumed to be 00:00:00.500 seconds, and the detection time of vehicle 7 by vehicle 2 is assumed to be 00:00:00.200 seconds. In other words, there is a 300-millisecond difference between the two detection times due to reasons such as communication delay from vehicle 2 to vehicle 1. For this reason, the time difference correction unit 108 calculates the amount of movement of vehicle 7 (direction of movement and distance of movement) over 300 milliseconds based on the movement speed and direction of movement of vehicle 7 included in the received target list. The time difference correction unit 108 calculates detection position 22E by moving detection position 22D by the calculated amount of movement. As a result, the time shift correction unit 108 estimates the detection position 22E of vehicle 7 that vehicle 2 would have detected at the detection time of vehicle 7 by the object detection unit 102, which is 00:00:00.500 seconds. The time shift correction unit 108 corrects the received target list by rewriting the detection time and position coordinates of the vehicle 7 target included in the received target list to the detection time of 00:00:00.500 seconds and the position coordinates of the detection position 22E, respectively. As a result, the time shift correction unit 108 corrects the time shift of the targets included in the received target list.

[0062] [Regarding the first effective target selection unit 104] The effective target first selection unit 104 excludes targets detected by other parties (in this case, vehicle 2) that are within the object detection range of the on-board sensor 101 from the received target list, based on the grid map managed by the blind spot area management unit 103 and the received target list after time shift correction by the time shift correction unit 108. The effective target first selection unit 104 then selects the remaining targets that were not excluded from the received target list (effective target first selection process). The effective target first selection process by the effective target first selection unit 104 will be described below with reference to the drawings.

[0063] The effective target first selection unit 104 acquires, for example, the grid map 30 shown in Figure 4. In the grid map 30, the areas other than the object observation areas 32A, 32B, 32C and the blind spot areas 33A, 33B, 33C are referred to as the empty areas 34. In the areas other than the blind spot areas 33A, 33B, 33C, the on-board sensor 101 can detect objects. Therefore, the combined area of ​​the object observation areas 32A, 32B, 32C and the empty areas 34 becomes the area in which the on-board sensor 101 can detect objects. However, the area in which the vehicle 1 is located is also included in the empty areas 34.

[0064] Figures 8 and 9 illustrate an example of the effective target first selection process. Figures 8 and 9 show a grid map 30.

[0065] Referring to Figure 8, the effective target first selection unit 104 overlays the detection positions of objects included in the time-shift corrected received target list onto the grid map 30 shown in Figure 4. Self-position 21 indicates the position of vehicle 2 included in the received target list. Detection positions 22A, 22B, 22C, 22D, and 22E indicate the positions of objects detected by vehicle 2. Detection position 22A indicates the detection position of pedestrian 4, and detection positions 22B and 22C indicate the detection positions of vehicle 3. Here, it is assumed that two detection positions for vehicle 3 have been detected. Detection position 22D indicates the detection position of vehicle 1. Detection position 22E indicates a falsely detected detection position.

[0066] Referring to Figure 9, the first effective target selection unit 104 excludes detection positions 22C, 22D, and 22E from the detection positions 22A, 22B, 22C, 22D, and 22E that are within the detectable area of ​​the grid map 30. In other words, the first effective target selection unit 104 excludes detection position 22C because it overlaps with the object observation area 32A. The first effective target selection unit 104 also excludes detection positions 22D and 22E because they overlap with the empty area 34. The first effective target selection unit 104 selects the information of the objects indicating the remaining detection positions 22A and 22B that were not excluded as effective targets. The first effective target selection unit 104 also selects the information of the vehicle 2 indicating its own position 21 as an effective target.

[0067] [Regarding the second effective target selection unit 106] The second effective target selection unit 106 identifies the object detected by the object detection unit 102 with the object detected by the vehicle 2 indicated by the effective target, based on the object tracking results by the object tracking unit 105 and the effective targets selected by the first effective target selection unit 104. From among the effective targets selected by the first effective target selection unit 104, it further selects an effective target (second effective target selection process). Note that identifying object A and object B means determining that object A and object B are the same object. The object tracking process by the object tracking unit 105 will be described later. The second effective target selection process by the second effective target selection unit 106 will be explained below with reference to the drawings.

[0068] Figure 10 shows an example of the object tracking results by the object tracking unit 105. The tracking results include the current (current frame) object tracking positions 41A, 41B, and 41C predicted by the object tracking unit 105 based on past object tracking results. The tracking results also include the detection positions 12A, 12B, and 12C of the object in the current frame detected by the object detection unit 102. Furthermore, the tracking results include validation gates 51A, 51B, and 51C for identifying the object at each of the tracking positions 41A, 41B, and 41C. The validation gates indicate boundaries for object identification, and multiple objects included in the same validation gate are identified as the same object. Here, validation gates are set with tracking positions 41A, 41B, and 41C as the center positions. The method for determining the validation gates will be described later.

[0069] Figures 11 and 12 illustrate an example of the second selection process for effective targets. Referring to Figure 11, the second effective target selection unit 106 superimposes the vehicle's own position 21 and the object's detection positions 22A and 22B, indicated by the effective target selected by the first effective target selection unit 104, onto the object tracking results shown in Figure 10.

[0070] Referring to Figure 12, the second valid target selection unit 106 identifies multiple objects within the same validation gate 51A, 51B, and 51C as the same object. For example, the second valid target selection unit 106 identifies the object corresponding to detection position 12B within validation gate 51B as the same object as the object corresponding to detection position 22B. Furthermore, the second valid target selection unit 106 identifies the vehicle 2 corresponding to its own position 21 within validation gate 51C as the same object as the object corresponding to detection position 12C.

[0071] The second valid target selection unit 106 selects valid targets for objects in a validation gate containing multiple objects that have been determined to be the same by selecting a valid object from among the objects detected by the vehicle 2 within the validation gate. Specifically, the second valid target selection unit 106 calculates the Mahalanobis distance from the tracking position within the validation gate to each object. The second valid target selection unit 106 selects an object detected by the vehicle 2 whose Mahalanobis distance from the tracking position is smaller than the Mahalanobis distance from the tracking position to the object detected by the object detection unit 102, and determines that the selected object's target is valid. The second valid target selection unit 106 also determines that targets of objects detected by the vehicle 2 whose Mahalanobis distance from the tracking position is larger than or equal to the Mahalanobis distance from the tracking position to the object detected by the object detection unit 102 are invalid.

[0072] For example, with respect to validation gate 51B, the Mahalanobis distance from tracking position 41B to detection position 22B is greater than the Mahalanobis distance from tracking position 41B to detection position 12B. Therefore, the second valid target selection unit 106 determines that the target of the object corresponding to detection position 22B is invalid. On the other hand, with respect to validation gate 51C, the Mahalanobis distance from tracking position 41C to self position 21 is less than the Mahalanobis distance from tracking position 41C to detection position 12C. Therefore, the second valid target selection unit 106 determines that the target of vehicle 2 corresponding to self position 21 is valid. The second valid target selection unit 106 always determines that the target of vehicle 1 corresponding to detection position 12C within validation gate 51C is valid. The second valid target selection unit 106 may also select targets detected by vehicle 1.

[0073] The detection position 22A is outside the validation gate. Therefore, the second valid target selection unit 106 determines that the target of the object corresponding to the detection position 22A is valid.

[0074] [About validation gates] Next, we will explain validation gates. Figure 13 is a conceptual diagram of a validation gate. Figure 13 shows a validation gate for the tracking position 41 of an object, and the curve where the Mahalanobis distance from the tracking position 41 is γ is set as the validation gate 51.

[0075] Furthermore, the left-hand side of Equation 1 below represents the Mahalanobis distance from the tracking position 41, and the right-hand side of Equation 1 represents the validation gate threshold γ. The threshold γ is calculated, for example, based on the inverse chi-squared cumulative distribution.

[0076]

number

number

number

[0077] As described above, the second valid target selection unit 106 determines that the object corresponding to the detection position 22 and the object corresponding to the tracking position 41 are the same object if the detection position 22 is located inside the validation gate 51 (i.e., if Equation 1 is satisfied). On the other hand, the second valid target selection unit 106 determines that the object corresponding to the detection position 22 and the object corresponding to the tracking position 41 are different objects if the detection position 22 is located outside the validation gate 51 (i.e., if Equation 1 is not satisfied).

[0078] The validation gate 51 is adjusted to match the characteristics of the on-board sensor 101. Therefore, the second valid target selection unit 106 can set the validation gate to a range in which it predicts that the detection position 22 of an object should be obtainable from another party. This allows the second valid target selection unit 106 to select a valid target from among the valid targets selected by the first valid target selection unit 104.

[0079] [Regarding the object tracking unit 105] The object tracking unit 105 tracks objects around the vehicle 1 based on the object detection results from the object detection unit 102 and the effective targets selected by the effective target second selection unit 106.

[0080] Figure 14 is a diagram illustrating an example of object tracking processing. The object tracking unit 105 corrects the tracking positions 41A, 41B, and 41C of the object in the current frame, which were predicted based on past object tracking results, based on the object targets detected by the object detection unit 102 and the effective targets selected by the effective target second selection unit 106.

[0081] Specifically, the object tracking unit 105 corrects the tracking positions 41A, 41B, and 41C of the object in the current frame, which were predicted using filtering processes such as a Kalman filter, based on the object positions detected by the object detection unit 102 and the object positions indicated by the effective target second selection unit 106.

[0082] For example, the object tracking unit 105 corrects the tracking position 41A predicted using the Kalman filter to the detection position 12A within the validation gate 51A of the tracking position 41A. Similarly, the object tracking unit 105 corrects the tracking position 41B predicted using the Kalman filter to the detection position 12B within the validation gate 51B of the tracking position 41B.

[0083] Furthermore, the object tracking unit 105 corrects the tracking position 41C predicted using the Kalman filter, using the detected position 12C and its own position 21 within the validation gate 51C of the tracking position 41C. When correcting the tracking position based on multiple positions in this way, the object tracking unit 105 uses a probabilistic weighting method such as PDA (Probabilistic Data Association) or JPDA (Joint Probabilistic Data Association) to weight the multiple positions (for example, the detected position 12C and its own position 21), and corrects the tracking position 41C based on the weighted positions.

[0084] Furthermore, the object tracking unit 105 sets the detection position 22A, which is not included in any of the validation gates for any of the detection positions, as a new tracking position 41D and starts tracking from the current frame. A validation gate 51D is also set for the tracking position 41D.

[0085] The object tracking unit 105 tracks the object by performing the above-described tracking position update process for each frame. The same object target in different frames is assigned the same target ID. This enables object tracking across frames.

[0086] [Regarding the Transmit Target Selection Unit 109] The transmitting target selection unit 109 selects targets that indicate tracking results to be broadcast to devices outside of vehicle 1, based on the received target list received from vehicle 2 and whose time difference has been corrected by the time difference correction unit 108, and the object tracking results from the object tracking unit 105. The target selection process by the transmitting target selection unit 109 will be described below with reference to the drawings.

[0087] Figures 15 and 16 illustrate an example of the target selection process performed by the target selection unit 109.

[0088] Figure 15 shows the object tracking results from the object tracking unit 105 overlaid with the object's position indicated by the target after time shift correction by the time shift correction unit 108. In other words, the tracking positions 41A, 41B, and 41C are shown as the object tracking results. In addition, the object detection positions 22A, 22B, 22C, 22D, and 22E, as well as the vehicle's own position 21, are shown as the positions indicated by the target after time shift correction following reception from vehicle 2. The vehicle's own position 11 is also shown.

[0089] Referring to Figure 16, the transmitting target selection unit 109 determines whether the location indicated by the target received from another party, including vehicle 2, is included within the validation gate for the tracking location. If the location indicated by the target received from another party is included, the transmitting target selection unit 109 decides not to transmit the target indicating the tracking location. On the other hand, if the location indicated by the target received from another party is not included, the transmitting target selection unit 109 decides to transmit the target indicating the tracking location.

[0090] For example, the validation gate 51A for tracking location 41A does not include the location indicated by a target received from another party. Therefore, the transmitting target selection unit 109 determines the target indicating tracking location 41A as the transmission target.

[0091] On the other hand, the validation gate 51B for tracking position 41B includes the detection positions 22B and 22C of the object indicated by the target received from another party. Therefore, the transmitting target selection unit 109 decides not to transmit the target indicating tracking position 41B. Similarly, the validation gate 51C for tracking position 41C includes the vehicle's own position 21 indicated by the target received from another party. Therefore, the transmitting target selection unit 109 decides not to transmit the target indicating tracking position 41C. Furthermore, the target selection unit 109 always selects targets that indicate the vehicle's own position 11 as targets for transmission.

[0092] [Regarding the target transmission unit 110] The target transmission unit 110 transmits the targets selected for transmission by the target selection unit 109 to an external device by broadcasting them.

[0093] Figure 17 is a diagram illustrating an example of the transmission process by the target transmission unit 110. The target transmission unit 110 broadcasts a target indicating the tracking position 41A of the object to be transmitted, and a target indicating the vehicle's own position 11.

[0094] [Processing procedure of the in-vehicle system 10] Figure 18 is a flowchart showing an example of the processing procedure of the in-vehicle system 10. Referring to Figure 18, the effective target first selection unit 104 obtains a grid map 30 of the area around vehicle 1 as shown in Figure 4 from the blind spot area management unit 103 (step S1).

[0095] The effective target second selection unit 106 obtains a tracked target list from the object tracking unit 105, which shows the object tracking results by the object tracking unit 105 (step S2).

[0096] The time shift correction unit 108 obtains a list of received targets from the target receiving unit 107 (step S3).

[0097] The time difference correction unit 108 corrects the received target list by correcting the time difference between the detection time of the object indicated in the received target list acquired in step S3 and the detection time of the object detected by the object detection unit 102 (step S4).

[0098] The first effective target selection unit 104, the second effective target selection unit 106, and the time shift correction unit 108 store the grid map 30, the tracked target list, and the corrected received target list in the memory of the relay device 130 (step S5). The effective target first selection unit 104 performs the effective target first selection process (step S6).

[0099] Figure 19 is a flowchart showing an example of the processing procedure for the effective target first selection process (step S6 in Figure 18).

[0100] Referring to Figure 19, the effective target first selection unit 104 retrieves the grid map and correction received target list held in the memory from the relay device 130 (step S61).

[0101] The effective target first selection unit 104 places the detection positions 22A, 22B, 22C, 22D, 22E and its own position 21 of the objects included in the received target list on the grid map 30, as shown in Figure 8 (step S62).

[0102] The effective target first selection unit 104 deletes detection positions 22D and 22E on the empty region 34 of the grid map 30, as shown in Figure 9 (step S63).

[0103] The effective target first selection unit 104 deletes the detection position 22C that overlaps with the object observation area 32A, as shown in Figure 9 (step S64).

[0104] The first effective target selection unit 104 selects the information of the objects indicating the remaining detection positions 22A and 22B, and the information of the vehicle 2 indicating its own position 21, as effective targets, and stores the list of selected effective targets in the memory of the relay device 130 (step S65).

[0105] Referring again to Figure 18, the effective target second selection unit 106 performs the effective target second selection process (step S7).

[0106] Figure 20 is a flowchart showing an example of the processing procedure for the second effective target selection process (step S7 in Figure 18).

[0107] Referring to Figure 20, the second effective target selection unit 106 obtains from the memory of the relay device 130 the tracked target list held in the memory and the effective target list selected as a result of the first effective target selection process (step S71).

[0108] The effective target second selection unit 106 executes the processes from step S72 to step S75 for each tracking result shown in the tracking target list (loop A).

[0109] In other words, as shown in Figure 12, the effective target second selection unit 106 calculates the Mahalanobis distance from the tracking position indicated by the tracking result of interest to the detection positions of all objects detected by the object detection unit 102 within the validation gate of the tracking position (step S72).

[0110] The effective target second selection unit 106 calculates the smallest Mahalanobis distance among the Mahalanobis distances calculated in step S72 (step S73).

[0111] The valid target second selection unit 106 calculates the Mahalanobis distance from the tracking position described above to the detection position of the object shown in the received target list within the validation gate described above (step S74).

[0112] The second effective target selection unit 106 determines that an object target corresponding to a detection position where the Mahalanobis distance calculated in step S74 is smaller than the minimum Mahalanobis distance calculated in step S73 is an effective target (step S75).

[0113] After the processing of loop A is completed, the second valid target selection unit 106 outputs a list of valid targets that it has determined to be valid to the object tracking unit 105 (step S76). In addition, as shown in Figure 14, the object tracking unit 105 updates the tracked target list based on the object detection results by the object detection unit 102 and the valid target list.

[0114] Referring again to Figure 18, the transmitting target selection unit 109 performs a transmitting target selection process to select a transmitting target that indicates the tracking result of an object to be broadcast to a device outside the vehicle 1 (step S8). Figure 21 is a flowchart showing an example of the processing procedure for the target selection process (step S8 in Figure 18).

[0115] Referring to Figure 21, the transmitting target selection unit 109 obtains the tracked target list and the corrected received target list from the memory of the relay device 130 (step S81).

[0116] The transmitted target selection unit 109 executes the processes in steps S82 and S83 for each tracked target included in the tracked target list (loop B).

[0117] In other words, as shown in Figure 16, the transmitting target selection unit 109 determines whether the location indicated by the target received from another party, including vehicle 2, is included within the validation gate of the tracking location of the object indicated by the tracking target (step S82).

[0118] If the target selection unit 109 includes a location indicated by a target received from another party (YES in step S82), it sets a non-transmission flag for the tracked target to be processed (step S83).

[0119] After processing in loop B, the transmit target selection unit 109 selects a tracked target for which the non-transmit flag is not set as the transmit target (step S84).

[0120] Referring again to Figure 18, the transmitter target selection unit 109 stores the transmitter target ID, which is the identifier of the transmitter target selected in the transmitter target selection process (step S8), in the memory of the relay device 130 (step S9).

[0121] The transmitting target selection unit 109 obtains a tracked target list from the object tracking unit 105 (step S10). The tracked target list obtained from the object tracking unit 105 is an updated tracked target list that reflects the object detection results by the object detection unit 102 in the current frame.

[0122] The transmitted target selection unit 109 selects a tracking target having the same target ID as the transmitted target ID stored in memory in step S9 from the updated tracking target list acquired in step S10 (step S11). This makes it possible to select a tracking target that indicates the latest tracking location.

[0123] The target selection unit 109 outputs the selected tracking target and the target indicating the vehicle's own position 11 to the target transmission unit 110 (step S12). This allows the target transmission unit 110 to broadcast the tracking target and the target indicating the vehicle's own position 11, which it has received from the target selection unit 109, to an external device.

[0124] [Effects of the embodiments of this disclosure] As explained above, the effective target second selection unit 106 uses the object position tracking results to identify the first object detected by the on-board sensor 101 and the second object detected by another party. Therefore, by considering the continuity of the object's position from the past, the first and second objects can be accurately identified. This prevents misidentification of one object as multiple objects, thus preventing unnecessary vehicle driving control and enabling efficient vehicle driving control.

[0125] Furthermore, if a second object detected by another party exists within the object detection area of ​​the on-board sensor 101, excluding the blind spot area, the effective target first selection unit 104 can determine that the second object was mistakenly detected by another party and exclude the second object. This allows for the identification of the first and second objects by prioritizing the object detection results from the on-board sensor 101.

[0126] Furthermore, the second valid target selection unit 106 always selects the first object if multiple objects are included within the validation gate. This allows the object tracking unit 105 to continuously track the first object.

[0127] Furthermore, the second valid target selection unit 106 selects the object closest to the tracking position indicated by the tracking result from among the multiple objects included in the validation gate. This enables the object tracking unit 105 to perform highly continuous object tracking with smoothly connected object positions.

[0128] Furthermore, the object tracking unit 105 updates the tracking position of an object based on the probabilistically weighted positions of multiple objects contained within the validation gate. This allows the tracking results to be updated based on the probabilistically weighted positions of objects, even when multiple objects are present within the validation gate, enabling highly continuous object tracking with smoothly connected object positions.

[0129] Furthermore, the target selection unit 109 can choose not to transmit location information indicated by the tracking result to an external device if a second object is present within the validation gate. Since this information is already transmitted by other vehicles as second information, there is no need to transmit it redundantly. This suppresses the transmission of unnecessary information and reduces the amount of communication traffic.

[0130] Furthermore, the time shift correction unit 108 can correct the position of the second object by taking into account the difference between the detection time of the first object and the detection time of the second object. This allows the first and second objects to be identified based on their detection positions at the same time. As a result, the first and second objects can be accurately identified.

[0131] <Variation> In the above-described embodiment, the effective target first selection unit 104 excludes targets of objects detected by others that are located within the object-detectable area of ​​the on-board sensor 101 from the received target list.

[0132] In this modified example, the targets included in the received target list represent the area (occupied area) of an object. In this case, the first effective target selection unit 104 will exclude the target of an object from the received target list if at least a portion of the area of ​​the object indicated by the target included in the received target list overlaps with the detectable area of ​​the on-board sensor 101.

[0133] This allows for the identification of the first and second objects by prioritizing the object detection results from the first sensor.

[0134] [Note] Some or all of the components constituting each of the above-described devices may consist of one or more semiconductor devices such as system LSIs.

[0135] Furthermore, the computer programs described above may be recorded on a computer-readable non-temporary recording medium, such as an HDD, CD-ROM, or semiconductor memory, and distributed. Alternatively, the computer programs may be transmitted and distributed via telecommunications lines, wireless or wired communication lines, networks such as the Internet, or data broadcasting. Furthermore, each of the above devices may be implemented using multiple computers or multiple processors.

[0136] Furthermore, some or all of the functions of each of the above devices may be provided by cloud computing. In other words, some or all of the functions of each device may be implemented by a cloud server. Furthermore, at least some of the above embodiments and modifications may be combined as desired.

[0137] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims, not in the sense described above, and all modifications within the meaning and scope equivalent to the claims are intended. [Explanation of Symbols]

[0138] 1 vehicle 2 vehicles 3 vehicles 4 Pedestrians 5 road 6 vehicles 7 vehicles 10 In-vehicle systems 11 Self-location 12A Detection position 12B Detection position 12C detection position 12D detection position 21 Self-location 22 Detection position 22A Detection position 22B Detection position 22C Detection position 22D Detection position 22E Detection position 30 Grid Map 31 grid 32A Object observation area 32B Object observation area 32C Object Observation Area 33A Blind spot area 33B Blind spot area 33C Blind spot area 34 Empty area 41 Tracking location 41A Tracking location 41B Tracking location 41C Tracking location 41D Tracking location 51 Validation Gate 51A Validation Gate 51B Validation Gate 51C Validation Gate 51D Validation Gate 101 Automotive Sensors 101A Automotive Sensor 101B Automotive Sensor 102 Object detection unit 103 Blind spot area management department 104 Effective Target First Selection Unit (Information Selection Unit) 105 Object Tracking Unit 106 Effective Target Second Selection Unit (Identification Unit) 107 Target Receiving Unit 108 Time drift correction unit (correction unit) 109 Transmit Target Selection Unit (Transmit Target Determination Unit) 110 Target Transmitter 111 Operation Control Unit 120 Control device 130 Relay device 140 Communication equipment

Claims

1. An in-vehicle device installed in a vehicle, An identification unit that performs identification to determine that the first object and the second object are the same object, based on first information indicating the position of a first object detected by a first sensor mounted on the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and the tracking result of the position of the first object or the second object. The system includes an information selection unit that selects the second information from which the position of the second object located within the detectable area of ​​the first sensor is excluded, The identification unit is an in-vehicle device that identifies the first object and the second object based on the tracking result, the first information, and the second information selected by the information selection unit.

2. The second piece of information further indicates the area occupied by the second object. The in-vehicle device according to claim 1, wherein the information selection unit excludes the second object whose occupied area overlaps with the detectable area, at least a portion of the area of ​​the second object.

3. An in-vehicle device mounted on a vehicle, The system includes an identification unit that performs identification to determine that the first object and the second object are the same object, based on first information indicating the position of a first object detected by a first sensor mounted on the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and the result of tracking the position of the first object or the second object. The identification unit prioritizes selecting the first object from among the first and second objects, which have been identified as the same object. The in-vehicle device further comprises an object tracking unit that modifies the tracking result based on the position of the object selected by the identification unit.

4. The in-vehicle device according to claim 3, wherein the identification unit selects the object closest to the position indicated by the tracking result from among the first object and the second object included in the setting area set based on the tracking result.

5. The in-vehicle device according to claim 4, wherein the object tracking unit updates the tracking result by calculating the positions of the first object and the second object included in the set area, which are probabilistically weighted.

6. An in-vehicle device mounted on a vehicle, An identification unit that performs identification to determine that the first object and the second object are the same object, based on first information indicating the position of a first object detected by a first sensor mounted on the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and the tracking result of the position of the first object or the second object. An in-vehicle device comprising: a transmission target determination unit that, when the second object is not included within a setting area set based on the tracking result, determines that the position information indicated by the tracking result is to be transmitted to a device outside the vehicle.

7. The first information further indicates the detection time, movement speed, and movement direction of the first object. The second information further indicates the detection time, movement speed, and movement direction of the second object. The in-vehicle device according to claim 1 or claim 2, further comprising a correction unit that corrects the position of the second object based on the first information and the second information.

8. A method for identifying an object using an in-vehicle device mounted on a vehicle, A step of identification in which the first object and the second object are the same object, based on first information indicating the position of a first object detected by a first sensor mounted on the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and the result of tracking the position of the first object or the second object. The step of selecting the second information from which the position of the second object located within the detectable area of ​​the first sensor is excluded, An identification method comprising the step of performing the identification, in which the first object and the second object are identified based on the tracking result, the first information, and the second information selected in the step of selecting the second information.

9. The computer installed in the vehicle, An identification unit that performs identification to determine that the first object and the second object are the same object, based on first information indicating the position of a first object detected by a first sensor mounted on the vehicle, second information indicating the position of a second object detected by a second sensor outside the vehicle, and the tracking result of the position of the first object or the second object. A computer program for functioning as an information selection unit that selects second information from the second information, excluding the position of the second object that is within the detectable area of ​​the first sensor, wherein The identification unit is a computer program that identifies the first object and the second object based on the tracking result, the first information, and the second information selected by the information selection unit.

10. An in-vehicle system installed in a vehicle, A receiving unit that receives information indicating the position of an object detected by the sensor outside the vehicle, An object tracking unit that tracks objects around the vehicle, An identification unit performs identification based on information indicating the position of an object detected by a sensor mounted on the vehicle, information received by the receiving unit, and the tracking results from the object tracking unit, to determine that the object detected by the sensor mounted on the vehicle and the object detected by the sensor outside the vehicle are the same object. The receiving unit includes an information selection unit that selects information from the received information excluding the positions of objects detected by external sensors that are within the object detection range of the sensors mounted on the vehicle, The identification unit is an in-vehicle system that identifies objects detected by sensors mounted on the vehicle and objects detected by sensors outside the vehicle, based on the tracking results, information indicating the location of objects detected by sensors mounted on the vehicle, and the information selected by the information selection unit.

Images