Information processing device, information processing method, and program

Abstract

An information processing device according to one embodiment of the present technology comprises a calculation unit and a detection unit. The calculation unit calculates an index indicating a probability as to whether an object is a moving object on the basis of the Doppler velocity at the reflection point of the object. The determination unit determines whether or not a prescribed threshold is exceeded by data based on a plurality of indices calculated at prescribed intervals by the calculation unit. As a result, an object having a Doppler velocity and obviously being in motion is promptly detected as a moving object, whereas, for an object for which no Doppler velocity can be acquired, moving object detection is conducted on the basis of a more accurately estimated velocity by performing tracking for a relatively long period of time, thus making it possible to enhance accuracy in the discrimination between moving object and stationary object.

Description

Information processing device, information processing method, and program

[0001] The present technology relates to an information processing device, an information processing method, and a program that can be applied to object detection and the like.

[0002] Patent Document 1 describes an object detection device that is mounted on a vehicle, sends out a transmission wave, receives reflected waves of the transmission wave from objects present around the vehicle, receives detection result information indicating the intensity, direction, and Doppler velocity of the reflected wave obtained by the radar device, classifies the detection result information into either first detection result information corresponding to a moving object or second detection result information corresponding to a stationary object, and outputs the distance from the radar device to each reflection point of the moving object and the direction of the reflection point, thereby improving the detection accuracy of objects present around the vehicle.

[0003] JP 2018-100899 A

[0004] However, radar can generally only measure the velocity in the radial direction (Doppler velocity) seen from the sensor. In other words, it cannot measure the velocity in the tangential direction on a concentric circle centered on the sensor. Therefore, it is difficult to measure the velocity of a moving object, such as a pedestrian crossing the road in front of a moving vehicle, using a sensor on the vehicle. In addition, there is a possibility that a moving object may be mistaken for a stationary object, making it difficult to detect and track the moving object.

[0005] There is a demand for a technology that can detect moving objects early and improve the accuracy of distinguishing stationary objects.

[0006] In view of the above circumstances, an object of the present technology is to provide an information processing device, an information processing method, and a program that are capable of early detection of moving objects and improving accuracy in discriminating stationary objects.

[0007] In order to achieve the above object, an information processing device according to one embodiment of the present technology includes a calculation unit and a determination unit. The calculation unit calculates an index indicating a likelihood that the object is a moving object based on the Doppler velocity at a reflection point of the object. The determination unit determines whether data based on the multiple indices calculated by the calculation unit at predetermined time intervals exceeds a predetermined threshold.

[0008] This information processing device calculates an index indicating the likelihood that an object is moving based on the Doppler velocity at the object's reflection point. It then determines whether data based on multiple indexes calculated at predetermined time intervals exceeds a predetermined threshold. This enables early detection of moving objects and improved accuracy in distinguishing stationary objects.

[0009] According to one aspect of the present technology, there is provided an information processing method executed by a computer system, the information processing method including: calculating an index indicating a likelihood that an object is a moving object based on a Doppler velocity at a reflection point of the object; and determining whether data based on a plurality of the indexes calculated at predetermined time intervals exceeds a predetermined threshold.

[0010] A program according to one embodiment of the present technology causes a computer system to execute the following steps: calculating an index indicating the likelihood that an object is a moving object based on the Doppler velocity at a reflection point of the object, and determining whether data based on a plurality of the indexes calculated at predetermined time intervals exceeds a predetermined threshold.

[0011] FIG. 1 is a block diagram showing an example of the configuration of a moving body according to the first embodiment. FIG. 2 is a block diagram showing an example of the configuration of a signal processing unit. FIG. 3 is a flowchart showing the operation of a moving object index calculation unit. FIG. 4 is a schematic diagram showing the case where stationary object detection is performed. FIG. 5 is a schematic diagram showing the case where moving object detection is performed. FIG. 6 is a diagram showing the relationship between corrected Doppler velocity VD' and moving object index L according to the first embodiment. FIG. 7 is a diagram showing the environment in which evaluation of the present embodiment was performed and detection results. FIG. 8 is a diagram showing the relationship between corrected Doppler velocity VD' and moving object index L according to the second embodiment. FIG. 9 is a block diagram showing the configuration of a calculation device. FIG. 10 is a diagram showing changes in the relationship between corrected Doppler velocity VD' and moving object index L. FIG. 11 is a schematic diagram showing the case where moving object detection is performed using two radars. FIG. 12 is a block diagram showing an example of the configuration of a vehicle control system. FIG. 13 is a diagram showing an example of a sensing area.

[0012] Hereinafter, embodiments of the present technology will be described with reference to the drawings.

[0013] First Embodiment FIG. 1 is a block diagram showing an example of the configuration of a moving object 1 according to a first embodiment of the present technology.

[0014] As shown in FIG. 1, a mobile object 1 includes an information processing device 10 , a moving speed observation device 90 , and a mobile object control system 100 .

[0015] For example, the mobile body 1 includes an automobile, a drone, etc. In this embodiment, an example is given of detecting an object while the mobile body 1 is moving. Note that the information processing device may be mounted on a device fixed at a predetermined position, such as a traffic light, other than a mobile body.

[0016] The information processing device 10 is mounted on the moving object 1 and includes a radio signal transmitting unit 11 , a radio signal receiving unit 12 , a demodulating unit 13 , an A / D converting unit 14 , a signal processing unit 20 , and a moving object output unit 15 .

[0017] The radio signal transmitter 11 generates a transmission signal and radiates it into space as a transmission wave. The radio signal transmitter 11 also supplies the transmission signal to the demodulator 13.

[0018] The wireless signal receiving unit 12 receives, as reflected waves, transmitted waves reflected at reflection points of objects, including moving objects and stationary objects, present around the mobile body 1. The wireless signal receiving unit 12 also supplies the received signals to the demodulating unit 13. Note that the number of moving objects and stationary objects is not limited, and may be one, multiple, or none at all.

[0019] The demodulator 13 demodulates the received signal using the transmitted signal to generate a demodulated signal, and supplies the demodulated signal to the A / D converter 14.

[0020] The A / D converter 14 converts the demodulated signal into a digital value to generate a digital radar signal, which is then supplied to the signal processor 20. Depending on the radar system, the transmitted signal and the received signal may first be digitized by the A / D converter 14, and then the demodulator 13 may generate the digital radar signal.

[0021] The signal processing unit 20 generates object tracking information of the moving object based on the digital radar signal and self-moving speed information, which is the moving speed of the moving body 1 (information processing device 10), supplied from a moving speed observation device 90 (described later). The signal processing unit 20 also supplies the object tracking information to the moving object output unit 15.

[0022] The moving object output unit 15 outputs the object tracking information to an external device.

[0023] The moving speed observation device 90 observes self-moving speed information including the moving speed of the moving body 1. For example, if the moving body 1 is an automobile, a vehicle speed sensor or the like that observes the speed of the automobile functions as the moving speed observation device 90. In addition to this, an inertial measurement unit (IMU) or the like may also be used as the moving speed observation device 90. Position measurement may also be performed using a satellite positioning system or the like, and the moving speed may be calculated from the results. The self-moving speed information includes various information such as the speed, direction of travel, angular velocity, pitch, roll, and yaw of the moving body 1.

[0024] In addition, in this embodiment, since the information processing device 10 is mounted on the moving body 1, the movement speed of the moving body 1 corresponds to the movement speed of the information processing device 10. Hereinafter, descriptions regarding the position and speed of the information processing device 10 may be treated as corresponding to the position and speed of the moving body 1.

[0025] The mobile object control system 100 controls the movement, stopping, and predetermined operation of the mobile object 1. In Fig. 12 and Fig. 13 described below, a vehicle is given as an example of the mobile object 1. That is, the mobile object control system 100 functions as a vehicle control system 100 that performs processing related to vehicle driving assistance and driving automation.

[0026] FIG. 2 is a block diagram showing an example of the configuration of the signal processing unit 20 shown in FIG.

[0027] As shown in FIG. 2, the signal processing unit 20 includes a reflection point detection unit 21 , a moving object index calculation unit 22 , an object tracking unit 23 , a tracking result storage unit 24 , and a moving object determination unit 25 .

[0028] The reflection point detection unit 21 detects reflection points from the digital radar signal and supplies reflection point information including the distance, direction, Doppler velocity, etc. of the detected reflection points to the moving object index calculation unit 22 and the object tracking unit 23.

[0029] Specifically, in the case of FCM (Fast Chirp Modulation) radar, a range / velocity spectrum is generated by performing FFT (Fast Fourier Transform) processing on a digital radar signal. CFAR (Constant False Alarm Rate) processing is also performed on the range / velocity spectrum to detect spectral peaks as reflection points, and the range and Doppler velocity are obtained from their positions. Furthermore, the direction of the reflection point is estimated using direction-of-arrival estimation algorithms such as digital beamforming and MUSIC (Multiples Signal Classification) using information on the phase difference between complex signals corresponding to multiple transmitting or receiving antennas.

[0030] The moving object index calculation unit 22 calculates a moving object index L, which is an index indicating the likelihood that an object is a moving object, based on the reflection point information and the self-moving speed information supplied from the moving speed observation device 90. In this embodiment, the moving object index calculation unit 22 calculates the moving object index L corresponding to each reflection point for each frame (predetermined time). The calculated moving object index L is output to the object tracking unit 23.

[0031] The index indicating the likelihood that an object is a moving object includes likelihood. Note that the method for calculating likelihood is not limited, and any method may be used for calculation.

[0032] The index may also be a parameter not based on Doppler velocity. For example, the index may include the result of estimating the object type, such as a vehicle, pedestrian, or guardrail, based on the reflection intensity or distribution of reflection points observed by the radar. Of course, Doppler velocity may also be used when estimating the object type.

[0033] The object tracking unit 23 tracks the object and outputs it as object tracking information using the reflection point information, the moving object index L, the self-moving speed information, and the object tracking information of the previous time frame supplied from the tracking result storage unit 24 described later. Specifically, the reflection point information is input to a state estimation filter such as a Kalman filter, and the current position and speed Vtrk(T) of the moving object are estimated by integrating it with past object tracking information.

[0034] The object tracking information includes various information about a moving or stationary object, such as its position, speed, direction of travel, and size. In addition, the information may include the rotation, tilt, vibration, etc. of the moving object. Furthermore, if the information processing device 10 is equipped with a camera or the like capable of capturing an image of a moving object, the tracking information may include the class of the moving object, such as a car or a person, based on image recognition.

[0035] The object tracking information also includes an accumulated moving object index Laccum, which is a value obtained by accumulating the moving object index L in each frame from the moving object index L calculated in a frame at a certain time (e.g., T=1) to the moving object index L calculated in a frame at a certain time (e.g., T=N).

[0036] That is, the integrated moving object index Laccum is added and updated every time a new moving object index L is input. The integrated moving object index Laccum at this time is expressed by the following equation (Equation 1).

[0037] [Math. 1] Laccum(T)=Laccum(T-1)+L

[0038] Here, T indicates the current time frame, and T-1 indicates the previous time frame. If an object has not been detected in the previous time frame and is now detected for the first time, Laccum(T-1)=0 is set, and Laccum(T)=L is output.

[0039] The state estimation filter may be an extended Kalman filter, an unscented Kalman filter, a particle filter, etc. Also, in consideration of the case where multiple reflection points are obtained from one moving object, multiple reflection points that are close in distance, direction, or Doppler velocity may be integrated (clustered) and input to the state estimation filter as a single object.

[0040] Furthermore, since the relative positional relationship between the information processing device 10 and the object changes not only due to the movement of the object but also due to the movement of the information processing device 10, tracking is performed by correcting the influence of the information processing device 10 using self-movement speed information.

[0041] The tracking result storage unit 24 stores the object tracking information, and outputs the object tracking information to the object tracking unit 23 in the next time frame.

[0042] The moving object determination unit 25 outputs, as a moving object detection result, an object that satisfies the following (Equation 2) and (Equation 3) from the object tracking information.

[0043] [Equation 2] |Vtrk(T)|>Vth

[0044] [Math 3] Laccum(T)>Lth

[0045] Here, Vtrk(T) is the estimated velocity of the object included in the object tracking information, and Vth and Lth are predetermined thresholds.

[0046] FIG. 3 is a flowchart showing the operation of the moving object index calculation unit 22.

[0047] 3, reflection point information is acquired by the reflection point detection unit 21 (step 101). Self-moving speed information is acquired by the moving speed observation device 90 (step 102). The reflection point information and self-moving speed information are input to the moving object index calculation unit 22, which performs a loop operation for each reflection point included in the reflection point information and executes the processing of steps 103 to 107 to calculate the moving object index L corresponding to each reflection point.

[0048] In this embodiment, as shown in FIGS. 4 to 6, the moving object index L is calculated from the corrected Doppler velocity VD'.

[0049] Fig. 4 is a schematic diagram showing the case where a stationary object is detected. In Fig. 4, a radar 5 is attached to a moving object 1 in a direction 6 of travel of the moving object 1 (for example, on the front side of the moving object 1). This radar 5 corresponds to a radio signal transmitter 11 and a radio signal receiver 12. The number and attachment positions of the radars 5 are not limited, and they may be attached at will.

[0050] As shown in Fig. 4, a mobile object 1 equipped with an information processing device 10 is moving at a speed Vego in a traveling direction 6. In Fig. 4, the direction in which a stationary object 25 exists relative to the traveling direction is set to θ.

[0051] At this time, the information processing device 10 observes the Doppler velocity VD of the stationary object 25 within the irradiation range 7. VD is given by the following equation (Equation 4).

[0052] [Equation 4] VD=-Vegocosθ Note that the sign of VD is positive in the direction away from the information processing device 10.

[0053] FIG. 5 is a schematic diagram showing a case where a moving object is detected.

[0054] In FIG. 5, a moving object 26 moves at a velocity Vtgt, and the angle formed between the direction 8 in which the moving object 26 is observed from the information processing device 10 and Vtgt is φ.

[0055] In this case, the Doppler velocity VD observed from the moving object 26 includes, in addition to the above-mentioned Vego, a component due to the velocity Vtgt of the moving object 26. Specifically, this is expressed by the following equation (Equation 5).

[0056] [Math. 5] VD=-Vegocosθ+Vtgtcosφ

[0057] That is, from (Equation 4) and (Equation 5), it is possible to determine whether the reflection point is a moving object or a stationary object depending on whether the corrected Doppler velocity VD' = VD + Vegocosθ corrected by the own moving speed of the moving object 1 is close to zero.

[0058] In reality, it should be noted that when cosφ in (Equation 5) becomes small, the corrected Doppler velocity VD′ approaches zero even for a moving object 26 having a velocity in the tangential direction of a concentric circle centered on the sensor.

[0059] 6 is a diagram showing the relationship between the corrected Doppler velocity VD' according to the first embodiment and the moving object index L. In Fig. 6, the vertical axis represents the moving object index L, and the horizontal axis represents the corrected Doppler velocity |VD'|.

[0060] 3 and 6, in this embodiment, when |VD'|≧VDth, L=L2 (YES in step 104, step 105), and when |VD'|<VDth, L=L1 (NO in step 104, step 106). Here, L2>L1>0. VDth is a threshold value.

[0061] When the loop processing of steps 103 to 107 is completed for all reflection points, the moving object index L at each reflection point is output (step 108).

[0062] Objects with a small corrected Doppler velocity |VD'| include both stationary objects and moving objects with a velocity in the tangential direction on a concentric circle centered on the sensor. To distinguish between the two, the estimated velocity Vtrk(T) estimated by the object tracking unit 23 is used to make a determination from (Equation 2). However, the position of the radar reflection point is generally unstable and subject to fluctuations, and due to issues with tracking accuracy, the estimated velocity Vtrk(T) of a stationary object may temporarily increase as a result of object tracking. Therefore, it is necessary to perform tracking for a longer period of time to obtain a more accurate estimation result of Vtrk(T).

[0063] In this embodiment, a relatively small moving object index L is assigned to an object with a small corrected Doppler velocity |VD'|, and therefore it takes a relatively long time to satisfy (Equation 3). In other words, by using the estimated velocity Vtrk(T) obtained as a result of long-term tracking in (Equation 2) to determine whether the object is a moving object, it is possible to distinguish between stationary objects and moving objects with high accuracy.

[0064] Furthermore, for objects with a large corrected Doppler velocity |VD'| (objects that are clearly moving), a relatively large moving object index L is given, and therefore, the moving object determination unit 25 satisfies (Equation 2) and (Equation 3) early on, and the object is detected as a moving object early on.

[0065] In this embodiment, the moving object index calculation unit 22 corresponds to a calculation unit that calculates an index indicating the likelihood that an object is moving, based on the Doppler velocity at the object's reflection point. In this embodiment, the tracking result storage unit 24 corresponds to an object tracking unit that tracks an object based on reflection point information about the object's reflection point, the index, and the speed of the moving body. In this embodiment, the moving object determination unit 25 corresponds to a determination unit that determines whether data based on multiple indexes calculated by the calculation unit at predetermined time intervals exceeds a predetermined threshold.

[0066] 7 is a diagram showing the environment in which the evaluation of this embodiment was conducted and the detection results. As shown in Fig. 7, an area of ​​7.4 m (in the Y-axis direction) on both sides of the moving body 1 was set as the pedestrian detection rate evaluation area, and the detection rate of a pedestrian 9 walking laterally 11 m ahead of the moving body 1 (passing through the pedestrian detection rate evaluation area) was evaluated.

[0067] The conventional method used for comparison discards all reflection points that do not have a corrected Doppler velocity VD', assuming that they are reflections from stationary objects. When a pedestrian 9 crosses in front of the moving object 1, it moves in a tangential direction on a concentric circle centered on the sensor 5, making it impossible to obtain the corrected Doppler velocity VD'. In other words, the conventional method cannot detect the pedestrian 9 when it crosses in front of the moving object 1 (moves in a tangential direction on a concentric circle centered on the sensor 5), and therefore there are cases where detection fails even within the pedestrian detection rate evaluation area.

[0068] On the other hand, in this embodiment, a moving object index L is assigned even to reflection points that do not have a corrected Doppler velocity VD', and moving objects are detected based on the estimated velocity Vtrk(T) obtained as a result of tracking. Therefore, detection continues even at the moment when a pedestrian 9 is walking in front of the moving object 1, and a detection rate of 100% is achieved within the pedestrian detection rate evaluation area. Furthermore, not only in conventional methods but also in this embodiment, there is no erroneous detection of stationary objects as moving objects.

[0069] As described above, the information processing device 10 according to this embodiment calculates an index indicating the likelihood that an object is moving based on the Doppler velocity at the object's reflection point, and determines whether data based on multiple indexes calculated at predetermined time intervals exceeds a predetermined threshold. This enables early detection of moving objects and improved accuracy in distinguishing stationary objects.

[0070] Conventionally, when extracting moving speed based on the Doppler velocity measured from reflection points obtained by radar, depending on the direction of movement of a moving object, the Doppler velocity may not be obtained and the object may be determined to be stationary.

[0071] This technology performs object tracking after assigning moving object indices corresponding to the Doppler velocity to the reflection points of the object, and when the integrated moving object indices exceed a threshold, the object whose estimated velocity obtained by object tracking exceeds a velocity threshold is detected as a moving object. In other words, an object that has a Doppler velocity and is clearly moving will have its integrated indices exceed the threshold early, and will be detected as a moving object early, while an object for which a Doppler velocity cannot be obtained will be tracked for a relatively long time and moving object detection will be performed based on an estimated velocity with increased accuracy, thereby improving the accuracy of distinguishing between moving and stationary objects.

[0072] Second Embodiment In the first embodiment, the relationship between the corrected Doppler velocity VD′ and the moving object index L varies discretely as shown in Fig. 6. In the second embodiment, the moving object index L is calculated based on a curve of the moving object index L that varies continuously with changes in the corrected Doppler velocity VD′.

[0073] In the following description, the description of the same configurations and operations as those of the information processing device 10 described in the above embodiment will be omitted or simplified.

[0074] 8 is a diagram showing the relationship between the corrected Doppler velocity VD' according to the second embodiment and the moving object index L. In Fig. 8, the vertical axis represents the moving object index L, and the horizontal axis represents the corrected Doppler velocity |VD'|.

[0075] 8, the moving object index L changes continuously in response to changes in the corrected Doppler velocity VD', thereby enabling more flexible changes in the time until detection determination. Note that the relationship between the corrected Doppler velocity VD' and the moving object index L may be stored in the form of a lookup table, and the moving object index L corresponding to the corrected Doppler velocity VD' may be read out from the lookup table.

[0076] In addition, when the moving object index L changes continuously in response to changes in the corrected Doppler velocity VD', as in the examples of Figure 8 and Figure 10 described later, the moving object index L corresponds one-to-one to a predetermined corrected Doppler velocity VD', so steps 103 to 106 in the flowchart shown in Figure 3 become one step.

[0077] <Third Embodiment> In the third embodiment, the moving object index calculation unit 22 calculates the moving object index L using a relation curve between the corrected Doppler velocity VD′ and the moving object index L, which is calculated in advance by a calculation device 30 described later, or a lookup table.

[0078] FIG. 9 is a block diagram showing the configuration of the computing device 30.

[0079] As shown in FIG. 9 , the calculation device 30 includes a measurement data recording unit 31, a moving object index calculation unit 32, an object tracking unit 33, a tracking result storage unit 34, a moving object determination unit 35, a detection result evaluation unit 36, and a moving object index setting unit 37.

[0080] The measurement data recording unit 31 records the reflection point information, the subject's own moving speed information, and the correct answer data regarding the presence of a moving object in the measurement environment when measurements were previously performed by the information processing device 10. The measurement data recording unit 31 also shares the reflection point information and the subject's own moving speed information with the moving object index calculation unit 32 and the object tracking unit 33, and outputs the correct answer data to the detection result evaluation unit 36.

[0081] The moving object index calculation unit 32 calculates the moving object index L from the corrected Doppler velocity VD′ calculated from the reflection point information and the self-moving velocity information, based on the relationship between the corrected Doppler velocity VD′ supplied in advance from the moving object index setting unit 37 and the moving object index L, and outputs the calculated moving object index L to the object tracking unit 33.

[0082] The object tracking unit 33 uses the reflection point information, the moving object index L, the self-moving speed information, and the object tracking information of the previous time frame supplied from the tracking result storage unit 34 to track an object present in a measurement environment prepared in advance, and outputs the object tracking information.

[0083] The tracking result storage unit 34 stores the object tracking information, and outputs the object tracking information to the object tracking unit 33 in the next time frame.

[0084] The moving object determination unit 35 outputs, as a moving object detection result, an object that satisfies (Equation 2) and (Equation 3) from the object tracking information.

[0085] The detection result evaluation unit 36 ​​evaluates the moving object detection performance based on the moving object detection result and the ground truth data. For example, the detection result evaluation unit 36 ​​calculates and outputs evaluation results such as Multi-Object Tracking Accuracy (MOTA).

[0086] The moving object index setting unit 37 modifies the relationship between the corrected Doppler velocity VD′ and the moving object index L based on predetermined conditions. For example, the predetermined conditions include various information such as the evaluation result of the moving object detection performance (detection rate of moving objects), erroneous detection of a moving object, the distance to the reflection point, an error in the self-moving speed, and the positional relationship between a moving object or a stationary object and the self-device (information processing device 10).

[0087] In the third embodiment, the moving object index setting unit 37 modifies the relationship between the corrected Doppler velocity VD' and the moving object index L based on the evaluation result of the moving object detection performance.

[0088] 10 is a diagram showing changes in the relationship between the corrected Doppler velocity VD' and the moving object index L. In Fig. 10, the vertical axis represents the moving object index L, and the horizontal axis represents the corrected Doppler velocity |VD'|.

[0089] 10 , when the moving object detection rate is low, the moving object index setting unit 37 sets the state to transition from the initial state setting (solid line 40) to a state (hereinafter referred to as a first state (dotted line 41)) in which a large moving object index L is obtained with a smaller corrected Doppler velocity VD′ (see arrow 43). On the other hand, when a non-existent moving object is erroneously detected, the moving object index setting unit 37 sets the state to transition from the initial state setting (solid line 40) to a state (hereinafter referred to as a second state (dotted line 42)) in which the moving object index L obtained for the corrected Doppler velocity VD′ is smaller (see arrow 44).

[0090] The initial state refers to the relationship between the corrected Doppler velocity VD′ and the moving object index L before being set by the moving object index setting unit 37. For example, the initial state may be an optimal setting at the time of shipment from the factory, or may be a setting that is appropriately set depending on the type of moving object.

[0091] By repeating the above evaluation results of the moving object detection performance and the setting of the relationship between the corrected Doppler velocity VD' and the moving object index L, it is possible to obtain a relationship between the corrected Doppler velocity VD' and the moving object index L that optimizes the moving object detection performance.

[0092] According to the third embodiment, the moving object index calculation unit 32 detects a moving object using the relationship between the corrected Doppler velocity VD′ and the moving object index L, which is set in the calculation device 30 based on the pre-measurement data, and therefore, appropriate moving object detection performance can be obtained.

[0093] Fourth Embodiment In a fourth embodiment, the moving object index calculation unit 32 calculates the moving object index L using the relationship between the corrected Doppler velocity VD′ and the moving object index L, which is set based on the position of the reflection point.

[0094] For example, when a moving object exists near the information processing device 10 (mobile body 1), it is necessary to detect the presence of the object early in order to avoid a collision, etc. On the other hand, when the moving object exists far from the information processing device 10, there is a time leeway.

[0095] 10, the moving object index setting unit 37 transitions from the initial state setting to the first state when there is a reflection point that is close to the information processing device 10. On the other hand, for a reflection point that is far from the information processing device 10, the moving object index setting unit 37 transitions from the initial state setting to the second state.

[0096] That is, when the distance to the reflection point is short, early detection is possible by setting a small corrected Doppler velocity VD' to obtain a large moving object index L. On the other hand, when the distance to the reflection point is long, the tracking time until detection is lengthened by reducing the moving object index L obtained for the corrected Doppler velocity VD', and moving objects are detected based on a more accurate estimated velocity Vtrk(T).

[0097] According to the fourth embodiment, the moving object index calculation unit 32 detects a moving object using the relationship between the corrected Doppler velocity VD′ and the moving object index L, which is set based on the position of the reflection point, and therefore it is possible to achieve a detection time and detection accuracy appropriate for the position of the moving object.

[0098] <Fifth Embodiment> In the fifth embodiment, the moving object index calculation unit 32 calculates the moving object index L using the relationship between the corrected Doppler velocity VD′ and the moving object index L, which is set based on the amount of error included in the self-moving speed information output by the moving speed observation device 90.

[0099] If the self-movement speed information contains an error, an error will be included when calculating the corrected Doppler speed VD' from the Doppler speed VD, so the corrected Doppler speed VD' of a stationary object will have a large value, and a large moving object index L will be given, which may result in the object being detected as a moving object.

[0100] 10 , when the amount of error included in the self-moving speed information is small, the moving object index setting unit 37 transitions from the initial state setting to the first state, and when the amount of error included in the self-moving speed information is large, the moving object index setting unit 37 transitions from the initial state setting to the second state.

[0101] That is, when the amount of error contained in the self-moving velocity information is large, the moving object index L obtained for the corrected Doppler velocity VD' is reduced to suppress erroneous detection of a moving object. On the other hand, when the amount of error contained in the self-moving velocity information is small, the detection rate of a moving object is increased by setting so that a large moving object index L can be obtained with a smaller corrected Doppler velocity VD'.

[0102] The amount of error included in the self-moving speed information may be determined in advance according to the accuracy of the moving speed observation device 90, or may be determined based on an amount of error measured in advance. Furthermore, when the moving speed observation device 90 estimates the self-moving speed information using a Kalman filter or the like, it is possible to output the error variance of the self-moving speed in addition to the estimated value of the self-moving speed, and therefore the relationship between the corrected Doppler speed VD' and the moving object index L may be adaptively changed based on the error variance of the self-moving speed.

[0103] According to the fifth embodiment, the moving object index calculation unit 32 detects a moving object using the relationship between the corrected Doppler velocity VD′ and the moving object index L, which is set based on the amount of error included in the self-moving speed information output by the moving speed observation device 90, and therefore, appropriate moving object detection performance can be obtained.

[0104] Sixth Embodiment In a sixth embodiment, the moving object index calculation unit 32 calculates the moving object index L using the relationship between the corrected Doppler velocity VD′ and the moving object index L based on the detection status of reflection points from moving objects and stationary objects in the previous time frame or earlier.

[0105] 10 , if a reflection point with a large corrected Doppler velocity VD′ (an object that is likely to be a moving object) has been detected in the vicinity of the detected reflection point in the previous time frame or earlier, the moving object index setting unit 37 transitions from the initial state setting to the first state. If a reflection point with a small corrected Doppler velocity VD′ (an object that is likely to be a stationary object) has been detected in the vicinity of the detected reflection point in the previous time frame or earlier, the moving object index setting unit 37 transitions from the initial state setting to the second state.

[0106] That is, if a reflection point with a large corrected Doppler velocity VD' has been detected near the detected reflection point in the previous time frame or earlier, the detection rate of the moving object is increased by setting a smaller corrected Doppler velocity VD' to obtain a large moving object index L. Also, even if the corrected Doppler velocity VD' becomes small due to the positional relationship between the information processing device and the moving object, the detection rate of the moving object is increased.

[0107] On the other hand, if a reflection point with a large corrected Doppler velocity VD' has been detected near the detected reflection point in the previous time frame or earlier, erroneous detection of a moving object is suppressed by reducing the moving object index L obtained for the corrected Doppler velocity VD'. Also, even if a large corrected Doppler velocity VD' is calculated from a reflection point from a stationary object due to measurement accuracy issues, erroneous detection of a moving object can be suppressed.

[0108] Seventh Embodiment In the seventh embodiment, two radars 5 are arranged at different positions on a moving body. Fig. 11 is a schematic diagram showing a case where a moving object is detected using radars 5a and 5b. For simplicity, the moving velocity of the moving body is set to Vego = 0, and the Doppler velocity VD is equal to the corrected Doppler velocity VD'.

[0109] In FIG. 11, the angle between the radar 5a and the moving object 50 is θa (the angle between the lines 51a and 52a), and the angle between the radar 5b and the moving object 50 is θb (the angle between the lines 51b and 52b).

[0110] 11 , when a moving object 50 is moving at a velocity Vtgt, the radars 5a and 5b observe different Doppler velocities VDa' and VDb', respectively. In this case, Vtgt > VDa' and Vtgt > VDb', so the larger of the Doppler velocities VDa' and VDb' has a magnitude closer to Vtgt. Therefore, the moving object index calculation unit 22 calculates the moving object index L using the larger of the Doppler velocities VDa' and VDb'.

[0111] According to the seventh embodiment, depending on the positional relationship between the moving object 50 and the radars 5a and 5b, it may be impossible to observe the corrected Doppler velocity VD' with either radar, but the detection rate of the moving object 50 can be increased by calculating the moving object index L using the observation value of the other radar.

[0112] 11 , in the eighth embodiment, straight lines 55a and 55b perpendicular to the vectors of Doppler velocity VDa' and Doppler velocity VDb' are drawn so as to pass through the tips of the vectors of Doppler velocity VDa' and Doppler velocity VDb', and an intersection 56 of the straight lines 55a and 55b is determined. The vector from the moving object to the intersection 56 is Vtgt. The moving object index calculation unit 22 calculates the moving object index L using Vtgt determined from the intersection.

[0113] According to the eighth embodiment, the detection rate of a moving object can be increased by calculating the moving object index L by estimating the velocity Vtgt of the moving object instead of the corrected Doppler velocity VD'.

[0114] Other Embodiments The present technology is not limited to the above-described embodiments, and various other embodiments can be realized.

[0115] In the above embodiment, the moving speed observation device 90 obtained its own speed from the moving object control system 100. Here, an example of the moving object control system 100 in which the moving object 1 is a vehicle will be given. In other words, the moving object control system 100 is a system that performs processing related to vehicle driving assistance and driving automation. Hereinafter, the moving object control system 100 in which the moving object 1 is a vehicle will be referred to as the vehicle control system 100.

[0116] <Configuration Example of Vehicle Control System> FIG. 12 is a block diagram showing a configuration example of a vehicle control system 100, which is an example of a mobility device control system to which the present technology is applied.

[0117] The vehicle control system 100 is provided in the vehicle 1 and performs processing related to automated driving of the vehicle 1. This automated driving includes automated driving of levels 1 to 5, and remote driving and remote assistance of the vehicle 1 by a remote driver.

[0118] The vehicle control system 100 includes a vehicle control ECU (Electronic Control Unit) 121, a communication unit 122, a map information storage unit 123, a location information acquisition unit 124, an external recognition sensor 125, an in-vehicle sensor 126, a vehicle sensor 127, a memory unit 128, a driving automation control unit 129, a DMS (Driver Monitoring System) 130, an HMI (Human Machine Interface) 131, and a vehicle control unit 132.

[0119] The vehicle control ECU 121, communication unit 122, map information storage unit 123, position information acquisition unit 124, external recognition sensor 125, in-vehicle sensor 126, vehicle sensor 127, memory unit 128, driving automation control unit 129, DMS 130, HMI 131, and vehicle control unit 132 are connected to each other so as to be able to communicate with each other via a communication network 141. The communication network 141 is configured, for example, by an in-vehicle communication network or bus conforming to a digital two-way communication standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), FlexRay (registered trademark), or Ethernet (registered trademark). Different communication networks 141 may be used depending on the type of data being transmitted. For example, CAN may be used for data related to vehicle control, and Ethernet may be used for large-volume data. In addition, each part of the vehicle control system 100 may be directly connected without going through the communication network 141, using wireless communication intended for communication over relatively short distances, such as near field communication (NFC) or Bluetooth (registered trademark).

[0120] In the following description, when each unit of the vehicle control system 100 communicates via the communication network 141, the description of the communication network 141 will be omitted. For example, when the vehicle control ECU 121 and the communication unit 122 communicate via the communication network 141, it will simply be described as the vehicle control ECU 121 and the communication unit 122 communicating with each other.

[0121] The vehicle control ECU 121 is configured by various processors such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc. The vehicle control ECU 121 controls the entire or part of the functions of the vehicle control system 100.

[0122] The communication unit 122 communicates with various devices inside and outside the vehicle, other vehicles, servers, base stations, etc., and transmits and receives various types of data. At this time, the communication unit 122 can communicate using multiple communication methods.

[0123] The following provides an overview of communication with the outside of the vehicle that can be performed by the communication unit 122. The communication unit 122 communicates with a server (hereinafter referred to as an external server) or the like present on an external network via a base station or an access point using a wireless communication method such as 5G (fifth generation mobile communication system), LTE (Long Term Evolution), or DSRC (Dedicated Short Range Communications). The external network with which the communication unit 122 communicates is, for example, the Internet, a cloud network, or a network specific to a carrier. The communication method used by the communication unit 122 with the external network is not particularly limited as long as it is a wireless communication method that enables digital two-way communication at a communication speed equal to or higher than a predetermined distance.

[0124] Furthermore, for example, the communication unit 122 can communicate with a terminal located near the vehicle using P2P (Peer to Peer) technology. The terminal located near the vehicle can be, for example, a terminal worn by a mobile object 1 that moves at a relatively slow speed, such as a pedestrian or a bicycle, a terminal installed at a fixed location in a store, or an MTC (Machine Type Communication) terminal. Furthermore, the communication unit 122 can also perform V2X communication. V2X communication refers to communication between the vehicle and others, such as vehicle-to-vehicle communication with another vehicle, vehicle-to-infrastructure communication with a roadside unit, vehicle-to-home communication, and vehicle-to-pedestrian communication with a terminal carried by a pedestrian.

[0125] The communication unit 122 can receive, for example, a program for updating software that controls the operation of the vehicle control system 100 from the outside (over the air). The communication unit 122 can also receive map information, traffic information, information about the surroundings of the vehicle 1, and the like from the outside. For example, the communication unit 122 can also transmit information about the vehicle 1 and information about the surroundings of the vehicle 1 to the outside. Information about the vehicle 1 that the communication unit 122 transmits to the outside includes, for example, data indicating the state of the vehicle 1 and the recognition result by the recognition unit 173. Furthermore, for example, the communication unit 122 performs communication corresponding to a vehicle emergency notification system such as e-call.

[0126] For example, the communication unit 122 receives electromagnetic waves transmitted by a road traffic information and communication system (VICS (Vehicle Information and Communication System) (registered trademark)) such as a radio beacon, an optical beacon, or FM multiplex broadcasting.

[0127] The following provides an overview of communication with the vehicle interior that can be performed by the communication unit 122. The communication unit 122 can communicate with each device in the vehicle using, for example, wireless communication. The communication unit 122 can communicate with each device in the vehicle using a communication method that enables bidirectional digital communication at a predetermined communication speed or higher via wireless communication, such as wireless LAN, Bluetooth, NFC, or Wireless USB (WUSB). The communication unit 122 can also communicate with each device in the vehicle using wired communication. For example, the communication unit 122 can communicate with each device in the vehicle using wired communication via a cable connected to a connection terminal (not shown). The communication unit 122 can communicate with each device in the vehicle using a communication method that enables bidirectional digital communication at a predetermined communication speed or higher via wired communication, such as Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI) (registered trademark), or Mobile High-Definition Link (MHL).

[0128] Here, the in-vehicle device refers to, for example, a device in the vehicle that is not connected to the communication network 141. Examples of the in-vehicle device include a mobile device or a wearable device carried by a user in the vehicle, such as a driver, and an information device brought into the vehicle and temporarily installed therein.

[0129] The map information storage unit 123 stores one or both of a map acquired from an external source and a map created by the vehicle 1. For example, the map information storage unit 123 stores a three-dimensional high-precision map, a global map that is less accurate than a high-precision map and covers a wide area, and the like.

[0130] Examples of high-precision maps include dynamic maps, point cloud maps, and vector maps. A dynamic map is a map consisting of four layers of dynamic information, quasi-dynamic information, quasi-static information, and static information, and is provided to the vehicle 1 from an external server or the like. A point cloud map is a map made up of a point cloud (point cloud data). A vector map is a map adapted to automated driving by associating traffic information such as the positions of lanes and traffic lights with the point cloud map.

[0131] The point cloud map and the vector map may be provided, for example, from an external server or the like, or may be created by the vehicle 1 based on sensing results from the camera 151, radar 152, LiDAR 153, etc. as a map for matching with a local map described later, and stored in the map information storage unit 123. Furthermore, when a high-precision map is provided from an external server or the like, map data of, for example, an area of ​​several hundred square meters related to the planned route along which the vehicle 1 will travel is acquired from the external server or the like in order to reduce communication capacity.

[0132] The location information acquisition unit 124 receives GNSS (Global Navigation Satellite System) signals from GNSS satellites and acquires location information of the vehicle 1. The acquired location information is supplied to the driving automation control unit 129. Note that the location information acquisition unit 124 is not limited to a method using GNSS signals, and may acquire location information using a beacon, for example.

[0133] The external recognition sensor 125 includes various sensors used to recognize the situation outside the vehicle 1, and supplies sensor data from each sensor to each unit of the vehicle control system 100. The type and number of sensors included in the external recognition sensor 125 are arbitrary.

[0134] For example, the external recognition sensor 125 includes a camera 151, a radar 152, a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) 153, and an ultrasonic sensor 154. Without being limited to this, the external recognition sensor 125 may be configured to include one or more types of sensors selected from the camera 151, the radar 152, the LiDAR 153, and the ultrasonic sensor 154. The number of cameras 151, radars 152, LiDARs 153, and ultrasonic sensors 154 is not particularly limited as long as the number is a number that can be realistically installed on the vehicle 1. Furthermore, the types of sensors included in the external recognition sensor 125 are not limited to this example, and the external recognition sensor 125 may include other types of sensors. Examples of sensing areas of the sensors included in the external recognition sensor 125 will be described later.

[0135] The imaging method of the camera 151 is not particularly limited. For example, cameras of various imaging methods, such as a time-of-flight (ToF) camera, a stereo camera, a monocular camera, and an infrared camera, which are imaging methods capable of distance measurement, can be applied to the camera 151 as needed. However, the camera 151 may simply acquire a photographed image without being related to distance measurement.

[0136] Furthermore, for example, the external recognition sensor 125 may include an environmental sensor for detecting the environment for the vehicle 1. The environmental sensor is a sensor for detecting the environment such as weather, climate, brightness, etc., and may include various sensors such as a raindrop sensor, a fog sensor, a sunlight sensor, a snow sensor, and an illuminance sensor.

[0137] Furthermore, for example, the external recognition sensor 125 includes a microphone used to detect sounds around the vehicle 1 and the location of sound sources.

[0138] The interior sensor 126 includes various sensors for detecting information inside the vehicle, and supplies sensor data from each sensor to each unit of the vehicle control system 100. The types and number of the various sensors included in the interior sensor 126 are not particularly limited as long as they are of types and numbers that can be realistically installed in the vehicle 1.

[0139] For example, the interior sensor 126 may include one or more types of sensors selected from the group consisting of a camera, radar, a seating sensor, a steering wheel sensor, a microphone, and a biometric sensor. The camera included in the interior sensor 126 may be a camera using any of various imaging methods capable of measuring distances, such as a Time of Flight (ToF) camera, a stereo camera, a monocular camera, or an infrared camera. The camera included in the interior sensor 126 may also be a camera simply for acquiring captured images, regardless of distance measurement. The biometric sensor included in the interior sensor 126 is provided, for example, on a seat, a steering wheel, or the like, and detects various types of biometric information of the user.

[0140] The vehicle sensor 127 includes various sensors for detecting the state of the vehicle 1, and supplies sensor data from each sensor to each unit of the vehicle control system 100. The types and number of the various sensors included in the vehicle sensor 127 are not particularly limited as long as they are of types and numbers that can be realistically installed on the vehicle 1.

[0141] For example, the vehicle sensor 127 includes a speed sensor, an acceleration sensor, an angular velocity sensor (gyro sensor), and an inertial measurement unit (IMU) that integrates these sensors. For example, the vehicle sensor 127 includes a steering angle sensor that detects the steering angle of the steering wheel, a yaw rate sensor, an accelerator sensor that detects the amount of accelerator pedal operation, and a brake sensor that detects the amount of brake pedal operation. For example, the vehicle sensor 127 includes a rotation sensor that detects the number of rotations of the engine or motor, an air pressure sensor that detects tire air pressure, a slip ratio sensor that detects tire slip ratio, and a wheel speed sensor that detects the rotation speed of the wheels. For example, the vehicle sensor 127 includes a battery sensor that detects the remaining battery charge and temperature, and an impact sensor that detects external impacts.

[0142] The storage unit 128 includes at least one of a non-volatile storage medium and a volatile storage medium, and stores data and programs. The storage unit 128 is used, for example, as an electrically erasable programmable read-only memory (EEPROM) and a random access memory (RAM). Examples of storage media that can be used include a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, and a magneto-optical storage device. The storage unit 128 stores various programs and data used by each component of the vehicle control system 100. For example, the storage unit 128 includes an event data recorder (EDR) and a data storage system for automated driving (DSSAD), and stores information about the vehicle 1 before and after an event such as an accident, and information acquired by the in-vehicle sensor 126.

[0143] The driving automation control unit 129 controls the driving automation function of the vehicle 1. For example, the driving automation control unit 129 includes an analysis unit 161, an action planning unit 162, and an operation control unit 163.

[0144] The analysis unit 161 performs an analysis process of the vehicle 1 and the surrounding situation. The analysis unit 161 includes a self-position estimation unit 171, a sensor fusion unit 172, and a recognition unit 173.

[0145] The self-position estimation unit 171 estimates the self-position of the vehicle 1 based on the sensor data from the external recognition sensor 125 and the high-precision map stored in the map information storage unit 123. For example, the self-position estimation unit 171 generates a local map based on the sensor data from the external recognition sensor 125 and matches the local map with the high-precision map to estimate the self-position of the vehicle 1. The position of the vehicle 1 is based on, for example, the center of the rear wheel pair axle.

[0146] The local map is, for example, a three-dimensional high-precision map or an occupancy grid map created using a technique such as SLAM (Simultaneous Localization and Mapping). The three-dimensional high-precision map is, for example, the point cloud map described above. The occupancy grid map is a map in which the three-dimensional or two-dimensional space around the vehicle 1 is divided into grids of a predetermined size and the occupancy state of objects is indicated on a grid-by-grid basis. The occupancy state of objects is indicated, for example, by the presence or absence of an object and its probability of existence. The local map is also used, for example, in the detection process and recognition process of the situation outside the vehicle 1 by the recognition unit 173.

[0147] The self-position estimation unit 171 may estimate the self-position of the vehicle 1 based on the position information acquired by the position information acquisition unit 124 and the sensor data from the vehicle sensor 127 .

[0148] The sensor fusion unit 172 performs sensor fusion processing to obtain information by combining multiple different types of sensor data (for example, image data supplied from the camera 151 and sensor data supplied from the radar 152). Methods for combining different types of sensor data include compounding, integration, fusion, and association.

[0149] The recognition unit 173 executes a detection process for detecting the situation outside the vehicle 1 and a recognition process for recognizing the situation outside the vehicle 1 .

[0150] For example, the recognition unit 173 performs detection processing and recognition processing of the situation outside the vehicle 1 based on information from the external recognition sensor 125, information from the self-position estimation unit 171, information from the sensor fusion unit 172, etc.

[0151] Specifically, for example, the recognition unit 173 performs detection processing and recognition processing of objects around the vehicle 1. The object detection processing is, for example, processing to detect the presence or absence, size, shape, position, movement, etc. of an object. The object recognition processing is, for example, processing to recognize attributes such as the type of object, or to identify a specific object. However, the detection processing and the recognition processing are not necessarily clearly separated, and may overlap.

[0152] For example, the recognition unit 173 detects objects around the vehicle 1 by performing clustering to classify a point cloud based on sensor data from the radar 152, the LiDAR 153, or the like into clusters of points. This allows the presence, size, shape, and position of objects around the vehicle 1 to be detected.

[0153] For example, the recognition unit 173 performs tracking to follow the movement of clusters of point clouds classified by clustering, thereby detecting the movement of objects around the vehicle 1. As a result, the speed and traveling direction (movement vector) of the objects around the vehicle 1 are detected.

[0154] For example, the recognition unit 173 detects or recognizes vehicles, people, bicycles, obstacles, structures, roads, traffic lights, traffic signs, road markings, etc. based on image data supplied from the camera 151. Furthermore, the recognition unit 173 may recognize the type of object around the vehicle 1 by performing recognition processing such as semantic segmentation.

[0155] For example, the recognition unit 173 can perform a recognition process of traffic rules around the vehicle 1 based on the map stored in the map information storage unit 123, the estimation result of the self-position by the self-position estimation unit 171, and the recognition result of the objects around the vehicle 1 by the recognition unit 173. Through this process, the recognition unit 173 can recognize the positions and states of traffic lights, the contents of traffic signs and road markings, the contents of traffic regulations, and lanes that can be driven on.

[0156] For example, the recognition unit 173 can perform a recognition process of the environment around the vehicle 1. The surrounding environment to be recognized by the recognition unit 173 may include weather, temperature, humidity, brightness, and road surface conditions.

[0157] The behavior planning unit 162 creates a behavior plan for the vehicle 1. For example, the behavior planning unit 162 creates the behavior plan by performing route planning and route following processing.

[0158] The route planning includes global path planning and local path planning. Global path planning includes a process of planning a rough route from a start to a goal. Local path planning, also called trajectory planning, includes a process of generating a trajectory that allows the vehicle 1 to proceed safely and smoothly in the vicinity of the vehicle 1 on the planned route, taking into account the motion characteristics of the vehicle 1.

[0159] Path following is a process of planning an operation for safely and accurately traveling along a route planned by a route plan within a planned time. The behavior planning unit 162 can, for example, calculate a target speed and a target angular velocity of the vehicle 1 based on the results of this path following process.

[0160] The operation control unit 163 controls the operation of the vehicle 1 in order to realize the action plan created by the action planning unit 162 .

[0161] For example, the operation control unit 163 controls the steering control unit 181, the brake control unit 182, and the drive control unit 183 included in the vehicle control unit 132 described later, to perform lateral vehicle motion control and longitudinal vehicle motion control so that the vehicle 1 travels along the trajectory calculated by the trajectory plan. For example, the operation control unit 163 performs control aimed at driver assistance functions such as collision avoidance or impact mitigation, following driving, vehicle speed maintenance driving, collision warning for the host vehicle, and lane departure warning for the host vehicle, as well as control aimed at driving automation such as driving without operation by the driver or a remote driver.

[0162] The DMS 130 performs processes such as authenticating the driver and recognizing the driver's state based on sensor data from the in-vehicle sensor 126 and input data input to the HMI 131 (described later). Examples of the driver's state to be recognized include physical condition, level of alertness, level of concentration, level of fatigue, line of sight, level of intoxication, driving operation, and posture.

[0163] The DMS 130 may be configured to perform authentication processing for users other than the driver and recognition processing for the status of the users. Furthermore, for example, the DMS 130 may be configured to perform recognition processing for the status inside the vehicle based on sensor data from the in-vehicle sensor 126. Examples of the status inside the vehicle that may be recognized include temperature, humidity, brightness, and odor.

[0164] The HMI 131 inputs various data and instructions and presents various data to the user.

[0165] The following provides an overview of data input via the HMI 131. The HMI 131 includes an input device for a person to input data. The HMI 131 generates input signals based on data, instructions, and the like input via the input device and supplies the signals to each component of the vehicle control system 100. The HMI 131 includes, as input devices, controls such as a touch panel, buttons, switches, and levers. The HMI 131 may also include input devices that allow information to be input by voice, gestures, or other means other than manual operation. Furthermore, the HMI 131 may use, as input devices, externally connected devices such as a remote control device using infrared or radio waves, or a mobile or wearable device compatible with the operation of the vehicle control system 100.

[0166] The presentation of data by the HMI 131 will be briefly described. The HMI 131 generates visual information, auditory information, and tactile information for the user or the outside of the vehicle. The HMI 131 also performs output control, controlling the output, output content, output timing, output method, etc. of each piece of generated information. The HMI 131 generates and outputs, as visual information, information indicated by images or lights, such as an operation screen, a status display of the vehicle 1, a warning display, and a monitor image showing the situation around the vehicle 1. The HMI 131 also generates and outputs, as auditory information, information indicated by sounds, such as voice guidance, warning sounds, and warning messages. The HMI 131 also generates and outputs, as tactile information, information imparted to the user's sense of touch by force, vibration, movement, etc.

[0167] The output device to which the HMI 131 outputs visual information may be, for example, a display device that presents visual information by displaying an image itself or a projector device that presents visual information by projecting an image. The display device may be a device that displays visual information within the user's field of view, such as a head-up display, a transmissive display, or a wearable device with an augmented reality (AR) function, in addition to a display device having a normal display. The HMI 131 may also use display devices included in a navigation system, an instrument panel, a camera monitoring system (CMS), an electronic mirror, a lamp, or the like provided in the vehicle 1 as output devices that output visual information.

[0168] As an output device for the HMI 131 to output auditory information, for example, an audio speaker, headphones, or earphones can be applied.

[0169] For example, a haptic element using haptic technology can be applied as an output device for outputting tactile information from the HMI 131. The haptic element is provided on a part that the user touches, such as a steering wheel or a seat.

[0170] The vehicle control unit 132 controls each unit of the vehicle 1. The vehicle control unit 132 includes a steering control unit 181, a brake control unit 182, a drive control unit 183, a body system control unit 184, a light control unit 185, and a horn control unit 186.

[0171] The steering control unit 181 detects and controls the state of the steering system of the vehicle 1. The steering system includes, for example, a steering mechanism including a steering wheel, an electric power steering, etc. The steering control unit 181 includes, for example, a steering ECU that controls the steering system, an actuator that drives the steering system, etc.

[0172] The brake control unit 182 detects and controls the state of the brake system of the vehicle 1. The brake system includes, for example, a brake mechanism including a brake pedal, an antilock brake system (ABS), a regenerative brake mechanism, etc. The brake control unit 182 includes, for example, a brake ECU that controls the brake system, an actuator that drives the brake system, etc.

[0173] The drive control unit 183 detects and controls the state of the drive system of the vehicle 1. The drive system includes, for example, an accelerator pedal, a drive force generating device for generating drive force such as an internal combustion engine or a drive motor, and a drive force transmission mechanism for transmitting the drive force to the wheels. The drive control unit 183 includes, for example, a drive ECU for controlling the drive system, and an actuator for driving the drive system.

[0174] The body system control unit 184 detects and controls the states of the body system systems of the vehicle 1. The body system systems include, for example, a keyless entry system, a smart key system, a power window device, a power seat, an air conditioning system, an airbag, a seat belt, a shift lever, etc. The body system control unit 184 includes, for example, a body system ECU that controls the body system systems, an actuator that drives the body system systems, etc.

[0175] The light control unit 185 detects and controls the states of various lights of the vehicle 1. Examples of lights to be controlled include headlights, backlights, fog lights, turn signals, brake lights, projections, and bumper displays. The light control unit 185 includes a light ECU that controls the lights, an actuator that drives the lights, and the like.

[0176] The horn control unit 186 detects and controls the state of the car horn of the vehicle 1. The horn control unit 186 includes, for example, a horn ECU that controls the car horn, an actuator that drives the car horn, and the like.

[0177] Fig. 13 is a diagram showing an example of a sensing area by the camera 151, radar 152, LiDAR 153, ultrasonic sensor 154, etc. of the external recognition sensor 125 in Fig. 12. Note that Fig. 13 schematically shows the vehicle 1 as seen from above, with the left end side being the front end (front) side of the vehicle 1 and the right end side being the rear end (rear) side of the vehicle 1.

[0178] Sensing area 101F and sensing area 101B are examples of sensing areas of the ultrasonic sensors 154. Sensing area 101F covers the periphery of the front end of the vehicle 1 with a plurality of ultrasonic sensors 154. Sensing area 101B covers the periphery of the rear end of the vehicle 1 with a plurality of ultrasonic sensors 154.

[0179] The sensing results in the sensing area 101F and the sensing area 101B are used, for example, for parking assistance for the vehicle 1.

[0180] Sensing area 102F to sensing area 102B show examples of sensing areas of a short-range or medium-range radar 152. Sensing area 102F covers a position farther in front of the vehicle 1 than sensing area 101F. Sensing area 102B covers a position farther behind the vehicle 1 than sensing area 101B. Sensing area 102L covers the periphery behind the left side of the vehicle 1. Sensing area 102R covers the periphery behind the right side of the vehicle 1.

[0181] The sensing results in sensing area 102F are used, for example, to detect vehicles, pedestrians, and the like that are present in front of the vehicle 1. The sensing results in sensing area 102B are used, for example, for a collision prevention function behind the vehicle 1. The sensing results in sensing area 102L and sensing area 102R are used, for example, to detect objects in blind spots on the sides of the vehicle 1.

[0182] Sensing areas 103F to 103B show examples of sensing areas sensed by camera 151. Sensing area 103F covers a position farther in front of vehicle 1 than sensing area 102F. Sensing area 103B covers a position farther in the rear of vehicle 1 than sensing area 102B. Sensing area 103L covers the periphery of the left side of vehicle 1. Sensing area 103R covers the periphery of the right side of vehicle 1.

[0183] The sensing results in the sensing area 103F can be used for, for example, recognition of traffic lights and traffic signs, lane departure prevention assistance systems, and automatic headlight control systems. The sensing results in the sensing area 103B can be used for, for example, parking assistance and surround view systems. The sensing results in the sensing areas 103L and 103R can be used for, for example, surround view systems.

[0184] Sensing area 104 shows an example of the sensing area of ​​LiDAR 153. Sensing area 104 covers a position farther ahead of vehicle 1 than sensing area 103F. On the other hand, sensing area 104 has a narrower range in the left-right direction than sensing area 103F.

[0185] The sensing results in the sensing area 104 are used to detect objects such as surrounding vehicles, for example.

[0186] Sensing area 105 shows an example of the sensing area of ​​long-range radar 152. Sensing area 105 covers a position further ahead of vehicle 1 than sensing area 104. On the other hand, sensing area 105 has a narrower range in the left-right direction than sensing area 104.

[0187] The sensing results in the sensing area 105 are used for, for example, adaptive cruise control (ACC), emergency braking, collision avoidance, and the like.

[0188] The sensing areas of the cameras 151, radar 152, LiDAR 153, and ultrasonic sensors 154 included in the external recognition sensor 125 may have various configurations other than those shown in FIG. 13 . Specifically, the ultrasonic sensors 154 may also sense the sides of the vehicle 1, and the LiDAR 153 may sense the rear of the vehicle 1. The installation positions of the sensors are not limited to the above-described examples. The number of each sensor may be one or more.

[0189] The configurations of the signal processing unit, reflection point detection unit, moving object tracking unit, etc., and the control flow of the moving object extraction unit, etc., which have been described with reference to the drawings, are merely one embodiment and can be modified as desired without departing from the spirit of the present technology. In other words, any other configurations, algorithms, etc. for implementing the present technology may be adopted.

[0190] It should be noted that the effects described in this disclosure are merely examples and are not limiting, and other effects may also be present. The description of multiple effects above does not necessarily mean that these effects are exhibited simultaneously. It means that at least one of the effects described above can be obtained depending on the conditions, etc., and of course, effects not described in this disclosure may also be exhibited.

[0191] It is also possible to combine at least two of the characteristic features of each embodiment described above. In other words, the various characteristic features described in each embodiment may be combined in any manner without distinguishing between the embodiments.

[0192] Note that the present technology can also adopt the following configurations. (1) An information processing device comprising: a calculation unit that calculates an index indicating the likelihood that an object is a moving object based on a Doppler velocity at a reflection point of the object; and a determination unit that determines whether data based on the plurality of indexes calculated by the calculation unit at predetermined time intervals exceeds a predetermined threshold. (2) The information processing device described in (1), wherein the determination unit determines whether an integrated value of the plurality of indexes in past frames and the index in a current frame exceeds the predetermined threshold. (3) The information processing device described in (2), wherein the determination unit determines that the object is a moving object when data based on the indexes exceeds a predetermined threshold. (4) The information processing device described in (1), wherein the information processing device is mounted on a moving object. (5) The information processing device described in (4), wherein the calculation unit calculates the index according to a corrected Doppler velocity based on the Doppler velocity and the velocity of the moving object. (6) The information processing device according to (5), further comprising an index setting unit that sets the index discretely or continuously in response to a change in the corrected Doppler velocity. (7) The information processing device according to (6), wherein the calculation unit calculates a first index when the corrected Doppler velocity does not exceed the threshold, and calculates a second index larger than the first index when the corrected Doppler velocity exceeds the threshold. (8) The information processing device according to (6), wherein the index setting unit sets a characteristic of the index determined by the corrected Doppler velocity to an initial state, a first state, or a second state based on a predetermined condition, wherein the first state is a state in which the index is larger than that in the initial state, and the second state is a state in which the index is smaller than that in the initial state. (9) An information processing device according to (8), wherein the predetermined conditions include a detection rate of a moving object, a false detection of a moving object, a distance between the reflection point and the moving body, an error in the speed of the moving body, or a positional relationship between a reflection point of a moving object, a reflection point of a stationary object, and the moving body.(10) The information processing device according to (9), wherein the index setting unit changes the initial state to the first state when the detection rate of the moving object is low, and changes the initial state to the second state when the moving object is erroneously detected. (11) The information processing device according to (9), wherein the index setting unit changes the initial state to the first state when the distance between the reflection point and the moving object is short, and changes the initial state to the second state when the distance between the reflection point and the moving object is long. (12) The information processing device according to (9), wherein the index setting unit changes the initial state to the first state when the error is small, and changes the initial state to the second state when the error is large. (13) The information processing device according to (9), wherein the index setting unit changes the initial state to the first state when the moving object is present near the moving object, and changes the initial state to the second state when the stationary object is present near the moving object. (14) The information processing device according to (4), further comprising an object tracking unit that tracks the object based on reflection point information related to reflection points of the object, the index, and the speed of the moving body, wherein the object tracking unit outputs object tracking information including at least an estimated speed of the object. (15) The information processing device according to (14), wherein the determination unit determines that the object is a moving object when data based on the index exceeds the predetermined threshold and when the estimated speed exceeds a speed threshold indicating a predetermined value. (16) The information processing device according to (4), wherein the moving body has at least two or more sensors that output a transmission wave to the object and receive a reflected wave reflected from the object, and the calculation unit calculates the index based on a corrected Doppler speed having a larger value from corrected Doppler speeds based on the Doppler speeds related to each of the sensors and the speed of the moving body.(17) An information processing method executed by a computer system, which calculates an index indicating the likelihood that an object is a moving object based on the Doppler velocity at the object's reflection point, and determines whether data based on a plurality of said indices calculated at predetermined time intervals exceeds a predetermined threshold. (18) A program that causes a computer system to execute the steps of: calculating an index indicating the likelihood that an object is a moving object based on the Doppler velocity at the object's reflection point; and determining whether data based on a plurality of said indices calculated at predetermined time intervals exceeds a predetermined threshold.

[0193] DESCRIPTION OF SYMBOLS 1... Moving object 10... Information processing device 11... Wireless signal transmitting unit 12... Wireless signal receiving unit 20... Signal processing unit 22... Moving object index calculating unit 23... Object tracking unit 25... Moving object determining unit 37... Moving object index setting unit 100... Moving object control system

Claims

1. An information processing device comprising: a calculation unit that calculates an index indicating the likelihood that an object is a moving object based on the Doppler velocity at the object's reflection point; and a determination unit that determines whether data based on multiple indexes calculated by the calculation unit at predetermined time intervals exceeds a predetermined threshold.

2. An information processing device according to claim 1, wherein the determination unit determines whether or not a value obtained by integrating the indexes in the past frames and the index in the current frame exceeds the predetermined threshold value.

3. An information processing device according to claim 2, wherein the determination unit determines that the object is a moving object when data based on the index exceeds a predetermined threshold.

4. An information processing device according to claim 1, wherein the information processing device is mounted on a moving object.

5. An information processing device according to claim 4, wherein the calculation unit calculates the index according to a corrected Doppler velocity based on the Doppler velocity and the velocity of the moving body.

6. An information processing device according to claim 5, further comprising an index setting unit that sets the index discretely or continuously in response to changes in the corrected Doppler velocity.

7. An information processing device according to claim 6, wherein the calculation unit calculates a first index when the corrected Doppler velocity does not exceed the threshold, and calculates a second index greater than the first index when the corrected Doppler velocity exceeds the threshold.

8. An information processing device according to claim 6, wherein the index setting unit sets the characteristics of the index determined by the corrected Doppler velocity to an initial state, a first state, or a second state based on predetermined conditions, the first state being a state in which a larger index is obtained compared to the initial state, and the second state being a state in which a smaller index is obtained compared to the initial state.

9. An information processing device according to claim 8, wherein the predetermined conditions include a detection rate of a moving object, a false detection of a moving object, a distance between the reflection point and the moving object, an error in the speed of the moving object, or a positional relationship between the reflection point of a moving object, a reflection point of a stationary object and the moving object.

10. An information processing device according to claim 9, wherein the index setting unit changes the initial state to the first state when the detection rate of the moving object is low, and changes the initial state to the second state when the moving object is erroneously detected.

11. An information processing device according to claim 9, wherein the index setting unit sets the initial state to the first state when the distance between the reflection point and the moving body is short, and sets the initial state to the second state when the distance between the reflection point and the moving body is long.

12. An information processing device according to claim 9, wherein the index setting unit changes the initial state to the first state when the error is small, and changes the initial state to the second state when the error is large.

13. An information processing device according to claim 9, wherein the indicator setting unit changes the initial state to the first state when the moving object is present in the vicinity of the moving body, and changes the initial state to the second state when the stationary object is present in the vicinity of the moving body.

14. An information processing device according to claim 4, further comprising an object tracking unit that tracks the object based on reflection point information relating to the reflection points of the object, the index, and the speed of the moving body, and the object tracking unit outputs object tracking information that includes at least the estimated speed of the object.

15. An information processing device according to claim 14, wherein the determination unit determines that the object is a moving object when the data based on the index exceeds the predetermined threshold and when the estimated speed exceeds a speed threshold indicating a predetermined value.

16. An information processing device according to claim 4, wherein the moving body has at least two sensors that output a transmission wave to the object and receive a reflected wave reflected from the object, and the calculation unit calculates the index based on a corrected Doppler velocity having a larger value from corrected Doppler velocities based on the Doppler velocities for each of the sensors and the velocity of the moving body.

17. An information processing method implemented by a computer system, which calculates an index indicating the likelihood that an object is a moving object based on the Doppler velocity at the object's reflection point, and determines whether data based on multiple such indices calculated at specified time intervals exceeds a specified threshold.

18. A program that causes a computer system to execute the steps of: calculating an index indicating the likelihood that an object is a moving object based on the Doppler velocity at the object's reflection point; and determining whether data based on multiple such indexes calculated at specified time intervals exceeds a specified threshold.

Images