Shape estimation device
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-18
- Publication Date
- 2026-04-02
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a shape estimation device that estimates the shape of an object. [Background technology]
[0002] For example, Non-Patent Document 1 discloses a technology for estimating the shape of an object using an on-board radar. Specifically, Non-Patent Document 1 proposes a method for estimating the shape of an object from a reflection intensity map that indicates the correspondence between the position and the reflection intensity, which is the signal strength of the reflected wave received by the radar, using an estimator that is generated in advance by deep learning. [Prior art documents] [Non-patent literature]
[0003] [Non-Patent Document 1] FE Nowruzi; D. Kolhatkar; P. Kapoor; F. Al Hassanat; EJ Heravi; R. Laganiere; J. Rebut; W. Malik, Deep Open Space Segmentation using Automotive Radar, 2020 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM) Summary of the Invention [Problem to be solved by the invention]
[0004] In the technology described in Non-Patent Document 1, when generating a reflection intensity map, it is necessary to perform azimuth estimation processing from the received signals of multiple antennas of the radar, which causes problems such as a high processing load and high costs due to the need for multiple antenna elements.
[0005] Therefore, the inventors have focused on the fact that, for example, a radar can observe the relative speed. More specifically, the relative speed of a stationary object existing around the vehicle is determined by the angle that a line connecting the position of the radar installed on the vehicle and a reflection point of the stationary object that reflects the transmitted wave forms with the traveling direction of the vehicle, and the speed of the vehicle. Therefore, if the speed of the vehicle and the above-mentioned relative speed are known, the position of the reflection point of the stationary object can be identified. The inventors have focused on this fact.
[0006] On the other hand, the distribution of reflection intensity with respect to position converted from the relative velocity obtained by radar is generally not suitable for estimating the shape of an object due to the spread of the radar signal, but shape estimation using deep learning can learn the signal spread as well, making it possible to estimate the shape of an object using the relative velocity.
[0007] In view of the above, the present invention aims to estimate the shape of a stationary object with low processing load and high accuracy by utilizing the fact that a distribution of reflection intensity versus relative distance and relative speed to a sensor unit such as a radar can be obtained based on a signal from the sensor unit. [Means for solving the problem]
[0008] In order to achieve the above object, the shape estimation device according to claim 1 comprises: A shape estimation device mounted on a moving body (80), a sensor unit (12) provided in the moving body, which transmits a transmission wave (Sw) which is an electromagnetic wave to the surroundings of the moving body and receives a reflected wave (Rw) generated by reflection of the transmission wave; a first map generator (16) that repeatedly generates a first reflection intensity map (M1) based on a transmitted wave and a reflected wave, the first reflection intensity map (M1) indicating a distribution of a signal intensity (Pw) of the reflected wave versus a relative distance (Rr) with respect to the sensor unit and a relative speed (Vr) with respect to the sensor unit; an ego-motion acquisition unit (18) for acquiring the velocity (Ve) and the position (Xe, Ye) of a moving object; a second map generating unit (20) that repeatedly generates a second reflection intensity map (M2) indicating a distribution of signal intensity with respect to coordinate position from the first reflection intensity map by converting a first coordinate system (CT1) indicating the relative distance and relative speed of the first reflection intensity map into a second coordinate system (CT2) indicating two-dimensional coordinate position (X, Y) using the speed and position of the moving body; a reflection intensity accumulation map generating unit (22) that calculates a signal intensity related value for each coordinate position of the second reflection intensity map according to a predetermined calculation rule based on the signal intensity in each of the plurality of second reflection intensity maps generated by the second map generating unit, and generates a reflection intensity accumulation map (Ms) that indicates a distribution of the signal intensity related values for the coordinate positions; The system further includes an estimation unit (24) that estimates an object shape map (Mf) representing the shape of a stationary object (82) existing around the moving object from the reflection intensity accumulation map using a trained model (32) of a neural network trained by supervised learning.
[0009] In this way, a second reflection intensity map showing the distribution of reflection intensity with respect to two-dimensional coordinate positions can be obtained from a first reflection intensity map showing the distribution of the signal strength (i.e., reflection intensity) of the reflected wave with respect to the relative distance and relative speed. That is, the second reflection intensity map that serves as the basis for estimating the shape of a stationary object can be obtained without performing an orientation estimation process for estimating the orientation with respect to the sensor unit. Therefore, it is possible to estimate the shape of a stationary object with a low processing load.
[0010] Since the reflection intensity accumulation map is generated based on the multiple second reflection intensity maps that are generated sequentially, the reflection intensity accumulation map can highlight the distribution state of the reflection intensity corresponding to the contour of the stationary object whose shape is to be estimated. This reflection intensity accumulation map is then used as an input to an estimation unit that performs shape estimation of the stationary object. Therefore, it is possible to estimate the shape of the stationary object with high accuracy, compared to, for example, a case where the second reflection intensity map, rather than the reflection intensity accumulation map, is used as an input to the estimation unit.
[0011] In addition, in each section of the application documents, each element may be given a reference symbol in parentheses. In this case, the reference symbol merely indicates an example of the correspondence between the element and the specific configuration described in the embodiment described later. Therefore, the present invention is not limited in any way by the description of the reference symbol. [Brief description of the drawings]
[0012] [Figure 1] 1 is a block diagram showing a schematic functional configuration of a shape estimation device in a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an object detection region of a sensor unit in the first embodiment. [Diagram 3] 3 is a view taken along the arrow III in FIG. 2, and is a front view showing a sensor unit alone in a schematic manner. [Figure 4] 4 is a flowchart showing a control process executed by an electronic control device included in the shape estimation device in the first embodiment. [Diagram 5] FIG. 5 is a diagram illustrating a first reflection intensity map generated in step S102 of FIG. 4. [Figure 6] FIG. 6 is a diagram for explaining the relationship between the relative distance and relative velocity shown in FIG. 5 and the coordinate position of a reflection point on a stationary object. [Figure 7] FIG. 5 is a diagram illustrating a second reflection intensity map generated in step S104 of FIG. [Figure 8] 5 is a flowchart showing a subroutine executed in step S104 of FIG. 4. [Figure 9] FIG. 5 is a diagram illustrating a reflection intensity accumulation map generated in step S105 of FIG. [Figure 10] 5 is a flowchart showing a subroutine executed in step S105 of FIG. 4. [Figure 11] FIG. 2 is a diagram illustrating a schematic configuration of a trained model of a neural network used to estimate an object shape map from a reflection intensity accumulation map in the first embodiment. [Figure 12A]FIG. 11 is a diagram showing an example of a reflection intensity accumulation map, which is one of a set of teacher data included in a plurality of sets of teacher data used for learning a trained model in the first embodiment. [Figure 12B] 12B is a diagram showing an example of a correct shape map which is the other of the set of training data in the first embodiment, that is, an example of a correct shape map which constitutes the set of training data together with the reflection intensity accumulation map of FIG. 12A. [Figure 13] FIG. 5 is a front view showing a sensor unit according to a second embodiment in a simplified manner, the view corresponding to FIG. 3. [Figure 14] FIG. 11 is a front view showing a sensor unit according to a third embodiment in a simplified manner, the view corresponding to FIG. [Figure 15] FIG. 13 is a diagram illustrating an object detection region of a sensor unit in a fourth embodiment, the diagram corresponding to FIG. 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, each embodiment will be described with reference to the drawings. In the following embodiments, the same reference numerals are given to parts that are the same or equivalent to each other.
[0014] (First embodiment) 1 and 2, a shape estimation device 10 of this embodiment is a device mounted on a vehicle 80, which is a moving body. The shape estimation device 10 includes a sensor unit 12 and an electronic control device 14. The shape estimation device 10 estimates the shape of a stationary object 82 existing around the vehicle 80, i.e., the shape of the stationary object 82 existing around the vehicle 80, by emitting radio waves from the sensor unit 12 to the surroundings of the vehicle 80 while the vehicle 80 is stopped or traveling.
[0015] Note that the double-ended arrows shown in Fig. 2 and Fig. 3 described later indicate the direction of the vehicle 80 on which the shape estimation device 10 is mounted. That is, in Fig. 2 and Fig. 3, a vehicle longitudinal direction D1, which is the front-rear direction of the vehicle 80, a vehicle lateral direction D2, which is the left-right direction of the vehicle 80, and a vehicle vertical direction D3, which is the up-down direction of the vehicle 80, are each indicated by a double-ended arrow. These directions D1, D2, and D3 intersect with each other, or strictly speaking, are perpendicular to each other. Also, in Fig. 3, a horizontal direction Dh is indicated by a double-ended arrow. Each of the vehicle longitudinal direction D1 and the vehicle lateral direction D2 is one direction of the horizontal direction Dh.
[0016] The sensor unit 12 is a sensor device fixed to a part of the vehicle 80. As shown in Fig. 1 to Fig. 3, the sensor unit 12 is, for example, an FCM type millimeter wave radar device. "FCM" is an abbreviation for "Fast Chirp Modulation". That is, the sensor unit 12 transmits a transmission wave Sw, which is an electromagnetic wave, to the periphery of the vehicle 80, and receives a reflected wave Rw generated by reflection of the transmission wave Sw. To this end, the sensor unit 12 has a transmitting antenna element 121, a receiving antenna element 122, and a signal processing circuit (not shown) that processes electrical signals related to the transmission of the transmission wave Sw and the reception of the reflected wave Rw.
[0017] The sensor unit 12 also transmits the transmission wave Sw and receives the reflected wave Rw, and outputs a transmission wave signal representing the transmission wave Sw and a reflected wave signal representing the reflected wave Rw to, for example, the electronic control device 14.
[0018] The transmitting antenna element 121 is an antenna element that transmits a transmission wave Sw, and the receiving antenna element 122 is an antenna element that receives a reflected wave Rw. The sensor unit 12 of this embodiment has one transmitting antenna element 121, and the sensor unit 12 also has one receiving antenna element 122. Therefore, the sensor unit 12 does not have a configuration in which multiple transmitting antenna elements 121 are arranged in the horizontal direction Dh, and does not have a configuration in which multiple receiving antenna elements 122 are arranged in the horizontal direction Dh. For example, in this embodiment, a single transmitting antenna element 121 and a single receiving antenna element 122 are arranged in the horizontal direction Dh.
[0019] Further, the sensor unit 12 is disposed, for example, biased toward the front side of the vehicle 80 in the vehicle longitudinal direction D1 and toward one side (specifically, the left side) in the vehicle lateral direction D2. The sensor unit 12 is attached to the vehicle 80 such that the object detection area As of the sensor unit 12 extends only to the left side of the vehicle 80 in the vehicle lateral direction D2. In short, the sensor unit 12 is attached to the vehicle 80 with the front surface of the sensor unit 12 facing toward the left side in the vehicle lateral direction D2.
[0020] The object detection area As of the sensor unit 12 is the maximum range in which the sensor unit 12 can detect the presence of a detectable object, such as a stationary object 82 or a moving object, by transmitting the transmission wave Sw and receiving the reflected wave Rw. The relative positional relationship of the object detection area As of the sensor unit 12 with respect to the vehicle 80 is determined by the specifications of the sensor unit 12 and the mounting orientation of the sensor unit 12 on the vehicle 80.
[0021] The electronic control unit 14 is configured as an in-vehicle microcomputer equipped with a CPU, RAM, ROM, non-volatile rewritable memory, etc. (not shown). That is, the electronic control unit 14 reads and executes a computer program stored in a recording medium such as a ROM or a non-volatile rewritable memory, which is a non-transient substantial recording medium. By executing this computer program, a method corresponding to the computer program is executed. That is, the electronic control unit 14 executes various control processes according to the computer program.
[0022] The electronic control device 14 has a first map generating section 16, an egomotion acquiring section 18, a second map generating section 20, a reflection intensity accumulated map generating section 22, and an estimating section 24 as functional sections that execute the respective functions of the electronic control device 14. The electronic control device 14 also has a storage section 26 that is a recording medium such as a RAM.
[0023] Fig. 4 is a flowchart showing the control process executed by the electronic control unit 14. The electronic control unit 14 repeats the process from steps S101 to S106 in Fig. 4 at a predetermined interval, with the process from steps S101 to S106 constituting one cycle.
[0024] 4, the electronic control device 14 first causes the sensor unit 12 to transmit a transmission wave Sw and receive a reflected wave Rw in step S101. For example, the sensor unit 12 executes a transmission / reception process of transmitting a transmission wave Sw and receiving a reflected wave Rw multiple times, and outputs an electrical signal indicating the transmission wave Sw and an electrical signal indicating the reflected wave Rw to the electronic control device 14. After step S101 in FIG. 4, the process proceeds to step S102.
[0025] In step S102, the first map generator 16 receives an electrical signal indicative of the transmitted wave Sw and an electrical signal indicative of the reflected wave Rw from the sensor unit 12. Then, the first map generator 16 generates a first reflection intensity map M1 of Fig. 5 based on the transmitted wave Sw and the reflected wave Rw. The first reflection intensity map M1 is a map showing the distribution of the signal intensity Pw of the reflected wave Rw with respect to the relative distance Rr with respect to the sensor unit 12 and the relative speed Vr with respect to the sensor unit 12, as shown in Fig. 5.
[0026] In the description of this embodiment, the signal intensity Pw of the reflected wave Rw is also referred to as the reflection intensity Pw. Strictly speaking, the relative distance Rr to the sensor unit 12 is the relative distance Rr to a sensor origin R0 (see FIG. 2) included in the sensor unit 12, and the relative velocity Vr to the sensor unit 12 is the relative velocity Vr to the sensor origin R0. At the sensor origin R0, the relative distance Rr is "Rr=0".
[0027] For example, the first map generator 16 performs frequency analysis processing, such as two FFT processes, based on the electrical signal representing the transmitted wave Sw and the electrical signal representing the reflected wave Rw to obtain a first reflection intensity map M1. The "FFT" in the description of the FFT processing is an abbreviation for "Fast Fourier Transform." After step S102 in FIG. 4, the process proceeds to step S103.
[0028] In step S103, the egomotion acquiring unit 18 acquires the host vehicle speed Ve, which is the speed Ve of the vehicle 80, and the host vehicle position, which is the position of the vehicle 80. For example, the egomotion acquiring unit 18 acquires the host vehicle speed Ve from the vehicle speed sensor 84 of FIG. 1 provided on the vehicle 80. The egomotion acquiring unit 18 may also acquire the host vehicle position by calculating it from the steering angle of the vehicle 80 and the host vehicle speed Ve, or may acquire the host vehicle position from a navigation system provided on the vehicle 80.
[0029] Strictly speaking, the host vehicle position is the position of the host vehicle position origin E0 indicating the representative position of the vehicle 80, as shown in Fig. 6. For example, in a second coordinate system CT2 described later, the host vehicle position, which is the position of the host vehicle position origin E0, is the coordinate position Xe, Ye of the host vehicle position origin E0. Therefore, in the following description, the host vehicle position may be described as the host vehicle position Xe, Ye. After step S103 in Fig. 4, the process proceeds to step S104.
[0030] In step S104, the second map generation unit 20 generates the second reflection intensity map M2 of Figure 7 based on the vehicle speed Ve and vehicle positions Xe, Ye acquired by the egomotion acquisition unit 18 and the first reflection intensity map M1 generated by the first map generation unit 16.
[0031] In detail, the second map generation unit 20 converts the first coordinate system CT1, which represents the relative distance Rr and the relative speed Vr in the first reflection intensity map M1 in Fig. 5, into a second coordinate system CT2, which represents two-dimensional coordinate positions X and Y, using the host vehicle speed Ve and the host vehicle positions Xe and Ye. Through this coordinate conversion, the second map generation unit 20 generates the second reflection intensity map M2 in Fig. 7 from the first reflection intensity map M1.
[0032] 7, the second reflection intensity map M2 is a map showing the distribution of reflection intensity Pw with respect to coordinate positions X and Y in the second coordinate system CT2. The second coordinate system CT2 is, for example, an orthogonal coordinate system in which the horizontal axis indicates coordinate position X and the vertical axis indicates coordinate position Y, and is a coordinate system that extends in the horizontal direction Dh. The second coordinate system CT2 is a fixed coordinate system that does not move even if the vehicle 80 moves.
[0033] For example, the second reflection intensity map M2 has grids 30 corresponding to a plurality of coordinate points indicating coordinate positions X, Y in the second coordinate system CT2, and the second reflection intensity map M2 stores a reflection intensity Pw for each of the plurality of grids 30. The reflection intensities Pw11 to Pw47 shown in FIG. 7 respectively represent the magnitude of the reflection intensity Pw stored in each grid 30.
[0034] The second reflection intensity map M2 generated in step S104 is formed so that the entire area occupied by the second reflection intensity map M2 is the same as the object detection area As of the sensor unit 12 in Fig. 2 or is an area within the range of the object detection area As. In other words, in this embodiment, the map target area Am, which is an area around the vehicle 80 that can be a target for disposing the second reflection intensity map M2, is the same as the object detection area As. In short, all coordinate positions X, Y of the second reflection intensity map M2 fall within the object detection area As of the sensor unit 12. This is because the reflection intensity Pw cannot be determined in an area where the sensor unit 12 cannot detect the presence of a detected object.
[0035] Also, referring to the object detection area As in Fig. 2, the object detection area As extends to one side (specifically, the left side) in the vehicle left-right direction D2 with respect to the imaginary plane PLv shown in Fig. 2, and does not extend to the other side in the vehicle left-right direction D2. As shown in Figs. 2, 5, and 6, the imaginary plane PLv passes through a position R0p where the relative distance Rr in the first coordinate system CT1 becomes zero, that is, the position R0p of the sensor origin R0, and is formed parallel to the vehicle front-rear direction D1 and the vehicle up-down direction D3. For example, as shown in Fig. 2, the object detection area As is formed in a sector shape or a substantially sector shape so as to expand in the circumferential direction centered on the position R0p as it moves away from the position R0p of the sensor origin R0 to the left in the vehicle left-right direction D2.
[0036] Here, since the first coordinate system CT1 is transformed into the second coordinate system CT2 as described above, the relationship between the first coordinate system CT1 and the second coordinate system CT2 will be described. As shown in Fig. 6, one reflection point Pr on a stationary object 82 that reflects the transmission wave Sw is selected. In this case, the coordinate positions Xw, Yw of the reflection point Pr in the second coordinate system CT2 are calculated from the following formulas F1 and F2. Then, the following formulas F3 and F4 are derived from the formulas F1 and F2.
number
number
[0037] In the above formulas F1 to F4, Xw is the coordinate position X of the reflection point Pr in the horizontal axis direction, Yw is the coordinate position Y of the reflection point Pr in the vertical axis direction, Xe is the coordinate position X of the vehicle position origin E0 in the horizontal axis direction, and Ye is the coordinate position Y of the vehicle position origin E0 in the vertical axis direction. Xrp is the horizontal axis direction distance from the vehicle position origin E0 to the sensor origin R0, which is a constant, and Yrp is the vertical axis direction distance from the vehicle position origin E0 to the sensor origin R0, which is a constant. Xr is the horizontal axis direction coordinate position of the reflection point Pr based on the sensor origin R0, that is, the horizontal axis direction distance from the sensor origin R0 to the reflection point Pr. Yr is the vertical axis direction coordinate position of the reflection point Pr based on the sensor origin R0, that is, the vertical axis direction distance from the sensor origin R0 to the reflection point Pr.
[0038] 6, the following formulas F5, F6, and F7 hold, and the following formula F8 is derived from the following formula F7. Furthermore, the following formulas F9 and F10 are derived from the following formulas F5, F6, and F8.
number
number
number
[0039] In the above formulas F5 to F10, Rr is the distance from the sensor origin R0 to the reflection point Pr shown in Fig. 6, i.e., the relative distance Rr shown in Fig. 5. Also, θ is the angle that the line segment connecting the sensor origin R0 and the reflection point Pr in Fig. 6 forms with the horizontal axis direction, Ve is the host vehicle speed Ve, and Vr is the relative speed of the reflection point Pr with respect to the sensor origin R0, i.e., the relative speed Vr shown in Fig. 5.
[0040] The following formulas F11 and F12 are derived from the above formulas F9 and F10.
number
[0041] The sign function in formula F12 is a function that gives "+1" if "Yr / Xr" in parentheses is a positive number, and gives "-1" if "Yr / Xr" is a negative number, and gives "0" if "Yr / Xr" is zero.
[0042] As described above, the formulas F3, F4, F11, and F12 are obtained. Therefore, if the vehicle speed Ve and the vehicle positions Xe and Ye are known, the coordinate positions Xw and Yw of the reflection point Pr in the second coordinate system CT2 in Fig. 6 can be converted into a combination of the relative distance Rr and the relative speed Vr of the reflection point Pr. In other words, the coordinate positions Xw and Yw of the reflection point Pr in the second coordinate system CT2 can be converted into the relative distance Rr and the relative speed Vr in the first coordinate system CT1 in Fig. 5.
[0043] Based on the fact that the above-mentioned formulas F3, F4, F11, and F12 have been obtained, specifically, in step S104, the second map generating unit 20 executes the subroutine of FIG. 8, thereby generating the second reflection intensity map M2 of FIG. 7 from the first reflection intensity map M1 of FIG. 5.
[0044] In step S201 of FIG. 8, the second map generating unit 20 determines whether or not the reflection intensities Pw have been stored for all the grids 30 of the second reflection intensity map M2, in other words, whether or not the reflection intensities Pw have been stored for all the grids 30.
[0045] In step S201, if it is determined that storage of the reflection intensities Pw for all the grids 30 of the second reflection intensity map M2 has been completed, the subroutine in Fig. 8 ends, and the process proceeds to step S105 in the flowchart in Fig. 4. On the other hand, if it is determined that storage of the reflection intensities Pw for all the grids 30 of the second reflection intensity map M2 has not yet been completed, the process proceeds to step S202.
[0046] In step S202, the second map generator 20 selects one grid 30 in which a reflection intensity Pw has not yet been stored from among all the grids 30 in the second reflection intensity map M2. The selected grid 30 is referred to as the selected grid 30a. After step S202, the process proceeds to step S203.
[0047] In step S203, the second map generating unit 20 calculates the relative distance Rr in Fig. 5 from the above-mentioned formulas F3, F4, and F11, which are predetermined calculation formulas, with the coordinate positions X and Y of the selection grid 30a in the second reflection intensity map M2 being the coordinate positions Xw and Yw of the reflection point Pr in Fig. 6. At this time, the host vehicle positions Xe and Ye obtained by the egomotion obtaining unit 18 are used. After step S203 in Fig. 8, the process proceeds to step S204.
[0048] In step S204, the second map generating unit 20 calculates the relative speed Vr in FIG. 5 from the above formulas F3, F4, and F12, which are predetermined calculation formulas, with the coordinate positions X and Y of the selection grid 30a in the second reflection intensity map M2 being the coordinate positions Xw and Yw of the reflection point Pr in FIG. 6. At this time, the vehicle speed Ve and the vehicle positions Xe and Ye acquired by the egomotion acquiring unit 18 are used. Note that the coordinate positions Xw and Yw of the above formulas F3 and F4 used in steps S203 and S204, that is, the coordinate positions X and Y of the selection grid 30a, can be said to be selected coordinate positions selected from the multiple coordinate positions X and Y included in the second reflection intensity map M2. Step S204 in FIG. 8 is followed by step S205.
[0049] In step S205, the second map generating unit 20 obtains the reflection intensity Pw for the relative distance Rr calculated in step S203 and the relative velocity Vr calculated in step S204 from the first reflection intensity map M1 in Fig. 5. Then, the second map generating unit 20 stores the obtained reflection intensity Pw in the selection grid 30a of the second reflection intensity map M2 in Fig. 7. In other words, the second map generating unit 20 stores the obtained reflection intensity Pw in the second reflection intensity map M2 as the reflection intensity Pw for the coordinate positions X, Y of the selection grid 30a. After step S205 in Fig. 8, the process returns to step S201.
[0050] In this manner, by repeatedly executing the processes of steps S201 to S205 in Fig. 8, the first coordinate system CT1 of the first reflection intensity map M1 in Fig. 5 is coordinate-converted into the second coordinate system CT2 of the second reflection intensity map M2 in Fig. 7. That is, the coordinate conversion from the first coordinate system CT1 to the second coordinate system CT2 to generate the second reflection intensity map M2 means performing the processes of steps S202 to S205 in Fig. 8 for each coordinate position X, Y for all coordinate positions X, Y of the second reflection intensity map M2.
[0051] 7 generated in step S104 is stored in the storage unit 26 of the electronic control unit 14 after completion of step S105, which will be described later, as shown in FIG 1. The storage unit 26 then stores the latest N cycles of second reflection intensity maps M2 among the second reflection intensity maps M2 stored in the storage unit 26. The "N cycles" refers to N cycles, with a series of processes from steps S101 to S106 in FIG 4 being one cycle, and therefore the number of second reflection intensity maps M2 stored in the storage unit 26 is N.
[0052] On the other hand, every time a new second reflection intensity map M2 is stored in the storage unit 26, second reflection intensity maps M2 older than the most recent N cycles are sequentially discarded from the storage unit 26.
[0053] In the above description of "N cycles," "N" is a positive integer. The storage limit of N cycles of the second reflection intensity map M2 is determined in advance so as to be equal to or less than the storage capacity of the storage unit 26 and so that a second reflection intensity map M2 that is too old is not used to generate a reflection intensity storage map Ms, which will be described later.
[0054] Returning to FIG. 4, in step S105, the reflection intensity accumulation map generating unit 22 reads the second reflection intensity map M2 for N cycles stored in the storage unit 26, and obtains the second reflection intensity map M2 generated in the immediately preceding step S104. Then, the reflection intensity accumulation map generating unit 22 generates the reflection intensity accumulation map Ms in FIG. 9 based on the second reflection intensity map M2 for N cycles and the second reflection intensity map M2 generated in the immediately preceding step S104. Therefore, in this embodiment, the number of second reflection intensity maps M2 on which the reflection intensity accumulation map Ms is based is N+1. Note that the second coordinate system CT2 of the reflection intensity accumulation map Ms and the second coordinate systems CT2 of the multiple second reflection intensity maps M2 on which the reflection intensity accumulation map Ms is based are both the same.
[0055] In detail, the reflection intensity accumulation map generating unit 22 calculates a reflection intensity related value Ps for each coordinate position X, Y of the second reflection intensity map M2 according to a predetermined calculation rule based on the reflection intensity Pw for each coordinate position X, Y of the multiple second reflection intensity maps M2 generated by the second map generating unit 20. This reflection intensity related value Ps corresponds to the signal intensity related value of the present disclosure. Then, the reflection intensity accumulation map generating unit 22 sequentially stores the calculated reflection intensity related value Ps for each coordinate position X, Y of the reflection intensity accumulation map Ms in association with the same coordinate position X, Y as the second reflection intensity map M2.
[0056] In this manner, the reflection intensity accumulation map generating unit 22 generates the reflection intensity accumulation map Ms of Fig. 9 from the multiple second reflection intensity maps M2. Therefore, the reflection intensity accumulation map Ms has the same second coordinate system CT2 as the second reflection intensity map M2, and is a map showing the distribution of the reflection intensity-related value Ps for the coordinate positions X, Y of the second coordinate system CT2.
[0057] The above-mentioned predetermined calculation rule is to set the reflection intensity related value Ps for each coordinate position X, Y of the reflection intensity accumulation map Ms to the maximum value of the reflection intensity Pw for each coordinate position X, Y in the multiple second reflection intensity maps M2 on which the reflection intensity accumulation map Ms is based.
[0058] For example, like the second reflection intensity map M2, the reflection intensity accumulation map Ms has grids 30 corresponding to a plurality of coordinate points indicating coordinate positions X, Y in the second coordinate system CT2. However, unlike the second reflection intensity map M2, the reflection intensity accumulation map Ms stores a reflection intensity related value Ps instead of a reflection intensity Pw for each of the plurality of grids 30. The reflection intensity related values Ps11 to Ps47 shown in FIG. 9 respectively represent the magnitude of the reflection intensity related value Ps stored in each grid 30.
[0059] Specifically, in step S105 in FIG. 4, the reflection intensity accumulation map generating unit 22 executes a subroutine in FIG. 10, thereby generating the reflection intensity accumulation map Ms in FIG. 9 from the multiple second reflection intensity maps M2.
[0060] In step S301 of FIG. 10, the reflection intensity accumulation map generating unit 22 determines whether the calculation of the reflection intensity related value Ps stored for each grid 30 in the reflection intensity accumulation map Ms has been completed for all grids 30.
[0061] In step S301, if it is determined that the calculation of the reflection intensity related value Ps has been completed for all the grids 30 of the reflection intensity accumulation map Ms, the subroutine in Fig. 10 ends, and the process proceeds to step S106 in the flowchart in Fig. 4. On the other hand, if it is determined that the calculation of the reflection intensity related value Ps has not yet been completed for all the grids 30, the process proceeds to step S302.
[0062] In step S302, the reflection intensity accumulation map generating unit 22 selects one grid 30 for which the reflection intensity related value Ps has not yet been calculated, that is, one grid 30 for which the reflection intensity related value Ps has not yet been stored, from among all the grids 30 in the reflection intensity accumulation map Ms. The selected grid 30 is called the selected grid 30a. After step S302, the process proceeds to step S303.
[0063] In step S303, the reflection intensity accumulation map generating unit 22 extracts a reflection intensity Pw for the same coordinate positions X, Y as the coordinate positions X, Y of the selection grid 30a from each of the second reflection intensity maps M2 on which the reflection intensity accumulation map Ms is based.Then, the reflection intensity accumulation map generating unit 22 calculates a reflection intensity maximum value that is the maximum value of the extracted reflection intensities Pw. After step S303 in FIG. 10, the process proceeds to step S304.
[0064] In step S304, the reflection intensity accumulation map generating unit 22 stores the reflection intensity maximum value calculated in step S303 in the selection grid 30a of the reflection intensity accumulation map Ms in Fig. 9. The reflection intensity maximum value stored in this selection grid 30a is the reflection intensity related value Ps. In other words, the reflection intensity accumulation map generating unit 22 stores the reflection intensity maximum value calculated in step S303 in the reflection intensity accumulation map Ms as the reflection intensity related value Ps for the coordinate position X, Y of the selection grid 30a. After step S304 in Fig. 10, the process returns to step S301.
[0065] In this manner, the processes of steps S301 to S304 in FIG. 10 are repeatedly executed, whereby the reflection intensity accumulation map Ms in FIG. 9 is generated.
[0066] Returning to Fig. 4, in step S106, the estimation unit 24 estimates an object shape map Mf from the reflection intensity accumulation map Ms obtained in step S105 by using the trained model 32 of the neural network in Fig. 11. The trained model 32 of the neural network used to estimate the object shape map Mf is included in the estimation unit 24, and is a trained model of the neural network trained by supervised learning. As shown in Fig. 11, the object shape map Mf obtained in step S106 is a map representing the shape of a stationary object 82 existing around the vehicle 80. After step S106 in Fig. 4, the process returns to step S101.
[0067] In addition, the reflection intensity accumulation map Ms shown in Fig. 11 is one in which the reflection intensity-related value Ps for each coordinate position X, Y is represented in a color corresponding to the magnitude of the reflection intensity-related value Ps. For example, the reflection intensity accumulation map Ms in Fig. 11 is displayed so that the larger the reflection intensity-related value Ps is, the darker the red color becomes, and the smaller the reflection intensity-related value Ps is, the darker the blue color becomes, but is displayed in monochrome in Fig. 11. In addition, "Conv." in Fig. 11 is an abbreviation for "Convolution", "DeConv." is an abbreviation for "DeConvolution", and "ReLU" is an abbreviation for "Rectified Linear Unit".
[0068] The trained model 32 in Fig. 11 is configured as a convolutional neural network. Specifically, the trained model 32 has a layer configuration having a plurality of layers 321 to 335 from a first layer 321 to a fifteenth layer 335. The first layer 321 is an input layer to which a reflection intensity accumulation map Ms is input, and has one layer having 400 x 400 nodes.
[0069] The second layer 322 has 42 layers with 400×400 nodes. The second layer 322 is calculated by performing a first calculation process Ca on the first layer 321. The first calculation process Ca is a convolution calculation using a kernel with a size of 11×11, followed by a calculation using ReLU, which is an activation function.
[0070] The third layer 323 and the fourth layer 324 each have 42 layers with 200×200 nodes. The third layer 323 is calculated by performing a second calculation process Cb on the second layer 322. The second calculation process Cb is a max pooling process using a kernel with a size of 11×11. The fourth layer 324 is calculated by performing a first calculation process Ca on the third layer 323.
[0071] The fifth layer 325 and the sixth layer 326 each have 42 layers with a node count of 100 x 100. The fifth layer 325 is calculated by performing the second calculation process Cb on the fourth layer 324. The sixth layer 326 is calculated by performing the first calculation process Ca on the fifth layer 325.
[0072] Each of the seventh layer 327 and the eighth layer 328 has 42 layers with a node count of 50 x 50. The seventh layer 327 is calculated by performing the second calculation process Cb on the sixth layer 326. The eighth layer 328 is calculated by performing the first calculation process Ca on the seventh layer 327.
[0073] The ninth layer 329 has 42 layers, each having a node count of 25 × 25. The ninth layer 329 is calculated by performing the second calculation process Cb on the eighth layer 328.
[0074] The tenth layer 330 has 42 layers with a node count of 50×50, and is calculated by performing a third calculation process Cc on the ninth layer 329. The third calculation process Cc is a process of performing a deconvolution calculation using a kernel with a size of 22×22, followed by a calculation using the ReLU activation function.
[0075] The 11th layer 331 has 42 pieces with a node count of 100 x 100, and the 12th layer 332 has 42 pieces with a node count of 200 x 200. The 11th layer 331 is calculated by performing the third calculation process Cc on the 10th layer 330, and the 12th layer 332 is calculated by performing the third calculation process Cc on the 11th layer 331.
[0076] The 13th layer 333 has 42 pieces of 400 x 400 nodes, and the 14th layer 334 has 5 pieces of 400 x 400 nodes. The 13th layer 333 is calculated by performing a third calculation process Cc on the 12th layer 332, and the 14th layer 334 is calculated by performing a fourth calculation process Cd on the 13th layer 333. The fourth calculation process Cd is a convolution calculation using a kernel of 1 x 1 size.
[0077] The 15th layer 335 is an output layer that outputs the object shape map Mf, and is configured to have five maps with 400 x 400 nodes. The 15th layer 335 is calculated by performing a fifth calculation process Ce on the 14th layer 334. The fifth calculation process Ce is a softmax process.
[0078] The first layer 321 to the ninth layer 329 are a convolutional neural network encoder 32a. The ninth layer 329 to the fifteenth layer 335 are a convolutional neural network decoder 32b.
[0079] Here, we will explain a method for creating the trained model 32. The trained model 32 is created in advance by a training model creation device other than the shape estimation device 10.
[0080] For example, when the learning model creation device creates the learned model 32 by having the learned model 32 learn, a plurality of sets of teacher data are prepared, each set being a reflection intensity accumulation map Ms and a correct shape map Ma corresponding to the reflection intensity accumulation map Ms. An example of the reflection intensity accumulation map Ms, which is one of the sets of teacher data, is shown in Fig. 12A, and an example of the correct shape map Ma, which is the other of the teacher data, is shown in Fig. 12B. The reflection intensity accumulation map Ms in Fig. 12A displays the reflection intensity-related value Ps in color in a manner similar to the display of the reflection intensity accumulation map Ms in Fig. 11, but is displayed in monochrome in Fig. 12A.
[0081] The correct shape map Ma is an object shape map Mf that is pre-configured to indicate the correct shape AF of the stationary object 82 derived from the reflection intensity accumulation map Ms, which together with the correct shape map Ma constitutes one set of teacher data. To be clear, the correct shape AF of the stationary object 82 is not a shape estimated from the reflection intensity accumulation map Ms, but a correct shape of the stationary object 82 that is obtained in advance by a method other than estimation from the reflection intensity accumulation map Ms.
[0082] There is no limitation on the method of acquiring the correct shape AF of the correct shape map Ma, but for example, the correct shape AF may be obtained by photographing the stationary object 82. Alternatively, the correct shape AF may be obtained by an automatic program or manual tracing operation using a laser point cloud obtained by irradiating the stationary object 82 with a laser by a LiDAR. The above "LiDAR" is an abbreviation for "Light Detection And Ranging."
[0083] Then, the learning model creation device uses a plurality of sets of pre-prepared teacher data to learn the trained model 32 (i.e., supervised learning) by a backpropagation method or the like. As a result of this learning, weights of a neural network corresponding to the trained model 32 are determined.
[0084] As described above, according to this embodiment, the second map generation unit 20 generates the second reflection intensity map M2 of Fig. 7 from the first reflection intensity map M1 of Fig. 5 by coordinate conversion from the first coordinate system CT1 of Fig. 5 to the second coordinate system CT2 of Fig. 7. Then, the estimation unit 24 estimates the object shape map Mf representing the shape of the stationary object 82 from the reflection intensity accumulation map Ms generated based on the multiple second reflection intensity maps M2.
[0085] Therefore, a second reflection intensity map M2 indicating the distribution of reflection intensity Pw with respect to two-dimensional coordinate positions X, Y can be obtained from a first reflection intensity map M1 indicating the distribution of reflection intensity Pw with respect to relative distance Rr and relative speed Vr. That is, the second reflection intensity map M2 on which the shape of the stationary object 82 is estimated can be obtained without performing an orientation estimation process for estimating the orientation with respect to the sensor unit 12. Therefore, it is possible to estimate the shape of the stationary object 82 with a low processing load.
[0086] Since the reflection intensity accumulation map Ms is generated based on the multiple second reflection intensity maps M2 that are generated sequentially, the reflection intensity accumulation map Ms can emphasize the distribution state of the reflection intensity Pw corresponding to the contour of the stationary object 82 whose shape is to be estimated. In short, the reflection intensity accumulation map Ms can increase the amount of information related to the shape of the stationary object 82 compared to the second reflection intensity map M2. The reflection intensity accumulation map Ms is then used as an input to the estimation unit 24 that performs shape estimation of the stationary object 82. Therefore, it is possible to estimate the shape of the stationary object 82 with high accuracy, as compared to the case where the second reflection intensity map M2 is used as an input to the estimation unit 24 instead of the reflection intensity accumulation map Ms, for example.
[0087] (1) Furthermore, according to this embodiment, the reflection intensity-related value Ps for the coordinate positions X, Y of the reflection intensity accumulation map Ms is set to the maximum value of the reflection intensities Pw for each coordinate position X, Y in the multiple second reflection intensity maps M2 on which the reflection intensity accumulation map Ms is based, for each coordinate position X, Y. Therefore, compared to a case in which the reflection intensity-related value Ps is set to an average value such as an arithmetic mean value of the reflection intensities Pw for each coordinate position X, Y, for example, it is easier to reflect the shape of the stationary object 82 that appears and disappears from the sensor unit 12 as the vehicle 80 moves in the reflection intensity-related value Ps. Therefore, there is an advantage in that it is easier to estimate the shape of the stationary object 82 with high accuracy.
[0088] (2) According to the present embodiment, the trained model 32 in Fig. 11 has an encoder 32a and a decoder 32b of a convolutional neural network. Therefore, a neural network suitable for estimating the shape of a stationary object 82 can be realized.
[0089] (3) According to this embodiment, the object detection area As of the sensor unit 12 in Fig. 2 extends to one side (specifically, the left side) in the vehicle left-right direction D2 with respect to the imaginary plane PLv shown in Fig. 2, and does not extend to the other side in the vehicle left-right direction D2. The imaginary plane PLv passes through the position R0p where the relative distance Rr in the first coordinate system CT1 becomes zero, that is, the position R0p of the sensor origin R0, and is formed parallel to the vehicle front-rear direction D1 and the vehicle up-down direction D3.
[0090] Therefore, no matter where the reflection point Pr on the stationary object 82 is located within the object detection area As, once the relative distance Rr and relative speed Vr of the first reflection intensity map M1 in Fig. 5 are determined, the coordinate positions X and Y of the reflection point Pr are uniquely determined. In this embodiment, all coordinate positions X and Y of the second reflection intensity map M2 are within the object detection area As of the sensor unit 12. In other words, once the relative distance Rr and relative speed Vr of Fig. 5 are determined, the coordinate positions X and Y of the second reflection intensity map M2 in Fig. 7 are uniquely determined. Therefore, it is possible to generate the second reflection intensity map M2 in Fig. 7 from the first reflection intensity map M1 in Fig. 5 while keeping the processing load low.
[0091] (4) Furthermore, according to this embodiment, the sensor unit 12 has one transmitting antenna element 121, and the sensor unit 12 has one receiving antenna element 122. That is, the sensor unit 12 does not have a configuration in which multiple transmitting antenna elements 121 are arranged in the horizontal direction Dh, and does not have a configuration in which multiple receiving antenna elements 122 are arranged in the horizontal direction Dh.
[0092] Therefore, when estimating the shape of the stationary object 82 based on the first reflection intensity map M1 in Fig. 5 without performing a calculation process to estimate the direction of the reflection point Pr, it is possible to configure the transmitting antenna elements 121 and the receiving antenna elements 122 in just the right amount. For example, the shape of the stationary object 82 can be estimated using an inexpensive sensor unit 12.
[0093] (5) According to the present embodiment, the coordinate conversion from the first coordinate system CT1 to the second coordinate system CT2 in step S104 in FIG. 4 specifically means performing the following process for each coordinate position X, Y of the grid 30 in the second reflection intensity map M2. That is, in this process, one grid 30 in which the reflection intensity Pw has not yet been stored, that is, the selected grid 30a, is selected from all the grids 30 in the second reflection intensity map M2. Next, the relative distance Rr and the relative speed Vr in the first coordinate system CT1 are calculated from the coordinate positions X, Y, the vehicle speed Ve, and the vehicle positions Xe, Ye of the selected grid 30a according to the above formulas F3, F4, F11, and F12. Then, the reflection intensity Pw in the first reflection intensity map M1 for the calculated relative distance Rr and relative speed Vr is stored as the reflection intensity Pw for the coordinate positions X, Y of the selected grid 30a in the second reflection intensity map M2.
[0094] Therefore, it is possible to keep the computation load for transforming the coordinates from the first coordinate system CT1 to the second coordinate system CT2 low.
[0095] Second embodiment Next, a second embodiment will be described. In this embodiment, differences from the first embodiment will be mainly described. Also, parts that are the same as or equivalent to the above-mentioned embodiment will be omitted or simplified. This also applies to the following embodiments.
[0096] 13, in this embodiment, unlike the first embodiment, the sensor unit 12 has a plurality of receiving antenna elements 122. Specifically, the sensor unit 12 has two receiving antenna elements 122. The plurality of receiving antenna elements 122 are not aligned in the horizontal direction Dh, but aligned in the vehicle up-down direction D3.
[0097] Except for the points described above, this embodiment is similar to the first embodiment. In this embodiment, the same effects as those of the first embodiment can be obtained from the configuration common to the first embodiment.
[0098] Third embodiment Next, a third embodiment will be described. In this embodiment, differences from the first embodiment will be mainly described.
[0099] 14, in this embodiment, unlike the first embodiment, the sensor unit 12 has a plurality of transmitting antenna elements 121. Specifically, the sensor unit 12 has two transmitting antenna elements 121. The plurality of transmitting antenna elements 121 are not aligned in the horizontal direction Dh, but aligned in the vehicle up-down direction D3.
[0100] Except for the points described above, this embodiment is similar to the first embodiment. In this embodiment, the same effects as those of the first embodiment can be obtained from the configuration common to the first embodiment.
[0101] (Fourth embodiment) Next, a fourth embodiment will be described. In this embodiment, differences from the first embodiment will be mainly described.
[0102] As shown in FIG. 15, in this embodiment, the mounting orientation of the sensor unit 12 with respect to the vehicle 80 is different from that in the first embodiment.
[0103] Specifically, the sensor unit 12 of this embodiment is attached to the vehicle 80 with the front surface of the sensor unit 12 facing the left side in the vehicle left-right direction D2 and diagonally forward in the vehicle front-rear direction D1. Therefore, the object detection area As of the sensor unit 12 has a one-side area A1 that extends to one side (specifically, the left side) in the vehicle left-right direction D2 with respect to the imaginary plane PLv in FIG. 15, and an other-side area A2 that extends to the other side in the vehicle left-right direction D2 with respect to the imaginary plane PLv. The other-side area A2 is narrower than the one-side area A1. The imaginary plane PLv in FIG. 15 passes through the position R0p of the sensor origin R0, like the imaginary plane PLv in FIG. 2, and is parallel to the vehicle front-rear direction D1 and the vehicle up-down direction D3.
[0104] The map target area Am in this embodiment is an area obtained by excluding an inverted other-side area A2r obtained by inverting the other-side area A2 to one side (specifically, the left side) in the vehicle left-right direction D2 with reference to the virtual plane PLv in Fig. 15 from the one-side area A1. Therefore, the map target area Am in this embodiment is a part of the object detection area As of the sensor unit 12, and is narrower than the object detection area As. Note that, in this embodiment, as in the first embodiment, the second reflection intensity map M2 is formed such that all coordinate positions X, Y of the second reflection intensity map M2 fall within the map target area Am.
[0105] (1) As described above, according to this embodiment, the object detection area As of the sensor unit 12 has a one-side area A1 that extends to one side in the vehicle left-right direction D2 with respect to the virtual plane PLv in FIG. 15, and an other-side area A2 that extends to the other side in the vehicle left-right direction D2 with respect to the virtual plane PLv. The other-side area A2 is narrower than the one-side area A1. The map target area Am is an area obtained by excluding the inverted other-side area A2r obtained by inverting the other-side area A2 to one side in the vehicle left-right direction D2 with respect to the virtual plane PLv from the one-side area A1. All coordinate positions X, Y of the second reflection intensity map M2 are included in the map target area Am.
[0106] Therefore, once the relative distance Rr and relative velocity Vr of the first reflection intensity map M1 in Fig. 5 are determined, the coordinate positions X, Y of the second reflection intensity map M2 in Fig. 7 are uniquely determined. Therefore, it is possible to generate the second reflection intensity map M2 in Fig. 7 from the first reflection intensity map M1 in Fig. 5 while keeping the processing load low.
[0107] Except for the points described above, this embodiment is similar to the first embodiment. In this embodiment, the same effects as those of the first embodiment can be obtained from the configuration common to the first embodiment.
[0108] It should be noted that although this embodiment is a modification based on the first embodiment, this embodiment can also be combined with the second or third embodiment described above.
[0109] (Other embodiments) (1) In each of the above-described embodiments, the second reflection intensity map M2 is formed to occupy a rectangular two-dimensional region as shown in Fig. 7, but this is only one example and is provided to make the second reflection intensity map M2 easier to understand. There is no limitation on the shape of the two-dimensional region occupied by the second reflection intensity map M2, and it may be, for example, a sector shape or an approximately sector shape. The same applies to the reflection intensity accumulation map Ms in Fig. 9.
[0110] (2) In each of the above-mentioned embodiments, the calculation rule adopted in step S105 in FIG. 4 is to set the reflection intensity related value Ps of the reflection intensity accumulation map Ms to the maximum value of the reflection intensities Pw for each coordinate position X, Y in the multiple second reflection intensity maps M2 for each coordinate position X, Y. However, this is only one example. For example, the calculation rule may be to set the reflection intensity related value Ps for each coordinate position X, Y to the arithmetic mean value or weighted mean value of the reflection intensities Pw for each coordinate position X, Y in the multiple second reflection intensity maps M2. For example, when the reflection intensity related value Ps for each coordinate position X, Y is set to the weighted mean value of the reflection intensities Pw, it is preferable that the weight in the weighted mean is increased as the second reflection intensity map M2 having the reflection intensity Pw is newer.
[0111] (3) In each of the above-described embodiments, the second coordinate system CT2 shown in Fig. 7 etc. is a fixed coordinate system that does not move even if the vehicle 80 moves, but this is just one example. If the second coordinate system CT2 is the same for all second reflection intensity maps M2 on which the reflection intensity accumulation map Ms is based, the second coordinate system CT2 does not need to be a fixed coordinate system.
[0112] (4) In each of the above-described embodiments, the sensor unit 12 shown in Fig. 2 is a millimeter wave radar device, but this is just one example. As long as it is possible to obtain the distribution of the reflection intensity Pw with respect to the relative distance Rr and the relative speed Vr shown in Fig. 5, the sensor unit 12 may be a sensor other than a radar device, such as a LiDAR.
[0113] (5) In each of the above-described embodiments, the electronic control unit 14 does not need to be an independent device, but may be a control unit included in the on-board control device as a functional part of the on-board control device.
[0114] (6) The present invention is not limited to the above-described embodiment, and can be practiced in various modified forms. Furthermore, the above-described embodiments are not unrelated to each other, and can be appropriately combined, except in cases where the combination is clearly impossible.
[0115] In each of the above embodiments, the elements constituting the embodiment are not necessarily essential, unless otherwise specified as essential or considered to be obviously essential in principle. In each of the above embodiments, when the numbers, values, amounts, ranges, etc. of the components of the embodiment are mentioned, they are not limited to the specific numbers, except when specified as essential or when they are obviously limited to a specific number in principle. In each of the above embodiments, when the materials, shapes, positional relationships, etc. of the components are mentioned, they are not limited to the materials, shapes, positional relationships, etc., except when specified as essential or when they are obviously limited to a specific material, shape, positional relationship, etc. in principle.
[0116] The electronic control unit 14 and the methodology described herein may be implemented by a special-purpose computer provided by configuring a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the electronic control unit 14 and the methodology described herein may be implemented by a special-purpose computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the electronic control unit 14 and the methodology described herein may be implemented by one or more special-purpose computers configured by a combination of a processor and memory programmed to perform one or more functions and a processor configured with one or more hardware logic circuits. The computer program may also be stored in a computer-readable non-transitory tangible recording medium as instructions executed by a computer. [Explanation of symbols]
[0117] 12 Sensor section 16 First map generator 18 Egomotion Knowledge Department 20 Second map generation section 22 Reflection intensity accumulation map generator 24 Estimation part M1 First reflection intensity map M2 Second reflection intensity map Ms reflection intensity accumulation map Mf Object shape map
Claims
1. A shape estimation device mounted on a mobile body (80), A sensor unit (12) is provided on the moving body and transmits a transmission wave (Sw), which is an electromagnetic wave, to the surroundings of the moving body, and receives a reflected wave (Rw) generated by the reflection of the transmission wave, A first map generation unit (16) repeatedly generates a first reflection intensity map (M1) that shows the distribution of the signal intensity (Pw) of the reflected wave with respect to the relative distance (Rr) to the sensor unit and the relative velocity (Vr) to the sensor unit, based on the transmitted wave and the reflected wave. An ego-motion knowing unit (18) that knows the velocity (Ve) and position (Xe, Ye) of the moving object, A second map generation unit (20) repeatedly performs the following: a coordinate transformation of a first coordinate system (CT1) representing the relative distance and relative velocity of the first reflectance intensity map into a second coordinate system (CT2) representing two-dimensional coordinate positions (X, Y) using the velocity and position of the moving object, thereby generating a second reflectance intensity map (M2) showing the distribution of the signal intensity with respect to the coordinate positions from the first reflectance intensity map; A reflection intensity accumulation map generation unit (22) calculates a signal intensity-related value obtained according to a predetermined calculation rule based on the signal intensity in each of the plurality of second reflection intensity maps generated by the second map generation unit for each of the coordinate positions of the second reflection intensity map, and generates a reflection intensity accumulation map (Ms) showing the distribution of the signal intensity-related value for that coordinate position, A shape estimation device comprising: an estimation unit (24) that estimates an object shape map (Mf) representing the shape of stationary objects (82) present around the moving object from the reflection intensity accumulation map using a trained model (32) of a neural network trained by supervised learning.
2. The shape estimation device according to claim 1, wherein the signal intensity related value for the coordinate position in the reflection intensity accumulation map is the maximum value among the signal intensities for each coordinate position in a plurality of second reflection intensity maps.
3. The shape estimation apparatus according to claim 1 or 2, wherein the trained model has an encoder (32a) and a decoder (32b) of a convolutional neural network.
4. The sensor unit is attached to the moving body so as to be able to detect the presence of the object within a predetermined object detection area (As). The object detection region extends to one side in the left-right direction (D2) of the moving object with respect to a virtual plane (PLv) that passes through the position (R0p) where the relative distance in the first coordinate system is zero and is parallel to the front-back direction (D1) and the up-down direction (D3) of the moving object, but does not extend to the other side in the left-right direction. The shape estimation device according to claim 1 or 2, wherein all of the coordinate positions of the second reflection intensity map fall within the object detection area.
5. The sensor unit is attached to the moving body so as to be able to detect the presence of the object within a predetermined object detection area (As). The object detection region has a one-sided region (A1) that extends to one side in the left-right direction (D2) of the moving body with respect to a virtual plane (PLv) that passes through the position (R0p) where the relative distance in the first coordinate system is zero and is parallel to the front-back direction (D1) and the up-down direction (D3) of the moving body, and a other-sided region (A2) that is narrower than the one-sided region and extends to the other side in the left-right direction with respect to the virtual plane. The shape estimation device according to claim 1 or 2, wherein all of the coordinate positions of the second reflection intensity map fall within the region (Am) obtained by excluding the inverted other-side region (A2r), which is obtained by inverting the other-side region toward the one-side region in the left-right direction with respect to the virtual plane, from the one-side region.
6. The sensor unit comprises one or more transmitting antenna elements (121) that emit the transmitted wave and one or more receiving antenna elements (122) that receive the reflected wave. The one or more transmitting antenna elements are not arranged horizontally (Dh), The shape estimation device according to claim 1 or 2, wherein the one or more receiving antenna elements are not arranged in the horizontal direction.
7. The moving body is a vehicle, The sensor unit has a plurality of the transmitting antenna elements, The shape estimation device according to claim 6, wherein the plurality of transmitting antenna elements are not arranged in the horizontal direction, but are arranged in the vertical direction (D3) of the moving body.
8. The shape estimation device according to claim 1 or 2, wherein the coordinate transformation of the first coordinate system to the second coordinate system is performed for each coordinate position in the second reflection intensity map, wherein the relative distance and relative velocity of the first coordinate system are calculated from a predetermined calculation formula (F3, F4, F11, F12) using a selected coordinate position (Xw, Yw) which is a coordinate position selected from a plurality of coordinate positions included in the second reflection intensity map, the velocity of the moving body and the position of the moving body, and the signal intensity of the first reflection intensity map with respect to the calculated relative distance and relative velocity is set to the signal intensity of the selected coordinate position in the second reflection intensity map.
9. The shape estimation device according to claim 1 or 2, wherein the supervised learning is learning using a plurality of training data, each set consisting of a reflection intensity accumulation map and a ground truth shape map (Ma) corresponding to the reflection intensity accumulation map.