Object detection device

Abstract

This object detecting device comprises a receiving unit (40B) that acquires a received signal for a wave that is transmitted as an ultrasonic wave and is reflected from an object. The object detecting device comprises an information processing unit (8) that acquires object information on the shape of the object from a learning model by inputting the feature amount of the waveform of the received signal and other pieces of measurement information to the learning model that has been learned to perform machine learning for estimating the shape of the object. The learning model includes a feature extraction unit (74) that extracts a time change of a characteristic element included in the received signal, as time characteristic data on the basis of the received signal and a predetermined object characteristic pattern, and a data compression unit (75) that compresses the time characteristic data to obtain a plurality of feature amounts. The learning model includes a determination unit (21) that determines the shape of the object by performing an operation while assigning a predetermined weight to the plurality of feature amounts and the measurement information. The measurement information is information that is correlated to a change of the waveform of the received signal or the shape of the object.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Patent Application No. 2022-114030, filed on July 15, 2022, the contents of which are incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to an object detection device. [Background technology]

[0003] Conventionally, there is known an object detection device that detects an object such as an obstacle based on a received signal corresponding to a wave reflected by an object from a transmitted ultrasonic wave (see, for example, Patent Document 1). The device described in Patent Document 1 detects an object by inputting compressed data of the received signal corresponding to the reflected wave into a neural network NNO. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 182963 Summary of the Invention

[0005] The present inventors are considering acquiring object information related to the shape of an object by inputting feature quantities contained in the waveform of a received signal corresponding to a reflected wave into a trained learning model that has undergone machine learning to estimate the shape of an object. Specifically, the inventors are considering extracting temporal changes in characteristic elements contained in the received signal as temporal feature data based on the received signal and a predetermined feature pattern of the object, and inputting the compressed temporal feature data into the learning model as feature quantities contained in the received signal.

[0006] However, according to the inventors' research, it was found that since the received signal corresponding to the reflected wave is affected by changes in the surrounding environment of the object, including the positional relationship with the object, it is difficult to obtain highly accurate object information even if only the features contained in the received signal are input into the learning model. An object of the present disclosure is to provide an object detection device that can obtain object information with high accuracy.

[0007] According to one aspect of the present disclosure, The object detection device detects an object, a receiving unit that acquires a received signal corresponding to a reflected wave of an ultrasonic wave transmitted by an object; an information processing unit that inputs waveform features of the received signal and other measurement information into a trained learning model that has undergone machine learning to estimate the shape of the object, thereby acquiring object information relating to the shape of the object from the trained model; The learning model is a feature extraction unit that extracts, as time feature data, a time change of a characteristic element included in the received signal based on the received signal and a predetermined feature pattern of an object; a data compression unit that compresses the temporal feature data to obtain a plurality of feature quantities; a determination unit that determines the shape of an object by performing calculations while weighting the plurality of feature amounts and measurement information with predetermined weights, The measurement information is information that has a correlation with the change in the waveform of the received signal or the shape of the object. the law of nature, A plurality of receiving units, each including a receiving section and a feature extracting section, are arranged at different positions; When a plurality of feature quantities obtained by compressing time feature data based on received signals acquired by some of the plurality of receiving units are defined as a plurality of first feature quantities, and a plurality of feature quantities obtained by compressing time feature data based on received signals acquired by other receiving units in the same time period as the acquisition of the received signals by some of the receiving units are defined as a plurality of second feature quantities, the measurement information includes at least some of the second feature amounts; The determination unit determines the shape of the object by performing calculations while weighting at least some of the plurality of first feature amounts and the plurality of second feature amounts with predetermined weights.

[0008] In this way, by using not only the features contained in the received signal but also measurement information that is correlated with changes in the waveform of the received signal, it is possible to obtain object information with higher accuracy than by using only the features contained in the received signal as input to the learning model.

[0009] The reference symbols in parentheses attached to each component indicate an example of the correspondence between the component and the specific components described in the embodiments described below. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a plan view showing a schematic configuration of a vehicle equipped with an in-vehicle system that constitutes an object detection device according to a first embodiment. [Figure 2] FIG. 2 is an explanatory diagram for explaining a connection mode between each ultrasonic sensor and an electronic control device. [Figure 3] 1 is a block diagram showing a schematic functional configuration of an in-vehicle system according to a first embodiment. [Figure 4] 2 is a block diagram showing the functional configuration of an ultrasonic sensor and an electronic control unit of the in-vehicle system according to the first embodiment. FIG. [Figure 5] FIG. 10 is an explanatory diagram for explaining extraction of time feature data from a received signal. [Figure 6] 4 is an explanatory diagram for explaining the relationship between the amplitude waveform of a received signal and the height and distance measurement distance of an object. FIG. [Figure 7] FIG. 10 is an explanatory diagram for explaining weighting based on a plurality of feature amounts and distance information when the distance to the object is short. [Figure 8] FIG. 10 is an explanatory diagram for explaining weighting based on a plurality of feature amounts and distance information when the distance to the object is long. [Figure 9] 2 is an explanatory diagram for explaining functions of an ultrasonic sensor and an electronic control unit of the in-vehicle system according to the first embodiment. FIG. [Figure 10] FIG. 10 is a block diagram showing the functional configuration of an ultrasonic sensor and an electronic control unit of an in-vehicle system according to a second embodiment. [Figure 11] FIG. 10 is an explanatory diagram for explaining functions of an ultrasonic sensor and an electronic control unit of an in-vehicle system according to a third embodiment. [Figure 12] FIG. 10 is a block diagram showing a schematic functional configuration of an in-vehicle system according to a fourth embodiment. [Figure 13]FIG. 10 is an explanatory diagram for explaining functions of an ultrasonic sensor and an electronic control unit of an in-vehicle system according to a fourth embodiment. [Figure 14] FIG. 10 is an explanatory diagram for explaining measurement information combined on the electronic control unit side. [Figure 15] 10 is an explanatory diagram for explaining a modified example of measurement information combined on the electronic control device side. FIG. [Figure 16] FIG. 10 is a block diagram showing a schematic functional configuration of an in-vehicle system according to a fifth embodiment. [Figure 17] FIG. 10 is an explanatory diagram for explaining functions of an ultrasonic sensor and an electronic control unit of an in-vehicle system according to a fifth embodiment. [Figure 18] FIG. 10 is an explanatory diagram for explaining measurement information that is connected in a fully connected layer. [Figure 19] FIG. 10 is an explanatory diagram illustrating a modified example of measurement information combined in a fully connected layer. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following embodiments, parts that are the same as or equivalent to those described in the preceding embodiments will be given the same reference numerals, and their description may be omitted. Furthermore, in the embodiments, when only some of the components are described, the components described in the preceding embodiments can be applied to the remaining components. The following embodiments can be partially combined with each other, even if not specifically stated, as long as there is no particular problem with the combination.

[0012] (First embodiment) This embodiment will be described with reference to FIGS. 1 to 9. In this embodiment, an example will be described in which an object detection device according to the present disclosure is applied to an in-vehicle system 1. As shown in FIG. 1, the in-vehicle system 1 is mounted on a vehicle C as a moving body. The vehicle C is a so-called four-wheeled automobile, and has a box-shaped body C1 that is formed into a substantially rectangular shape in a plan view. The shape of each part of the vehicle C in a "plan view" refers to the shape of that part when viewed from the line of sight in the same direction as the direction of gravity, with the vehicle C stably placed on a horizontal surface so as to be able to travel. The vehicle C equipped with the in-vehicle system 1 according to this embodiment will hereinafter be referred to as the "host vehicle."

[0013] Hereinafter, the imaginary line that passes through the center of the host vehicle in the vehicle width direction in a plan view and is parallel to the vehicle length direction of the host vehicle will be referred to as the vehicle center line LC. The vehicle length direction is a direction that is perpendicular to the vehicle width direction and perpendicular to the vehicle height direction. The vehicle height direction is a direction that defines the vehicle height of the host vehicle and is a direction parallel to the direction of gravity when the host vehicle is stably placed on a horizontal surface so that it can travel. In addition, "front," "rear," "left," "right," and "up" are defined as shown by the arrows in Figure 1. In other words, the vehicle length direction is synonymous with the front-to-rear direction. In addition, the vehicle width direction is synonymous with the left-to-right direction.

[0014] The in-vehicle system 1 includes an electronic control unit 2 and an ultrasonic sensor 3. The electronic control unit 2 is an in-vehicle microcomputer that may also be referred to as an ECU, and includes a CPU, ROM, RAM, non-volatile rewritable memory, etc. (not shown). ECU stands for Electronic Control Unit. The non-volatile rewritable memory is a memory that allows information to be rewritten while the power is on but retains information in an unrewritable manner while the power is off, such as a flash ROM. The ROM, RAM, and non-volatile rewritable memory are non-transient physical storage media. The electronic control unit 2 is mounted inside a vehicle body C1.

[0015] The electronic control unit 2 is connected to the ultrasonic sensors 3 via an on-board information communication line so as to be able to send and receive information. In this embodiment, the host vehicle is equipped with a plurality of ultrasonic sensors 3. The electronic control unit 2 is configured to read and execute a control program stored in a ROM or a non-volatile rewritable memory, thereby controlling the overall operation of the on-board system 1, including the timing of the transmission and reception of ultrasonic waves by each of the plurality of ultrasonic sensors 3. In other words, the on-board system 1 constituting the object detection device according to this embodiment is configured to detect an object B around the host vehicle based on the results of transmission and reception of ultrasonic waves by the ultrasonic sensors 3 while mounted on the host vehicle.

[0016] The front bumper of the host vehicle, i.e., the bumper C2 on the front side of the vehicle body C1, is equipped with a first front sensor 3A, a second front sensor 3B, a third front sensor 3C, and a fourth front sensor 3D as ultrasonic sensors 3. Similarly, the rear bumper of the host vehicle, i.e., the bumper C2 on the rear side of the vehicle body C1, is equipped with a first rear sensor 3E, a second rear sensor 3F, a third rear sensor 3G, and a fourth rear sensor 3H as ultrasonic sensors 3.

[0017] The first front sensor 3A is provided at the right end of the front bumper so as to emit a transmission wave to the right front of the vehicle. The second front sensor 3B is disposed between the first front sensor 3A and the vehicle center line LC in the vehicle width direction so as to emit a transmission wave substantially ahead of the vehicle. The third front sensor 3C is disposed in a position substantially symmetrical to the second front sensor 3B across the vehicle center line LC. The third front sensor 3C is disposed between the vehicle center line LC and the fourth front sensor 3D in the vehicle width direction so as to emit a transmission wave substantially ahead of the vehicle. The fourth front sensor 3D is disposed in a position substantially symmetrical to the first front sensor 3A across the vehicle center line LC. The fourth front sensor 3D is provided at the left end of the front bumper so as to emit a transmission wave to the left front of the vehicle.

[0018] The first rear sensor 3E is provided at the right end of the rear bumper so as to emit a transmission wave to the right rear of the vehicle. The second rear sensor 3F is disposed between the first rear sensor 3E and the vehicle center line LC in the vehicle width direction so as to emit a transmission wave to approximately rear of the vehicle. The third rear sensor 3G is disposed in a position approximately symmetrical to the second rear sensor 3F across the vehicle center line LC. The third rear sensor 3G is disposed between the vehicle center line LC and the fourth rear sensor 3H in the vehicle width direction so as to emit a transmission wave to approximately rear of the vehicle. The fourth rear sensor 3H is disposed in a position approximately symmetrical to the first rear sensor 3E across the vehicle center line LC. The fourth rear sensor 3H is provided at the left end of the rear bumper so as to emit a transmission wave to the left rear of the vehicle.

[0019] In recent years, the number of wire harnesses in the vehicle C has been reduced in order to achieve carbon neutrality and reduce costs. Taking this into consideration, the in-vehicle system 1 of this embodiment has a plurality of ultrasonic sensors 3 and an electronic control unit 2 connected to each other via a bus-type network, as shown in FIG.

[0020] Next, the schematic configuration of the ultrasonic sensor 3 will be described with reference to Fig. 3. In Fig. 3, for the sake of simplicity, only one of the multiple ultrasonic sensors 3 connected to the electronic control device 2 is shown, and the others are omitted.

[0021] The ultrasonic sensor 3 is configured to transmit ultrasonic waves as transmission waves toward the outside of the vehicle, and is configured to detect an object B present in the vicinity and acquire the distance to the object B based on a reception signal corresponding to a reception wave including a wave of the transmission wave reflected by the object B.

[0022] Specifically, the ultrasonic sensor 3 includes a transmitter / receiver 4, a drive signal generator 5, a received signal processor 6, and a sensor controller 7. In this embodiment, the transmitter / receiver 4, drive signal generator 5, received signal processor 6, and sensor controller 7 are supported by a single sensor housing made of synthetic resin or the like.

[0023] In this embodiment, the ultrasonic sensor 3 is provided with only one transmitter / receiver 4, and is configured to perform the transmission and reception functions using this transmitter / receiver 4. That is, the transmitter / receiver 4 has a function as a transmitter 40A that transmits transmission waves to the outside, and a function as a receiver 40B that receives reception waves. Specifically, one transmitter / receiver 4 has one transducer 41. The transmitter 40A and receiver 40B are configured to use the common transducer 41 to perform the transmission function and the reception function, respectively.

[0024] The transducer 41 is configured to function as a transmitter that transmits a transmission wave to the outside and as a receiver that receives a reflected wave. The transducer 41 has a configuration as a so-called resonance type ultrasonic microphone that incorporates an electromechanical energy conversion element such as a piezoelectric element.

[0025] The transmitter / receiver unit 4 includes a transducer 41, a transmission circuit 42, and a reception circuit 43. That is, the transmitter unit 40A includes the transducer 41 and the transmission circuit 42. The receiver unit 40B includes the transducer 41 and the reception circuit 43. The transducer 41 is electrically connected to the transmission circuit 42 and the reception circuit 43.

[0026] The transmission circuit 42 is configured to drive the transducer 41 based on the input drive signal, thereby causing the transducer 41 to emit a transmission wave in the ultrasonic band. Specifically, the transmission circuit 42 has a digital / analog conversion circuit and the like. That is, the transmission circuit 42 is configured to perform processing such as digital / analog conversion on the drive signal output from the drive signal generation unit 5, and apply the AC voltage generated thereby to the transducer 41.

[0027] The receiving circuit 43 is configured to generate a receiving signal corresponding to the reception result of the ultrasonic wave at the transducer 41, and to output the generated receiving signal to the receiving signal processing unit 6. Specifically, the receiving circuit 43 has an amplifier circuit, an analog / digital conversion circuit, etc. That is, the receiving circuit 43 is configured to perform signal processing such as amplification and analog / digital conversion on the voltage signal input from the transducer 41, thereby generating and outputting a receiving signal corresponding to the frequency, phase, and amplitude of the received ultrasonic wave.

[0028] The drive signal generating unit 5 is provided to generate a drive signal for driving the transmitting unit 40 A. The drive signal is a signal for driving the transmitting unit 40 A to cause the transducer 41 to transmit a transmission wave.

[0029] The received signal processing unit 6 is configured to perform various signal processing such as filtering and quadrature detection on the received signal output from the receiving circuit 43. The received signal processing unit 6 is also configured to output a processed signal, which is a result of the various signal processing, to the sensor control unit 7.

[0030] The sensor control unit 7 is communicably connected to the electronic control unit 2 so as to cooperate with the electronic control unit 2 to control the operation of the ultrasonic sensor 3. That is, the sensor control unit 7 is configured to control the output of the drive signal from the drive signal generation unit 5 to the transmission unit 40A, and to detect the object B based on the processed signal output from the received signal processing unit 6.

[0031] The sensor control unit 7 is configured as an in-vehicle microcomputer including a CPU, ROM, RAM, non-volatile rewritable memory, etc. (not shown). That is, the sensor control unit 7 is configured to read and execute a control program stored in the ROM or non-volatile rewritable memory to control the operation of the ultrasonic sensor 3. Specifically, the sensor control unit 7 includes, as functional components realized on the ultrasonic sensor 3, a drive control unit 71, a reflected wave detection unit 72, and a distance acquisition unit 73.

[0032] The drive control unit 71 controls the emission state of the transmission wave from the transmitter 40A by outputting a control signal to the drive signal generation unit 5. The control signal is a signal for controlling the output characteristics of the drive signal output from the drive signal generation unit 5 to the transmitter / receiver 4, specifically, the output timing, frequency, etc. In other words, the drive control unit 71 controls the output timing, frequency, etc. of the drive signal generated and output by the drive signal generation unit 5.

[0033] The reflected wave detection unit 72 detects the received wave, which includes the reflected wave of the transmitted wave contained in the received signal by the object B. The reflected wave detection unit 72 determines whether or not there is a characteristic portion exceeding a predetermined amplitude value in the amplitude waveform of the received signal obtained by passing it through a predetermined filter, and detects the reflected wave based on the determination result.

[0034] The distance acquisition unit 73 is configured to acquire the measured distance, which is the distance to the object B, based on the received signal. Specifically, in this embodiment, for example, the distance acquisition unit 73 is configured to calculate the measured distance based on the reception time of a peak in the amplitude signal included in the received signal when the reflected wave is detected by the reflected wave detection unit 72.

[0035] In this embodiment, an in-vehicle system 1 equipped with an electronic control unit 2 and an ultrasonic sensor 3 is configured to detect the height of object B as part of the shape of object B based on a signal received by the ultrasonic sensor 3. The in-vehicle system 1 of this embodiment includes an information processing unit 8 that acquires object information related to the shape of object B from a trained learning model that has undergone machine learning to estimate the shape of object B by inputting feature quantities of the waveform of the received signal and other measurement information. The learning model of this information processing unit 8 is configured as a neural network.

[0036] The information processing unit 8 is configured to include not only the electronic control device 2 but also a portion of the sensor control unit 7 included in the ultrasonic sensor 3. The learning model of the information processing unit 8 is a trained model trained using multiple pieces of training data in which input features of the waveform of the received signal and other measurement information are paired with the shape of the object B. In the learning model, the part that inputs the features of the waveform of the received signal and other measurement information from the ultrasonic sensor 3 constitutes the input layer, and the part that outputs the object information from the electronic control device 2 constitutes the output layer. The learning model includes the feature extraction unit 74 and data compression unit 75 of the sensor control unit 7 and the determination unit 21 of the electronic control device 2 as intermediate layers. At least a portion of this intermediate layer performs arithmetic processing on the input values ​​using predetermined weights, and then outputs the calculated value obtained by the arithmetic processing via an activation function. The activation function is a function for performing nonlinear transformation processing.

[0037] The feature extraction unit 74 extracts temporal changes of characteristic elements contained in the received signal as temporal feature data based on the received signal and a predetermined characteristic pattern of object B. The feature extraction unit 74 functions as a convolutional layer in a neural network. The feature extraction unit 74 constitutes part of a learning model. As shown in FIG. 4, the feature extraction unit 74 extracts temporal changes of characteristic elements contained in the IQ signal, which is the received signal, as temporal feature data by performing a convolution operation using multiple trained filters corresponding to the characteristic pattern of object B. The IQ signal is a signal containing two components: in-phase I and quadrature Q. In this way, if the configuration is such that temporal feature data is extracted from the IQ signal, the temporal feature data will contain not only amplitude changes but also phase changes, making it possible to extract more appropriate data than if temporal feature data were simply extracted from the amplitude waveform of the received signal.

[0038] Here, Fig. 5 is an explanatory diagram for explaining extraction of time feature data from a received signal. For ease of understanding, Fig. 5 shows an example in which time feature data is extracted from the amplitude component of an IQ signal, but in reality, time feature data is extracted from the IQ signal. As shown in Fig. 5, the feature extraction unit 74 identifies a characteristic element present in a portion of the amplitude waveform of the received signal that exceeds a predetermined threshold. The feature extraction unit 74 then defines a predetermined period that includes the characteristic element as a reflected wave region, and extracts the time change of the characteristic element in the reflected wave region as temporal feature data. The reflected wave region may be set to a period longer than the period in which the amplitude waveform of the received signal exceeds the predetermined threshold, or may be set to a period equal to the period in which the amplitude waveform exceeds the predetermined threshold. In this way, by extracting data from a reflected wave region including characteristic elements, it is possible to reduce the processing load on the feature extraction unit 74. Note that the feature extraction unit 74 can identify characteristic elements by using, for example, a correlation filter that has correlation with the reflected wave. Note that the feature extraction unit 74 may extract time feature data from the amplitude component of the IQ signal, rather than extracting time feature data from the IQ signal.

[0039] The data compression unit 75 compresses the temporal feature data extracted by the feature extraction unit 74 to obtain multiple feature quantities. The data compression unit 75 functions as a fully connected layer in a neural network. The number of nodes in the fully connected layer that constitutes the data compression unit 75 is smaller than the number of nodes in the convolution layer that constitutes the feature extraction unit 74, so that the data compression unit 75 compresses the data. The data compressed by the data compression unit 75 is transferred to the electronic control unit 2 via the bus-type network shown in FIG. 2.

[0040] Here, while a bus-type network has the advantage of being able to reduce the wire harness between the ultrasonic sensor 3 and the electronic control unit 2, it also has the trade-off of being prone to excessive communication traffic due to concentration of communication load.

[0041] In contrast, the in-vehicle system 1 of this embodiment is configured to transfer data compressed on the ultrasonic sensor 3 side to the electronic control unit 2. This makes it possible to reduce the wire harness between the ultrasonic sensor 3 and the electronic control unit 2 while also transferring data with reduced communication traffic.

[0042] The determination unit 21 determines the shape of object B by performing calculations while weighting the measurement information and multiple feature quantities obtained by compression in the data compression unit 75 with predetermined weights. Specifically, the determination unit 21 of this embodiment determines whether object B is a tall object or a short object. The determination unit 21 functions as a fully connected layer in a neural network. The determination unit 21 may be configured as one fully connected layer or multiple fully connected layers. In the determination unit 21, it is desirable that the number of nodes in the fully connected layer be greater than the number of nodes in the fully connected layer constituting the data compression unit 75, so as to improve the determination accuracy. The weights used in the determination unit 21 are set when a learning model is obtained using training data.

[0043] Here, the characteristics of the signal received by the ultrasonic sensor 3 change nonlinearly depending on the shape of the object B, the surrounding environment, etc. Below, the relationship between the amplitude waveform of the received signal and the height and measuring distance of the object B will be explained with reference to FIG.

[0044] For example, if object B is a tall object such as a wall or pole, the amplitude waveform of the received signal will have two peaks: one corresponding to the front reflection and one corresponding to the base reflection, as shown in the upper part of Figure 6. If object B is a short object such as a curb, the amplitude waveform of the received signal will have one peak corresponding to the base reflection, as shown in the lower part of Figure 6. In this way, there is a certain correlation between the amplitude waveform of the received signal and the shape of object B.

[0045] Furthermore, the reflected wave from object B theoretically has the same waveform as the transmitted wave, but a phase shift may occur relative to the transmitted wave depending on the positional relationship between the ultrasonic sensor 3 and object B. Such a phase shift affects the amplitude waveform of the received signal. In other words, there is a correlation between the phase change of the reflected wave and the change in the waveform of the received signal.

[0046] Furthermore, as shown on the left side of Figure 6, when the measurement distance is short, the two peaks for a tall object are clearly separated. In contrast, as shown on the right side of Figure 6, when the measurement distance is long, the two peaks for a tall object are close to each other. The two peaks for a tall object may sometimes combine to form almost a single peak. As such, there is a certain correlation between the measurement distance of object B and changes in the waveform of the received signal. In particular, when object B is tall, the influence on the waveform of the received signal corresponding to the reflected wave due to differences in the distance from object B tends to be significant. Note that the magnitude of the amplitude waveform of the received signal not only attenuates with the distance to object B, but also fluctuates due to changes in the angle of the root reflection in response to changes in the distance to object B. Therefore, there is also a certain correlation between the magnitude of the amplitude waveform of the received signal and the shape (e.g., height) of object B.

[0047] Taking these factors into consideration, the determination unit 21 of this embodiment is configured to determine the shape of object B by performing calculations using the measured distance of object B as measurement information, weighting it with predetermined weights together with multiple feature amounts obtained by compression in the data compression unit 75. Then, the determination unit 21 is configured to output, for example, information such as whether object B is "tall" or "short" as the determination result of the shape of object B, as shown in Figs. 7 and 8.

[0048] Here, the determination unit 21 is configured to receive input of the measured distance expressed as a one-hot vector. In the examples shown in Fig. 7 and Fig. 8, a state in which object B is close and the measured distance is small is represented as "(1, 0)", and a state in which object B is far and the measured distance is large is represented as "(0, 1)". Note that what is shown in Fig. 7 and Fig. 8 is just an example, and different vectors may be used. For example, a state in which the measured distance is small may be represented as "(1, 0, 0)", a state in which the measured distance is large as "(0, 1, 0)", and a state in which the measured distance is medium as "(0, 0, 1)".

[0049] In the determination unit 21 configured in this manner, when the measured distance is close, the information of the node indicating close affects the output, but the information of the node indicating far does not affect the output, as shown in Fig. 7. Also, when the measured distance is far, the information of the node indicating close does not affect the output, but the information of the node indicating far does affect the output, as shown in Fig. 8. This allows the determination unit 21 to appropriately determine whether object B is a tall "tall object" or a short "short object."

[0050] Next, the object detection process by the in-vehicle system 1 will be described with reference to Fig. 9. When a predetermined object detection condition is met, the electronic control unit 2 causes each of the multiple ultrasonic sensors 3 to repeatedly execute an object detection operation at a predetermined cycle. When the ultrasonic sensor 3 receives a command signal from the electronic control unit 2, it repeatedly executes the detection process for object B at a predetermined cycle.

[0051] When the detection process for object B starts, the ultrasonic sensors 3 emit transmission waves, which are ultrasonic waves, in a predetermined order. In addition, the ultrasonic sensors 3 periodically receive reception waves that include reflected waves of the transmission waves by object B.

[0052] Next, the ultrasonic sensor 3 calculates the measured distance based on the reception time of the peak in the amplitude signal included in the received signal when the reflected wave is detected. The ultrasonic sensor 3 also extracts temporal feature data, which is the change over time of characteristic elements included in the received signal, based on the received signal corresponding to the received wave and the predetermined characteristic pattern of object B. As shown in FIG. 9 , the ultrasonic sensor 3 of this embodiment extracts temporal feature data using a trained filter in the convolutional layer of the learning model. The convolutional layer of the learning model can extract features that retain information indicating the positions of the peaks in the waveform of the received signal. Therefore, when two peaks appear in the amplitude waveform of the received signal, characteristic elements such as the distance information between the two peaks can be extracted.

[0053] Then, the ultrasonic sensor 3 compresses the time feature data in the fully connected layer of the learning model to obtain multiple features, and then transfers the data including the multiple features and the measured distance to the electronic control device 2 via a bus-type network.

[0054] When the electronic control device 2 receives data including multiple feature amounts and measured distances from the ultrasonic sensor 3, it performs calculations while weighting the data with learned weights in the fully connected layer of the learning model, to determine the shape of object B. Then, the electronic control device 2 outputs information such as whether object B is "tall" or "short" as the determination result of the shape of object B. Note that the electronic control device 2 may be configured to output information other than information such as whether object B is "tall" or "short" as the determination result of the shape of object B.

[0055] The in-vehicle system 1 described above is configured to determine the shape of the object B using not only the feature amounts included in the received signal corresponding to the wave reflected by the object B, but also measurement information that has a correlation with changes in the waveform of the received signal. This makes it possible to obtain object information with higher accuracy than a configuration in which only the feature amounts included in the received signal are input to the learning model.

[0056] The in-vehicle system 1 of this embodiment also has the following features.

[0057] (1) The ultrasonic sensor 3 includes a distance acquisition unit 73 that acquires the measured distance, which is the distance to object B, based on the received signal. The electronic control device 2 is configured to determine the shape of object B using not only the feature values ​​included in the received signal corresponding to the reflected wave from object B, but also the measured distance. The inventors' investigations have revealed that the waveform of the received signal corresponding to the reflected wave changes depending on the distance to object B. In particular, when object B is a tall object, the influence of the difference in distance from object B on the waveform of the received signal corresponding to the reflected wave tends to be significant. Thus, it can be said that there is a certain correlation between the distance to object B and the change in the waveform of the received signal. Therefore, if the shape of object B is determined by using the distance to object B as measurement information and performing calculations on multiple feature values ​​and the distance to object B while weighting them with predetermined weights, the shape of object B can be appropriately determined.

[0058] In particular, the in-vehicle system 1 of this embodiment performs "feature extraction by convolution" and "data compression" on the ultrasonic sensor 3 side, and inputs distance information in one-hot vector format to the learning model on the electronic control unit 2 side. This makes it possible to determine the height of object B with high accuracy.

[0059] (2) The feature extraction unit 74 identifies characteristic elements present in parts of the amplitude waveform of the received signal that exceed a predetermined threshold, and extracts time feature data that represents the time change of the characteristic elements over a predetermined period of time. This allows the amount of data processed by the information processing unit 8 to be efficiently reduced.

[0060] (3) The data compression unit 75 and the determination unit 21 that constitute the information processing unit 8 are provided in different components. Specifically, in the information processing unit 8, the data compression unit 75 is provided in the ultrasonic sensor 3, and the determination unit 21 is provided in the electronic control unit 2. The plurality of feature quantities compressed by the data compression unit 75 are transferred to the determination unit 21 via a communication network. In this manner, if the configuration is such that the plurality of compressed feature quantities are transferred from the data compression unit 75 to the determination unit 21 via a communication network, the amount of data communication within the communication network can be reduced compared to when unprocessed RAW data is transferred.

[0061] (4) The determination unit 21 receives the measured distance expressed as a one-hot vector as measurement information. In this way, if the configuration uses measurement information expressed as a one-hot vector, variables other than numerical values, such as far or near, can be used as measurement information, and the variables input to the determination unit 21 can be treated equally. This contributes to the appropriate determination of the shape of object B using a learning model. Furthermore, measurement information expressed as a one-hot vector does not require normalization or standardization. This contributes to improving the processing speed and reducing the load on the information processing unit 8.

[0062] (Second embodiment) Next, a second embodiment will be described with reference to Fig. 10. In this embodiment, differences from the first embodiment will be mainly described.

[0063] The learning model of the information processing unit 8 of this embodiment is configured with a convolutional neural network (so-called CNN). As shown in Fig. 10, the data compression unit 75 functions as a pooling layer in the neural network, rather than a fully connected layer, which reduces the data by capturing only representative feature amounts from the temporal feature data extracted by the feature extraction unit 74. Unlike a fully connected layer, the pooling layer does not have parameters such as weights, and therefore can reduce excess memory usage in the ultrasonic sensor 3.

[0064] The data compression unit 75 of this embodiment compresses the temporal feature data by, for example, max pooling, extracting the maximum value of multiple intervals in the temporal feature data extracted by the feature extraction unit 74. Note that the data compression unit 75 may also compress the temporal feature data by average spooling.

[0065] The rest of the configuration is the same as that of the first embodiment. The in-vehicle system 1 of this embodiment can obtain the same effects as those of the first embodiment that are achieved by the configuration common to or equivalent to that of the first embodiment.

[0066] The in-vehicle system 1 of this embodiment also has the following features.

[0067] (1) The data compression unit 75 of this embodiment functions as a pooling layer. This allows the time feature data extracted by the feature extraction unit 74 to be compressed and transferred to the electronic control unit 2 while reducing unnecessary memory usage in the ultrasonic sensor 3.

[0068] (Third embodiment) Next, a third embodiment will be described with reference to Fig. 11. In this embodiment, differences from the first embodiment will be mainly described.

[0069] The waveform of the received signal corresponding to the reflected wave changes nonlinearly due to fluctuations in the speed of sound, which depend on the temperature, humidity, and wind speed around the ultrasonic sensor 3 on which the receiving unit 40B is mounted. Therefore, there is a certain correlation between the temperature, humidity, and wind speed around the ultrasonic sensor 3 and changes in the waveform of the received signal.

[0070] Furthermore, the waveform of the received signal corresponding to the reflected wave changes depending on the positional relationship between the position where the transmitted wave is emitted and the position where the received signal is received. For example, the waveform of the received signal corresponding to the reflected wave of the transmitted wave emitted by the first front sensor 3A by object B differs between the first front sensor 3A and the second front sensor 3B. In this way, there is a certain correlation between the transmission and reception information for identifying the positional relationship between the position where the transmitted wave is emitted and the position where the received signal is received and the waveform of the received signal corresponding to the reflected wave.

[0071] In consideration of these points, the in-vehicle system 1 of this embodiment is configured to determine the shape of object B using not only the feature values ​​contained in the received signal corresponding to the wave reflected by object B, but also the distance measurement, the temperature, humidity, wind speed, and transmission and reception information around the ultrasonic sensor 3, as shown in FIG. 11.

[0072] The ultrasonic sensor 3 of this embodiment is configured to be able to measure the temperature, humidity, and wind speed around the ultrasonic sensor 3. The ultrasonic sensor 3 is also configured to be able to output data that associates its own mounting position in the vehicle C with the transmission and reception of signals as transmission and reception information.

[0073] When the detection process for object B starts, the ultrasonic sensors 3 emit ultrasonic transmission waves in a predetermined sequence upon receiving a command signal from the electronic control device 2. The ultrasonic sensors 3 also periodically receive reception waves that include reflected waves of the transmission waves by object B.

[0074] Next, the ultrasonic sensor 3 calculates the measured distance and extracts time feature data. The ultrasonic sensor 3 also measures the temperature, humidity, and wind speed around the ultrasonic sensor 3. The ultrasonic sensor 3 then compresses the time feature data in the fully connected layer of the learning model to obtain multiple feature quantities, and then transfers data including the multiple feature quantities, the measured distance, temperature, humidity, wind speed, and transmission / reception information to the electronic control device 2 via the bus-type network.

[0075] When the electronic control device 2 receives data including multiple feature amounts, ranging distance, temperature, humidity, wind speed, and transmission / reception information from the ultrasonic sensor 3, it performs calculations while weighting the data with learned weights in the fully connected layer of the learning model to determine the shape of object B. Then, the electronic control device 2 outputs information such as whether object B is "tall" or "short" as the result of the determination of the shape of object B. Note that the determination unit 21 of the electronic control device 2 is configured to receive input of measurement information expressed as a one-hot vector.

[0076] The rest of the configuration is the same as that of the first embodiment. The in-vehicle system 1 of this embodiment can obtain the same effects as those of the first embodiment that are achieved by the configuration common to or equivalent to that of the first embodiment.

[0077] The in-vehicle system 1 of this embodiment also has the following features.

[0078] (1) The electronic control device 2 determines the shape of object B using not only the feature values ​​contained in the received signal corresponding to the wave reflected by object B, but also the temperature, humidity, and wind speed around the ultrasonic sensor 3. There is a certain correlation between the temperature, humidity, and wind speed and changes in the waveform of the received signal. Therefore, if the temperature, humidity, and wind speed around the above-mentioned device are used as measurement information and the shape of object B is determined by performing calculations while weighting the multiple feature values, temperature, humidity, and wind speed with predetermined weights, the shape of object B can be appropriately determined.

[0079] (2) The electronic control device 2 determines the shape of the object B using not only the feature values ​​contained in the received signal corresponding to the wave reflected by the object B, but also the transmission and reception information for specifying the position at which the received signal is received relative to the position at which the transmitted wave is emitted. The waveform of the received signal corresponding to the reflected wave changes depending on the position at which the received signal is received relative to the position at which the transmitted wave is emitted. Therefore, if the transmission and reception information is used as measurement information and the shape of the object B is determined by performing calculations while weighting the multiple feature values ​​and the transmission and reception information with predetermined weights, the shape of the object B can be appropriately determined.

[0080] (Modification of the third embodiment) The electronic control device 2 may be configured to use one or two of the temperature, humidity, and wind speed around the ultrasonic sensor 3 as measurement information, and perform calculations while weighting the plurality of feature quantities and one or two of the temperature, humidity, and wind speed with predetermined weights to determine the shape of the object B. The electronic control device 2 acquires measurement information such as temperature, humidity, and wind speed from the ultrasonic sensor 3, but may also acquire it from a device other than the ultrasonic sensor 3.

[0081] It is desirable that the electronic control device 2 is configured to use the transmitted and received information as measurement information, and to perform calculations while weighting the multiple feature amounts and the transmitted and received information with predetermined weights to determine the shape of object B, but this is not necessarily the case.

[0082] (Fourth embodiment) Next, a fourth embodiment will be described with reference to Figures 12 to 14. In this embodiment, differences from the first embodiment will be mainly described.

[0083] The multiple feature amounts obtained by compressing the temporal feature data extracted by the feature extraction unit 74 are based on characteristic elements that have a correlation with the shape of object B, and are information that has a certain correlation with the shape of object B. This applies not only to the multiple feature amounts obtained by compressing the temporal feature data currently extracted by the feature extraction unit 74 (hereinafter also referred to as current feature amounts), but also to the multiple feature amounts obtained by compressing the temporal feature data previously extracted by the feature extraction unit 74 (hereinafter also referred to as past feature amounts). Therefore, the past feature amounts are information that has a certain correlation with the shape of object B.

[0084] Furthermore, since the waveform of the received signal corresponding to the reflected wave changes depending on the position at which the received signal is received, it can be said that the position change information indicating the change in position has a certain correlation with the change in the shape of the received signal.

[0085] Taking these factors into consideration, the in-vehicle system 1 of this embodiment is configured to determine the shape of object B using not only the current feature values ​​contained in the received signal corresponding to the wave reflected by object B, but also the past feature values ​​and position change information, as shown in Figures 12 and 13.

[0086] When the electronic control unit 2 receives multiple feature quantities obtained by compressing the temporal feature data extracted by the feature extraction unit 74, the electronic control unit 2 stores the multiple feature quantities in memory. As a result, the electronic control unit 2 accumulates N feature quantities, including current feature quantities and past feature quantities, in memory. The electronic control unit 2 also acquires wheel speed pulses as position change information from the wheel speed sensors. The determination unit 21 of the electronic control unit 2 then weights the N feature quantities, ranging distances, and position change information, combining the current feature quantities and past feature quantities acquired from the ultrasonic sensors 3, with weights learned in the fully connected layer of the learning model, and combines the data, such as the N feature quantities. The determination unit 21 determines the shape of the object B based on the data, such as the N feature quantities. Measurement information expressed as a one-hot vector is input to the determination unit 21 of the electronic control unit 2. The number of previous feature quantities to be input to the learning model is determined appropriately depending on the traveling speed of the vehicle C, the processing power of the electronic control unit 2, the memory capacity, and other factors.

[0087] Specifically, as shown in FIG. 14 , the determination unit 21 inputs, into the learning model, among the plurality of current feature amounts and the plurality of past feature amounts, those based on signals whose amplitude waveforms contain feature portions exceeding a predetermined amplitude value. The determination unit 21 then performs calculations while weighting, with a predetermined weight, those among the plurality of current feature amounts and the plurality of past feature amounts that are based on signals whose amplitude waveforms contain feature portions exceeding a predetermined amplitude value. The electronic control unit 2 of this embodiment outputs information such as whether the height of object B is "tall" or "short" as the determination result of the shape of object B. Note that the electronic control unit 2 may be configured to output information other than information such as whether the height of object B is "tall" or "short" as the determination result of the shape of object B.

[0088] The rest of the configuration is the same as that of the first embodiment. The in-vehicle system 1 of this embodiment can obtain the same effects as those of the first embodiment that are achieved by the configuration common to or equivalent to that of the first embodiment.

[0089] The in-vehicle system 1 of this embodiment also has the following features.

[0090] (1) The determination unit 21 determines the shape of object B by performing calculations on at least some of the multiple current feature quantities and multiple past feature quantities while weighting them with predetermined weights. The past feature quantities are based on characteristic elements that are correlated with the shape of object B and are included in previously acquired received signals. Similar to the current feature quantities, the past feature quantities are information that has a certain correlation with the shape of object B. Therefore, if at least some of the multiple past feature quantities are used as measurement information and the shape of object B is determined by performing calculations on at least some of the multiple current feature quantities and past feature quantities while weighting them with predetermined weights, the shape of object B can be appropriately determined. In particular, if, as in the determination unit 21 of this embodiment, not only the current feature quantities but also the past feature quantities are input as measurement information to the learning model, changes in the feature quantities corresponding to changes in the amplitude and phase of the received signal are also input to the learning model. This allows for more appropriate determination of the shape of object B.

[0091] (2) The determination unit 21 determines the shape of object B by performing calculations using not only the multiple feature amounts but also position change information indicating a change in the position at which the received signal is received while weighting it with a predetermined weight. In this way, by using position change information as measurement information in addition to the multiple feature amounts and performing calculations using the multiple feature amounts and position change information while weighting it with a predetermined weight, the shape of object B can be determined appropriately.

[0092] (3) The determination unit 21 performs calculations while weighting, with a predetermined weight, those of the multiple current feature amounts and multiple past feature amounts that are based on signals whose amplitude waveforms contain feature portions exceeding a predetermined amplitude value, thereby determining the shape of the object B. In this way, if the configuration is such that the shape of the object B is determined from feature amounts that are based on signals whose amplitude waveforms contain feature portions exceeding a predetermined amplitude value among the received signals, the amount of data processed by the information processing unit 8 can be efficiently reduced.

[0093] (Modification of the fourth embodiment) When object B is in a position that affects the received signal, the amplitude waveform of the received signal normally increases, but in a real environment, for example, the amplitude waveform of the received signal may decrease due to air current fluctuations, wave interference, etc., as shown in the center of Figure 15. When object B is in a position that affects the received signal, even if the amplitude waveform of the received signal is small, there is a high probability that the amplitude waveform of the received signal includes changes caused by object B.

[0094] Therefore, when it is estimated that object B is in a position that affects the received signal, the determination unit 21 may determine the shape of object B using, among the multiple feature amounts, not only those based on signals having characteristic portions but also those based on signals having no characteristic portions. With this configuration, the shape of object B can be determined appropriately.

[0095] Here, whether or not object B is in a position that will affect the received signal can be estimated based on the amplitude waveform and position change information of the previously received signal. For example, the determination unit 21 may be configured to estimate that object B is in a position that will affect the received signal if the amplitude waveform of the previously received signal contains a characteristic portion that exceeds a predetermined amplitude value and the change in position of the ultrasonic sensor 3 since the previous time is equal to or less than a predetermined value.

[0096] Furthermore, as in the fourth embodiment described above, it is desirable that the judgment unit 21 judges the shape of the object B by performing calculations using not only multiple feature amounts but also the ranging distance and position change information while weighting them with predetermined weights, but this is not necessarily required.

[0097] (Fifth embodiment) Next, a fifth embodiment will be described with reference to Figures 16 to 18. In this embodiment, differences from the first embodiment will be mainly described.

[0098] The ultrasonic sensor 3 constitutes a receiving unit RU that includes a receiving section 40B and a feature extraction section 74. In the in-vehicle system 1, a plurality of receiving units RU are arranged at different positions in the vehicle C. Adjacent receiving units RU may receive a reflected wave from the same object B. For example, as shown in FIG. 1, if an object B is present in front of the vehicle C, the first front sensor 3A and the second front sensor 3B may both receive a reflected wave from the object B.

[0099] When a first feature is defined as a plurality of feature amounts obtained by compressing temporal feature data based on a received signal acquired by one of adjacent receiving units RU, the first feature is based on a characteristic element contained in the received signal that has a correlation with the shape of object B. When a second feature is defined as a plurality of feature amounts obtained by compressing temporal feature data based on a received signal acquired by the other receiving unit RU, the second feature is based on a characteristic element contained in the received signal that has a correlation with the shape of object B. Therefore, the first feature and the second feature have a certain correlation with the shape of object B.

[0100] Here, if the distance between the receiving unit 40B and the transmitting unit 40A that emits the transmitted wave is small, the strength of the received signal corresponding to the wave reflected by the object B is likely to be large. Furthermore, if the distance between the multiple receiving units 40B is short, the difference in strength of the received signal corresponding to the wave reflected by the object B is likely to be small. In this way, there is a certain correlation between the positions of the multiple receiving units 40B relative to the transmitting unit 40A and the positional relationship between the multiple receiving units 40B.

[0101] Taking these factors into consideration, the in-vehicle system 1 of this embodiment is configured to determine the shape of object B using not only the first feature contained in the received signal corresponding to the wave reflected by object B, but also the second feature and sensor information, as shown in Figures 16 and 17.

[0102] The ultrasonic sensor 3 of this embodiment is configured to be able to output data as sensor information that associates its own mounting position in the vehicle C with the transmission and reception of signals. Upon receiving a command signal from the electronic control unit 2, the multiple ultrasonic sensors 3 emit ultrasonic transmission waves in a predetermined order. The ultrasonic sensors 3 also periodically receive reception waves that include waves of the transmission waves reflected by the object B. The ultrasonic sensors 3 then compress the time feature data in the fully connected layer of the learning model to obtain multiple feature amounts, and then transfer data including the multiple feature amounts, ranging distances, and sensor information to the electronic control unit 2 via a bus-type network.

[0103] When the electronic control unit 2 receives data including multiple feature quantities and the like from the multiple ultrasonic sensors 3, it weights the data with learned weights in the fully connected layer of the learning model and combines the feature quantity and other data transferred from adjacent receiving units RU. The determination unit 21 determines the shape of the object B based on the feature quantity and other data transferred from adjacent receiving units RU. Note that measurement information expressed as a one-hot vector is input to the determination unit 21 of the electronic control unit 2.

[0104] Specifically, as shown in FIG. 18 , the determination unit 21 inputs, into the learning model, among the plurality of first feature quantities and the plurality of second feature quantities, those based on signals whose amplitude waveforms contain characteristic portions exceeding a predetermined amplitude value. In other words, the determination unit 21 does not input, into the learning model, among the plurality of first feature quantities and the plurality of second feature quantities, those based on signals whose amplitude waveforms contain characteristic portions exceeding a predetermined amplitude value. The determination unit 21 then performs calculations while weighting, with a predetermined weight, those among the plurality of first feature quantities and the plurality of second feature quantities that are based on signals whose amplitude waveforms contain characteristic portions exceeding a predetermined amplitude value. The electronic control unit 2 of this embodiment outputs information such as whether the height of object B is “tall” or “short” as the determination result of the shape of object B. Note that the electronic control unit 2 may be configured to output information other than information such as whether the height of object B is “tall” or “short” as the determination result of the shape of object B.

[0105] The rest of the configuration is the same as that of the first embodiment. The in-vehicle system 1 of this embodiment can obtain the same effects as those of the first embodiment that are achieved by the configuration common to or equivalent to that of the first embodiment.

[0106] The in-vehicle system 1 of this embodiment also has the following features.

[0107] (1) The determination unit 21 determines the shape of object B by performing calculations on at least some of the multiple first feature amounts and multiple second feature amounts while weighting them with predetermined weights. The first feature amounts and second feature amounts are based on characteristic elements contained in the received signal that have a correlation with the shape of object B, and are information that has a certain correlation with the shape of object B. Therefore, if the configuration is such that at least some of the multiple first feature amounts and second feature amounts are used as measurement information and at least some of the multiple first feature amounts and second feature amounts are weighted with predetermined weights and calculated to determine the shape of object B, the shape of object B can be appropriately determined.

[0108] (2) The determination unit 21 determines the shape of the object B by performing calculations using not only the multiple feature amounts but also the positions of the multiple receiving units 40B relative to the transmitting unit 40A that emits the transmission wave, and sensor information indicating the positional relationship between the multiple receiving units 40B, while weighting the calculations with predetermined weights. In this way, by using the sensor information as measurement information in addition to the multiple feature amounts and performing calculations using predetermined weights to determine the shape of the object B, the shape of the object B can be appropriately determined.

[0109] (3) The determination unit 21 performs calculations on the plurality of first feature amounts and the plurality of second feature amounts, weighting with a predetermined weight, those based on signals whose amplitude waveforms contain feature portions exceeding a predetermined amplitude value, to determine the shape of the object B. In this way, by using a configuration in which the shape of the object B is determined from feature amounts based on signals whose amplitude waveforms contain feature portions exceeding a predetermined amplitude value among received signals, the amount of data processed by the information processing unit 8 can be efficiently reduced.

[0110] (Modification of the fifth embodiment) If the amplitude waveform of the received signal at one of the adjacent receiving units RU becomes large, even if the amplitude waveform of the received signal at the other adjacent receiving unit RU is small, there is a high probability that the amplitude waveform of the received signal contains a change caused by object B.

[0111] 19, when a characteristic portion is present in the received signal at one receiving unit RU, the first characteristic amount and the second characteristic amount may be used to determine the shape of the object B even when a characteristic portion is not present in the received signal at the other receiving unit RU. With this configuration, the shape of the object B can be determined appropriately.

[0112] Furthermore, as in the fifth embodiment described above, it is desirable that the judgment unit 21 judges the shape of the object B by performing calculations using not only multiple feature amounts but also the ranging distance and sensor information while weighting them with predetermined weights, but this is not necessarily required.

[0113] (Other embodiments) The rest of the configuration is the same as that of the first embodiment. The in-vehicle system 1 of this embodiment can obtain the same effects as those of the first embodiment that are achieved by the configuration common to or equivalent to that of the first embodiment.

[0114] The in-vehicle system 1 of this embodiment also has the following features. Representative embodiments of the present disclosure have been described above, but the present disclosure is not limited to the above-described embodiments and can be modified in various ways, for example, as follows.

[0115] As in the above-described embodiment, it is desirable that the determination unit 21 inputs the measured distance as measurement information into the learning model, but this is not necessarily required.

[0116] As in the above embodiment, the feature extraction unit 74 identifies a characteristic element present in a portion of the amplitude waveform of the received signal that exceeds a predetermined threshold, and extracts the time change of the characteristic element as time feature data, but this is not essential. For example, the feature extraction unit 74 may identify the characteristic element of the received signal by comparing the waveform of the received signal with a waveform pattern prepared in advance.

[0117] As in the above embodiment, it is desirable that the data compression unit 75 and the determination unit 21 are provided in different components, but this is not limiting and they may be provided in the same component. For example, the data compression unit 75 and the determination unit 21 may be provided in the electronic control unit 2. Note that the ultrasonic sensor 3 and the electronic control unit 2 may be connected via, for example, a star network instead of a bus network.

[0118] As in the above-described embodiment, it is desirable that measurement information expressed as a one-hot vector is input to the judgment unit 21, but this is not limited thereto, and for example, measurement information expressed in other formats may be input.

[0119] The learning model of the information processing unit 8 is not limited to the above, and may be configured, for example, as a recurrent neural network (so-called RNN). The learning model may also be configured as a model other than a neural network, but is preferably configured as a non-linear model.

[0120] In the above-described embodiment, various types of measurement information are input to the learning model, but the measurement information is not limited to these. For example, the measurement information may include information about object B detected by an in-vehicle camera, weather information, time information, etc.

[0121] In the above embodiment, an example in which the object detection device of the present disclosure is applied to the in-vehicle system 1 has been described, but the object detection device can also be applied to systems other than the in-vehicle system 1.

[0122] In the above-described embodiments, it goes without saying that the elements constituting the embodiments are not necessarily essential unless they are specifically stated as essential or are clearly considered essential in principle.

[0123] In the above-described embodiments, when numerical values ​​such as the number, values, amounts, ranges, etc. of components of the embodiments are mentioned, they are not limited to the specific numbers unless they are specifically stated as essential or are clearly limited to a specific number in principle.

[0124] In the above-described embodiments, when referring to the shapes, positional relationships, etc. of components, etc., the shapes, positional relationships, etc. are not limited to those unless otherwise specified or when they are fundamentally limited to specific shapes, positional relationships, etc.

[0125] The controller and method of the present disclosure may be implemented on a special-purpose computer by configuring a processor and memory programmed to perform one or more functions embodied in a computer program. The controller and method of the present disclosure may be implemented on a special-purpose computer by configuring a processor with one or more dedicated hardware logic circuits. The controller and method of the present disclosure may be implemented on one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. The computer program may also be stored on a computer-readable non-transitory tangible storage medium as instructions executed by a computer.

[0126] (Aspects of the present disclosure) The present specification discloses the following aspects.

[0127] [First viewpoint] An object detection device that detects an object (B), a receiving unit (40B) that acquires a received signal corresponding to a reflected wave of a transmitted ultrasonic wave by the object; an information processing unit (8) that inputs waveform features of the received signal and other measurement information into a trained learning model that has undergone machine learning to estimate the shape of the object, thereby acquiring object information relating to the shape of the object from the trained learning model; The learning model is a feature extraction unit (74) that extracts, based on the received signal and a predetermined feature pattern of the object, a time change of a feature element included in the received signal as time feature data; a data compression unit (75) that compresses the time feature data to obtain a plurality of the feature amounts; a determination unit (21) that determines the shape of the object by performing calculations while weighting the plurality of feature amounts and the measurement information with predetermined weights, The object detection device, wherein the measurement information is information that has a correlation with a change in the waveform of the received signal or a shape of the object.

[0128] [Second perspective] a distance acquisition unit (72) that acquires the distance to the object based on the received signal, The object detection device according to a first aspect, wherein the measurement information includes the distance acquired by the distance acquisition unit.

[0129] [Third Perspective] The object detection device according to the first or second aspect, wherein the measurement information includes at least one of temperature, humidity, and wind speed in the vicinity of a device in which the receiving unit is mounted.

[0130] [Fourth viewpoint] a plurality of feature quantities obtained by compressing the temporal feature data currently extracted by the feature extraction unit are defined as a plurality of current feature quantities, and a plurality of feature quantities obtained by compressing the temporal feature data extracted in the past are defined as a plurality of past feature quantities, the measurement information includes at least some of the past feature amounts; The object detection device according to any one of the first to third aspects, wherein the determination unit determines the shape of the object by performing calculations on at least some of the plurality of current feature amounts and the plurality of past feature amounts while weighting them with predetermined weights.

[0131] [Fifth viewpoint] The object detection device according to a fourth aspect, wherein the measurement information includes position change information indicating a change in a position at which the reception signal is received.

[0132] [Sixth viewpoint] The object detection device according to the fourth or fifth aspect, wherein the determination unit determines the shape of the object by performing a calculation on the plurality of current feature amounts and the plurality of past feature amounts based on signals whose amplitude waveforms contain feature portions exceeding a predetermined amplitude value while weighting them with a predetermined weight.

[0133] [Seventh viewpoint] The object detection device according to the fourth or fifth aspect, wherein, when it is estimated that the object is in a position that will affect the received signal, the determination unit determines the shape of the object by performing calculations while weighting, with a predetermined weight, one of the plurality of current feature amounts and one of the plurality of past feature amounts that is based on a signal whose amplitude waveform contains a feature portion exceeding a predetermined amplitude value and one that is based on a signal whose amplitude waveform does not contain the feature portion.

[0134] [Eighth viewpoint] a plurality of receiving units, each including the receiving unit and the feature extraction unit, are arranged at different positions; When the plurality of feature quantities obtained by compressing the time feature data based on the reception signals acquired by some of the reception units among the plurality of reception units are defined as the plurality of first feature quantities, and the plurality of feature quantities obtained by compressing the time feature data based on the reception signals acquired by other reception units in the same time period as the acquisition of the reception signals by some of the reception units are defined as the plurality of second feature quantities, the measurement information includes at least some of the second feature amounts; The object detection device according to any one of the first to seventh aspects, wherein the determination unit determines the shape of the object by performing calculations on at least some of the plurality of first features and the plurality of second features while weighting them with predetermined weights.

[0135] [Ninth viewpoint] The object detection device according to an eighth aspect, wherein the measurement information includes sensor information indicating the positions of the plurality of receiving units relative to a transmitting unit (40A) that emits the transmission wave and the positional relationship between the plurality of receiving units.

[0136] [10th viewpoint] The object detection device according to the eighth or ninth aspect, wherein the determination unit determines the shape of the object by performing a calculation on the plurality of first feature amounts and the plurality of second feature amounts based on signals whose amplitude waveforms contain feature portions exceeding a predetermined amplitude value while weighting them with a predetermined weight.

[0137] [11th viewpoint] The object detection device according to the eighth or ninth aspect, wherein when a characteristic portion exceeding a predetermined amplitude value is present in the amplitude waveform of the received signal received by one of the adjacent receiving units, the determination unit determines the shape of the object by performing calculations while weighting the first characteristic amount and the second characteristic amount with a predetermined weight, even if the characteristic portion is not present in the amplitude waveform of the received signal received by the other receiving unit.

[0138] [12th viewpoint] The object detection device according to any one of the first to eleventh aspects, wherein the feature extraction unit identifies the characteristic element present in a part of the amplitude waveform of the received signal that exceeds a predetermined threshold, and extracts the time change of the characteristic element during a predetermined period that includes the characteristic element as the time feature data.

[0139] [13th viewpoint] The object detection device according to any one of the first to twelfth aspects, wherein the data compression unit and the determination unit are provided in different components, and the plurality of features compressed by the data compression unit are transferred to the determination unit via a communication network.

[0140] [14th viewpoint] The object detection device according to any one of the first to thirteenth aspects, wherein the determination unit receives the measurement information expressed as a one-hot vector.

Claims

1. An object detection device for detecting an object (B), a receiving unit (40B) that acquires a received signal corresponding to a reflected wave of an ultrasonic wave transmitted by the object; an information processing unit (8) that inputs waveform features of the received signal and other measurement information into a trained learning model that has undergone machine learning to estimate the shape of the object, thereby acquiring object information relating to the shape of the object from the trained learning model; The learning model is a feature extraction unit (74) that extracts, as time feature data, a time change of a characteristic element included in the received signal based on the received signal and a predetermined characteristic pattern of the object; a data compression unit (75) that compresses the time feature data to obtain a plurality of the feature amounts; a determination unit (21) that determines the shape of the object by performing calculations while weighting the plurality of feature amounts and the measurement information with predetermined weights, the measurement information is information correlating with a change in the waveform of the received signal or a shape of the object, a plurality of receiving units, each including the receiving unit and the feature extraction unit, are arranged at different positions; When a plurality of the feature quantities obtained by compressing the time feature data based on the reception signals acquired by some of the reception units among the plurality of the reception units are defined as a plurality of first feature quantities, and a plurality of the feature quantities obtained by compressing the time feature data based on the reception signals acquired by other reception units in the same time period as the acquisition of the reception signals by some of the reception units are defined as a plurality of second feature quantities, the measurement information includes at least some of the second feature amounts; The object detection device, wherein the determination unit determines the shape of the object by performing calculations while weighting at least some of the plurality of first feature amounts and the plurality of second feature amounts with predetermined weights.

2. a distance acquisition unit (72) that acquires a distance to the object based on the received signal; The object detection device according to claim 1 , wherein the measurement information includes the distance acquired by the distance acquisition unit.

3. The object detection device according to claim 1 , wherein the measurement information includes at least one of temperature, humidity, and wind speed around the device in which the receiving unit is mounted.

4. a plurality of feature quantities obtained by compressing the temporal feature data currently extracted by the feature extraction unit are defined as a plurality of current feature quantities, and a plurality of feature quantities obtained by compressing the temporal feature data extracted in the past are defined as a plurality of past feature quantities, the measurement information includes at least some of the past feature amounts; The object detection device according to claim 1 , wherein the determination unit determines the shape of the object by performing calculations on at least some of the plurality of current feature amounts and the plurality of past feature amounts while weighting them with predetermined weights.

5. The object detection device according to claim 4 , wherein the measurement information includes position change information indicating a change in a position at which the reception signal is received.

6. 5. The object detection device according to claim 4, wherein the determination unit determines the shape of the object by performing a calculation while weighting, with a predetermined weight, those of the plurality of current feature amounts and the plurality of past feature amounts that are based on signals whose amplitude waveforms include feature portions that exceed a predetermined amplitude value.

7. 5. The object detection device according to claim 4, wherein, when it is estimated that the object is in a position that will affect the received signal, the determination unit determines the shape of the object by performing calculations on the plurality of current feature amounts and the plurality of past feature amounts while weighting, with a predetermined weight, those based on signals whose amplitude waveforms contain feature portions exceeding a predetermined amplitude value and those based on signals whose amplitude waveforms do not contain the feature portions.

8. 2. The object detection device according to claim 1, wherein the measurement information includes sensor information indicating the positions of the plurality of receiving units relative to a transmitting unit (40A) that emits the transmission wave and the positional relationship between the plurality of receiving units.

9. 2. The object detection device according to claim 1, wherein the determination unit determines the shape of the object by performing a calculation while weighting, with a predetermined weight, those of the plurality of first feature amounts and the plurality of second feature amounts that are based on signals whose amplitude waveforms contain feature portions that exceed a predetermined amplitude value.

10. 2. The object detection device according to claim 1, wherein, when a characteristic portion exceeding a predetermined amplitude value is present in the amplitude waveform of the received signal received by one of the adjacent receiving units, the determination unit determines the shape of the object by performing calculations while weighting the first characteristic amount and the second characteristic amount with a predetermined weight, even if the characteristic portion is not present in the amplitude waveform of the received signal received by the other receiving unit.

11. 10. The object detection device according to claim 1, wherein the feature extraction unit identifies the characteristic element present in a portion of the amplitude waveform of the received signal that exceeds a predetermined threshold, and extracts a time change of the characteristic element during a predetermined period that includes the characteristic element as the time feature data.

12. 11. The object detection device according to claim 1, wherein the data compression unit and the determination unit are provided in different components, and the plurality of features compressed by the data compression unit are transferred to the determination unit via a communication network.

13. The object detection device according to claim 1 , wherein the determination unit receives the measurement information expressed as a one-hot vector.

Images