Object detection device and vehicle control system including the object detection device

Abstract

An object detection device includes a first sensor, a second sensor, a calculation range selector, and an estimator. The first sensor outputs a radio frequency (RF) signal, receives a reflected RF signal reflected from an object, and obtains a first measurement value for the object based on the received reflected RF signal. The second sensor obtains a second measurement value for the object by sensing a physical feature from the object. The physical feature sensed by the second sensor is different from the feature of the object measured as the first measurement value obtained by the first sensor. The calculation range selector sets a first reference range based on the second measurement value. The first reference range represents an execution range of a first calculation for detecting a position of an object using a first algorithm. The estimator performs a first calculation only on the first reference range using the first measurement value, and generates a first result value as a result of performing the first calculation. The first result value represents the position of the object.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the priority benefit of Korean Patent Application No. 10-2020-0007258 filed on January 20, 2020 in the Korean Intellectual Property Office (KIPO), the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Example embodiments of the present disclosure generally relate to semiconductor integrated circuits. More particularly, example embodiments of the present disclosure relate to an object detection device that detects an object around a vehicle and a vehicle control system including the object detection device. Background Art

[0004] Recently, cars have rapidly become more and more intelligent due to the convergence of information communication technology and the automotive industry. As a result, cars have evolved from simple mechanisms that are completely controlled by the driver to smart cars. Smart cars can be controlled by the driver with the assistance of advanced driver assistance systems (ADAS), and sometimes can be completely controlled by autonomous driving systems. ADAS and autonomous driving systems are particularly prominent as core technologies of smart cars.

[0005] For ADAS and autonomous driving systems, various technologies are required, such as technology for recognizing the driving environment (e.g., lanes, surrounding vehicles, pedestrians, etc.); technology for determining driving conditions; control technology such as steering, acceleration / deceleration, etc. In addition, for these various technologies, it is necessary to accurately and effectively recognize and / or detect objects around the vehicle. Summary of the invention

[0006] At least one example embodiment of the present disclosure provides an object detection device capable of accurately, quickly, and efficiently identifying and / or detecting objects around a vehicle.

[0007] At least one example embodiment of the present disclosure provides a vehicle control system including an object detection device.

[0008] According to an example embodiment, an object detection device includes a first sensor, a second sensor, a calculation range selector, and an estimator. The first sensor outputs a radio frequency (RF) signal, receives a reflected RF signal reflected from an object, and obtains a first measurement value for the object based on the received RF signal. The second sensor obtains a second measurement value for the object by sensing a physical feature from the object. The physical feature sensed by the second sensor is different from the feature of the object measured as the first measurement value obtained by the first sensor. The calculation range selector sets a first reference range based on the second measurement value. The first reference range represents an execution range of a first calculation for detecting a position of an object using a first algorithm. The estimator performs a first calculation only on the first reference range using the first measurement value, and generates a first result value as a result of performing the first calculation. The first result value represents the position of the object.

[0009] According to an example embodiment, a vehicle control system includes an object detection device and a controller. The object detection device detects an object around the vehicle and generates a first result value indicating the position of the object. The controller controls the vehicle based on the first result value. The object detection device includes a first sensor, a second sensor, a calculation range selector, and an estimator. The first sensor outputs a radio frequency (RF) signal, receives a reflected RF signal reflected from an object, and obtains a first measurement value for the object based on the received reflected RF signal. The second sensor obtains a second measurement value for the object by sensing a physical feature from the object. The physical feature sensed by the second sensor is different from the feature of the object measured as the first measurement value obtained by the first sensor. The calculation range selector sets a first reference range based on the second measurement value. The first reference range represents an execution range of a first calculation for detecting the position of an object using a first algorithm. The estimator performs a first calculation only on the first reference range using the first measurement value, and generates a first result value as a result of performing the first calculation.

[0010] According to an exemplary embodiment, an object detection device includes a first sensor, a second sensor, a third sensor, a calculation range selector, and an estimator. The first sensor outputs a radio frequency (RF) signal, receives a reflected RF signal reflected from an object, divides the received reflected RF signal into an in-phase component and an orthogonal phase component, calculates the phase of the received RF signal based on the in-phase component and the orthogonal phase component, and generates a phase vector based on the calculated phase. The second sensor obtains image information for the object based on reflected light reflected from the object. The third sensor outputs a laser pulse, receives a reflected laser pulse reflected from the object, and obtains measurement information for the object based on the received reflected laser pulse. The calculation range selector sets a first reference range based on at least one of the image information obtained by the second sensor and the measurement information obtained by the third sensor. The first reference range is a portion of the entire execution range that can be used to select a multiple signal classification (MUSIC) algorithm. The estimator performs calculation based on the MUSIC algorithm only for the first reference range using the phase vector, and estimates the azimuth of the object representing the position of the object. The estimator selects a steering vector corresponding to a first reference range from an array manifold (AM) vector including a plurality of predicted steering vectors, calculates a covariance matrix based on the selected steering vector and a phase vector, obtains MUSIC spectrum values ​​by normalizing the covariance matrix, calculating determinants of the normalized covariance matrix, and reciprocalizing the determinants, extracts peaks from the MUSIC spectrum values, and estimates the azimuth of the object based on the peaks of the vectors in the spatial spectrum.

[0011] In an object detection device and a vehicle control system according to an example embodiment, a first calculation based on a first algorithm for detecting the position of an object can be performed using a first measurement value obtained from a first sensor. In addition, a portion of the entire execution range that can be used to select the first calculation can be set as a reference range using a second measurement value obtained from a second sensor, and the first calculation can be performed only on a reference range corresponding to a portion of the entire execution range that can be used to select the first calculation. The second sensor can be a sensor of a different type from the first sensor and can sense a physical feature that is different from a feature of an object measured as a first measurement value obtained by the first sensor. Therefore, the amount of calculation can be reduced, objects can be detected quickly and efficiently, and the performance and accuracy of detecting objects can be maintained without degradation. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Illustrative, non-limiting example embodiments will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings.

[0013] Figure 1 is a block diagram illustrating an object detection apparatus according to example embodiments.

[0014] Figure 2 and Figure 3 is a diagram for describing a configuration and operation of a first sensor included in an object detection apparatus according to an example embodiment.

[0015] Figure 4A and Figure 4B is a diagram for describing an operation of an estimator included in an object detection apparatus according to an example embodiment.

[0016] Figure 5 is a flowchart illustrating an object detection method according to example embodiments.

[0017] Figure 6 and Figure 7 It is shown Figure 5 A flowchart of an example of setting a first reference range in FIG.

[0018] Figure 8 is shown only for Figure 5 Flowchart of an example of performing a first calculation within a first reference range in FIG.

[0019] Fig. 9A and Fig. 9B Is used to describe Figure 8 Graph of the operation.

[0020] Fig.10 is a block diagram illustrating an object detection apparatus according to example embodiments.

[0021] Fig.11 is a flowchart illustrating an object detection method according to example embodiments.

[0022] Fig.12 and Fig.13 It is shown Fig.11 A flowchart of an example of setting a first reference range in FIG.

[0023] Fig.14 is a block diagram illustrating an object detection apparatus according to example embodiments.

[0024] Fig.15 is a flowchart illustrating an object detection method according to example embodiments.

[0025] Fig.16 is a block diagram illustrating a vehicle control system according to an example embodiment.

[0026] Fig.17 is a block diagram illustrating an electronic device including a vehicle control system according to an example embodiment.

[0027] Fig.18 is a diagram illustrating an example in which an electronic device according to an example embodiment is installed in a vehicle. DETAILED DESCRIPTION

[0028] Various example embodiments will be described more fully with reference to the accompanying drawings, in which embodiments are shown. However, the present disclosure can be implemented in many different forms and should not be construed as being limited to the embodiments set forth herein. In this application, the same reference numerals represent the same elements.

[0029] Figure 1 is a block diagram illustrating an object detection apparatus according to example embodiments.

[0030] Reference Figure 1 , the object detection apparatus 100 includes a first sensor 110 , a second sensor 120 , a calculation range selector 130 , and an estimator 140 .

[0031] The first sensor 110 obtains or acquires a first measurement value MV1 for the object 10. The second sensor 120 obtains or acquires a second measurement value MV2 for the object 10. The first sensor 110 and the second sensor 120 may be different types of sensors. For example, the first sensor 110 and the second sensor 120 may respectively obtain the first measurement value MV1 and the second measurement value MV2 by sensing different physical features (or attributes) from the object 10. In other words, the physical feature sensed by the second sensor 120 may be different from the feature of the object 10 measured by the first measurement value obtained by the first sensor 110 or otherwise reflected in the first measurement value. That is, the physical feature sensed by the second sensor 120 is not a physical feature or attribute sensed by the first sensor 110. The physical feature sensed by the second sensor 120 is obtained by the second sensor 120 based on a basis that is independent of and different from the physical feature or attribute of the object 10 obtained by the first sensor 110.

[0032] In some example embodiments, the first sensor 110 may be a radio detection and ranging (RADAR) sensor. For example, the first sensor 110 may output or transmit a radio frequency (RF) signal, and may receive a reflected RF signal reflected from the object 10. The first sensor 110 may obtain a first measurement value MV1 for the object 10 based on the received reflected RF signal. The detailed configuration of the first sensor 110 and the first measurement value MV1 obtained from (or by) the first sensor 110 will be referred to in detail. Figure 2 and Figure 3 describe.

[0033] In some example embodiments, the second sensor 120 may be an image sensor. For example, the second sensor 120 may include a charge coupled device (CCD) image sensor, a complementary metal oxide semiconductor (CMOS) image sensor, etc. The second sensor 120 may obtain image information for the object 10 based on the reflected light reflected from the object 10. The image information obtained by or from the second sensor 120 may correspond to the second measurement value MV2 for the object 10. When the second sensor 120 is an image sensor, the physical feature measured as the second measurement value MV2 may be a feature from the image information, such as size, shape, reflectivity, color, or other features of the object 10 that may be represented as image information.

[0034] In other example embodiments, the second sensor 120 may be a light detection and ranging (LIDAR) sensor. For example, the second sensor 120 may output or transmit a laser pulse, and may receive a reflected laser pulse reflected from the object 10. The second sensor 120 may obtain a second measurement value MV2 for the object 10 based on the received reflected laser pulse.

[0035] In other example embodiments, the second sensor 120 may include both an image sensor and a LIDAR sensor. Fig.10 and Fig.14 As described, the object detection apparatus 100 according to example embodiments may further include two or more sensors of different types in addition to the first sensor 110 .

[0036] The calculation range selector 130 sets a reference range RRNG based on the second measurement value MV2. The reference range RRNG represents an execution range of a first calculation (or arithmetic operation) for detecting a position (or location) of the object 10 using the first algorithm. Figure 6 and Figure 7 , a detailed operation of setting the reference range RRNG by the calculation range selector 130 will be described. An example of the reference range RRNG is a subset of the steering vectors SV1, ..., SVn of the array manifold vector AMV including a plurality of steering vectors SV1, ..., SVn. The steering vectors SV1, ..., SVn and the array manifold vector AMV are described below with respect to, for example Figure 4A and Figure 4B Description. An execution scope may refer to a range of degrees in a spatial spectrum, and may be less than all degrees in the spatial spectrum. Thus, an execution scope as described herein may refer to a subset of information that undergoes computation within a larger set of information that may undergo computation. An execution scope may refer to vectors corresponding to a finite subset of vectors within an arc of 360 degrees in a circle.

[0037] The estimator 140 performs a first calculation (e.g., a calculation based on the first algorithm) based on the first algorithm only for the reference range RRNG using the first measurement value MV1, and generates a result value RV as a result of performing the first calculation. For example, the result value RV may include a first result value indicating the position of the object 10. The result value RV may also include a second result value indicating the distance to the object 10 and the speed of the object 10. Figure 8 Detailed operations of performing the first calculation and generating the result value RV by the estimator 140 are described.

[0038] In some example embodiments, the first algorithm may be a multiple signal classification (MUSIC) algorithm. The MUSIC algorithm is a representative algorithm for estimating the direction of incidence (i.e., the direction of arrival (DOA) of the signal). In the MUSIC algorithm, the spatial spectrum can be obtained by using the property that the signal subspace and the noise subspace are orthogonal to each other. The DOA is associated with and estimated by the peak of the vector in the spatial spectrum. The two subspaces can be obtained by eigendecomposition of the covariance matrix. When the first algorithm is the MUSIC algorithm, the first result value may represent the azimuth (or azimuth angle) of the object 10.

[0039] In some example embodiments, at least a portion of the calculation range selector 130 and / or the estimator 140 may be implemented as hardware. For example, at least a portion of the calculation range selector 130 and / or the estimator 140 may be included in a computer-based electronic system. In other example embodiments, at least a portion of the calculation range selector 130 and / or the estimator 140 may be implemented as an instruction code or program routine (e.g., a software program). For example, the instruction code or program routine may be executed by a computer-based electronic system and may be stored in any storage device located inside or outside the computer-based electronic system.

[0040] In the object detection device 100 according to the example embodiment, a first calculation based on a first algorithm for detecting the position of the object 10 may be performed using a first measurement value MV1 obtained from the first sensor 110. In addition, a second measurement value MV2 obtained from the second sensor 120 may be used to set a portion of the entire execution range available for selection of the first calculation as a reference range RRNG. The entire execution range available for selection of the first calculation may refer to a potential range that may be subject to the calculation, and the reference range RRNG may be a subset of the potential range, such as a limited subset of vectors within an arc of 360 degrees in a circle. Therefore, the reference range RRNG may include information such as a limited subset of vectors that are actually calculated outside the entire execution range potentially available for calculation, and this is useful, for example, in the following Figure 4A and Figure 4BThe first calculation may be performed only on a reference range RRNG corresponding to a portion of the entire execution range that can be used to select the first calculation. The second sensor 120 may be a sensor of a different type from the first sensor 110, and may sense physical features of the object 10 that are different from the features of the first measurement value MV1 obtained by the first sensor 110 or otherwise reflected in the first measurement value MV1. Therefore, the amount of calculation (or operation) can be reduced, the object 10 can be detected quickly and efficiently, and the performance and accuracy of detecting the object 10 can be maintained without degradation.

[0041] Hereinafter, the operation of the object detection device according to the example embodiment will be described based on the MUSIC algorithm. However, the example embodiment is not limited thereto. For example, the object detection device according to the example embodiment may estimate the azimuth of the object 10 based on the Bartlett beamforming algorithm, the Capon algorithm, etc. Alternatively, the object detection device according to the example embodiment may estimate the position of the object 10 based on at least one of various algorithms for estimating various parameters associated with or related to the position of the object 10.

[0042] Figure 2 and Figure 3 is a diagram for describing a configuration and operation of a first sensor included in an object detection apparatus according to example embodiments.

[0043] Reference Figure 2 The first sensor 110 may be a RADAR sensor and may include a transmitting antenna 112 , a plurality of receiving antennas 114 a and 114 b , and a signal processor 116 .

[0044] The transmitting antenna 112 may output or transmit an RF signal TX_RF. For example, the frequency of the RF signal TX_RF may be implemented in various ways, such as about 24 GHz or 77 GHz. For example, when the frequency of the RF signal TX_RF is about 77 GHz, the first sensor 110 may have a relatively high performance, but may be relatively expensive and have a relatively complex configuration. When the frequency of the RF signal TX_RF is about 24 GHz, the first sensor 110 may be relatively cheap, but may have a relatively low performance. The RF signal TX_RF output from the transmitting antenna 112 may reach the object 10, and may then be reflected by the object 10.

[0045] The plurality of receiving antennas 114a and 114b may receive the reflected RF signal reflected from the object 10. Since the positions of the receiving antennas 114a and 114b are different from each other and the distances between the object 10 and the receiving antennas 114a and 114b are different from each other, the received RF signals RX_RF1 and RX_RF2 obtained by the receiving antennas 114a and 114b may have a phase difference PDIFF. Figure 2 Only two receiving antennas 114a and 114b are shown in FIG. 1 , but example embodiments are not limited thereto. The first sensor 110 may include m receiving antennas, where m is a natural number greater than or equal to 2.

[0046] The signal processor 116 may divide each of the received RF signals RX_RF1 and RX_RF2 obtained from the multiple receiving antennas 114a and 114b into an in-phase component and an orthogonal phase component. The signal processor 116 may calculate the phase of each of the received RF signals RX_RF1 and RX_RF2 based on the in-phase component and the orthogonal phase component. The signal processor 116 may generate a first measurement value MV1 based on the calculated phase.

[0047] Reference Figure 3 When the first sensor 110 includes m receiving antennas, the first measurement value MV1 obtained from the first sensor 110 may correspond to a phase vector PV, which includes m components pd 1 、pd 2 、...、pd m , and is represented by an m*1 matrix.

[0048] In some example embodiments, the first receiving antenna (eg, the frontmost receiving antenna) among the m receiving antennas included in the first sensor 110 may be set as a reference receiving antenna. In this case, the m components pd included in the phase vector PV 1 、pd 2 、...、pd m In the example, the kth component may represent the phase difference between the RF signal received by the first receiving antenna and the RF signal received by the kth receiving antenna among the m receiving antennas, where k is a natural number greater than or equal to 1 and less than or equal to m. For example, the m components pd 1 、pd 2 、...、pd m The first component pd 1 It can be said that the phase difference is always zero as long as it is calculated as the difference of the same RF signal, for example, the difference between the RF signal received by the first receiving antenna and the RF signal received by the second receiving antenna. 1 、pd 2 、...、pdm The second component pd 2 It can represent the phase difference between the RF signal received by the first receiving antenna and the RF signal received by the second receiving antenna. 1 、pd 2 、...、pd m The mth component pd m The phase difference between the RF signal received by the first receiving antenna and the RF signal received by the mth receiving antenna may be represented.

[0049] Figure 1 The estimator 140 in can be used Figure 3 The phase vector PV shown in is used to generate a first result value representing the position of the object 10 (eg, the azimuth angle of the object 10 ).

[0050] Despite Figure 2 and Figure 3 , but when the second sensor 120 is a LIDAR sensor, the LIDAR sensor may have Figure 2 The configuration of the RADAR sensor shown in FIG. 1 is similar to that of the LIDAR sensor shown in FIG. For example, the LIDAR sensor may include a transmitting antenna, a plurality of receiving antennas, and a signal processor. The transmitting antenna may output or transmit a laser pulse. The plurality of receiving antennas may receive a reflected laser pulse reflected from the object 10 based on the transmitted laser pulse. The signal processor may obtain a measurement value based on the received reflected laser pulse.

[0051] Figure 4A and Figure 4B is a diagram for describing an operation of an estimator included in an object detection apparatus according to an example embodiment.

[0052] Reference Figure 4A and Figure 4B , the estimator 140 included in the object detection device 100 may include an array manifold (AM) vector AMV, and the AMV includes a plurality of steering vectors SV1, ..., SVn. The plurality of steering vectors SV1, ..., SVn and the AM vector AMV may be pre-measured (or previously measured) using the first sensor 110 and may be stored in the estimator 140.

[0053] For example, Figure 4AAs shown, the object detection device 100 can be set in a default environment such as a blank space where there is no specific object. A virtual circle can be set relative to the object detection device 100. Multiple measurement positions (or locations) P1, P2, P3, P4, P5, P6, P7, P8, P9, P10, P11, P12, P13, P14, P15 and P16 can be set at regular intervals along the circumference of the virtual circle. The RF signal sent by the multiple measurement positions P1 to P16 or otherwise output from the multiple measurement positions P1 to P16, or reflected by the multiple measurement positions P1 to P16 can be received by the object detection device 100. Therefore, multiple steering vectors SV1, ..., SVn can be obtained based on the received RF signal. A steering vector can be obtained based on an RF signal, and therefore the number of multiple measurement positions P1 to P16 can be substantially equal to the number of multiple steering vectors SV1, ..., SVn. A plurality of steering vectors SV1, ..., SVn and AM vector AMV may be measured, obtained and stored in advance as comparison targets and / or default values ​​for performing object detection and estimation operations. For example, a plurality of steering vectors SV1, ..., SVn and AM vector AMV may be pre-set, such as in a calibration mode of the object detection device 100.

[0054] Since the plurality of measurement positions P1 to P16 are arranged at regular intervals along the circumference of the virtual circle, all angles between two adjacent measurement positions can be substantially the same as each other. Figure 4A When the arrangement is shown, all angles between two adjacent measurement positions (e.g., the angle between two measurement positions P1 and P2) may be about 22.5 degrees. However, example embodiments are not limited thereto, and the number of measurement positions and the angle between two adjacent measurement positions may vary according to example embodiments. As the number of measurement positions (e.g., the number of steering vectors) increases, the accuracy of the estimation operation may increase, however, the amount of calculation may increase, thereby increasing the time required for the estimation operation.

[0055] When the first sensor 110 includes m receiving antennas, each of the plurality of steering vectors SV1, ..., SVn may be represented by an m×m matrix, such as Figure 4B For example, the steering vector SV1 obtained at the first measurement position can be composed of components s1 11 s1 12 , ..., s1 1m s1 21 s1 22 , ..., s1 2m , ..., s1 m1 s1 m2 , ..., s1 mmThe steering vector SVn obtained at the last measured position can be represented by the components sn 11 、sn 12 、...、Sn 1m 、sn 21 、sn 22 、...、Sn 2m 、...、Sn m1 、sn m2 、...、Sn mm The m×m matrix representation of .

[0056] In some example embodiments, the estimator 140 may include a memory that stores an AM vector AMV including a plurality of steering vectors SV1, ..., SVn. For example, the memory may include any non-volatile memory, such as an electrically erasable programmable read-only memory (EEPROM), a flash memory, a phase random access memory (PRAM), a resistive random access memory (RRAM), a nano-floating gate memory (NFGM), a polymer random access memory (PoRAM), a magnetic random access memory (MRAM), a ferroelectric random access memory (FRAM), a thyristor random access memory (TRAM), etc., and / or any volatile memory such as a dynamic random access memory (DRAM), a static random access memory (SRAM), etc.

[0057] Figure 1 The estimator 140 in can be used Figure 4B , SVn shown in is used to generate a first result value representing the position of the object 10 (eg, the azimuth angle of the object 10 ).

[0058] Figure 5 is a flowchart illustrating an object detection method according to example embodiments.

[0059] Reference Figure 1 and Figure 5 In the object detection method according to the example embodiment, a first measurement value MV1 for the object 10 is obtained using the first sensor 110 (step S100). For example, the first sensor 110 may be a RADAR sensor, which may receive a reflected RF signal reflected from the object 10, and may obtain a phase vector corresponding to the first measurement value MV1 based on the received RF signal (e.g., Figure 3 The phase vector PV).

[0060] A second measurement value MV2 for the object 10 is obtained using the second sensor 120 (step S200). The second sensor 120 may be a sensor of a different type from the first sensor 110, and may sense a physical feature of the object 10 that is different from a feature of the object 10 measured by or reflected in the first measurement value MV1 obtained by the first sensor 110. For example, the second sensor 120 may be an image sensor, and may obtain image information corresponding to the second measurement value MV2 based on reflected light reflected from the object 10.

[0061] A first reference range is set based on the second measurement value MV2 (step S300). The first reference range represents an actual execution range of the first calculation based on the first algorithm, and the actual execution range may be a subset of the entire execution range potentially available for the first calculation. Step S300 may be performed by the calculation range selector 130.

[0062] A first calculation based on a first algorithm is performed only on a first reference range using the first measurement value MV1 (step S400), and a first result value is generated as a result of performing the first calculation (step S500). The first result value represents the position of the object 10, for example, the first algorithm may be a MUSIC algorithm, and the first result value may be an azimuth of the object 10. Steps S400 and S500 may be performed by the estimator 140.

[0063] In some example embodiments, when the first result value is generated in step S500, steps S100, S200, S300 and S400 may be repeatedly performed. A plurality of result values ​​obtained by repeating steps S100, S200, S300 and S400 may be statistically processed, and then the first result value may be finally determined.

[0064] Figure 6 and Figure 7 It is shown Figure 5 A flowchart of an example of setting a first reference range in FIG.

[0065] Reference Figure 1 , Figure 5 and Figure 6 When setting the first reference range (step S300), the first range may be set based on the second measurement value MV2 obtained from the second sensor 120 (step S310). The first reference range matching the first range may be selected (step S320).

[0066] For example, when the reliability of the first range is higher than the reference value, such as when the reliability of the second sensor 120 is relatively high, the first reference range that is the same as the first range may be selected. As another example, when a check operation is required on the first range, such as when a review of the first range is required, the first reference range that is the same as the first range may be selected. Figure 5 The first calculation in step S400 may be performed only within the range determined by the second sensor 120 .

[0067] Reference Figure 1 , Figure 5 and Figure 7 , when setting the first reference range (step S300), the first range may be set based on the second measurement value MV2 obtained from the second sensor 120 (step S310). A first reference range that matches a second range other than the first range in the entire range available for selection may be selected (step S330).

[0068] For example, when the reliability of the first range is lower than the reference value, such as when the reliability of the second sensor 120 is relatively low, the first reference range that is the same as the second range other than the first range may be selected. As another example, when a check operation is required for the second range other than the first range, such as when a blind spot other than the first range needs to be checked, the first reference range that is the same as the second range other than the first range may be selected. Therefore, Figure 5 The first calculation in step S400 may be performed only outside the range determined by the second sensor 120. For example, Figure 5 The first calculation in step S400 may be performed only outside the coverage range of the second sensor 120 .

[0069] Although the reference Figure 6 and Figure 7 The operation and specific scenario for selecting the first reference range are described, but example embodiments are not limited thereto. The operation and specific scenario for selecting the first reference range may vary according to example embodiments.

[0070] Figure 8 is shown only for Figure 5 Flowchart of an example of performing a first calculation within a first reference range in FIG. Fig. 9A and Fig. 9B Is used to describe Figure 8 Graph of the operation.

[0071] Reference Figure 1 , Figure 5 , Figure 8 , Fig. 9A and Fig. 9B , step S400 may be performed based on the MUSIC algorithm.

[0072] As described above, the MUSIC algorithm may be an algorithm for estimating the DOA of a signal by using the property that the signal subspace and the noise subspace are orthogonal to each other, and the two subspaces may be obtained by eigendecomposition of the covariance matrix. For example, when the first sensor 110 includes m receiving antennas, and when P signals are incident on the m receiving antennas, where P is a natural number greater than or equal to 2, the received signal may be defined by Equation 1.

[0073] [Equation 1]

[0074]

[0075] In Equation 1, “A” represents the steering vector a For example, “A” may have the form of Equation 2.

[0076] [Equation 2]

[0077] A=[a(φ 1 )a(φ 2 )···a(φ P )]

[0078] In Equation 2, “a(φ)” can be defined as a(φ)=e -j2πdsinφ / λ , where "d" represents the distance between the corresponding receiving antennas. The covariance matrix used to obtain the signal subspace and the noise subspace can be obtained based on Equation 3.

[0079] [Equation 3]

[0080] R=E[XX H ]=AR s A H +δ 2 I

[0081] Equation 3 represents the covariance matrix of the signal. In the MUSIC algorithm, the direction of the signal can be estimated by the noise subspace of the signal. The noise subspace of the signal E N It can be obtained by eigendecomposition of “R”. For example, the DOA of the signal can be obtained in the MUSIC algorithm based on Equation 4.

[0082] [Equation 4]

[0083]

[0084] In the MUSIC algorithm, "φ" satisfying Equation 4 can be estimated as the DOA of the signal. The spatial spectrum can be obtained based on Equation 5.

[0085] [Equation 5]

[0086]

[0087] As can be seen from Equation 5, the spatial spectrum may have a peak at a vector "φ" satisfying Equation 4 as much as possible. Therefore, the vector "φ" having a peak in the spatial spectrum may be estimated as the DOA of the signal.

[0088] When the first calculation is performed only on the first reference range using the first measurement value (step S400), a steering vector corresponding to the first reference range may be selected from the AM vectors including the plurality of steering vectors of the predicted amount (step S410). A covariance matrix may be calculated based on the selected steering vector and the phase vector corresponding to the first measurement value (step S420).

[0089] For example, Fig. 9A As shown, among all the steering vectors of the plurality of steering vectors SV1, SV2, ..., SV(k-1), SVk, SV(k+1), ..., SVn measured and stored in advance, the steering vectors SV1, SV2, ..., SV(k-1), SVk included in the first reference range set in step S300 can be selected. The covariance matrix COV1 can be obtained by multiplying the selected steering vector SV1 and the phase vector PV. The covariance matrix COV2 can be obtained by multiplying the selected steering vector SV2 and the phase vector PV. The covariance matrix COV(k-1) can be obtained by multiplying the selected steering vector SV(k-1) and the phase vector PV. The covariance matrix COVk can be obtained by multiplying the selected steering vector SVk and the phase vector PV. As shown in FIG. Figure 3 and Figure 4B As described above, when the phase vector PV is represented by an m×1 matrix and each of the steering vectors SV1 to SVn is represented by an m×m matrix, each of the covariance matrices COV1, COV2, ..., COV(k-1), COVk can be represented by an m×1 matrix.

[0090] When the covariance matrices COV1, COV2, ..., COV(k-1), COVk are obtained by multiplying the selected steering vectors SV1, SV2, ..., SV(k-1), SVk and the phase vector PV according to an exemplary embodiment, the amount of calculation can be reduced and the calculation speed can be increased compared to the conventional technical case 2 in which the covariance matrices COV1, COV2, ..., COV(k-1), COVk, COV(k+1), ..., COVn are obtained by multiplying all of the multiple steering vectors SV1, SV2, ..., SV(k-1), SVk, SV(k+1), ..., SVn and the phase vector PV.

[0091] In other words, when calculations based on the MUSIC algorithm are performed only on a partial range (or a portion of the entire range available for selection) according to an example embodiment, the speed of the calculations can be increased by narrowing the range of the calculations. For example, when information that an object is located at approximately 10 degrees, 20 degrees, and 70 degrees is obtained from the second sensor 120, the calculations can be performed using only the steering vector corresponding to approximately 0 degrees to 80 degrees. ADAS and autonomous driving systems may encounter many objects at any time, so that calculations based on the MUSIC algorithm can be performed in parallel for multiple different objects detected by ADAS and autonomous driving systems. As described herein, calculations for object detection can be reduced by narrowing the calculation range for one or more objects, thereby increasing the calculation speed.

[0092] Afterwards, the covariance matrix may be normalized (step S430). The MUSIC algorithm may be an algorithm that considers only the phase difference between the received RF signals. Therefore, the covariance matrices corresponding to the received RF signals may be normalized to have the same size.

[0093] Thereafter, the determinant of the normalized covariance matrix may be calculated (step S440). The determinant may be inverted. Thus, the MUSIC spectrum value may be obtained (step S450). By inverting or negating the determinant in step S450, the items with high correlation may be reduced, and a peak may be formed within the MUSIC spectrum value.

[0094] Thereafter, a peak is extracted from the MUSIC spectrum value (step S460). An azimuth angle of the object corresponding to the position of the object may be estimated based on the peak (step S470).

[0095] For example, you can Fig. 9B The MUSIC spectrum values ​​are derived as shown. Fig. 9B The MUSIC spectrum value may include peaks PK1, PK2, and PK3 and azimuths ANG1, ANG2, and ANG3 corresponding thereto. Among the azimuths ANG1, ANG2, and ANG3 corresponding to the peaks PK1, PK2, and PK3, the azimuth included in the first reference range may be estimated as the azimuth of the object.

[0096] In other words, according to an example embodiment, when calculations based on the MUSIC algorithm are performed only on a partial range, relatively accurate and reliable detection can be performed, and even if there are a large number of peaks in the generated spectrum, the ambiguity of the detection can be resolved. For example, when information that an object is located at approximately 10 degrees, 20 degrees, and 70 degrees is obtained from the second sensor 120, the MUSIC spectrum can be obtained by performing calculations using only the steering vector corresponding to approximately 0 degrees to 80 degrees. In addition, when the azimuths corresponding to the peaks in the MUSIC spectrum are approximately 18 degrees and approximately 43 degrees, depending on r, the azimuth of the object can be estimated as one of two peaks such as approximately 18 degrees or approximately 43 degrees.

[0097] As a result, when calculation based on the MUSIC algorithm is performed only on a partial range according to an example embodiment, the disadvantage of the MUSIC algorithm of obtaining several azimuth angles when the used bandwidth is relatively narrow or the number of receiving antennas is relatively small can be effectively overcome.

[0098] Fig.10 is a block diagram showing an object detection device according to an example embodiment. Figure 1 Duplicate description.

[0099] Reference Fig.10 , the object detection apparatus 100 a includes a first sensor 110 , a second sensor 120 , a third sensor 150 , a calculation range selector 130 a , and an estimator 140 .

[0100] In addition to the object detection apparatus 100a further including the third sensor 150 and partially changing the operation of the calculation range selector 130a, Fig.10 The object detection device 100a can be used with Figure 1 The object detection apparatus 100 is substantially the same.

[0101] The third sensor 150 may obtain a third measurement value MV3 for the object 10. The first sensor 110, the second sensor 120, and the third sensor 150 may be different types of sensors. For example, the first sensor 110, the second sensor 120, and the third sensor 150 may respectively obtain the first measurement value MV1, the second measurement value MV2, and the third measurement value MV3 by sensing different physical features from the object 10.

[0102] In some example embodiments, the first sensor 110 may be a RADAR sensor, and one of the second sensor 120 and the third sensor 150 may be an image sensor. The other of the second sensor 120 and the third sensor 150 may be a LIDAR sensor.

[0103] The calculation range selector 130a sets a reference range RRNG based on the second measurement value MV2 and the third measurement value MV3. The reference range RRNG represents an execution range of the first calculation for detecting the position of the object 10 using the first algorithm. Fig.12 and Fig.13 A detailed operation of setting the reference range RRNG by the calculation range selector 130a is described.

[0104] Fig.11 is a flowchart illustrating an object detection method according to an example embodiment. Figure 5 Duplicate description.

[0105] Reference Fig.10 and Fig.11 , in the object detection method according to the example embodiment, Fig.11 Steps S100, S400 and S500 in Figure 5 Steps S100, S400 and S500 in the embodiment are substantially the same.

[0106] A second measurement value MV2 and a third measurement value MV3 for the object 10 are obtained using the second sensor 120 and the third sensor 150 (step S200a). Each of the second sensor 120 and the third sensor 150 may be a sensor of a different type from the first sensor 110. Each of the second sensor 120 and the third sensor 150 may sense a physical feature of the object 10 that is different from a feature of the object 10 measured by the first measurement value MV1 obtained by the first sensor 110 or otherwise reflected in the first measurement value MV1. A first reference range is set based on the second measurement value MV2 and the third measurement value MV3 (step S300a). The first reference range represents an execution range of a first calculation based on the first algorithm. Step S300a may be performed by the calculation range selector 130a.

[0107] Fig.12 and Fig.13 It is shown Fig.11 A flowchart of an example of setting a first reference range in FIG.

[0108] Reference Fig.10 , Fig.11 , Fig.12 and Fig.13 When setting the first reference range (step S300a), the first range may be set based on the second measurement value MV2 obtained from the second sensor 120 (step S350), and the second range may be set based on the third measurement value MV3 obtained from the third sensor 150 (step S360). The first reference range may be selected based on the first range and the second range (step S370).

[0109] When the first reference range is selected based on the first range and the second range (step S370 ), in the case where the first range and the second range do not match each other (step S370 : Yes), the first reference range including the first range and the second range may be selected (step S374 ).

[0110] For example, when the determination result of the second sensor 120 and the determination result of the third sensor are different from each other, the first reference range which is the same as the sum of the first range and the second range may be selected. Fig.11 The first calculation in step S400 makes it finally determined which of the determination result of the second sensor 120 and the determination result of the third sensor is correct. For example, step S374 may be used when the image sensor as the second sensor 120 is contaminated and / or when a picture drawn on a floor or a wall is misleadingly recognized as an object rather than the floor or the wall itself.

[0111] When the first range and the second range match each other (step S370 : No), a first reference range including a third range other than the first range and the second range in the entire selectable range available for selection may be selected (step S376 ).

[0112] For example, when the determination result of the second sensor 120 and the determination result of the third sensor are identical to each other, the first reference range identical to the third range other than the first range and the second range may be selected. Fig.11 The first calculation in step S400 makes the determination result of the second sensor 120 and the determination result of the third sensor 120 credible, and checks the remaining range.

[0113] Although the reference Fig.12 and Fig.13 The operation and specific scenario for selecting the first reference range are described, but example embodiments are not limited thereto and the operation and specific scenario for selecting the first reference range may vary according to example embodiments.

[0114] Fig.14 is a block diagram showing an object detection device according to an example embodiment. Figure 1 and Fig.10 Duplicate description.

[0115] Reference Fig.14 , the object detection device 100b includes a first sensor 110, a second sensor 120, a third sensor 150, an Xth sensor 160, a calculation range selector 130b and an estimator 140, where X is a natural number greater than or equal to 4.

[0116] In addition to the object detecting apparatus 100b further including the X-th sensor 160 and partially changing the operation of the calculation range selector 130b, Fig.14 The object detection device 100b can be used with Fig.10 The object detection device 100a is substantially the same.

[0117] The Xth sensor 160 may obtain an Xth measurement value MVX for the object 10. The first sensor 110, the second sensor 120, the third sensor 150, and the Xth sensor 160 may be sensors of different types. For example, the first sensor 110, the second sensor 120, the third sensor 150, and the Xth sensor 160 may respectively obtain a first measurement value MV1, a second measurement value MV2, a third measurement value MV3, and an Xth measurement value MVX by sensing different physical features from the object 10.

[0118] The calculation range selector 130b sets the reference range RRNG based on the second measurement value MV2, the third measurement value MV3, and the Xth measurement value MVX. The reference range RRNG represents the execution range of the first calculation for detecting the position of the object 10 using the first algorithm. The operation of setting the reference range RRNG by the calculation range selector 130b may be similar to the reference range RRNG. Figure 6 , Figure 7 , Fig.12 and Fig.13 Described operations are implemented.

[0119] In some example embodiments, when the object detection devices 100a and 100b are not limited to the following Fig.10 and Fig.14 When two or more sensors of different types are included in addition to the first sensor 110 shown, the object detection devices 100a and 100b can be implemented to also include an environmental sensor and a weight setting unit. The environmental sensor may include at least one of a temperature sensor, a humidity sensor, a pressure sensor, a motion sensor, a time sensor, a space sensor, an illumination sensor, an acceleration sensor, a vibration sensor, a mechanical stress sensor, an impact sensor, etc., for sensing information associated with the surrounding environment. The weight setting unit may assign different weights (or reliabilities) to the measurements obtained from sensors other than the first sensor 110 based on the information associated with the surrounding environment sensed by the environmental sensor. For example, in a foggy or fuzzy environment, the weight setting unit may assign a relatively low weight to the measurement value of the image sensor and a relatively high weight to the measurement value of the LIDAR sensor. The calculation range selectors 130a and 130b may select a first reference range based on the measurement value and the weight.

[0120] Fig.15 is a flowchart illustrating an object detection method according to an example embodiment. Figure 5 Duplicate description.

[0121] Reference Figure 1 and Fig.15 In the object detection method according to the exemplary embodiment, a first measurement value MV1 for the object 10 is periodically obtained using the first sensor 110 (step S100c). A second measurement value MV2 for the object 10 is periodically obtained using the second sensor 120 (step S200c). A first reference range is periodically set based on the second measurement value MV2 (step S300c). A first calculation is periodically performed only on the first reference range using the first measurement value MV1 (step S400c). A first result value is periodically generated (step S500c). In addition to steps S100c, S200c, S300c, S400c, and S500c being periodically performed, Fig.15 Steps S100c, S200c, S300c, S400c and S500c in Figure 5 Steps S100, S200, S300, S400 and S500 in the embodiment are substantially the same.

[0122] A second result value is also generated based on the periodically generated first result value (step S600c). The second result value represents the distance to the object 10 and the speed of the object 10. For example, when the first result value is the azimuth of the object 10, the change amount of the azimuth can be estimated. Therefore, the distance to the object 10 and / or the speed of the object 10 can be calculated. Step S600c can be performed by the estimator 140.

[0123] In some example embodiments, Fig.15 As shown, it is possible to Fig.11 The object detection method additionally performs the operation of executing the cycle and generating a second result value.

[0124] Fig.16 is a block diagram illustrating a vehicle control system according to an example embodiment.

[0125] Reference Fig.16 The vehicle control system 500 includes a controller 600 and an object detection device 700 which are installed, equipped or loaded on a vehicle 501 .

[0126] The object detection device 700 detects objects around the vehicle 501 and generates a first result value indicating the position of the object. Figures 1 to 15The object detection device of the example embodiment described. For example, the object detection device 700 may include a sensor SEN1, one or more sensors SEN2, ..., SENX, a calculation range selector CRS, and an estimator. Each of the one or more sensors SEN2, ..., SENX may be a sensor of a different type from the sensor SEN1. The calculation range selector CRS may set a reference range based on the measurement values ​​of the sensors SEN2, ..., SENX. The estimator EST may perform a first calculation based on a first algorithm (e.g., a MUSIC algorithm) only on the reference range using the measurement value of the sensor SEN1, and may generate a first result value.

[0127] The controller 600 controls the vehicle 501 based on the first result value. For example, the controller 600 may include any processing unit such as a central processing unit (CPU), an application processor (AP), and the like.

[0128] As will be appreciated by those skilled in the art, the present invention may be implemented as a system, method, computer program product and / or a computer program product implemented in one or more computer-readable media on which a computer-readable program code is implemented. The computer-readable program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be any tangible medium that may include or store a program used by or in conjunction with an instruction execution system, device, or equipment. For example, the computer-readable medium may be a non-transitory computer-readable medium.

[0129] Fig.17 is a block diagram illustrating an electronic device including a vehicle control system according to an example embodiment.

[0130] Reference Fig.17 , the electronic device 1000 includes a processing circuit 1010 and an object detection device 1020. The electronic device 1000 may also include a memory 1100, an output interface 1200 (e.g., including an output circuit), a sensor 1400, a communication interface 1500 (e.g., including a communication circuit), an A / V input interface 1600 (audio / video input interface) (e.g., including an A / V input circuit), and an input interface 1700 (e.g., including an input circuit). As will be described with reference to Fig.18 As described, the electronic device 1000 may be an ADAS, an autonomous driving system, etc. installed, equipped, or loaded in a vehicle.

[0131] The processing circuit 1010 includes various processing circuits and controls the general operation of the electronic device 1000. The object detection device 1020 is controlled by the processing circuit 1010 and performs an object detection operation. The processing circuit 1010 and the object detection device 1020 may correspond to Fig.16 The controller 600 and the object detection device 700 in Fig.16 At least a part of the sensors in the object detection device 700 may be included in the sensor 1400, the A / V input interface 1600, and the like.

[0132] The input interface 1700 may receive an input for controlling operations of modules installed in the vehicle.

[0133] The output interface 1200 may include various circuits to output audio signals, video signals, and / or vibration signals. The output interface 1200 may include a display 1210, a sound output interface 1220 (e.g., including a sound output circuit), and a vibration motor 1230. For example, the output interface 1200 may output a notification message as audio, video, and / or vibration.

[0134] The display 1210 may display and output information processed in the electronic device 1000. For example, the display 1210 may display a notification message on a head-up display (HUD) of a vehicle. The sound output interface 1220 may include various circuits to output audio data received from the communication interface 1500 or stored in the memory 1100. In addition, the sound output interface 1220 may output a sound signal (e.g., a call signal reception sound, a message reception sound, a notification sound) related to a function executed in the electronic device 1000.

[0135] The processing circuit 1010 can generally control the input interface 1700, the output interface 1200, the sensor 1400, the communication interface 1500, the A / V input interface 1600, etc. by executing the program stored in the memory 1100. In addition, the processing circuit 1010 can perform the functions of the electronic device 1000 by executing the program stored in the memory 1100. The processing circuit 1010 may include at least one processor. The processing circuit 1010 may include multiple processors or one integrated processor based on its functions and operations.

[0136] The sensor 1400 may include various sensors and sensing circuits to sense a state of the electronic device 1000, a state of a user, or a state around the electronic device 1000. The sensor 1400 may transmit the sensed information to the processing circuit 1010.

[0137] The sensor 1400 may include various sensing circuits. For example, the sensor 1400 may include, but is not limited to, at least one of a magnetic sensor 1410, an acceleration sensor 1420, a temperature / humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, a position sensor (e.g., a global positioning system (GPS)) 1460, an atmospheric sensor 1470, a proximity sensor 1480, and an RGB sensor 1490.

[0138] The communication interface 1500 may include various communication circuits, including at least one component configured to enable the electronic device 1000 to communicate with another electronic device (not shown) and a server (not shown). The other electronic device may be a computing device or a sensor, but is not limited thereto. For example, the communication interface 1500 may include a short-range wireless communication interface 1510, a mobile communication interface 1520, and a broadcast reception interface 1530.

[0139] The short-range wireless communication interface 1510 may include a Bluetooth communication interface, a Bluetooth low energy (BLE) communication interface, a near field communication and / or radio frequency identification (NFC / RFID) interface, a WLAN (e.g., Wi-Fi) communication interface, a Zigbee communication interface, an Infrared Data Association (IrDA) communication interface (not shown), a Wi-Fi Direct (WFD) communication interface, an ultra-wideband (UWB) communication interface, an Ant+ communication interface, etc., but is not limited to these.

[0140] The A / V input interface 1600 may include various A / V interface circuits and is configured to input an audio signal or a video signal. The A / V input interface 1600 may include a camera 1610, a microphone 1620, etc. The camera 1610 may obtain an image frame such as a still image or a video via an image sensor in a video phone mode or a shooting mode. The image captured by the image sensor may be processed by the processing circuit 1010 or an additional image processor (not shown).

[0141] The microphone 1620 may receive an external sound signal and process the external sound signal into electrical sound data. For example, the microphone 1620 may receive a sound signal from an external device or a user. The microphone 1620 may use various noise removal algorithms to remove noise generated in the process of receiving the external sound signal.

[0142] The memory 1100 may store a program for processing and controlling the operation of the processing circuit 1010. The memory 1100 may store data input to or output from the electronic device 1000.

[0143] The memory 1100 may include at least one type of storage medium of a flash memory type, a hard disk type, and a multimedia card micro type. The programs stored in the memory 1100 may be divided into a plurality of modules based on their functions. For example, the program may be divided into a UI module 1110 (user interface module), a touch screen module 1120, and a notification module 1130.

[0144] The UI module 1110 may provide a dedicated UI, a graphical user interface (GUI), etc. for each application synchronized with the electronic device 1000. The touch screen module 1120 may sense a touch gesture on the touch screen via a user and transmit information related to the touch gesture to the processing circuit 1010. The touch screen module 1120 may be implemented as additional hardware including a controller.

[0145] The notification module 1130 may generate a signal to notify the occurrence of an event. The notification module 1130 may output the notification signal as a video signal via the display 1210 , as an audio signal via the sound output interface 1220 , or as a vibration signal via the vibration motor 1230 .

[0146] Fig.18 is a diagram showing an example in which an electronic device according to an example embodiment is installed in a vehicle.

[0147] Reference Fig.18 , the electronic device 1000 may be an ADAS, an autonomous driving system, etc. included (e.g., installed) in the vehicle 2000. The electronic device 1000 may include various instances of circuits and components configured to receive a video sequence including a stereoscopic image, a reflected wave (e.g., a reflected electromagnetic wave), or a reflected light from a camera installed in the vehicle 2000 and determine the occurrence of various events associated with the vehicle 2000. The various events may include object detection, object tracking, and scene segmentation. The electronic device 1000 may generate an output signal including a notification message based on determining the occurrence of one or more events, which may be presented to an occupant (e.g., a user) of the vehicle 2000 via one or more user interfaces of the vehicle 2000. The electronic device 1000 may generate an output signal based on determining the occurrence of one or more events, which causes the vehicle control system of the vehicle 2000 to control the driving (e.g., driving trajectory) of the vehicle 2000.

[0148] Although it is described that the electronic device 1000 receives a video sequence from a camera installed in the vehicle 2000, example embodiments are not limited thereto. The electronic device 1000 may receive a video sequence from a camera to capture the surrounding environment of the vehicle 2000. The surrounding environment of the vehicle 2000 (also referred to herein as a driving environment associated with the vehicle 2000) may include, for example, a front side, a lateral side, and a rear side.

[0149] In some example embodiments, the electronic device 1000 may detect an event based on the location of the event by tracking a bounding box of a specified object. Therefore, the electronic device 1000 may recognize the importance level of the type of the object differently based on the location of the object, thereby determining whether the event occurs based on the location of the object.

[0150] In some example embodiments, the electronic device 1000 may detect at least one video sequence 2003 (or a stereoscopic image) including an object from a plurality of video sequences, and may obtain a wave (e.g., a reflected electromagnetic wave) or a reflected light (not shown) reflected from a RADAR sensor. The reflected wave may be captured at one or more sensors at the vehicle 2000, and may be reflected from one or more objects located in the surrounding environment (e.g., a driving environment). The electronic device 1000 may detect a road 2002 including a fixed pattern and another vehicle 2001 moving according to time by analyzing at least one video sequence 2003. In some example embodiments, the electronic device 1000 may analyze the position of other vehicles 2001 by analyzing the coordinates of other vehicles 2001 in at least one video sequence 2003, thereby determining the occurrence of an event based on the detection of other vehicles 2001. The electronic device 1000 may also generate an output signal based on the determination, wherein the output signal, when processed by the control system of the vehicle 2000, causes a specific notification message to be presented to the occupant of the vehicle 2000 via the user interface of the vehicle 2000. Additionally or alternatively, the output signal generated based on the determination may cause the driving of the vehicle 2000 to be controlled so that the vehicle 2000 drives along a specific driving path (e.g., on a driving trajectory) through the surrounding environment. For example, the output signal generated based on the determination may cause the driving of the vehicle 2000 to be autonomously controlled by the autonomous driving system as an autonomous vehicle.

[0151] In some example embodiments, vehicle 2000 may include any means of transportation. For example, vehicle 2000 may be, but is not limited to, a car, bus, truck, train, bicycle, motorcycle, scooter, etc. Any instantiation of vehicle 2000 may provide communication functionality, data processing functionality, and / or transportation functionality consistent with the description herein.

[0152] The various example embodiments described above are implemented by hardware components, software components, or a combination of hardware components and software components. Where appropriate, examples of hardware components that can be used to perform the operations described in this application include controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more hardware components in the hardware components that perform the operations described in this application are implemented by computing hardware, such as by one or more processors or computers. A processor or computer can be implemented by one or more processing elements, such as a logic gate array, a controller and an arithmetic logic unit, a digital signal processor, a microcomputer, a programmable logic controller, a field programmable gate array, a programmable logic array, a microprocessor, or a combination of any other devices or devices that are configured to respond and execute instructions in a limited manner to achieve a desired result. In one example, a processor or computer includes or is connected to one or more memories storing instructions or software executed by a processor or computer. The hardware components implemented by a processor or a computer can execute instructions or software, such as an operating system (OS) and one or more software applications running on the OS, to perform the operations described in this application. The hardware components can also access, manipulate, process, create and store data in response to the execution of instructions or software. For simplicity, the singular term "processor" or "computer" can be used in the description of the examples described in this application, but in other examples, multiple processors or computers can be used, or the processor or computer can include multiple processing elements or multiple types of processing elements, or both. For example, a single hardware component or two or more hardware components can be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components can be implemented by one or more processors, or a processor and a controller, and one or more other hardware components can be implemented by one or more other processors, or another processor and another controller. One or more processors, or a processor and a controller, can implement a single hardware component, or two or more hardware components.

[0153] The inventive concept may be applied to various devices and / or systems including an object detection device and a vehicle control system. For example, the inventive concept may be applied to systems such as an advanced driver assistance system (ADAS), an autonomous driving system, and the like.

[0154] The above is an illustration of example embodiments and should not be construed as limiting thereof. Although some example embodiments have been described, those skilled in the art will readily appreciate that many modifications may be made in the example embodiments without materially departing from the novel teachings and advantages of the example embodiments. Therefore, all such modifications are intended to be included within the scope of the example embodiments as defined in the claims. Therefore, it should be understood that the foregoing is an illustration of various example embodiments and should not be construed as limiting the disclosed example embodiments, and modifications to the disclosed example embodiments as well as other example embodiments are intended to be included within the scope of the appended claims.

Claims

1. An object detection device, comprising: a first sensor configured to output a radio frequency (RF) signal, receive a reflected RF signal reflected from an object, and obtain a first measurement value for the object based on the received reflected RF signal; a second sensor configured to obtain a second measurement for the object by sensing a physical characteristic from the object, the physical characteristic sensed by the second sensor being different from the characteristic of the object measured as the first measurement obtained by the first sensor; a calculation range selector configured to set a first reference range based on the second measurement value, the first reference range representing an execution range of a first calculation for detecting a position of the object using a first algorithm; as well as An estimator is configured to perform the first calculation only over the first reference range using the first measurement value and generate a first result value as a result of performing the first calculation, the first result value indicating the position of the object.

2. The object detection device of claim 1, wherein the calculation range selector is configured to: setting a first range based on the second measurement value obtained from the second sensor; and A first reference range is selected that matches the first range.

3. The object detection device according to claim 2, wherein: When the reliability of the first range is higher than a reference value or when a check operation is required for the first range, a first reference range matching the first range is selected.

4. The object detection device of claim 1, wherein the calculation range selector is configured to: setting a first range based on the second measurement value obtained from the second sensor; and Among the entire range available for selection of the first reference range, a first reference range matching a second range other than the first range is selected.

5. The object detection device according to claim 4, wherein: When the reliability of the first range is lower than a reference value or when a check operation is required for the second range other than the first range, a first reference range matching the second range is selected.

6. The object detection device of claim 1, wherein the first sensor is a Radio Detection and Ranging (RADAR) sensor.

7. The object detection device of claim 6, wherein the first sensor comprises: a transmitting antenna configured to output the RF signal; a plurality of receiving antennas configured to receive the reflected RF signals reflected from the object; as well as A signal processor is configured to divide a received RF signal obtained from each of the plurality of receiving antennas into an in-phase component and an orthogonal-phase component, calculate a phase of the received RF signal based on the in-phase component and the orthogonal-phase component, and generate a phase vector corresponding to the first measurement value based on the calculated phase.

8. The object detection device of claim 7, wherein: The phase vector includes m components, where m is a natural number greater than or equal to 2, and Among the m components, the kth component represents a phase difference between an RF signal received by a first receiving antenna among the multiple receiving antennas and an RF signal received by a kth receiving antenna, where k is a natural number greater than or equal to 1 and less than or equal to n1.

9. The object detection device of claim 1, wherein: The second sensor is an image sensor that obtains image information for the object based on reflected light reflected from the object; as well as The image information obtained by the second sensor corresponds to the second measurement value.

10. The object detection apparatus of claim 1, wherein the first algorithm is a Multi-Signal Classification (MUSIC) algorithm.

11. The object detection device of claim 10, wherein the estimator is configured to: selecting a steering vector corresponding to the first reference range from an array manifold AM vector including a plurality of steering vectors measured in advance; calculating a covariance matrix based on the selected steering vector and a phase vector corresponding to the first measurement; The MUSIC spectrum value is obtained by normalizing the covariance matrix, calculating the determinant of the normalized covariance matrix, and taking the inverse of the determinant; Extracting peak values ​​from the MUSIC spectrum values; as well as An azimuth angle of the object corresponding to the position of the object is estimated based on the peak value.

12. The object detection device of claim 1, further comprising: a third sensor configured to obtain a third measurement for the object by sensing a physical characteristic from the object, the physical characteristic sensed by the third sensor being different from the physical characteristics sensed by the first sensor and the second sensor, and The calculation range selector is configured to set the first reference range based on the second measurement value and the third measurement value.

13. The object detection device of claim 12, wherein the calculation range selector is configured to: setting a first range based on the second measurement value obtained from the second sensor; setting a second range based on the third measurement value obtained from the third sensor; and The first reference range is selected based on the first range and the second range.

14. The object detection device according to claim 13, wherein: When the first range and the second range do not match each other, the first reference range including the first range and the second range is selected.

15. The object detection device according to claim 13, wherein: When the first range and the second range match each other, the first reference range including a third range other than the first range and the second range is selected among the entire range available for selection.

16. The object detection device of claim 12, wherein the third sensor is a light detection and ranging (LIDAR) sensor that outputs laser pulses, receives reflected laser pulses reflected from the object, and obtains the third measurement value based on the received reflected laser pulses.

17. The object detection device of claim 1, wherein: The first sensor and the second sensor are configured to periodically obtain a first measurement value and a second measurement value; The calculation range selector is configured to periodically set the first reference range; as well as The estimator is configured to periodically perform the first calculation and periodically generate the first result value.

18. The object detection device of claim 17, wherein: The estimator is configured to generate a second result value further based on the periodically generated first result value; and The second result value represents the distance to the object and the speed of the object.

19. A vehicle control system comprising: an object detection device configured to detect an object around the vehicle and generate a first result value representing a position of the object; as well as a controller configured to control the vehicle based on the first result value, Wherein, the object detection device comprises: a first sensor configured to output a radio frequency (RF) signal, receive a reflected RF signal reflected from the object, and obtain a first measurement value for the object based on the received reflected RF signal; a second sensor configured to obtain a second measurement for the object by sensing a physical characteristic from the object, the physical characteristic sensed by the second sensor being different from the characteristic of the object measured as the first measurement obtained by the first sensor; a calculation range selector configured to set a first reference range based on the second measurement value, the first reference range representing a range of execution of a first calculation for detecting a position of the object using a first algorithm; and An estimator is configured to perform the first calculation only over the first reference range using the first measurement value and generate the first result value as a result of performing the first calculation.

20. An object detection device comprising: a first sensor configured to output a radio frequency (RF) signal, receive a reflected RF signal reflected from an object, divide the received reflected RF signal into an in-phase component and a quadrature-phase component, calculate a phase of the received RF signal based on the in-phase component and the quadrature-phase component, and generate a phase vector based on the calculated phase; a second sensor configured to obtain image information for the object based on reflected light reflected from the object; a third sensor configured to output laser pulses, receive reflected laser pulses reflected from the object, and obtain measurement information for the object based on the received reflected laser pulses; a calculation range selector configured to set a first reference range based on at least one of the image information obtained by the second sensor and the measurement information obtained by the third sensor, the first reference range being a portion of an entire execution range of a multiple signal classification (MUSIC) algorithm available for selection; as well as an estimator configured to perform calculation based on the MUSIC algorithm only on the first reference range using the phase vector and estimate an azimuth angle of the object representing a position of the object, and The estimator is configured to: select a steering vector corresponding to the first reference range from an array manifold AM vector including a plurality of steering vectors measured in advance; calculate a covariance matrix based on the selected steering vector and the phase vector; obtain a MUSIC spectrum value by normalizing the covariance matrix, calculating a determinant of the normalized covariance matrix, and taking the inverse of the determinant; extracting a peak value from the MUSIC spectrum value; and estimating an azimuth angle of the object based on the peak value.

Images