Complex radar device and multi-doppler-based object detection device

Abstract

A complex radar device according to embodiments of the present disclosure may comprise: a primary object detection unit, which transmits a transmitted signal so as to receive, as a received signal, a signal in which the transmitted signal is reflected from the surroundings, extracts a first Doppler component and / or a second Doppler component from the received signal, and obtains primary object information about an object in the surroundings on the basis of the extracted result; a secondary object detection unit for estimating a height center value of the object on the basis of the primary object information, and obtaining secondary object information on the basis of the estimated result; and an object integration detection unit for obtaining object integration information on the basis of the secondary object information.

Description

Composite radar device and multi-Doppler-based object detection device

[0001] The embodiments of the present disclosure relate to a composite radar device and a multi-Doppler-based object detection device.

[0002]

[0003] Nowadays, various object detection technologies are being developed in diverse fields to detect objects such as people and objects. These diverse object detection technologies employ various detection methods. For example, various object detection technologies may include infrared-based detection technology, image or photo-based detection technology, or radio frequency identification (RFID)-based detection technology.

[0004] However, all object detection technologies currently under development may have several issues. For example, infrared detection can lead to erroneous judgments due to the influence of shielding or temperature. As another example, image or photo-based detection technologies can also lead to misjudgments due to the influence of lighting or human posture, and raise significant concerns regarding privacy infringement. Furthermore, current object detection technologies may not meet required detection accuracy standards and may have limitations in their application across various fields.

[0005] Embodiments of the present disclosure may provide a composite radar device and a multi-Doppler-based object detection device that perform object detection using heterogeneous composite Doppler components.

[0006] Embodiments of the present disclosure may provide a composite radar device and a multi-Doppler-based object detection device capable of precisely and rapidly detecting various information, states, or actions (movements) of an object using heterogeneous composite Doppler components.

[0007] Since the embodiments of the present disclosure are not based on images, a composite radar device and a multi-Doppler-based object detection device can be provided that can perform object detection without infringing on privacy or exposing personal information.

[0008] Embodiments of the present disclosure can provide a composite radar device and a multi-Doppler-based object detection device capable of providing various application functions using radar-based object detection technology.

[0009] The problems of the embodiments of the present disclosure are not limited to those mentioned in this specification, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0010] A composite radar device according to embodiments of the present disclosure may include a primary object detection unit configured to transmit a transmission signal and receive a signal reflected from the surroundings as a reception signal, extract at least one of a first Doppler component and a second Doppler component from the reception signal, and obtain primary object information about an object in the surroundings based on the extraction result; a secondary object detection unit configured to estimate the height center value of an object based on the primary object information and obtain secondary object information based on the estimation result; and an object integration detection unit configured to obtain object integration information based on the secondary object information.

[0011] A multi-Doppler-based object detection device according to embodiments of the present disclosure may include a multi-Doppler component extraction unit that extracts a plurality of Doppler components corresponding to different types from a received signal that is received by reflecting a transmitted signal from the surroundings; a first detection unit that obtains first detection information including two-dimensional position information of an object existing in the surroundings based on the plurality of Doppler components; a second detection unit that obtains second detection information including velocity information of the object along with two-dimensional position information based on the plurality of Doppler components; and a third detection unit that obtains third detection information including a height center value of the object along with the second detection information based on the plurality of Doppler components.

[0012] According to embodiments of the present disclosure, a composite radar device and a multi-Doppler-based object detection device that perform object detection using heterogeneous composite Doppler components can be provided.

[0013] According to embodiments of the present disclosure, a composite radar device and a multi-Doppler-based object detection device can be provided that can precisely and rapidly detect various information, states, or actions (movements) of an object using heterogeneous composite Doppler components.

[0014] According to embodiments of the present disclosure, a composite radar device and a multi-Doppler-based object detection device can be provided that can perform object detection without infringing on privacy or exposing personal information because they are not based on images.

[0015] According to embodiments of the present disclosure, a composite radar device and a multi-Doppler-based object detection device capable of providing various application functions using radar-based object detection technology can be provided.

[0016] The effects of the embodiments of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description in the claims.

[0017] The content of this disclosure will be more fully understood from the detailed description and accompanying drawings provided below, which are provided solely for illustrative purposes and are not intended to limit the content of this disclosure.

[0018] FIG. 1 is a block diagram of a composite radar device according to embodiments of the present disclosure.

[0019] FIG. 2 is a detailed block diagram of a composite radar device according to embodiments of the present disclosure.

[0020] FIG. 3 illustrates the operation procedure of a composite radar device according to embodiments of the present disclosure.

[0021] FIGS. 4 and FIGS. 5 are diagrams illustrating the object motion detection function of a composite radar device according to embodiments of the present disclosure.

[0022] FIGS. 6 and 7 are diagrams illustrating the object movement tracking function of a composite radar device according to embodiments of the present disclosure.

[0023] FIGS. 8 and 9 are diagrams illustrating the three-dimensional shape recognition function of a composite radar device according to embodiments of the present disclosure.

[0024] FIG. 10 is an additional block diagram of a composite radar device according to embodiments of the present disclosure.

[0025] FIG. 11 is a diagram showing the object count measurement function of a composite radar device according to embodiments of the present disclosure.

[0026] FIG. 12 is a diagram showing the abnormal situation detection function of a composite radar device according to embodiments of the present disclosure.

[0027] FIG. 13 is an additional block diagram of a composite radar device according to embodiments of the present disclosure.

[0028] FIGS. 14 and 15 are diagrams illustrating the object presence detection function of a composite radar device according to embodiments of the present disclosure.

[0029] FIG. 16 is a diagram showing the object abnormality detection function of a composite radar device according to embodiments of the present disclosure.

[0030] FIG. 17 is a block diagram of a multi-Doppler-based object detection device according to embodiments of the present disclosure.

[0031] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, the same components may have the same reference numeral as much as possible, even if they are shown in different drawings. Furthermore, in describing the present disclosure, if it is determined that a detailed description of related known components or functions may obscure the essence of the present disclosure, such detailed description may be omitted. Where terms such as "comprising," "having," or "consisting of" are used in this specification, other parts may be added unless "only" is used. Where a component is expressed in the singular, it may include a plural unless there is a special explicit description otherwise.

[0032] Additionally, terms such as first, second, A, B, (a), (b), etc., may be used to describe the components of the present disclosure. These terms are used merely to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by such terms.

[0033] In describing the positional relationship of components, where it is stated that two or more components are "connected," "combined," or "joined," it should be understood that while the two or more components may be directly "connected," "combined," or "joined," they may also be "connected," "combined," or "joined" with other components "intervened." Here, the other components may be included in one or more of the two or more components that are "connected," "combined," or "joined" with one another.

[0034] In describing the temporal flow relationship regarding components, methods of operation, or methods of production, for example, when the temporal or sequential relationship is described using "after," "following," "next," or "before," it may include cases where the relationship is not continuous unless "immediately" or "directly" is used.

[0035] Meanwhile, where numerical values ​​or corresponding information regarding a component (e.g., levels, etc.) are mentioned, even without separate explicit notation, the numerical values ​​or corresponding information may be interpreted as including a range of error that may occur due to various factors (e.g., process factors, internal or external shocks, noise, etc.).

[0036] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0037] FIG. 1 is a block diagram of a composite radar device (100) according to embodiments of the present disclosure.

[0038] Referring to FIG. 1, a composite radar device (100) according to embodiments of the present disclosure is a device that detects an object by combining a Doppler signal (Doppler component) and a micro-Doppler signal (micro-Doppler component), and may include a primary object detection unit (110), a secondary object detection unit (120), and an object integration detection unit (130), etc.

[0039] The primary object detection unit (110) may be configured to transmit a transmission signal and receive a signal reflected from the surroundings as a reception signal, extract at least one of a first Doppler component and a second Doppler component from the reception signal, and obtain primary object information about an object in the surroundings based on the extraction result.

[0040] The secondary object detection unit (120) may be configured to estimate the height center value of an object based on primary object information and to obtain secondary object information based on the estimation result.

[0041] The object integration detection unit (130) may be configured to acquire object integration information based on secondary object information.

[0042] The first object detection unit (110) can extract a Doppler component from a received signal as a first Doppler component during a first time period, and extract a Doppler component from a received signal as a second Doppler component during a second time period. Here, the second time period may be set to be longer than the first time period. The second Doppler component may be a micro-Doppler component.

[0043] The primary object detection unit (110) can determine the type of object as one of a Doppler object having a speed greater than or equal to a predefined threshold speed, and a micro-Doppler object having a speed less than or equal to the threshold speed or being stationary.

[0044] The primary object detection unit (110) can determine that an object determined to be a Doppler object or a micro-Doppler object has a height component, and that object is a 4-dimensional Doppler object.

[0045] The primary object detection unit (110) can obtain primary object information including information about the height of the object based on the extraction result.

[0046] The secondary object detection unit (120) can estimate the height center value of the object from the height of the object identified from the primary object information.

[0047] The secondary object detection unit (120) can calculate a height change amount indicating the degree to which the estimated height center value changes over time, determine the state of the object based on the calculation result, and obtain secondary object information including information about the state.

[0048] For example, the composite radar device (100) according to the embodiments of the present disclosure may utilize frequency modulated continuous waves (FMCW) technology and may provide accurate measurement results regarding the distance and speed of an object. The composite radar device (100) according to the embodiments of the present disclosure may use a chirp signal, and the frequency of the chirp signal may increase linearly over time. The presence of an object may be further estimated based on the phase difference between two chirp signals in a radar echo wave. The composite radar device (100) according to the embodiments of the present disclosure may perform multi-object detection.

[0049] A composite radar device (100) according to embodiments of the present disclosure can find characteristic information of an object (meaning object information such as location, height, and state) by utilizing the frequency difference between a transmitted signal and a received signal reflected from an object. Here, the object may be a person or an object, and may have periodic / non-periodic motion or movement, be stationary or moving, or have a sudden change of state.

[0050] By analyzing radar Doppler frequency characteristics, the momentum of an object (person or thing) can be monitored to determine the object's state.

[0051] A received signal reflected from at least one object surrounding the composite radar device (100) may have a frequency different from the transmitted signal frequency. Here, the difference in frequency between the transmitted signal and the received signal may be called the Doppler frequency. The Doppler frequency measured by the composite radar device (100) can be used as a parameter to obtain motion information, including the velocity of the object.

[0052] A composite radar device (100) according to embodiments of the present disclosure can comprehensively analyze not only a Doppler signal (Doppler component) but also a micro-Doppler signal (micro-Doppler component) in order to accurately determine the motion state (movement state) of an object (person or thing). Doppler frequency is a Doppler frequency generated by the macroscopic movement (movement, trajectory) of an object, and micro-Doppler frequency can be viewed as one of these Doppler frequencies and may be a Doppler frequency generated by microscopic movement (fine movement) occurring between macroscopic movements (movement, trajectory).

[0053] Micro-Doppler signals may be used to obtain information on motion (movement) such as shaking, vibration, or rotation of an object (person or thing) or small movements of the object, and micro-Doppler signals can be widely utilized for object (person or thing) recognition, motion characteristics, or motion patterns analysis. A composite radar device (100) according to embodiments of the present disclosure can detect various motion characteristics of an object through spectrogram analysis of the micro-Doppler component (micro-Doppler signal).

[0054] A composite radar device (100) according to embodiments of the present disclosure can collect one-dimensional micro-Doppler signals for a moving object in one plane using single MIMO (Multiple Input Multiple Output) or beamforming technology. Additionally, the transmitting antenna and the receiving antenna of the composite radar device (100) according to embodiments of the present disclosure may be configured spatially together or spatially separated to separate the transmission and reception signals of the radar.

[0055] A composite radar device (100) according to embodiments of the present disclosure can collect 3D micro-Doppler frequency information about an object using a plurality of beamforming systems and can detect object information by configuring the collected information into time-series data.

[0056] A composite radar device (100) according to embodiments of the present disclosure can acquire object information by collecting micro-Doppler signals over time and receiving three-dimensional time series data as input, which has frequency changes over time as features.

[0057] A composite radar device (100) according to embodiments of the present disclosure can perform object detection based on a three-dimensional Doppler signal (three-dimensional micro-Doppler signal) to further acquire object information including not only the distance of an object but also speed information.

[0058] A composite radar device (100) according to embodiments of the present disclosure can perform object detection based on a 4-dimensional Doppler signal (3-dimensional micro-Doppler signal) to obtain object information including not only distance or speed information of an object, but also the height of the object.

[0059] Hereinafter, the basic detection function of the composite radar device (100) according to the embodiments of the present disclosure and the application function utilizing it will be described in more detail.

[0060] FIG. 2 is a detailed block diagram of a composite radar device (100) according to embodiments of the present disclosure.

[0061] The secondary object detection unit (120) of the composite radar device (100) according to the embodiments of the present disclosure may include an object movement tracking unit (220) and a three-dimensional shape recognition unit (230), etc.

[0062] The object movement tracking unit (220) can obtain the position and trajectory of an object using the distance and velocity of the object identified from the primary object information, and the first Doppler component and the second Doppler component, and can obtain second object information including information about the position and trajectory of the object. Here, the position of the object may mean coordinates (2D coordinates) on a two-dimensional plane, and the trajectory of the object may mean a change in coordinates (2D coordinates) on a two-dimensional plane, and may also be referred to as the movement trajectory or movement of the object.

[0063] The 3D shape recognition unit (230) can recognize the 3D shape of an object based on the distance, speed, and height of the object identified from the 1st object information and the height center value, and obtain 2nd object information including 3D shape recognition information. Here, having a 3D shape of an object may mean that the object has a certain height. The 3D shape recognition information may include the height or height center value of the object, and may include information about the position or size of the object. The size of an object may be the area of ​​the region occupied by the points (positions) detected for an object.

[0064] Referring to FIG. 2, the secondary object detection unit (120) may further include an object motion detection unit (210). The object motion detection unit (210) can detect whether an object is moving based on the type of Doppler component extracted from the first Doppler component and the second Doppler component from the received signal.

[0065] For example, the movement of an object may refer to small changes, such as minute vibrations, rotations, or tremors of the object or a part thereof, without a change in the object's position (i.e., movement). As another example, the movement of an object may refer to minute changes in position (i.e., movement) where the change in the object's position falls below a predefined threshold.

[0066] The object integration detection unit (130) can obtain object integration information for the object by integrating all output information (secondary object information) of each of the object movement detection unit (210), object movement tracking unit (220), and 3D shape recognition unit (230). For example, the object integration information may include information about the location, trajectory, and state of the object.

[0067] FIG. 3 illustrates the operation procedure of a composite radar device (100) according to embodiments of the present disclosure.

[0068] Referring to FIG. 3, a composite radar device (100) according to embodiments of the present disclosure may include a first object detection step (S110), a second object detection step (S120), and an integrated object detection step (S130).

[0069] In the first object detection step (S110), the first object detection unit (110) may be configured to transmit a transmission signal and receive a signal reflected from the surroundings as a reception signal, extract at least one of a first Doppler component and a second Doppler component from the reception signal, and obtain first object information about an object in the surroundings based on the extraction result.

[0070] In the first object detection step (S110), the first object detection unit (110) may extract a Doppler component from a received signal as a first Doppler component during a first time period, and extract a Doppler component from a received signal as a second Doppler component during a second time period. Here, the second time period may be set to be longer than the first time period. The second Doppler component may be a micro-Doppler component.

[0071] The first Doppler component is called the general Doppler component, and the second Doppler component can be called the micro-Doppler component.

[0072] Each of the first Doppler component and the second Doppler component may include a Doppler frequency, which is the frequency at which there is a difference between the transmitted signal and the received signal. The first Doppler component may include a Doppler frequency generated by the movement of an object (macroscopic movement), and the second Doppler component may include a micro-Doppler frequency generated by the movement of an object (microscopic movement).

[0073] The primary object information obtained from the primary object detection unit (110) includes information about the distance of the object and may further include information about at least one of speed and height.

[0074] The primary object detection unit (110) can determine that an object is a Doppler object if the object in the vicinity has a speed greater than or equal to a predefined threshold speed.

[0075] The primary object detection unit (110) can determine that an object in the vicinity has a speed below a threshold speed or is a stationary micro-Doppler object (S111). Here, the micro-Doppler object may be an object with small movements such as fine vibration, rotation, or shaking.

[0076] The primary object detection unit (110) can determine that an object determined to be a Doppler object or a micro-Doppler object has an additional height component, and that object is a 4-dimensional Doppler object (S112). Here, the 4-dimensional Doppler object may be an object in which height is detected. For example, the 4-dimensional Doppler object may be an object in which 2-dimensional position information, velocity information, and height information are detected.

[0077] The second object detection step (S120) may include an object movement detection step (S121), an object movement tracking step (S122), and a three-dimensional shape recognition step (S123).

[0078] In the object motion detection step (S121), the object motion detection unit (210) can detect whether an object is moving based on the type of Doppler component extracted from the received signal, among the first Doppler component and the second Doppler component. For example, the movement of an object may refer to a small change, such as a minute vibration, rotation, or shaking of the object or a part thereof, without a change in the object's position (i.e., movement). For another example, the movement of an object may refer to a minute change in position (i.e., movement) where the change in the object's position is below a predefined threshold value.

[0079] The object motion detection unit (210) can detect that the object is in a state of no movement if only the second Doppler component among the first Doppler component and the second Doppler component is extracted from the received signal.

[0080] The object motion detection unit (210) can detect that the object is in a state of motion when the first Doppler component is extracted from the received signal or when both the first Doppler component and the second Doppler component are extracted.

[0081] In other words, when the object motion detection unit (210) detects that the object is not moving, the received signal may include only the second Doppler component (micro-Doppler component). When the object motion detection unit (210) detects that the object is moving, the received signal may include both the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component).

[0082] In the object movement tracking step (S122), the object movement tracking unit (220) can obtain the position and trajectory of an object using the distance and velocity of the object identified from the primary object information, and the first Doppler component and the second Doppler component, and obtain second object information including information about the position and trajectory of the object.

[0083] Here, the position of an object may refer to coordinates (2D coordinates) on a 2D plane, and the trajectory of an object may refer to changes in coordinates (2D coordinates) on a 2D plane, and may also be referred to as the movement trajectory or movement of an object.

[0084] In the 3D shape recognition step (S123), the 3D shape recognition unit (230) can recognize the 3D shape of an object based on the distance, speed, and height of the object identified from the 1st object information and the height center value, and obtain 2nd object information including 3D shape recognition information.

[0085] Here, the fact that an object has a three-dimensional shape may mean that the object has a certain height. The three-dimensional shape recognition information may include the height or height center value of the object, and may include information about the position or size of the object. The size of an object may be the area of ​​the region occupied by the detected points (positions) for an object.

[0086] For example, the 3D shape recognition unit (230) can collect information in the form of a 3D point cloud based on 3D position ((x, y, z) coordinates) and velocity information to obtain 3D shape recognition information of an object (e.g., height, width, size, etc. of the object).

[0087] In the 3D shape recognition step (S123), the 3D shape recognition unit (230) estimates the height center value of the object based on primary object information, calculates a height change amount indicating the degree to which the estimated height center value changes over time, and monitors the height change amount to determine the state of the object.

[0088] In the three-dimensional shape recognition step (S123), the three-dimensional shape recognition unit (230) can generate three-dimensional shape recognition information by detecting the three-dimensional shape or three-dimensional shape change of an object based on at least one of a height center value, a height change amount, and a state.

[0089] For example, the 3D shape recognition information may include information on at least one of the height center value, the height change amount, and the state.

[0090] In the integrated object detection step (S130), the object integration detection unit (130) can obtain object integration information for the object by integrating all output information (secondary object information) of each of the object movement detection step (S121), object movement tracking step (S122), and 3D shape recognition step (S123). For example, the object integration information may include information about the location, trajectory, and state of the object.

[0091] FIGS. 4 and FIGS. 5 are diagrams illustrating the object movement detection function of a composite radar device (100) according to embodiments of the present disclosure.

[0092] FIG. 4 illustrates an actual detection situation image (400_REAL) showing a detection area actually detected by an object movement detection unit (210) of a composite radar device (100) according to embodiments of the present disclosure, and a detection result screen (400) showing an object movement detection result by an object movement detection unit (210) of a composite radar device (100).

[0093] Referring to FIG. 4, a composite radar device (100) is installed in a space, and the composite radar device (100) detects a detection area in which a person, who is an object (OBJ_REAL), is sitting on a chair in a state of no movement (or a state of almost no movement).

[0094] Referring to the detection result screen (400), if only the second Doppler component (micro-Doppler component) among the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component) is extracted from the received signal received by the surroundings (object (OBJ_REAL) and objects in its vicinity) after the transmission signal transmitted from the composite radar device (100), the object motion detection unit (210) can detect that the person, who is the object (OBJ_REAL), is in a state of no movement.

[0095] An object image (OBJ) corresponding to the detected object (OBJ_REAL) may be displayed on the detection result screen (400). The object image (OBJ) may have a predefined simple three-dimensional shape. For example, the object image (OBJ) may be displayed as a three-dimensional shape such as a rectangular prism or a cylinder.

[0096] Additionally, the detection result screen (400) may display micro-Doppler points (Pmd) where a second Doppler component (micro-Doppler component) is generated.

[0097] Micro-Doppler points (Pmd) are points where very fine movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0098] FIG. 5 illustrates an actual detection situation image (500_REAL) showing a detection area actually detected by an object movement detection unit (210) of a composite radar device (100) according to embodiments of the present disclosure, and a detection result screen (500) showing an object movement detection result by an object movement detection unit (210) of a composite radar device (100).

[0099] Referring to FIG. 5, a composite radar device (100) is installed in a space, and the object movement detection unit (210) of the composite radar device (100) detects a detection area in which a person, who is an object (OBJ_REAL), is sitting on a chair in a moving state.

[0100] Referring to the detection result screen (500), when both the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component) are extracted from the received signal received by the surroundings (object (OBJ_REAL) and objects in its vicinity) after the transmission signal transmitted from the composite radar device (100) is reflected, the object motion detection unit (210) can detect that the person, who is the object (OBJ_REAL), is in a state of movement.

[0101] An object image (OBJ) corresponding to the detected object (OBJ_REAL) may be displayed on the detection result screen (500). The object image (OBJ) may have a simple predefined three-dimensional shape. For example, the object image (OBJ) may be displayed as a three-dimensional shape such as a rectangular prism or a cylinder.

[0102] Additionally, the detection result screen (500) may display Doppler points (Pd) where a first Doppler component (general Doppler component) is generated and micro-Doppler points (Pmd) where a second Doppler component (micro-Doppler component) is generated.

[0103] Doppler points (Pd) may be points where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, if an arm or hand moves, or if the sitting posture changes slightly significantly, points with such large movements may be marked as Doppler points (Pd).

[0104] Micro-Doppler points (Pmd) are points where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0105] The object movement detection unit (210) of the composite radar device (100) can detect an object (OBJ_REAL) that is stationary or in a state of ultra-low speed movement through the object movement detection function.

[0106] FIGS. 6 and FIGS. 7 are diagrams illustrating the object movement tracking function of a composite radar device (100) according to embodiments of the present disclosure.

[0107] FIG. 6 shows an actual detection situation image (600_REAL_t1) representing a detection area detected at a first time point (t1) by an object movement tracking unit (220) of a composite radar device (100) according to embodiments of the present disclosure, and a stereoscopic detection result screen (600_t1) and a planar detection result screen (600_PLN_t1) representing the object movement tracking result at the first time point (t1) by the object movement tracking unit (220) of the composite radar device (100).

[0108] FIG. 7 shows an actual detection situation image (600_REAL_t2) representing a detection area detected at a second time point (t2) by an object movement tracking unit (220) of a composite radar device (100) according to embodiments of the present disclosure, and a stereoscopic detection result screen (600_t2) and a planar detection result screen (600_PLN_t2) representing the object movement tracking result at the second time point (t2) by the object movement tracking unit (220) of the composite radar device (100).

[0109] Referring to FIG. 6, the composite radar device (100) is installed in a certain space, and the object movement tracking unit (220) of the composite radar device (100) can detect a situation in which a person, who is an object (OBJ_REAL), enters the detection area at a first time point (t1).

[0110] Referring to FIG. 6, when referring to the stereoscopic detection result screen (600_t1) and the planar stereoscopic detection result screen (600_PLN_t1) based on object movement tracking at the first time point (t1), the position information of the object (OBJ_REAL) can be detected by using the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component) extracted from the received signal that is received by the transmission signal transmitted from the composite radar device (100) being reflected from the surroundings (object (OBJ_REAL) and objects in its vicinity).

[0111] Referring to FIG. 6, the object image (OBJ_t1) corresponding to the detected object (OBJ_REAL) can be displayed on the stereoscopic detection result screen (600_t1) and the planar stereoscopic detection result screen (600_PLN_t1).

[0112] Referring to FIG. 6, in the stereoscopic detection result screen (600_t1), the object image (OBJ_t1) may have a predefined simple stereoscopic shape. For example, the object image (OBJ) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0113] Referring to FIG. 6, in the plane detection result screen (600_PLN_t1), the object image (OBJ_t1) may have a predefined simple plane shape. For example, the object image (OBJ) may be displayed as a plane shape such as a rectangle or a circle.

[0114] Additionally, the stereoscopic detection result screen (600_t1) and the planar stereoscopic detection result screen (600_PLN_t1) may display Doppler points (Pd) where a first Doppler component (general Doppler component) is generated and micro-Doppler points (Pmd) where a second Doppler component (micro-Doppler component) is generated.

[0115] Doppler points (Pd) may be points where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, when an arm or leg moves to walk, the points where the object moves can be indicated as Doppler points (Pd) in situations where there is a change in the object's position and the object's trajectory is detected.

[0116] Micro-Doppler points (Pmd) are points where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0117] Referring to FIG. 6, in the object information (OBJ_INFO_t2) for the corresponding object ID (0) obtained as a result of object movement tracking at the first time point (t1), the (x, y, z) coordinate values ​​of the object (OBJ_REAL) are (-0.938, 2.461, 1.82).

[0118] Referring to FIG. 7, a composite radar device (100) is installed in a certain space, and the object movement tracking unit (220) of the composite radar device (100) can detect a situation in which a person, who is an object (OBJ_REAL), enters the detection area at a second time point (t2). Here, the second time point (t2) may be a time point after the first time point (t1).

[0119] Referring to FIG. 7, when referring to the stereoscopic detection result screen (600_t2) and the planar stereoscopic detection result screen (600_PLN_t2) based on object movement tracking at the second time point (t2), the position information of the object (OBJ_REAL) can be detected by using the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component) extracted from the received signal that is received by the transmission signal transmitted from the composite radar device (100) being reflected from the surroundings (object (OBJ_REAL) and objects in its vicinity).

[0120] Referring to FIG. 7, the object image (OBJ_t2) corresponding to the object (OBJ_REAL) whose movement (position change) and trajectory were detected can be displayed on the stereoscopic detection result screen (600_t2) and the planar stereoscopic detection result screen (600_PLN_t2).

[0121] Referring to FIG. 7, in the stereoscopic detection result screen (600_t2), the object image (OBJ_t2) may have a predefined simple stereoscopic shape. For example, the object image (OBJ) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0122] Referring to FIG. 7, in the plane detection result screen (600_PLN_t2), the object image (OBJ_t1) may have a predefined simple plane shape. For example, the object image (OBJ) may be displayed as a plane shape such as a rectangle or a circle.

[0123] Additionally, the stereoscopic detection result screen (600_t2) and the planar stereoscopic detection result screen (600_PLN_t2) may display Doppler points (Pd) where a first Doppler component (general Doppler component) is generated and micro-Doppler points (Pmd) where a second Doppler component (micro-Doppler component) is generated.

[0124] Doppler points (Pd) may be points where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, when an arm or leg moves to walk, the points where the object moves can be indicated as Doppler points (Pd) in situations where there is a change in the object's position and the object's trajectory is detected.

[0125] Micro-Doppler points (Pmd) are points where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0126] It can be seen that the position where the object image (OBJ_t2) is displayed in the stereoscopic detection result screen (600_t2) and the planar stereoscopic detection result screen (600_PLN_t2) of Fig. 7 has changed compared to the position where the object image (OBJ_t1) is displayed in the stereoscopic detection result screen (600_t1) and the planar stereoscopic detection result screen (600_PLN_t1) of Fig. 6.

[0127] Referring to FIG. 7, in the object information (OBJ_INFO_t2) for the corresponding object ID (0) obtained as a result of object movement tracking at the second time point (t2), the (x, y, z) coordinate values ​​of the object (OBJ_REAL) are (-0.637, 3.159, 1.971). The (x, y, z) coordinates of the object (OBJ_REAL) detected at the second time point (t2) are changed compared to the (x, y, z) coordinates of the object (OBJ_REAL) detected at the first time point (t1).

[0128] The object movement tracking unit (220) of the composite radar device (100) can detect information about the trajectory (movement trajectory) of the object (OBJ_REAL) by calculating the change between the (x, y, z) coordinates of the object (OBJ_REAL) detected at the first time point (t1) and the (x, y, z) coordinates of the object (OBJ_REAL) detected at the second time point (t2).

[0129] FIGS. 8 and FIGS. 9 are diagrams illustrating the three-dimensional shape recognition function of a composite radar device (100) according to embodiments of the present disclosure.

[0130] FIG. 8 shows an actual detection situation image (800_REAL_t1) representing a detection area detected at a first time point (t1) by a three-dimensional shape recognition unit (230) of a composite radar device (100) according to embodiments of the present disclosure, and a stereoscopic detection result screen (800_t1) and a planar detection result screen (800_PLN_t1) representing a three-dimensional shape recognition result at a first time point (t1) by a three-dimensional shape recognition unit (230) of a composite radar device (100).

[0131] FIG. 9 shows an actual detection situation image (800_REAL_t2) representing a detection area detected at a second time point (t2) by a three-dimensional shape recognition unit (230) of a composite radar device (100) according to embodiments of the present disclosure, and a stereoscopic detection result screen (600_t2) and a planar detection result screen (800_PLN_t2) representing the three-dimensional shape recognition result at the second time point (t2) by the three-dimensional shape recognition unit (230) of the composite radar device (100).

[0132] Referring to FIG. 8, the composite radar device (100) is installed in a designated space, and the 3D shape recognition unit (230) of the composite radar device (100) can recognize the 3D shape of a person, which is an object (OBJ_REAL), within the space at a first time point (t1).

[0133] Referring to FIG. 8, when referring to the stereoscopic detection result screen (800_t1) and the planar stereoscopic detection result screen (800_PLN_t1) based on three-dimensional shape recognition at the first time point (t1), the position information of the object (OBJ_REAL) can be detected by using the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component) extracted from the received signal that is received by the transmission signal transmitted from the composite radar device (100) being reflected from the surroundings (object (OBJ_REAL) and objects in its vicinity).

[0134] Referring to FIG. 8, the stereoscopic detection result screen (800_t1) and the planar stereoscopic detection result screen (800_PLN_t1) may display a spatial image (801) representing the space and an object image (OBJ_t1) corresponding to the detected object (OBJ_REAL).

[0135] In the stereoscopic detection result screen (800_t1) and the planar stereoscopic detection result screen (800_PLN_t1), an object image (802) representing an object within the space may be further displayed.

[0136] Referring to FIG. 8, in the stereoscopic detection result screen (800_t1), the object image (OBJ_t1) may have a predefined simple stereoscopic shape. For example, the object image (OBJ_t1) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0137] Referring to FIG. 8, in the plane detection result screen (800_PLN_t1), the object image (OBJ_t1) may have a predefined simple plane shape. For example, the object image (OBJ_t1) may be displayed as a plane shape such as a rectangle or a circle.

[0138] Referring to FIG. 8, in the stereoscopic detection result screen (800_t1), each of the spatial image (801) and the object image (802) may have a predefined simple stereoscopic shape. For example, each of the spatial image (801) and the object image (802) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0139] Referring to FIG. 8, in the plane detection result screen (800_PLN_t1), the space image (801) and the object image (802) may each have a predefined simple plane shape. For example, the space image (801) and the object image (802) may each be displayed as a plane shape such as a square or a circle.

[0140] Additionally, the stereoscopic detection result screen (800_t1) and the planar stereoscopic detection result screen (800_PLN_t1) may display Doppler points (Pd) where a first Doppler component (general Doppler component) is generated and micro-Doppler points (Pmd) where a second Doppler component (micro-Doppler component) is generated.

[0141] Doppler points (Pd) may be points where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, when an arm or leg moves to walk, the points where the object moves can be indicated as Doppler points (Pd) in situations where there is a change in the object's position and the object's trajectory is detected.

[0142] Micro-Doppler points (Pmd) are points where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0143] The 3D shape recognition unit (230) can obtain 3D shape recognition information (e.g., height, width, size, etc. of an object) for a single object by grouping 3D points (Pd, Pmd) based on 3D position ((x, y, z) coordinates) and velocity information. That is, the object image (OBJ_t1) can be displayed in an area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL).

[0144] Referring to FIG. 8, in the object information (OBJ_INFO_t1) for the corresponding object ID (0) obtained as a result of 3D shape recognition for the object (OBJ_REAL) at the first time point (t1), the (x, y, z) coordinate values ​​of the corresponding object (OBJ_REAL) are (-0.166, 0.251, 1.646).

[0145] Referring to FIG. 8, in the (x, y, z) coordinate values, the x value may be the center value of the x values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL), and the y value may be the center value of the y values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the z values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the height of the object (OBJ_REAL) at the first time point (t1).

[0146] Referring to FIG. 9, the composite radar device (100) is installed in a designated space, and the 3D shape recognition unit (230) of the composite radar device (100) can recognize the 3D shape of a person, which is an object (OBJ_REAL), within the space at a second time point (t2).

[0147] Referring to FIG. 9, when referring to the stereoscopic detection result screen (800_t2) and the planar stereoscopic detection result screen (800_PLN_t2) based on three-dimensional shape recognition at the second time point (t2), the position information of the object (OBJ_REAL) can be detected by using the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component) extracted from the received signal that is received by the transmission signal transmitted from the composite radar device (100) being reflected from the surroundings (object (OBJ_REAL) and objects in its vicinity).

[0148] Referring to FIG. 9, the stereoscopic detection result screen (800_t2) and the planar stereoscopic detection result screen (800_PLN_t2) may display a spatial image (801) representing the space and an object image (OBJ_t1) corresponding to the detected object (OBJ_REAL).

[0149] In the stereoscopic detection result screen (800_t2) and the planar stereoscopic detection result screen (800_PLN_t2), an object image (802) representing an object within the space may be further displayed.

[0150] Referring to FIG. 9, in the stereoscopic detection result screen (800_t2), the object image (OBJ_t2) may have a predefined simple stereoscopic shape. For example, the object image (OBJ_t2) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0151] Referring to FIG. 9, in the plane detection result screen (800_PLN_t2), the object image (OBJ_t2) may have a predefined simple plane shape. For example, the object image (OBJ_t2) may be displayed as a plane shape such as a rectangle or a circle.

[0152] Referring to FIG. 9, in the stereoscopic detection result screen (800_t2), each of the spatial image (801) and the object image (802) may have a predefined simple stereoscopic shape. For example, each of the spatial image (801) and the object image (802) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0153] Referring to FIG. 9, in the plane detection result screen (800_PLN_t2), the space image (801) and the object image (802) may each have a predefined simple plane shape. For example, the space image (801) and the object image (802) may each be displayed as a plane shape such as a square or a circle.

[0154] Additionally, the stereoscopic detection result screen (800_t2) and the planar stereoscopic detection result screen (800_PLN_t2) may display Doppler points (Pd) where a first Doppler component (general Doppler component) is generated and micro-Doppler points (Pmd) where a second Doppler component (micro-Doppler component) is generated.

[0155] Doppler points (Pd) may be points where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, when an arm or leg moves to walk, the points where the object moves can be indicated as Doppler points (Pd) in situations where there is a change in the object's position and the object's trajectory is detected.

[0156] Micro-Doppler points (Pmd) are points where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0157] The object image (OBJ_t2) can be displayed in the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL).

[0158] Referring to FIG. 9, in the object information (OBJ_INFO_t2) for the corresponding object ID (0) obtained as a result of 3D shape recognition for the object (OBJ_REAL) at the second time point (t2), the (x, y, z) coordinate values ​​of the corresponding object (OBJ_REAL) are (0.073, 0.376, 1.278).

[0159] Referring to FIG. 9, in the (x, y, z) coordinate values, the x value may be the center value of the x values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL), and the y value may be the center value of the y values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the z values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the height of the object (OBJ_REAL) at the first time point (t1).

[0160] Referring to FIGS. 8 and 9, the 3D shape recognition unit (230) can recognize (detect) a change in the shape of the object (OBJ_REAL) by comparing the object information (OBJ_INFO_t1) at the first time point (t1) and the object information (OBJ_INFO_t2) at the second time point (t2). In particular, the 3D shape recognition unit (230) can recognize (detect) a change in the shape and state of the object (OBJ_REAL) by comparing the difference value (a change in height corresponding to the change in the height center value) between the z value (height center value) included in the object information (OBJ_INFO_t1) at the first time point (t1) and the z value (height center value) included in the object information (OBJ_INFO_t2) at the second time point (t2).

[0161] A composite radar device (1000) according to embodiments of the present disclosure can provide application functions (e.g., object count measurement function, abnormal situation detection function, object presence detection function, object abnormality detection function, etc.) by utilizing the functions described above (primary object detection function, secondary object detection function (object movement detection function, object movement detection function, 3D shape recognition function), and object integration detection function).

[0162] Hereinafter, application functions of a composite radar device (100) according to embodiments of the present disclosure are described. Application functions when a plurality of objects exist in a space where the composite radar device (100) is installed are described with reference to FIGS. 10 to 12, and application functions when a single object exists in a space where the composite radar device (100) is installed are described with reference to FIGS. 13 to 16.

[0163] FIG. 10 is an additional block diagram of a composite radar device (100) according to embodiments of the present disclosure. FIG. 11 is a diagram showing the object count measurement function of the composite radar device (100) according to embodiments of the present disclosure. FIG. 12 is a diagram showing the abnormal situation detection function of the composite radar device (100) according to embodiments of the present disclosure.

[0164] Referring to FIG. 10, a composite radar device (100) according to embodiments of the present disclosure may further include an installation space information management unit (1000) that sets, stores, and updates space information for a space where the composite radar device (100) is installed.

[0165] For example, spatial information may include at least one of size information for the space where the composite radar device (100) is installed, object information existing in the space where the composite radar device (100) is installed, and characteristic information for the space where the composite radar device (100) is installed.

[0166] At least one of the primary object detection unit (110), secondary object detection unit (120), and object integration detection unit (130) included in the composite radar device (100) can perform the corresponding detection operation by referring to spatial information.

[0167] The object integration detection unit (130) can determine the number of objects based on object integration information, assign an object ID to each object, and store the object ID and object integration information in conjunction.

[0168] Referring to FIG. 10, the composite radar device (100) according to embodiments of the present disclosure may further include an object count measuring unit (1010) that measures the number of objects in the space where the composite radar device (100) is installed, based on the number of object IDs and spatial information.

[0169] Referring to FIG. 10, a composite radar device (100) according to embodiments of the present disclosure may further include an abnormal situation detection unit (1020) that detects an abnormal situation in a space where the composite radar device (100) is installed, based on abnormal situation determination information identified based on object integration information associated with an object ID and spatial information. For example, the abnormal situation determination information mentioned above may include at least one of the position, trajectory, velocity, state, height center value, and height change amount of the height center value of the corresponding object for each object ID.

[0170] When an abnormal situation is detected, the abnormal situation detection unit (1020) can perform notification processing (e.g., notification processing such as sound, vibration, etc.) in a preset manner.

[0171] When an abnormal situation is detected, the abnormal situation detection unit (1020) can use a communication module included in the composite radar device (100) to send an emergency situation notification message to a recipient with pre-set emergency contact information.

[0172] Referring to FIG. 11, a composite radar device (100) according to embodiments of the present disclosure can perform an object count measurement function, which is an example of an application function, by utilizing a basic detection function. Here, the basic detection function may include a primary object detection function, a secondary object detection function (object movement detection function, object movement detection function, 3D shape recognition function), and an object integration detection function.

[0173] FIG. 11 shows a real-world image (1100_REAL) in which a composite radar device (100) according to embodiments of the present disclosure performs an object count measurement function, and a three-dimensional execution result screen (1100) and a planar execution result screen (1100_PLN) showing the results of the execution of the object count measurement function.

[0174] Referring to FIG. 11, a composite radar device (100) is installed in a space, and the composite radar device (100) can measure the number of objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) present in the space based on the execution result of a basic detection function.

[0175] Referring to FIG. 11, when referring to the three-dimensional execution result screen (1100) and the planar execution result screen (1100_PLN) according to the execution of the object count measurement function, the location information of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) can be detected by using the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component) extracted from the received signal that is received by the transmission signal transmitted from the composite radar device (100) being reflected from the surroundings (objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) and objects in the vicinity thereof).

[0176] Referring to FIG. 11, in the stereoscopic execution result screen (1100) and the planar execution result screen (1100_PLN), a spatial image (1101) representing the space and object images (OBJ1, OBJ2, OBJ3) corresponding to the detected objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) may be displayed.

[0177] Referring to FIG. 11, in the stereoscopic execution result screen (1100), each of the object images (OBJ1, OBJ2, OBJ3) may have a predefined simple stereoscopic shape. For example, each of the object images (OBJ1, OBJ2, OBJ3) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0178] Referring to FIG. 11, in the planar execution result screen (1100_PLN), each of the object images (OBJ1, OBJ2, OBJ3) may have a predefined simple planar shape. For example, each of the object images (OBJ1, OBJ2, OBJ3) may be displayed as a planar shape such as a rectangle or a circle.

[0179] Referring to FIG. 11, the stereoscopic execution result screen (1100) and the planar execution result screen (1100_PLN) may display Doppler points (Pd) where a first Doppler component (general Doppler component) is generated and micro-Doppler points (Pmd) where a second Doppler component (micro-Doppler component) is generated.

[0180] Doppler points (Pd) may be points where at least one large movement (e.g., movement of a part of an object) is detected among the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL). For example, when an arm or leg moves to walk, the points where there is movement of the object in a situation where there is a change in the object's position and the object's trajectory is detected may be indicated as Doppler points (Pd).

[0181] Micro-Doppler points (Pmd) may be points where at least one minute movement (e.g., slight shaking, rotation, vibration, small movement of a part of an object, etc.) of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) is detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0182] The object images (OBJ1, OBJ2, OBJ3) can be displayed in the area defined by the detected Doppler points (Pd) and micro-Doppler points (Pmd) for each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL).

[0183] Referring to FIG. 11, the composite radar device (100) can generate object information (OBJ_INFO) for objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) as a result of executing a basic detection function and an object count measurement function.

[0184] Object information (OBJ_INFO) may include an object ID for each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) and corresponding object location information (x, y, z) coordinate values. In the object location information (x, y, z) coordinate values, the x value may be the center value of the x values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object, and the y value may be the center value of the y values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object. The z value may be the center value of the z values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object. The z value may be the center value of the height of the object.

[0185] Referring to FIG. 12, a composite radar device (100) according to embodiments of the present disclosure can perform an abnormal situation detection function, which is an example of an application function, by utilizing a basic detection function. Here, the basic detection function may include a primary object detection function, a secondary object detection function (object motion detection function, object movement detection function, 3D shape recognition function), and an object integration detection function.

[0186] The composite radar device (100) can perform an abnormal situation detection function along with an object count measurement function.

[0187] FIG. 12 shows a real-world image (1200_REAL) in which a composite radar device (100) according to embodiments of the present disclosure performs an abnormal situation detection function, and a three-dimensional execution result screen (1200) and a planar execution result screen (1200_PLN) showing the results of the execution of the abnormal situation detection function.

[0188] Looking at the actual situation image (1200_REAL), in the space where the composite radar device (100) is installed, an abnormal situation occurs in which three people corresponding to the first to third objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) assault one person corresponding to the fourth object (OBJ4_REAL).

[0189] Referring to FIG. 12, the composite radar device (100) can detect an abnormal situation in the space based on the execution result of the basic detection function. In the example of FIG. 12, the abnormal situation is a situation where three people assault one person.

[0190] Referring to FIG. 12, when referring to the stereoscopic execution result screen (1200) and the planar execution result screen (1200_PLN) according to the execution of the abnormal situation detection function, the state of each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) can be detected by using the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component) extracted from the received signal that is received by the surroundings (objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) and objects in the vicinity) of the transmission signal transmitted from the composite radar device (100).

[0191] The composite radar device (100) monitors the change in height corresponding to the change in the height center value of each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) and the change in position corresponding to the change in position, and can detect that a sudden abnormal situation (e.g., a situation in which a person corresponding to the fourth object (OBJ4_REAL)) has occurred in relation to the object (e.g., a situation in which a person corresponding to the fourth object (OBJ4_REAL) is assaulted, a situation in which a person corresponding to the fourth object (OBJ4_REAL) suddenly collapses, etc.) has occurred when the height center value and / or position of at least one of the objects (e.g., the fourth object (OBJ4_REAL)) changes rapidly.

[0192] Referring to FIG. 12, in the stereoscopic execution result screen (1200) and the planar execution result screen (1200_PLN), a spatial image (1201) representing the space and object images (OBJ1, OBJ2, OBJ3, OBJ4) corresponding to the detected objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) may be displayed.

[0193] Referring to FIG. 12, in the stereoscopic execution result screen (1100), each of the object images (OBJ1, OBJ2, OBJ3, OBJ4) may have a predefined simple stereoscopic shape. For example, each of the object images (OBJ1, OBJ2, OBJ3, OBJ4) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0194] If the size of at least one of the object images (OBJ1, OBJ2, OBJ3, OBJ4) (e.g., OBJ4) suddenly decreases, it may be detected that an abnormal situation has occurred. In this way, when an abnormal situation is detected, the size of at least one of the object images (OBJ1, OBJ2, OBJ3, OBJ4) (e.g., OBJ4) may be different from the size of the others (e.g., OBJ1, OBJ2, OBJ3).

[0195] Referring to FIG. 12, in the planar execution result screen (1100_PLN), each of the object images (OBJ1, OBJ2, OBJ3, OBJ4) may have a predefined simple planar shape. For example, each of the object images (OBJ1, OBJ2, OBJ3, OBJ4) may be displayed as a planar shape such as a rectangle or a circle.

[0196] Referring to FIG. 12, the stereoscopic execution result screen (1200) and the planar execution result screen (1200_PLN) may display Doppler points (Pd) where a first Doppler component (general Doppler component) is generated and micro-Doppler points (Pmd) where a second Doppler component (micro-Doppler component) is generated.

[0197] Doppler points (Pd) may be points where at least one large movement (e.g., movement of a part of an object) is detected among the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL). For example, when an arm or leg moves to walk, the points where there is movement of the object in a situation where there is a change in the object's position and the object's trajectory is detected may be indicated as Doppler points (Pd).

[0198] Micro-Doppler points (Pmd) may be points where at least one minute movement (e.g., slight shaking, rotation, vibration, small movement of a part of an object, etc.) of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) is detected. For example, if a person blinks their eyes, moves their skin, or moves their glasses, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0199] The object images (OBJ1, OBJ2, OBJ3, OBJ4) can be displayed in the area defined by the detected Doppler points (Pd) and micro-Doppler points (Pmd) for each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL).

[0200] Referring to FIG. 12, the composite radar device (100) can generate object information (OBJ_INFO) for objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) as a result of executing a basic detection function and an abnormal situation detection function.

[0201] Object information (OBJ_INFO) may include an object ID for each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) and corresponding object location information (x, y, z) coordinate values. In the object location information (x, y, z) coordinate values, the x value may be the center value of the x values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object, and the y value may be the center value of the y values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object. The z value may be the center value of the z values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object. The z value may be the center value of the height of the object. In the object information where the object ID is 0, it can be seen that the z value is significantly smaller than the z value of other objects. The abnormal situation detection unit (1020) of the composite radar device (100) can recognize that the object with object ID 0 corresponds to the fallen fourth object (OBJ4) and is in a situation where it is being beaten by other people nearby.

[0202] FIG. 13 is an additional block diagram of a composite radar device (100) according to embodiments of the present disclosure. FIG. 14 and FIG. 15 are diagrams illustrating an object presence detection function of a composite radar device (100) according to embodiments of the present disclosure. FIG. 16 is a diagram illustrating an object anomaly detection function of a composite radar device (100) according to embodiments of the present disclosure.

[0203] Referring to FIG. 13, a composite radar device (100) according to embodiments of the present disclosure may further include an installation space information management unit (1000) that sets, stores, and updates spatial information for a space where the composite radar device (100) is installed. For example, the spatial information may include at least one of size information for a space where the composite radar device (100) is installed, object information existing in a space where the composite radar device (100) is installed, and characteristic information for a space where the composite radar device (100) is installed.

[0204] At least one of the primary object detection unit (110), secondary object detection unit (120), and object integration detection unit (130) included in the composite radar device (100) can perform the corresponding detection operation by referring to spatial information.

[0205] The object integration detection unit (130) can determine the number of objects based on object integration information, assign an object ID to each object, and store the object ID and object integration information in conjunction.

[0206] Referring to FIG. 13, a composite radar device (100) according to embodiments of the present disclosure may further include an object presence detection unit (1310) that detects whether an object exists in a space where the composite radar device (100) is installed, based on object integration information and spatial information.

[0207] Referring to FIG. 13, a composite radar device (100) according to embodiments of the present disclosure may further include an object abnormality detection unit (1320) that detects whether an object is abnormal based on object abnormality determination information identified based on object integration information when it is determined by an object presence detection unit (1310) that an object exists in the space where the composite radar device (100) is installed. For example, the object abnormality determination information may include at least one of the object's location, trajectory, velocity, state, height center value, and height change amount of the height center value.

[0208] If the object abnormality detection unit (1320) determines that the object is in an abnormal state (abnormal situation), it can perform notification processing (e.g., notification processing such as sound, vibration, etc.) in a preset manner.

[0209] When the object abnormality detection unit (1320) determines that the object is in an abnormal state (abnormal situation), it can use a communication module included in the composite radar device (100) to send an emergency situation notification message to a recipient with pre-set emergency contact information.

[0210] FIG. 14 shows an actual detection situation image (1400_REAL_t1) representing a detection area detected at a first time point (t1) by an object presence detection unit (1310) of a composite radar device (100) according to embodiments of the present disclosure, and a stereoscopic detection result screen (1400_t1) and a planar detection result screen (1400_PLN_t1) representing the object presence detection result at the first time point (t1) by the object presence detection unit (1310).

[0211] FIG. 15 shows an actual detection situation image (1400_REAL_t2) representing a detection area detected at a second time point (t2) by an object presence detection unit (1310) according to embodiments of the present disclosure, and a three-dimensional detection result screen (1400_t2) and a planar detection result screen (1400_PLN_t2) representing the object presence detection result at the second time point (t2) by a three-dimensional shape recognition unit (230) of a composite radar device (100).

[0212] Referring to FIG. 14, the composite radar device (100) is installed in a designated space (e.g., a restroom, a room, etc.), and the object presence detection unit (1310) of the composite radar device (100) can detect whether an object is present in the space at a first time point (t1).

[0213] Referring to FIG. 14, when referring to the stereoscopic detection result screen (1400_t1) and the planar stereoscopic detection result screen (1400_PLN_t1) based on object presence detection at the first time point (t1), the location information of an object can be detected by using the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component) extracted from the received signal that is received by the transmission signal transmitted from the composite radar device (100) being reflected from the surroundings (object (OBJ_REAL) and objects in its vicinity).

[0214] Referring to FIG. 14, since there is no object in the space at the first time point (t1), only the space image (1401) representing the space and the object image (1420) of an object (e.g., a toilet) existing in the space are displayed on the stereoscopic detection result screen (800_t1) and the planar stereoscopic detection result screen (800_PLN_t1), and no object image is displayed.

[0215] Therefore, the Doppler points (Pd) where the first Doppler component (general Doppler component) is generated and the micro-Doppler points (Pmd) where the second Doppler component (micro-Doppler component) is generated are not displayed on the stereoscopic detection result screen (1400_t1) and the planar stereoscopic detection result screen (1400_PLN_t1).

[0216] Referring to FIG. 14, the object information (OBJ_INFO_t1) at the first time point (t1) does not include any object ID and its corresponding information.

[0217] Referring to FIG. 15, when referring to the stereoscopic detection result screen (1400_t2) and the planar stereoscopic detection result screen (1400_PLN_t2) based on object presence detection at the second time point (t2), the location information of the object (OBJ_REAL) can be detected by using the first Doppler component (general Doppler component) and the second Doppler component (micro-Doppler component) extracted from the received signal that is received by the transmission signal transmitted from the composite radar device (100) being reflected from the surroundings (object (OBJ_REAL) and objects in its vicinity).

[0218] Referring to FIG. 15, the stereoscopic detection result screen (1400_t2) and the planar stereoscopic detection result screen (1400_PLN_t2) display a spatial image (1401) representing the space and an object image (1402) within the space, and an object image (OBJ_t2) corresponding to the detected object (OBJ_REAL) may be displayed.

[0219] Referring to FIG. 15, in the stereoscopic detection result screen (1400_t2), the object image (OBJ_t2) may have a predefined simple stereoscopic shape. For example, the object image (OBJ_t2) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0220] Referring to FIG. 15, in the plane detection result screen (1400_PLN_t2), the object image (OBJ_t2) may have a predefined simple plane shape. For example, the object image (OBJ_t2) may be displayed as a plane shape such as a rectangle or a circle.

[0221] Referring to FIG. 15, in the stereoscopic detection result screen (1400_t2), each of the spatial image (1401) and the object image (1402) may have a predefined simple stereoscopic shape. For example, each of the spatial image (1401) and the object image (1402) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0222] Referring to FIG. 15, in the plane detection result screen (1400_PLN_t2), the space image (1401) and the object image (1402) may each have a predefined simple plane shape. For example, the space image (1401) and the object image (1402) may each be displayed as a plane shape such as a square or a circle.

[0223] Additionally, in the stereoscopic detection result screen (1400_t2) and the planar stereoscopic detection result screen (1400_PLN_t2), Doppler points (Pd) where a first Doppler component (general Doppler component) is generated and micro-Doppler points (Pmd) where a second Doppler component (micro-Doppler component) is generated may be displayed.

[0224] Doppler points (Pd) may be points where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, when an arm or leg moves to walk, the points where the object moves can be indicated as Doppler points (Pd) in situations where there is a change in the object's position and the object's trajectory is detected.

[0225] Micro-Doppler points (Pmd) are points where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0226] The object image (OBJ_t2) can be displayed in the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL).

[0227] Referring to FIG. 15, in the object information (OBJ_INFO_t2) for the object (OBJ_REAL) obtained as a result of detecting the presence of the object at the second time point (t2), the (x, y, z) coordinate values ​​of the object (OBJ_REAL) are (0.073, 0.376, 1.278).

[0228] Referring to FIG. 15, in the (x, y, z) coordinate values, the x value may be the center value of the x values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL), and the y value may be the center value of the y values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the z values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the height of the object (OBJ_REAL) at the first time point (t1).

[0229] Referring to FIGS. 14 and 15, the object presence detection unit (1310) can determine that an object exists at the second time point (t2) because object information (OBJ_INFO_t2) occurred at the second time point (t2).

[0230] When the object presence detection unit (1310) is operating, the object abnormality detection unit (1320) can operate together. The object abnormality detection unit (1320) can determine whether the (x, y, z) coordinate values, which are the location information of an object (OBJ_REAL) included in the object information (OBJ_INFO_t2) which is the object detection result at the second time point (t2) in the object presence detection unit (1310), are included in the normal range.

[0231] In particular, the object abnormality detection unit (1320) can determine whether the object (OBJ_REAL) is in a normal state or an abnormal state by determining whether the z value (height center value) included in the object information (OBJ_INFO_t2) at the second time point (t2) falls within a predefined normal value range.

[0232] The object abnormality detection unit (1320) can determine that the object (OBJ_REAL) is in a normal state if the (x, y, z) coordinate values, which are location information of the object (OBJ_REAL), fall within the normal range.

[0233] The object abnormality detection unit (1320) can determine that the object (OBJ_REAL) is in an abnormal state (abnormal situation) if the (x, y, z) coordinate values, which are location information of the object (OBJ_REAL), are not included in the normal range.

[0234] Referring to FIG. 16, the object abnormality detection unit (1320) can detect, as a result of detecting the presence of an object at a third time point (t3), that an object (OBJ_REA) exists in the space, but the state of the object (OBJ_REAL) is abnormal.

[0235] Referring to FIG. 16, when assuming that at the third time point (t3) following the second time point (t2), a person who is the object (OBJ_REAL) collapses or loses consciousness in the space (e.g., restroom, room, etc.), the object abnormality detection unit (1320) can determine that the state of the object (OBJ_REAL) is abnormal based on the object integration information of the object (OBJ_REAL).

[0236] For example, the object abnormality detection unit (1320) can determine, based on the object integration information of the object (OBJ_REAL), that the height center value (z value or its corresponding value) of the object (OBJ_REAL) at the third time point (t3) has become smaller than the height center value (z value or its corresponding value) of the object (OBJ_REAL) at the second time point (t2), and determine that the state of the object (OBJ_REAL) is an abnormal state.

[0237] In this regard, at the third time point (t3), the stereoscopic detection result screen (1400_t3) and the planar stereoscopic detection result screen (1400_PLN_t3) continuously display a spatial image (1401) representing the space and an object image (1402) within the space, and an object image (OBJ_t2) corresponding to the detected object (OBJ_REAL) can also be continuously displayed.

[0238] However, in the stereoscopic detection result screen (1400_t3), the object image (OBJ_t3) at the third time point (t3) may have a smaller size than the object image (OBJ_t2) at the second time point (t2). This may mean that the object (OBJ_REAL) has fallen.

[0239] Referring to FIG. 16, in the object information (OBJ_INFO_t3) for the object (OBJ_REAL) obtained as a result of detecting the presence of the object at the third time point (t3), the (x, y, z) coordinate values ​​of the object (OBJ_REAL) are (0.107, 0.692, 0.84).

[0240] The object abnormality detection unit (1320) can compare the (x, y, z) coordinate values ​​(0.073, 0.376, 1.278), which are location information of the object (OBJ_REAL) at the second time point (t2), with the coordinate values ​​(0.107, 0.692, 0.84), which are location information of the object (OBJ_REAL) at the third time point (t3), and if there is a significant difference beyond a threshold level, it can determine that the object (OBJ_REAL) is in an abnormal state (abnormal situation).

[0241] That is, when comparing the (x, y, z) coordinate values ​​(0.073, 0.376, 1.278), which are the location information of the object (OBJ_REAL) at the second time point (t2), with the coordinate values ​​(0.107, 0.692, 0.84), which are the location information of the object (OBJ_REAL) at the third time point (t3), the x and y values ​​have increased, and the z value has decreased. This may mean that the height of the area corresponding to the person, which is the object (OBJ_REAL), has decreased and widened.

[0242] Meanwhile, the object abnormality detection unit (1320) may set different object information defining the abnormal state (abnormal situation) of an object according to the space information set in the installation space information management unit (1000). For example, the height center value of an object or the amount of change in height of an object indicating the abnormal state of an object may differ depending on the space information.

[0243] FIG. 17 is a block diagram of a multi-Doppler-based object detection device (1700) according to embodiments of the present disclosure.

[0244] Referring to FIG. 17, a multi-Doppler-based object detection device (1700) according to embodiments of the present disclosure may include: a multi-Doppler component extraction unit (1705) that transmits a transmission signal and receives a signal reflected from the surroundings as a reception signal, and extracts a plurality of Doppler components corresponding to different types from the reception signal; a first detection unit (1710) that obtains first detection information including two-dimensional position information of an object existing in the surroundings based on the plurality of Doppler components; a second detection unit (1720) that obtains second detection information including velocity information of the object along with two-dimensional position information based on the plurality of Doppler components; and a third detection unit (1730) that obtains third detection information including a height center value of the object along with the second detection information based on the plurality of Doppler components.

[0245] The first detection unit (1710) is called the first radar unit, the second detection unit (1720) is called the second radar unit (3D radar unit), and the third detection unit (1730) can be called the third radar unit (4D radar unit).

[0246] The multi-Doppler component extraction unit (1705) can extract a Doppler component from a received signal as a first Doppler component among a plurality of Doppler components during a first time period, and extract a Doppler component from a received signal as a second Doppler component among a plurality of Doppler components during a second time period. Here, the second time period may be set to be longer than the first time period.

[0247] The third sensing unit (1730) can estimate a height center value from height values ​​detected for an object and recognize the three-dimensional shape of the object based on changes over time for the estimated height center value.

[0248] The third sensing unit (1730) can determine the state of the object based on the recognition result of the object's three-dimensional shape.

[0249] The third detection unit (1730) can determine an abnormal situation of the object by monitoring changes in the object's state over time.

[0250] A multi-Doppler-based object detection device (1700) according to embodiments of the present disclosure may be a composite radar device (100). A first detection unit (1710) and a second detection unit (1720) may correspond to a first object detection unit (110), and a third detection unit (1730) may correspond to a second object detection unit (120).

[0251] A display device according to embodiments of the present disclosure can be described as follows.

[0252] A composite radar device according to embodiments of the present disclosure may include: a primary object detection unit configured to transmit a transmission signal and receive a signal reflected from the surroundings as a reception signal, extract at least one of a first Doppler component and a second Doppler component from the reception signal, and acquire primary object information about an object in the surroundings based on the extraction result; a secondary object detection unit configured to estimate the height center value of an object based on the primary object information and acquire secondary object information based on the estimation result; and an object integration detection unit configured to acquire object integration information based on the secondary object information.

[0253] The first object detection unit can extract a Doppler component from a received signal as a first Doppler component during a first time period, and extract a Doppler component from a received signal as a second Doppler component during a second time period. Here, the second time period may be set to be longer than the first time period.

[0254] The primary object detection unit determines the type of object as one of a Doppler object having a speed greater than or equal to a predefined threshold speed and a micro-Doppler object having a speed less than or equal to the threshold speed or being stationary, and if the Doppler object or micro-Doppler object has more height components, the Doppler object or micro-Doppler object can be determined as a 4-dimensional Doppler object.

[0255] The first object detection unit obtains first object information including information about the height of an object based on the extraction result, and the second object detection unit can estimate the center value of the height of an object from the height of an object identified from the first object information.

[0256] The secondary object detection unit can calculate a height change amount indicating the degree to which the estimated height center value changes over time, monitor the height change amount to identify the state of the object, and obtain secondary object information including information about the state.

[0257] The second object detection unit may include: an object movement tracking unit that obtains the position and trajectory of an object using the distance and velocity of an object identified from the first object information, and a first Doppler component and a second Doppler component, and obtains second object information including information about the position and trajectory; and a three-dimensional shape recognition unit that obtains second object information including three-dimensional shape recognition information by recognizing the three-dimensional shape of an object based on the distance, velocity, and height of an object identified from the first object information and a height center value.

[0258] The 3D shape recognition unit can estimate the height center value of an object based on primary object information, calculate a height change amount indicating the degree to which the estimated height center value changes over time, and monitor the height change amount to determine the state of the object.

[0259] The second object detection unit may further include an object motion detection unit that detects whether an object is moving based on the type of Doppler component extracted from the first Doppler component and the second Doppler component from the received signal.

[0260] The object motion detection unit detects that the object is in a state of no movement when only the second Doppler component among the first Doppler component and the second Doppler component is extracted from the received signal, and detects that the object is in a state of movement when the first Doppler component is extracted from the received signal or when both the first Doppler component and the second Doppler component are extracted.

[0261] The composite radar device may further include an installation space information management unit that sets, stores, and updates spatial information for the space where the composite radar device is installed.

[0262] Spatial information may include at least one of size information for the space where the composite radar device is installed, information on objects existing in the space where the composite radar device is installed, and characteristic information for the space where the composite radar device is installed.

[0263] At least one of the primary object detection unit, the secondary object detection unit, and the object integration detection unit can refer to spatial information.

[0264] The object integration detection unit can determine the number of objects based on object integration information, assign an object ID to each object, and store the object ID in conjunction with the object integration information.

[0265] A composite radar device according to embodiments of the present disclosure may further include an object count measuring unit that measures the number of objects in a space where the composite radar device is installed, based on the number of object IDs.

[0266] A composite radar device according to embodiments of the present disclosure may further include an abnormal situation detection unit that detects an abnormal situation in a space where the composite radar device is installed, based on abnormal situation determination information identified based on object integration information associated with an object ID and spatial information.

[0267] The abnormal situation judgment information may include at least one of the position, trajectory, velocity, state, height center value, and height change amount of the height center value for each object ID.

[0268] A composite radar device according to embodiments of the present disclosure may further include an object presence detection unit that detects whether an object is present in a space where the composite radar device is installed, based on object integration information.

[0269] A composite radar device according to embodiments of the present disclosure may further include an object abnormality detection unit that detects whether an object is abnormal based on object abnormality determination information identified based on object integration information when it is determined by an object presence detection unit that an object exists in the space where the composite radar device is installed.

[0270] Object abnormality determination information may include at least one of the object's position, trajectory, velocity, state, height center value, and height change amount of the height center value.

[0271] A multi-Doppler-based object detection device according to embodiments of the present disclosure may include: a multi-Doppler component extraction unit that extracts a plurality of Doppler components corresponding to different types from a received signal that is received by reflecting a transmitted signal from the surroundings; a first detection unit that obtains first detection information including two-dimensional position information of an object existing in the surroundings based on the plurality of Doppler components; a second detection unit that obtains second detection information including velocity information of the object along with two-dimensional position information based on the plurality of Doppler components; and a third detection unit that obtains third detection information including a height center value of the object along with the second detection information based on the plurality of Doppler components.

[0272] The multi-Doppler component extraction unit can extract a Doppler component from a received signal as a first Doppler component among a plurality of Doppler components during a first time period, and extract a Doppler component from a received signal as a second Doppler component among a plurality of Doppler components during a second time period. The second time period may be set to be longer than the first time period.

[0273] The third sensing unit can estimate the height center value from the height values ​​detected for the object and recognize the three-dimensional shape of the object based on the change in the height center value over time.

[0274] The third sensing unit can determine the state of the object based on the recognition result of the object's three-dimensional shape.

[0275] The third sensing unit can monitor changes in the object's state over time and determine abnormal conditions of the object.

[0276] According to the embodiments of the present disclosure described above, a composite radar device and a multi-Doppler-based object detection device that perform object detection using heterogeneous composite Doppler components can be provided.

[0277] According to embodiments of the present disclosure, a composite radar device and a multi-Doppler-based object detection device can be provided that can precisely and rapidly detect various information, states, or actions (movements) of an object using heterogeneous composite Doppler components.

[0278] According to embodiments of the present disclosure, a composite radar device and a multi-Doppler-based object detection device can be provided that can perform object detection without infringing on privacy or exposing personal information because they are not based on images.

[0279] According to embodiments of the present disclosure, a composite radar device and a multi-Doppler-based object detection device capable of providing various application functions using radar-based object detection technology can be provided.

[0280] The foregoing description is merely an illustrative explanation of the technical concept of the present disclosure, and those skilled in the art to which the present disclosure pertains may make various modifications and variations within the scope of the essential characteristics of the present disclosure. Furthermore, the embodiments disclosed in the present disclosure are intended to explain, not limit, the technical concept of the present disclosure, and thus the scope of the technical concept of the present disclosure is not limited by these embodiments.

[0281]

[0282] [Project ID] 2320000021

[0283] [Assignment No.] 00403630

[0284] [Department Name] National Police Agency

[0285] [Name of Project Management (Specialized) Agency] Science and Public Security Promotion Center

[0286] [Research Project Name] Autonomous Driving Technology Development Innovation Project

[0287] [Research Project Title] Development of an Autonomous Patrol Service for Proactive Traffic Accident Prevention and Response

[0288] [Contribution Rate] 100%

[0289] [Name of Project Performing Organization] Vaida Co., Ltd.

[0290]

[0291] CROSS-REFERENCE TO RELATED APPLICATION

[0292] This patent application claims priority pursuant to Section 119(a) of the U.S. Patent Act (35 USC §119(a)) to Korean Patent Application No. 10-2024-0196059 filed on December 24, 2024, the entire contents of which are incorporated by reference into this patent application. Additionally, this patent application claims priority in countries other than the United States for the same reasons as above, and the entire contents of which are incorporated by reference into this patent application.

Claims

1. A primary object detection unit configured to transmit a transmission signal and receive a signal reflected from the surroundings as a reception signal, extract at least one of a first Doppler component and a second Doppler component from the reception signal, and acquire primary object information about an object in the surroundings based on the extraction result; A secondary object detection unit configured to estimate the height center value of the object based on the primary object information and to acquire secondary object information based on the estimation result; and A composite radar device comprising an object integration detection unit configured to acquire object integration information based on the above secondary object information.

2. In Paragraph 1, The above primary object detection unit is, During the first time period, a Doppler component is extracted from the received signal as the first Doppler component, and A composite radar device that extracts a Doppler component from the received signal as the second Doppler component during a second time period, wherein the second time period is set to be longer than the first time period.

3. In Paragraph 1, The above primary object detection unit is, The type of the above object is determined to be one of a Doppler object having a speed greater than or equal to a predefined threshold speed, and a micro-Doppler object having a speed less than or equal to the threshold speed or being stationary. A composite radar device that determines the Doppler object or the micro-Doppler object as a 4-dimensional Doppler object when the Doppler object or the micro-Doppler object has an additional height component.

4. In Paragraph 1, The above primary object detection unit obtains primary object information including information about the height of the object based on the extraction result, and A composite radar device in which the secondary object detection unit estimates the height center value of the object from the height of the object identified from the primary object information.

5. In Paragraph 1, The above secondary object detection unit is, An object movement tracking unit that obtains the position and trajectory of the object using the distance and velocity of the object identified from the first object information, and the first Doppler component and the second Doppler component, and obtains the second object information including information regarding the position and the trajectory; and A composite radar device comprising a 3D shape recognition unit that recognizes the 3D shape of the object based on the distance, speed, and height of the object identified from the 1st object information and the height center value, and obtains the 2nd object information including 3D shape recognition information.

6. In Paragraph 5, The above secondary object detection unit is, A composite radar device further comprising an object motion detection unit that detects whether the object is moving based on the type of Doppler component extracted from the first Doppler component and the second Doppler component from the received signal.

7. In Paragraph 6, The object movement detection unit above is, If only the second Doppler component among the first Doppler component and the second Doppler component is extracted from the received signal, the object is detected as being in a state of no movement, and A composite radar device that detects that the object is in a moving state when the first Doppler component is extracted from the received signal or when both the first Doppler component and the second Doppler component are extracted.

8. In Paragraph 1, The object integration detection unit determines the number of objects based on the object integration information, assigns an object ID to each object, and stores the object ID in conjunction with the object integration information, in a composite radar device.

9. In Paragraph 8, A composite radar device further comprising an object count measuring unit that measures the number of objects in a space where the composite radar device is installed, based on the number of object IDs.

10. In Paragraph 8, A composite radar device further comprising an object presence detection unit that detects whether an object exists in a space where the composite radar device is installed, based on the object integration information above.

11. A multi-Doppler component extraction unit that extracts multiple Doppler components corresponding to different types from a received signal that is received by reflecting a transmitted signal from the surroundings; A first sensing unit that acquires first sensing information including two-dimensional position information of an object existing in the surroundings based on the above plurality of Doppler components; A second sensing unit that acquires second sensing information including velocity information of the object along with the two-dimensional position information based on the plurality of Doppler components; and A multi-Doppler-based object detection device comprising a third detection unit that acquires third detection information including the height center value of the object along with the second detection information, based on the plurality of Doppler components.

12. In Paragraph 11, The above multi-Doppler component extraction unit During the first time period, a Doppler component is extracted from the received signal as the first Doppler component among the plurality of Doppler components, and A multi-Doppler-based object detection device that, during a second time period, extracts a Doppler component from the received signal as a second Doppler component among the plurality of Doppler components, and the second time period is set to be longer than the first time period.

Images