Point cloud-based object detection device

Abstract

Embodiments of the present disclosure can provide a point cloud-based object detection device capable of accurately detecting whether a fall has occurred by detecting an object based on a point cloud.

Description

Technology Field

[0001] The embodiments of the present disclosure relate to a point cloud-based object detection device. Background Technology

[0002] Nowadays, various object detection technologies are being developed in diverse fields to detect objects such as people and objects. These diverse object detection technologies employ various detection methods. For example, various object detection technologies may include infrared-based detection technology, image or photo-based detection technology, or radio frequency identification (RFID)-based detection technology.

[0003] However, all object detection technologies currently under development may have several issues. For example, infrared detection can lead to erroneous judgments due to the influence of shielding or temperature. As another example, image or photo-based detection technologies can also lead to misjudgments due to the influence of lighting or human posture, and raise significant concerns regarding privacy infringement. Furthermore, current object detection technologies may not meet required detection accuracy standards and may have limitations in their application across various fields. The problem to be solved

[0004] Embodiments of the present disclosure may provide a point cloud-based object detection device that performs object detection based on a point cloud.

[0005] Embodiments of the present disclosure may provide a point cloud-based object detection device that detects the fall of an object based on a point cloud.

[0006] Embodiments of the present disclosure may provide a point cloud-based object detection device that performs object detection using heterogeneous composite Doppler components.

[0007] Embodiments of the present disclosure can provide a point cloud-based object detection device capable of precisely and rapidly detecting various information, states, or actions (movements) of an object using heterogeneous composite Doppler components.

[0008] Since the embodiments of the present disclosure are not based on images, a point cloud-based object detection device capable of performing object detection without infringing on privacy or exposing personal information can be provided.

[0009] Embodiments of the present disclosure can provide a point cloud-based object detection device capable of providing various application functions using radar-based object detection technology.

[0010] Embodiments of the present disclosure can provide a point cloud-based object detection device capable of accurately detecting whether a person, as an object, has fallen by combining composite Doppler components.

[0011] Embodiments of the present disclosure can provide a point cloud-based object detection device capable of accurately detecting whether there is an abnormality in the breathing state of a person object using composite Doppler components.

[0012] The problems of the embodiments of the present disclosure are not limited to those mentioned in this specification, and other problems not mentioned will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0013] A point cloud-based object detection device according to embodiments of the present disclosure comprises: a transmitting antenna device that transmits a transmission signal; a receiving antenna device that receives a signal reflected from at least one object located in the vicinity of the transmission signal as a receiving signal; a primary object detection unit that obtains primary object information from the receiving signal, wherein, based on the receiving signal, the primary object information includes distance information, azimuth information, and elevation angle information corresponding to each of the detection points, which are points where the transmission signal is reflected from the object, and obtains three-dimensional coordinate information for each of the detection points through the distance information, azimuth information, and elevation angle information for each of the detection points, and further obtains primary object information including three-dimensional coordinate information for each of the detection points; a secondary object detection unit that, based on the primary object information, generates a three-dimensional point cloud including detection points, clusters the detection points included in the three-dimensional point cloud into object units to distinguish and recognize each object, and obtains secondary object information for each object, wherein the secondary object information includes the centroid coordinates for each object based on the three-dimensional coordinate information for the detection points in each object; and for each object It may include an object integration detection unit that determines whether each object has fallen based on changes in the center of gravity coordinates and detects the determination result as object integration information.

[0014] A point cloud-based object detection device according to embodiments of the present disclosure comprises: a primary object detection unit that obtains primary object information from a received signal reflected from at least one object located in the vicinity of a transmitted signal, wherein, based on the received signal, the primary object information includes distance information, azimuth information, and elevation angle information corresponding to each of the detection points, which are points where the transmitted signal is reflected from the object, and obtains three-dimensional coordinate information for each of the detection points through the distance information, azimuth information, and elevation angle information for each of the detection points, and further obtains primary object information including three-dimensional coordinate information for each of the detection points; a secondary object detection unit that generates a three-dimensional point cloud including detection points based on the primary object information, clusters the detection points included in the three-dimensional point cloud into object units to distinguish and recognize each object, and obtains secondary object information for each object, wherein the secondary object information includes the center of gravity coordinates for each object based on the three-dimensional coordinate information for the detection points in each object; and determines whether each object has fallen based on a change in the center of gravity coordinates for each object, and integrates the determination result into objects It may include an object integration detection unit that detects as information. Effects of the invention

[0015] According to embodiments of the present disclosure, a point cloud-based object detection device that performs object detection based on a point cloud can be provided.

[0016] According to embodiments of the present disclosure, a point cloud-based object detection device for detecting the fall of an object based on a point cloud can be provided.

[0017] According to embodiments of the present disclosure, a point cloud-based object detection device and a multi-Doppler-based object detection device that perform object detection using heterogeneous composite Doppler components can be provided.

[0018] According to embodiments of the present disclosure, a point cloud-based object detection device capable of precisely and rapidly detecting various information, states, or actions (movements) of an object using heterogeneous composite Doppler components can be provided.

[0019] According to embodiments of the present disclosure, a point cloud-based object detection device can be provided that can perform object detection without infringing on privacy or exposing personal information because it is not based on images.

[0020] According to embodiments of the present disclosure, a point cloud-based object detection device capable of providing various application functions using radar-based object detection technology can be provided.

[0021] According to embodiments of the present disclosure, a point cloud-based object detection device capable of accurately detecting whether a person, which is an object, has fallen by combining composite Doppler components can be provided.

[0022] According to embodiments of the present disclosure, a point cloud-based object detection device capable of accurately detecting abnormalities in the breathing state of a person object using composite Doppler components can be provided.

[0023] The effects of the embodiments of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description in the claims. Brief explanation of the drawing

[0024] The content of this disclosure will be more fully understood from the detailed description and accompanying drawings provided below, which are provided solely for illustrative purposes and are not intended to limit the content of this disclosure. FIG. 1 is a block diagram of a point cloud-based object detection device according to embodiments of the present disclosure. FIG. 2a is a detailed block diagram of a point cloud-based object detection device according to embodiments of the present disclosure. FIG. 2b illustrates the operation procedure of a point cloud-based object detection device according to embodiments of the present disclosure. FIG. 3a illustrates the point cloud-based object detection function and application detection function of a point cloud-based object detection device according to embodiments of the present disclosure. FIG. 3b is a diagram illustrating the breath analysis function of a point cloud-based object detection device according to embodiments of the present disclosure. FIGS. 4 and 5 are diagrams illustrating the object motion detection function of a point cloud-based object detection device according to embodiments of the present disclosure. FIGS. 6 and 7 are diagrams illustrating the object movement tracking function of a point cloud-based object detection device according to embodiments of the present disclosure. FIGS. 8 and 9 are diagrams illustrating the three-dimensional shape recognition function of a point cloud-based object detection device according to embodiments of the present disclosure. FIG. 10 is an additional block diagram of a point cloud-based object detection device according to embodiments of the present disclosure. FIG. 11 is a diagram showing the object count measurement function of a point cloud-based object detection device according to embodiments of the present disclosure. FIG. 12 is a diagram showing the abnormal situation detection function of a point cloud-based object detection device according to embodiments of the present disclosure. FIG. 13 is an additional block diagram of a point cloud-based object detection device according to embodiments of the present disclosure. FIGS. 14 and 15 are diagrams illustrating the object presence detection function of a point cloud-based object detection device according to embodiments of the present disclosure. FIG. 16 is a diagram showing the object anomaly detection function of a point cloud-based object detection device according to embodiments of the present disclosure. FIG. 17 is a block diagram of a point cloud-based object detection device according to embodiments of the present disclosure. Specific details for implementing the invention

[0025] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, the same components may have the same reference numeral as much as possible, even if they are shown in different drawings. Furthermore, in describing the present disclosure, if it is determined that a detailed description of related known components or functions may obscure the essence of the present disclosure, such detailed description may be omitted. Where terms such as "comprising," "having," or "consisting of" are used in this specification, other parts may be added unless "only" is used. Where a component is expressed in the singular, it may include a plural unless there is a special explicit description otherwise.

[0026] Additionally, terms such as first, second, A, B, (a), (b), etc., may be used to describe the components of the present disclosure. These terms are used merely to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by such terms.

[0027] In describing the positional relationship of components, where it is stated that two or more components are "connected," "combined," or "joined," it should be understood that while the two or more components may be directly "connected," "combined," or "joined," they may also be "connected," "combined," or "joined" with other components "intervened." Here, the other components may be included in one or more of the two or more components that are "connected," "combined," or "joined" with one another.

[0028] In describing the temporal flow relationship regarding components, methods of operation, or methods of production, for example, when the temporal or sequential relationship is described using "after," "following," "next," or "before," it may include cases where the relationship is not continuous unless "immediately" or "directly" is used.

[0029] Meanwhile, where numerical values ​​or corresponding information regarding a component (e.g., levels, etc.) are mentioned, even without separate explicit notation, the numerical values ​​or corresponding information may be interpreted as including a range of error that may occur due to various factors (e.g., process factors, internal or external shocks, noise, etc.).

[0030] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0031] FIG. 1 is a block diagram of a point cloud-based object detection device (100) according to embodiments of the present disclosure.

[0032] Referring to FIG. 1, a point cloud-based object detection device (100) according to embodiments of the present disclosure is a device that detects objects based on a point cloud, and may be a device that can precisely and quickly detect various information, states, or actions (movements) of an object by selectively using heterogeneous composite Doppler components as needed.

[0033] Each object detected by the point cloud-based object detection device (100) according to the embodiments of the present disclosure may have one of the characteristics of a general Doppler object having a speed greater than a predefined threshold speed or motion characteristics greater than a predetermined level, and the characteristics of a micro-Doppler object having a speed less than a predefined threshold speed or motion characteristics less than a predetermined level or being stationary.

[0034] Additionally, each object detected by the point cloud-based object detection device (100) according to the embodiments of the present disclosure may have one of the characteristics of a 3D Doppler object having distance information, azimuth information, and elevation information, and a 4D Doppler object having additional velocity information along with distance information, azimuth information, and elevation information.

[0035] Velocity information in a 4-dimensional Doppler object refers to velocity in the form of vectors in the x-axis, y-axis, and z-axis directions, rather than the 1-dimensional Doppler velocity (relative velocity) of the object approaching or departing from a point cloud-based object detection device (100). A 3-dimensional Doppler object may have 1-dimensional Doppler velocity.

[0036] The heterogeneous composite Doppler component according to the embodiments of the present disclosure may include various types of Doppler components, such as a general Doppler component and a micro-Doppler component.

[0037] Doppler components refer to frequency shifts that occur when estimating velocity information of a moving object (object) through a point cloud-based object detection device (100). Doppler components may include general Doppler components and micro-Doppler components, and general Doppler components and micro-Doppler components may be distinguished according to the magnitude and characteristics of the movement.

[0038] The general Doppler component may refer to a Doppler frequency shift caused by the translational motion of an entire object. For example, a general Doppler component may occur when a person walks or a car moves in a certain direction. A point cloud-based object detection device (100) can be used to determine how quickly an object approaches or moves away from the point cloud-based object detection device (100). The general Doppler component may have a frequency spectrum of a single distinct center frequency component (narrow band). The general Doppler component can be used to estimate the overall velocity and direction of an object. However, the general Doppler component is not suitable for detecting the interior of an object or detailed movements of an object (e.g., arm swinging).

[0039] Micro-Doppler components can be Doppler components generated by sub-motions within an object, such as rotation, vibration, or shaking. For example, micro-Doppler components can be generated by the movement of a person's arms and legs, helicopter rotors, drone propellers, heartbeats, and respiration. Micro-Doppler components can provide information on the vibration velocity or period of local movements. Micro-Doppler components have the characteristic of spreading across various frequencies and can possess broadband components that vary over time. Micro-Doppler components can be utilized in biosignal detection, behavior recognition, and human identification. Micro-Doppler components can be used to identify the movement of specific details (e.g., the chest) within the same object.

[0040] For example, when a person (object) walks, the general Doppler corresponds to the speed at which the entire person moves forward (e.g., 1.5 m / s), and the micro-Doppler corresponds to small changes in speed (vibrations) when the feet touch the ground and fall, or when the arms swing back and forth.

[0041] A point cloud-based object detection device (100) according to embodiments of the present disclosure may include a transmitting antenna device (TX_ANT), a receiving antenna device (RX_ANT), a primary object detection unit (110), a secondary object detection unit (120), and an object integration detection unit (130), etc. Here, the point cloud-based object detection device (100) according to embodiments of the present disclosure is also referred to as a multi-Doppler-based object detection device in that it performs an object detection function using multi-Doppler components. Alternatively, circuit configurations including the primary object detection unit (110), the secondary object detection unit (120), and the object integration detection unit (130) included in the point cloud-based object detection device (100) are also referred to as a point cloud-based object detection device (100).

[0042] delete

[0043] delete

[0044] A transmitting antenna device (TX_ANT) includes at least one transmitting antenna and can transmit a radar signal (transmit signal). A receiving antenna device (RX_ANT) may include a plurality of receiving antennas. The receiving antenna device (RX_ANT) may have a multiple receiving antenna array structure in which a plurality of receiving antennas are arranged.

[0045] The primary object detection unit (110) may be configured to transmit a transmission signal through a transmitting antenna device (TX_ANT), receive a signal reflected from the surroundings as a receiving signal through a receiving antenna device (RX_ANT), extract at least one of a general Doppler component and a micro-Doppler component from the receiving signal, and obtain primary object information about an object in the surroundings based on the extraction result.

[0046] For example, primary object information may include distance information, velocity information, azimuth information, and elevation angle information (also called height information) for each of the detection points corresponding to the points where the transmitted signal is reflected. Primary object information may further include signal intensity information indicating the intensity of the signal reflected and received at each detection point. Primary object information may further include signal-to-noise ratio (SNR) information for the signal reflected and received at each detection point. Primary object information may be obtained from the general Doppler component and the micro-Doppler component, respectively.

[0047] A signal reflected from at least one object in the vicinity of a transmission signal transmitted from a transmission antenna device (TX_ANT) may arrive at multiple receiving antennas of a receiving antenna device (RX_ANT) with slightly different time delays and different phases.

[0048] The primary object detection unit (110) calculates the time difference and phase difference between signals received from multiple receiving antennas, and based on this, estimates the azimuth angle (horizontal direction angle) and elevation angle (vertical direction angle) of each reflected point (detection point frame), thereby obtaining Angle of Arrival (AoA) information including azimuth angle information and elevation angle information. The azimuth angle and elevation angle are two angles from different perspectives used to represent the position of a detection point in three-dimensional space. The azimuth angle is an angle indicating where it is in the horizontal direction, and may be an angle indicating which direction the detection point is in on the horizontal plane. The elevation angle is an angle indicating where it is in the up / down direction, and may be an angle indicating how much the detection point is above or below the horizontal line.

[0049] The primary object detection unit (110) can further acquire three-dimensional coordinate information representing three-dimensional coordinates (x, y, z) for each of the detection points as additional primary object information based on distance information, azimuth information, and elevation information for each of the detection points.

[0050] As mentioned above, the primary object information may include distance information, velocity information, azimuth information, and elevation information (height information) for each of the detection points, and three-dimensional coordinate information for each of the detection points. Additionally, the primary object information may further include signal strength information and signal-to-noise ratio information for each of the detection points.

[0051] The secondary object detection unit (120) can generate three-dimensional point cloud information for a three-dimensional point cloud as secondary object information by using at least one of the velocity information, signal strength information, and signal-to-noise ratio information included in the primary object information, along with three-dimensional coordinate information obtained from the distance information, azimuth information, and elevation angle information included in the primary object information.

[0052] A three-dimensional point cloud is a collection of detection points and can include all detection points for at least one object existing in the vicinity.

[0053] For example, a 3D point cloud may include multiple detection points for a first object, multiple detection points for a second object, multiple detection points for a third object, and so on. Here, surrounding objects such as the first object, the second object, and the third object may include people, cars, surrounding stationary objects, or backgrounds. Additionally, surrounding objects such as the first object, the second object, and the third object may be classified into static objects, micro-movement objects, and dynamic objects. Here, a dynamic object refers to an object having large movement (including movement), a static object refers to an object with no movement at all, and a micro-movement object refers to an object having micro-movement (vibration, shaking, rotation, etc.).

[0054] When generating three-dimensional point cloud information, the secondary object detection unit (120) can generate three-dimensional point cloud information by separating the three-dimensional point cloud into a static point cloud with no or almost no movement and a dynamic point cloud with movement.

[0055] The secondary object detection unit (120) can perform tracking and clustering processing after generating three-dimensional point cloud information as secondary object information. The secondary object detection unit (120) can perform tracking and clustering processing on multiple detection points included in the three-dimensional point cloud to generate at least one cluster by grouping the detection points, and can further generate cluster information for at least one cluster as secondary object information. Here, each cluster corresponds to a single identical object and may be a set of detection points that form a single identical object.

[0056] The secondary object detection unit (120) can separate and track multiple clusters by performing tracking and clustering processing on multiple detection points included in a three-dimensional point cloud. Here, the multiple clusters may include a human body cluster, which is a collection of detection points for a human body (object).

[0057] After the 3D point cloud information (3D point cloud data) is generated, the secondary object detection unit (120) calculates the distance between detection points based on the 3D coordinate information of each detection point, and groups all detection points included in the 3D point cloud into at least one cluster (group) based on the distance between detection points.

[0058] The secondary object detection unit (120) can track each detection point while considering the mobility of the object (i.e., while tracking the change in 3D coordinate information of each detection point), calculate the distance between detection points or the amount of change in distance between detection points, and group all detection points included in the 3D point cloud into at least one cluster (group) based on the distance between detection points or the amount of change in distance between detection points.

[0059] For example, the secondary object detection unit (120) can group detection points with a separation distance less than or equal to a certain distance value into the same cluster, and separate (classify) detection points with a separation distance greater than the certain distance value into different clusters.

[0060] As another example, the secondary object detection unit (120) can group detection points that are similar in terms of the change in distance from each other or below a certain level into the same cluster, and separate (classify) detection points that have different changes in distance from each other or have differences that exceed a certain level into different clusters.

[0061] Accordingly, for example, detection points included in a 3D point cloud can be grouped into a single cluster. For another example, detection points included in a 3D point cloud can be divided into two or more clusters and grouped.

[0062] The secondary object detection unit (120) may designate one cluster (one group) as one object and assign object identification information (object ID) to each object. The secondary object detection unit (120) may generate cluster information for each of at least one cluster, including various information such as object identification information, object location, object size, and shape, through clustering processing of detection points included in a three-dimensional point cloud. That is, the cluster information may include information such as the number of clusters (i.e., the number of objects), the location, size, and shape of each cluster.

[0063] The second object detection unit (120) can obtain the centroid coordinate information of each of the N (N is a natural number greater than or equal to 2) detection points forming each cluster (Xc, Yc, Zc) by using cluster information including the number of clusters (i.e., number of objects), the location, size, and shape of each cluster, and the three-dimensional coordinate information (Xi, Yi, Zi, i=1, 2, … , N) of each detection point forming each cluster, and calculate the average three-dimensional coordinate value (Xc, Yc, Zc) of each detection point forming the object (cluster) as the centroid coordinate (Xc, Yc, Zc) of the object.

[0064]

[0065] In Equation 1, N is the total number of detection points constituting an object, and (Xc, Yc, Zc) are the 3D coordinates for the object and may be the 3D coordinates for the centroid of the object. In (Xc, Yc, Zc), Xc is the x-axis coordinate value of the centroid of the object, Yc is the y-axis coordinate value of the centroid of the object, and Zc is the z-axis coordinate value of the centroid of the object. The 3D coordinates of the i-th detection point among the N detection points constituting the object are (xi, yi, zi).

[0066] In mathematical formula 1, Xc is the sum of the x-axis coordinate values ​​of each of the N detection points divided by N, Yc is the sum of the y-axis coordinate values ​​of each of the N detection points divided by N, and Zc is the sum of the z-axis coordinate values ​​of each of the N detection points divided by N.

[0067] In mathematical formula 1, Zc is the z-axis coordinate value of the object's center of gravity, which can be said to be the height center value of the object.

[0068] The secondary object detection unit (120) can calculate the ratio of the number of detection points (Nupper) located above a predetermined height to the total number of detection points (N) forming an object as the upper density of each object by using the 3D coordinate information and cluster information of each detection point. The upper density (ρ) of an object can be expressed by the following mathematical formula 2.

[0069]

[0070] In the above mathematical formula 2, ρ is the upper density, which is the ratio of the upper area of ​​an object to the total area of ​​an object, N is the number of all detection points constituting an object, and Nupper is the number of detection points located above a predetermined height among all detection points constituting an object.

[0071] As mentioned above, the secondary object information may include three-dimensional point cloud information, which is a set of all detection points, and cluster information, which is a set of detection points for each object.

[0072] The secondary object information may further include information on the center of gravity coordinates (Xc, Yc, Zc) for each object and the center of height value (Zc) for each object.

[0073] Secondary object information may further include upper density information of each object.

[0074] Information on the center of gravity coordinates (Xc, Yc, Zc) for each object, information on the center of height value (Zc) for each object, and information on the upper density for each object may be included in the 3D shape recognition information for each object.

[0075] The secondary object detection unit (120) can obtain secondary object information that includes additional state information for each object as more diverse information for each object.

[0076] Secondary object information may further include additional state information about the object. Additional state information about the object may include motion information, movement information, and additional 3D shape recognition information about the object.

[0077] For example, motion information for an object may include information about small changes, such as minute vibrations, rotations, or tremors of the object or a part thereof. Here, information about small changes, such as minute vibrations, rotations, or tremors, may include information about minute chest movements of the person object (information that can estimate the breathing cycle or breathing state). Secondary object information may be obtained from micro-Doppler components.

[0078] For example, movement information for an object may include information about the movement trajectory of the object.

[0079] For example, 3D shape recognition information for an object may include not only the centroid coordinate information and height centroid value information of the object, but also size information (volume information) for the object. The size information (volume information) of an object may be the area (volume) of the region (space) occupied by the detected points (detection points) for the object.

[0080] The object integration detection unit (130) may be configured to obtain object integration information for each object based on primary object information and secondary object information, that is, by analyzing primary object information and secondary object information. Here, the object integration information may include primary object information and secondary object information, and may further include additional object information obtained based on primary object information and secondary object information.

[0081] For example, additional object information may include various meaningful state information about the object, such as fall information, beating information, and breathing state information about the object (human body).

[0082] For example, fall information about an object may be information indicating whether the object fell, tipped over, or was crushed within the detection space.

[0083] For example, beating information about an object may be information indicating whether the object is being beaten (assaulted) by another object within the detection space.

[0084] For example, the respiration state information may include at least one of information indicating that respiration is in a normal state (respiration normal state information), information indicating that respiration is in a rapidly changing state (respiration rapidly changing state information), and information indicating that respiration is in a stopped state (respiration stopped state information).

[0085] The primary object detection unit (110) can extract a Doppler component as a general Doppler component from a received signal during a first time period defined for general Doppler component extraction, and extract a Doppler component as a micro-Doppler component from a received signal during a second time period defined for micro-Doppler component extraction. Here, the second time period may be set to be longer than the first time period.

[0086] In the embodiments of the present disclosure, the micro-Doppler component is an extended concept of the Doppler effect and refers to a frequency variation component caused by minute movements of an object (e.g., vibration, rotation, shaking, etc.). The general Doppler component refers to a frequency variation component due to a change in velocity that occurs when an object approaches or moves away from the point cloud-based object detection device (100). In contrast, the micro-Doppler component may refer to a frequency variation component that reflects even more precise movements of the object than the general Doppler component.

[0087] For example, a point cloud-based object detection device (100) can extract a micro-Doppler component from a received signal by performing a Short-Time Fourier Transform (STFT), a Wavelet Transform, and time-frequency analysis processing on a received signal (received signal) that is longer than a first time for general Doppler component extraction and is predefined for a second time.

[0088] In a point cloud-based object detection device (100) according to embodiments of the present disclosure, primary object information may include distance information, speed information, azimuth information, and elevation information (height information) for each of the detection points, and may further include three-dimensional coordinate information calculated from the distance information, azimuth information, and elevation information for each of the detection points. Additionally, primary object information may further include signal strength information and signal-to-noise ratio information for each of the detection points.

[0089] In a point cloud-based object detection device (100) according to embodiments of the present disclosure, secondary object information may include three-dimensional point cloud information which is a set of all detection points and cluster information which is a set of detection points for each object.

[0090] Additionally, the secondary object information may further include information on the center of gravity coordinates (Xc, Yc, Zc) for each object and the center of height value (Zc) for each object. The secondary object information may further include upper density information for each object. The secondary object information may further include additional state information for the object. The additional state information for the object may include motion information, translation information, and additional 3D shape recognition information for the object.

[0091] An object according to embodiments of the present disclosure may be classified into one of a general Doppler object and a micro-Doppler object according to the magnitude (degree) of the object's movement through at least one of a primary object detection unit (110) and a secondary object detection unit (120). For example, a general Doppler object may be an object having a speed greater than a predefined threshold speed or having movement characteristics greater than a predetermined level. A micro-Doppler object may be an object having a speed less than a threshold speed, having minute movement less than a predetermined level, or being stationary.

[0092] The secondary object detection unit (120) can classify the object into one of a general Doppler object and a micro-Doppler object by using both the primary object information obtained from the primary object detection unit (110) and the secondary object information generated therefrom.

[0093] The secondary object detection unit (120) can classify an object identified from cluster information, etc. included in the secondary object information as a general Doppler object if the amount of movement or the magnitude of movement of the object is above a certain level, and classify it as a micro Doppler object if the amount of movement or the magnitude of movement of the object is below a certain level, based on movement information and motion information, etc. of the object included in the secondary object information.

[0094] For example, the secondary object detection unit (120) can calculate the average value of the changes in three-dimensional coordinate information for the detection points forming the object as the amount of movement or movement size (degree of movement) of the object, and obtain movement information or movement information based on the calculated value.

[0095] When the amount of movement or the magnitude of movement of the object exceeds a certain level, the secondary object detection unit (120) calculates the average value of the three-dimensional coordinate information of the detection points forming the object as the center of gravity coordinate information of the object, and can obtain movement information or motion information based on the amount of change over time of the calculated value.

[0096] An object according to embodiments of the present disclosure may be classified into one of a 3D Doppler object and a 4D Doppler object according to the type of object information through at least one of a primary object detection unit (110) and a secondary object detection unit (120). For example, a 3D Doppler object may be an object having distance information, azimuth information, and elevation information, and a 4D Doppler object may be an object having distance information, azimuth information, elevation information, and velocity information. Here, the distance information, azimuth information, elevation information, and velocity information may be information included in the primary object information.

[0097] The secondary object detection unit (120) can classify the object into one of a three-dimensional Doppler object and a four-dimensional Doppler object by using both the primary object information obtained from the primary object detection unit (110) and the secondary object information generated therefrom.

[0098] The secondary object detection unit (120) can estimate the height center value (z-axis value in the center of gravity coordinate value) of the object from the height (elevation angle) of the detection points forming the object identified from the primary object information.

[0099] delete

[0100] delete

[0101] The secondary object detection unit (120) can calculate a height change amount indicating the degree to which the estimated height center value changes over time, determine the state of the object based on the calculation result, and obtain secondary object information including information about the state.

[0102] For example, the point cloud-based object detection device (100) according to the embodiments of the present disclosure may utilize frequency modulated continuous waves (FMCW) technology and may provide accurate measurement results regarding the distance and speed of an object. The point cloud-based object detection device (100) according to the embodiments of the present disclosure may use a chirp signal, and the frequency of the chirp signal may increase linearly over time. The presence of an object may be further estimated based on the phase difference between two chirp signals in a radar echo wave. The point cloud-based object detection device (100) according to the embodiments of the present disclosure may perform multi-object detection.

[0103] A point cloud-based object detection device (100) according to embodiments of the present disclosure can detect characteristic information of an object (meaning object information such as location, height, and state) by utilizing the frequency difference between a transmitted signal and a received signal reflected from an object. Here, the object may be a person or an object, and may have periodic / non-periodic motion or movement, be stationary or moving, or have a sudden change of state.

[0104] By analyzing radar Doppler frequency characteristics, the momentum of an object (person or thing) can be monitored to determine the object's state.

[0105] A received signal reflected from at least one object surrounding a point cloud-based object detection device (100) may have a frequency different from the transmitted signal frequency. Here, the difference in frequency between the transmitted signal and the received signal may be called the Doppler frequency. The Doppler frequency measured by the point cloud-based object detection device (100) can be used as a parameter to obtain motion information, including the velocity of the object.

[0106] A point cloud-based object detection device (100) according to embodiments of the present disclosure can comprehensively analyze not only a general Doppler signal (general Doppler component) but also a micro-Doppler signal (micro-Doppler component) in order to accurately determine the motion state (movement state) of an object (person or thing). The general Doppler frequency is a Doppler frequency generated by the macroscopic movement (movement, trajectory) of an object, and the micro-Doppler frequency may be a Doppler frequency generated by the microscopic movement (fine movement) occurring between the macroscopic movements (movement, trajectory).

[0107] Micro-Doppler signals can be used to obtain information on motion (movement) such as shaking, vibration, or rotation of an object (person or thing) or small movements of the object, and micro-Doppler signals can be widely utilized for object (person or thing) recognition, motion characteristics, or motion patterns analysis. A point cloud-based object detection device (100) according to embodiments of the present disclosure can detect various motion characteristics of an object through spectrogram analysis of micro-Doppler components (micro-Doppler signals).

[0108] A point cloud-based object detection device (100) according to embodiments of the present disclosure can collect one-dimensional micro-Doppler signals for a moving object in one plane using single MIMO (Multiple Input Multiple Output) or beamforming technology. Additionally, the transmitting antenna and the receiving antenna of the point cloud-based object detection device (100) according to embodiments of the present disclosure may be configured spatially together or spatially separated to separate the transmission and reception signals of the radar.

[0109] A point cloud-based object detection device (100) according to embodiments of the present disclosure can detect object information by collecting 3D micro-Doppler frequency information about an object using a plurality of beamforming systems and configuring the collected information into time-series data.

[0110] A point cloud-based object detection device (100) according to embodiments of the present disclosure can acquire object information by collecting micro-Doppler signals over time and receiving three-dimensional time series data as input, which has frequency changes over time as features.

[0111] A point cloud-based object detection device (100) according to embodiments of the present disclosure can perform object detection based on a three-dimensional Doppler signal (three-dimensional micro-Doppler signal) to further acquire object information including not only distances of objects but also speed information.

[0112] A point cloud-based object detection device (100) according to embodiments of the present disclosure can perform object detection based on a 4-dimensional Doppler signal (3-dimensional micro-Doppler signal) to obtain object information including not only distance or speed information of an object, but also the height of the object.

[0113] Hereinafter, the basic detection function of the point cloud-based object detection device (100) according to the embodiments of the present disclosure and the application function utilizing it will be described in more detail.

[0114] FIG. 2a is a detailed block diagram of a point cloud-based object detection device (100) according to embodiments of the present disclosure.

[0115] Regarding object classification based on the magnitude of object movement, etc., each object in the vicinity of the point cloud-based object detection device (100) according to the embodiments of the present disclosure may be one of a general Doppler object and a micro-Doppler object at a specific point in time.

[0116] Regarding object classification according to the type of object information, each object in the vicinity of the point cloud-based object detection device (100) according to the embodiments of the present disclosure may be one of a 3D Doppler object and a 4D Doppler object at a specific point in time.

[0117] Accordingly, the detection result obtained when the primary object detection unit (110) of the point cloud-based object detection device (100) according to the embodiments of the present disclosure performs a detection operation at a specific point in time may include at least one of a detection result for a general Doppler object, a detection result for a micro-Doppler object, a detection result for a three-dimensional Doppler object, and a detection result for a four-dimensional Doppler object.

[0118] Information obtained according to the detection results (general Doppler object detection result, micro Doppler object detection result, 3D Doppler object detection result, 4D Doppler object detection result) in which the primary object detection unit (110) performs a detection operation at a specific point in time is called primary object information.

[0119] The secondary object detection unit (120) of the point cloud-based object detection device (100) according to embodiments of the present disclosure may include an object movement tracking unit (220) and a three-dimensional shape recognition unit (230), etc., in order to acquire various secondary object information.

[0120] The object movement tracking unit (220) can obtain the position and trajectory of an object using the distance and velocity of the object identified from the primary object information, and the general Doppler component and the micro-Doppler component, and obtain secondary object information including information about the position and trajectory of the object.

[0121] The 3D shape recognition unit (230) can recognize the 3D shape of an object based on the distance, speed, and height of the object identified from the 1D object information and the height center value, and obtain 2D object information including 3D shape recognition information. Here, having a 3D shape of an object may mean that the object has a certain height. The 3D shape recognition information may include the height or height center value of the object, and may include information about the position or size of the object. The size of an object may be the area of ​​the region occupied by the points (positions) detected for an object.

[0122] Referring to FIG. 2a, the secondary object detection unit (120) may further include an object motion detection unit (210). The object motion detection unit (210) can detect whether an object is moving based on the type of Doppler component extracted from a received signal, which is either a general Doppler component or a micro-Doppler component.

[0123] For example, the movement of an object may refer to small changes, such as minute vibrations, rotations, or tremors of the object or a part thereof, without a change in the object's position (i.e., movement). As another example, the movement of an object may refer to minute changes in position (i.e., movement) where the change in the object's position falls below a predefined threshold.

[0124] The object integration detection unit (130) can obtain object integration information for the object by integrating all output information (secondary object information) of each of the object movement detection unit (210), object movement tracking unit (220), and 3D shape recognition unit (230). For example, the object integration information may include information about the location, trajectory, and state of the object, but may also include additional state information about the object (e.g., fall state information, beating state information, breathing state information, etc.).

[0125] FIG. 2b illustrates the operation procedure of a point cloud-based object detection device (100) according to embodiments of the present disclosure.

[0126] Referring to FIG. 2b, a point cloud-based object detection device (100) according to embodiments of the present disclosure may include a first object detection step (S110), a second object detection step (S120), and an integrated object detection step (S130).

[0127] In the first object detection step (S110), the first object detection unit (110) may be configured to transmit a transmission signal and receive a signal reflected from the surroundings as a reception signal, extract at least one of a general Doppler component and a micro-Doppler component from the reception signal, and obtain first object information about an object in the surroundings based on the extraction result.

[0128] In the first object detection step (S110), the first object detection unit (110) can extract a Doppler component from a received signal as a general Doppler component during a first time period, and extract a Doppler component from a received signal as a micro-Doppler component during a second time period. Here, the second time period for extracting the micro-Doppler component can be set to be longer than the first time period for extracting the general Doppler component.

[0129] Each of the general Doppler component and the micro-Doppler component may include a Doppler frequency, which is the frequency at which there is a difference between the transmitted signal and the received signal. The general Doppler component may include a Doppler frequency (general Doppler frequency) generated by the movement of an object (macroscopic movement), and the micro-Doppler component may include a Doppler frequency (micro-Doppler frequency) generated by the movement of an object (microscopic movement).

[0130] The primary object information obtained from the primary object detection unit (110) is information obtained primarily by receiving signals reflected from the surroundings, and may include various information (basic radar detection information) for each of the detection points corresponding to the reflected points. For example, the primary object information may include distance information, speed information, azimuth information, and elevation information (height information) for each of the detection points, and may further include three-dimensional coordinate information for each detection point obtained from the distance information, azimuth information, and elevation information. The primary object information may further include signal strength information and signal-to-noise ratio information for each of the detection points.

[0131] The primary object detection unit (110) can determine that an object is a Doppler object if the object in the vicinity has a speed greater than or equal to a predefined threshold speed.

[0132] The primary object detection unit (110) can determine that an object in the vicinity has a speed below a threshold speed or is a stationary micro-Doppler object (S111). Here, the micro-Doppler object may be an object with small movements such as fine vibration, rotation, or shaking.

[0133] The primary object detection unit (110) can determine that an object determined to be a Doppler object or a micro-Doppler object has an additional height component, and that object is a 4-dimensional Doppler object (S112).

[0134] In the second object detection step (S120), the point cloud-based object detection device (100) can calculate three-dimensional coordinate information for detection points as additional first object information based on the first object information (basic radar detection information) obtained in the first object detection step (S110), generate a three-dimensional point cloud which is a collection of detection points, and cluster the detection points into object units and perform tracking processing based on the generated three-dimensional point cloud to further obtain actual object information in object units as second object information.

[0135] For example, secondary object information may include 3D point cloud information, which is information on a set of all detection points, and cluster information, which is information on a set of detection points for each object. Secondary object information may further include centroid coordinates (Xc, Yc, Zc) information for each object and height centroid value (Zc) information for each object. Secondary object information may further include upper density information for each object. Secondary object information may further include additional state information for the object. Additional state information for the object may include motion information, movement information, and additional 3D shape recognition information for the object.

[0136] The second object detection step (S120) may include an object movement detection step (S121), an object movement tracking step (S122), and a three-dimensional shape recognition step (S123).

[0137] In the object motion detection step (S121), the object motion detection unit (210) can detect whether an object is moving based on the type of Doppler component extracted from the received signal, which is either a general Doppler component or a micro-Doppler component. For example, the movement of an object may refer to a small change, such as a minute vibration, rotation, or shaking of the object or a part thereof, without a change in the object's position (i.e., movement). For another example, the movement of an object may refer to a minute change in position (i.e., movement) where the change in the object's position is below a predefined threshold value.

[0138] The object motion detection unit (210) can detect that the object is in a state of no movement if only the micro-Doppler component among the general Doppler component and the micro-Doppler component is extracted from the received signal.

[0139] The object motion detection unit (210) can detect that the object is in a state of motion when a normal Doppler component is extracted from the received signal or when both a normal Doppler component and a micro-Doppler component are extracted.

[0140] In other words, when the object motion detection unit (210) detects that the object is not moving, the received signal may include only the micro-Doppler component. When the object motion detection unit (210) detects that the object is moving, the received signal may include both the general Doppler component and the micro-Doppler component.

[0141] In the object movement tracking step (S122), the object movement tracking unit (220) can obtain the position and trajectory of an object using the distance and velocity of the object identified from the primary object information, and the general Doppler component and the micro-Doppler component, and obtain secondary object information including information about the position and trajectory of the object.

[0142] Here, the position of an object may refer to coordinates (2D coordinates) on a 2D plane, and the trajectory of an object may refer to changes in coordinates (2D coordinates) on a 2D plane, and may also be referred to as the movement trajectory or movement of an object.

[0143] In the 3D shape recognition step (S123), the 3D shape recognition unit (230) can recognize the 3D shape of an object based on the distance, speed, and height of the object identified from the 1D object information and the height center value, and obtain 2D object information including 3D shape recognition information.

[0144] Here, the fact that an object has a three-dimensional shape may mean that the object has a certain height. The three-dimensional shape recognition information may include the height or height center value of the object, and may include information about the position or size of the object. The size of an object may be the area of ​​the region occupied by the detected points (positions) for an object.

[0145] For example, the 3D shape recognition unit (230) can collect information in the form of a 3D point cloud based on 3D position ((x, y, z) coordinates) and velocity information to obtain 3D shape recognition information of an object (e.g., height, width, size, etc. of the object).

[0146] In the 3D shape recognition step (S123), the 3D shape recognition unit (230) estimates the height center value of the object based on primary object information, calculates a height change amount indicating the degree to which the estimated height center value changes over time, and monitors the height change amount to determine the state of the object.

[0147] In the three-dimensional shape recognition step (S123), the three-dimensional shape recognition unit (230) can generate three-dimensional shape recognition information by detecting the three-dimensional shape or three-dimensional shape change of an object based on at least one of a height center value, a height change amount, and a state.

[0148] For example, the 3D shape recognition information may include information on at least one of the height center value, height change amount, and state.

[0149] In the integrated object detection step (S130), the object integration detection unit (130) can obtain object integration information for the object by integrating all output information (secondary object information) of each of the object movement detection step (S121), object movement tracking step (S122), and 3D shape recognition step (S123). For example, the object integration information may include information about the location, trajectory, and state of the object.

[0150] FIG. 3a shows the point cloud-based object detection function and the application detection function of the point cloud-based object detection device (100) according to embodiments of the present disclosure.

[0151] Referring to FIG. 3, a point cloud-based object detection device (100) according to embodiments of the present disclosure can perform a point cloud-based object detection function.

[0152] A point cloud-based object detection method of a point cloud-based object detection device (100) according to embodiments of the present disclosure may include a radar signal reception and preprocessing step (S310), a three-dimensional point cloud generation step (S320), and a tracking and clustering step (S330).

[0153] The radar signal reception and preprocessing step (S310) is executed by the primary object detection unit (110), and the three-dimensional point cloud generation step (S320) and the tracking and clustering step (S330) can be executed by the secondary object detection unit (120).

[0154] In the radar signal reception and preprocessing step (S310), the primary object detection unit (110) can extract distance information, azimuth information, elevation information (height information), and speed information as primary object information for each of the detection points by using the received signal (radar signal) received for each of the points (detection points) where the transmitted signal (radar signal) is reflected.

[0155] In the radar signal reception and preprocessing step (S310), the primary object detection unit (110) detects the signal strength of the received signal (radar signal) received at each of the points (detection points) where the transmitted signal (radar signal) is reflected, and based on this, can further obtain signal-to-noise ratio (SNR) information as primary object information.

[0156] In the radar signal reception and preprocessing step (S310), the primary object detection unit (110) can further acquire three-dimensional coordinate information for each of the detection points as primary object information based on distance information, azimuth information, and elevation information for each of the detection points.

[0157] In the 3D point cloud generation step (S320), the secondary object detection unit (120) can generate 3D point cloud information as secondary object information by using at least one of velocity information, signal strength information, and signal-to-noise ratio information included in the primary object information, along with 3D coordinate information obtained from distance information, azimuth information, and elevation angle information included in the primary object information.

[0158] In the 3D point cloud generation step (S320), the secondary object detection unit (120) may classify the 3D point cloud into one of a static point cloud and a dynamic point cloud, or separate the 3D point cloud into a static point cloud and a dynamic point cloud.

[0159] In the tracking and clustering step (S330), the secondary object detection unit (120) performs tracking and clustering processing on multiple detection points included in a three-dimensional point cloud to create at least one cluster by grouping the detection points, and may further create cluster information for at least one cluster as secondary object information. Here, each cluster corresponds to a single identical object and may be a set of detection points that form a single identical object.

[0160] Referring to FIG. 3, the point cloud-based object detection method of the point cloud-based object detection device (100) according to embodiments of the present disclosure may further include a center of gravity analysis step (S340), a Doppler velocity (or 3D velocity vector) analysis step (S350), a fall detection step (S370), and an additional danger situation detection and notification step (S380).

[0161] In the center of gravity analysis step (S340), the secondary object detection unit (120) of the point cloud-based object detection device (100) can calculate the average value of the three-dimensional coordinates (Xc, Yc, Zc) of each of the N (N is a natural number greater than or equal to 2) detection points forming the object (cluster) using cluster information including the number of clusters (i.e., number of objects), the location, size, and shape of each cluster, and the three-dimensional coordinate information (Xi, Yi, Zi, i=1, 2, … , N) of each detection point forming each cluster, as the center of gravity coordinates (Xc, Yc, Zc) of the object, and obtain center of gravity coordinate information. Here, the center of gravity coordinate information for the center of gravity coordinates (Xc, Yc, Zc) of the object may be included in additional secondary object information. The center of gravity coordinates (Xc, Yc, Zc) of an object can be expressed as shown in Equation 3 below.

[0162]

[0163] In Equation 3, N is the total number of detection points constituting an object, and (Xc, Yc, Zc) are the 3D coordinates of the object and may be the 3D coordinates of the centroid of the object. In (Xc, Yc, Zc), Xc is the x-axis coordinate value of the centroid of the object, Yc is the y-axis coordinate value of the centroid of the object, and Zc is the z-axis coordinate value of the centroid of the object. The 3D coordinates of the i-th detection point among the N detection points constituting the object are (xi, yi, zi).

[0164] In mathematical formula 3, Xc is the sum of the x-axis coordinate values ​​of each of the N detection points divided by N, Yc is the sum of the y-axis coordinate values ​​of each of the N detection points divided by N, and Zc is the sum of the z-axis coordinate values ​​of each of the N detection points divided by N.

[0165] In mathematical formula 3, Zc is the z-axis coordinate value of the object's center of gravity, which can be said to be the height center value of the object.

[0166] In the center of gravity analysis step (S340), the secondary object detection unit (120) of the point cloud-based object detection device (100) can determine the change in the center of gravity of each object after calculating the center of gravity coordinates (3D coordinates for the center of gravity) of each object as described above.

[0167] The secondary object detection unit (120) can analyze the change in the center of gravity of each object by calculating the change value of the center of gravity coordinates of each object and output the analysis result to the object integration detection unit (130).

[0168] The analysis results may include information on the change values ​​of the center of gravity coordinates of each object. Here, the change values ​​of the center of gravity coordinates may include change values ​​of the x-axis coordinate, change values ​​of the y-axis coordinate, and change values ​​of the z-axis coordinate. Each of the change values ​​of the x-axis coordinate, y-axis coordinate, and z-axis coordinate can have a positive or negative value. In particular, if the center of gravity of the object drops, the z-axis coordinate value suddenly decreases, and the change value of the z-axis coordinate may have a negative value.

[0169] The object integration detection unit (130) executes the fall detection step (S370) and detects whether the center of gravity of each object has dropped based on the change value of the center of gravity coordinates of each object output from the secondary object detection unit (120), and based on this, detects whether the object has fallen.

[0170] The object integration detection unit (130) can determine that the object has fallen if the z-axis coordinate change value (ΔZc=Zc(t)-Zc(t-Δt)), which represents the change in the z-axis coordinate among the center of gravity coordinates (Xc, Yc, Zc) of each object over a certain period of time, i.e., the z-axis coordinate change value (ΔZc=Zc(t)-Zc(t-Δt)) over a certain period of time (Δt), is less than a predetermined fall judgment threshold change value (-Tz). This can be expressed as shown in Equation 4 below.

[0171]

[0172] In Equation 4, Δt is the time for fall detection and can be set to a small time value to account for sudden falls. It represents the change in the z-axis coordinate of the object's center of gravity (the amount of change in the z-axis coordinate value) during Δt. Zc(t-Δt) is the z-axis coordinate value of the object's center of gravity at the first time point (t-Δt), and Zc(t) is the z-axis coordinate value of the object's center of gravity at the second time point (t), which is Δt elapsed from the first time point (t-Δt). The predetermined fall detection threshold change value (-Tz) has a negative value and may be the threshold value of the z-axis coordinate change value capable of detecting a fall. For example, the fall detection threshold change value may be set to a constant value (e.g., -Tz = -0.5m). As another example, the fall detection threshold change value (-Tz) may be set differently depending on the object size, which can be determined by the size of the area occupied by the detection points forming the object.

[0173] The object integration detection unit (130) may determine that an object has fallen if, among the change amount (magnitude of the change value (scalar value)) of the center of gravity coordinates (Xc, Yc, Zc) of each object, the change amount of the z-axis coordinate (|ΔZc| = |Zc(t) - Zc(t - Δt)|) during a certain time period (Δt) exceeds a predetermined fall judgment threshold change amount (|-Tz|). For example, the fall judgment threshold change amount (|-Tz| = Tz) may be set to a constant value (e.g., Tz = 0.5 m). As another example, the fall judgment threshold change amount (|-Tz| = Tz) may be set differently depending on the object size, which can be determined by the size of the area occupied by the detection points forming the object.

[0174] The integrated detection unit (130) can determine that an object has fallen based on the change value of the center of gravity coordinate (or the amount of change in the center of gravity coordinate), and can generate a sound or vibration to notify of the fall, or send a fall notification message to a designated device.

[0175] Meanwhile, in order to determine more precisely whether a fall has occurred, the secondary object detection unit (120) can execute a center of gravity analysis step (S340) to calculate the ratio of the number of detection points (Nupper) located above a predetermined height (a predetermined z-axis coordinate value) among the total number of detection points (N) forming an object as the upper density (ρ) of each object. The upper density (ρ) of an object can be expressed by the following mathematical formula 5.

[0176]

[0177] In the above mathematical formula 5, ρ is the upper density, which is the ratio of the upper area of ​​an object to the total area of ​​an object; N is the total number of all detection points constituting an object; and Nupper is the number of detection points located in the upper area above a predetermined height among all detection points constituting an object.

[0178] The object integration detection unit (130) executes a fall detection step (S370), and if the upper density (ρ) of each object output from the secondary object detection unit (120) changes to be less than a predetermined threshold density value (Tρ) for a certain period of time, it can determine that the object has fallen. This can be expressed by Equation 6.

[0179]

[0180] In the above mathematical formula 6, Tρ may be a threshold density value corresponding to the upper density value for determining whether a fall has occurred. For example, the threshold density value (Tρ) may be 0.2. This may mean that among the N detection points forming the object, 20% of the detection points (i.e., (N*0.2) detection points) are located in the upper region, and 80% of the detection points (i.e., (N*0.8) detection points) are located in the lower region.

[0181] The integrated detection unit (130) can generate a sound or vibration to notify of the fall or send a fall notification message to a designated device when it determines that an object has fallen based on the upper density.

[0182] The object integration detection unit (130) can finally determine whether an object has fallen by comprehensively considering the result of determining whether it has fallen based on the center of gravity coordinate change value (or amount of change in the center of gravity coordinate) and the result of determining whether it has fallen based on the upper density.

[0183] For example, the object integration detection unit (130) can finally determine that the object has fallen if both the result of determining whether it has fallen based on the center of gravity coordinate change value (or center of gravity coordinate change amount) and the result of determining whether it has fallen based on the upper density are determined to be a fall. For another example, the object integration detection unit (130) can finally determine whether the object has fallen if only one of the result of determining whether it has fallen based on the center of gravity coordinate change value (or center of gravity coordinate change amount) and the result of determining whether it has fallen based on the upper density is determined to be a fall, after a certain period of time has elapsed, by obtaining the result of determining whether it has fallen based on the center of gravity coordinate change value (or center of gravity coordinate change amount) and the result of determining whether it has fallen based on the upper density again.

[0184] When the integrated detection unit (130) finally determines that an object has fallen, it can generate a sound or vibration to notify of the fall, or send a fall notification message to a designated device.

[0185] Meanwhile, the secondary object detection unit (120) of the point cloud-based object detection device (100) may further perform a Doppler velocity analysis step (S350) that analyzes the Doppler velocity (including a three-dimensional velocity vector) of the object.

[0186] Through the Doppler velocity analysis step (S350), the secondary object detection unit (120) can output result information of analyzing the Doppler velocity of the object (included in the secondary object information) to the object integration detection unit (130). Here, the Doppler velocity (including the three-dimensional velocity vector) may include velocity in the z-axis direction. The secondary object detection unit (120) can detect distance information, azimuth information, and elevation angle information from the received signal, as well as velocity in the z-axis direction for each object.

[0187] The object integration detection unit (130) monitors the situation of sudden change in the Doppler velocity of the object, and if the situation of sudden change in the Doppler velocity of the object is detected, it can determine that the object has fallen. Here, the situation of sudden change in the velocity of the object in the z-axis direction may mean a case where the velocity in the z-axis direction changes by exceeding a predefined threshold velocity for a certain period of time. That is, the situation of sudden change in the Doppler velocity of the object may mean a case where the change in velocity in the z-axis direction exceeds the threshold velocity change amount for a certain period of time.

[0188] The object integration detection unit (130) can finally determine whether the object has fallen based on the result of determining whether it has fallen based on the change in the center of gravity coordinates and the result of determining whether it has fallen based on the Doppler velocity (velocity in the z-axis direction).

[0189] For example, the object integration detection unit (130) can make a final determination of whether the object has fallen if both the result of determining whether it has fallen based on the change in the center of gravity coordinates and the result of determining whether it has fallen based on the speed are determined to be a fall.

[0190] Meanwhile, the secondary object detection unit (120) can analyze the breathing signal of the object and output the breathing analysis result to the object integration detection unit (130) (S360). The integration object detection unit (130) can detect additional dangerous situations using the breathing analysis result and perform notification processing according to the detection result (S380).

[0191] The integrated object detection unit (130) can generate a sound or vibration to notify of a dangerous situation, or transmit a dangerous situation notification message to a designated device (e.g., a contact device of an emergency sensor) if, based on the results of the respiration analysis, the object's respiration state is determined to be abnormal (a sudden change in respiration state) or to be in a state of respiration cessation (S380).

[0192] FIG. 3b is a diagram illustrating the breath analysis function of a point cloud-based object detection device (100) according to embodiments of the present disclosure.

[0193] Referring to FIG. 3b, the secondary object detection unit (120) can perform a radar-based respiration monitoring function. The secondary object detection unit (120) can detect changes in coordinates or distances for each of the detection points corresponding to the chest (the chest may correspond to a micro-Doppler object) among the detection points forming the human body identified as an object, and can output a respiration-related information detection result including information on changes in phase or distances (changes in coordinates) for each of the detection points to the object integration detection unit (130).

[0194] The object integration detection unit (130) can detect minute chest movements (expansion / contraction of the chest due to breathing) based on the results of detecting breathing-related information (phase change or distance change (coordinate change), etc.) for each detection point corresponding to the chest (the chest may correspond to a micro-Doppler object), and can estimate the breathing cycle, breathing rate, breathing pattern, etc. based on the results.

[0195] When a person breathes, the chest moves back and forth periodically. Although this movement is a minute movement of a few millimeters, the secondary object detection unit (120) can detect phase changes or distance changes by using radar signals reflected from detection points in the chest. The object integration detection unit (130) can obtain breathing information (breathing cycle, breathing rate, breathing pattern, etc.) by using the detection results of phase changes or distance changes of the received radar signals, and can express the breathing information in a form such as the breathing signals (310, 320, 330) of FIG. 3.

[0196] For example, the change in distance (Δd) between the detection point on the chest and the point cloud-based object detection device (100) is the value obtained by dividing the wavelength (λ) of the radar signal by 4π and multiplying it by the change in phase (ΔΦ) of the radar signal reflected from the detection point on the chest (Δd=(λ / 4)* ΔΦ). From this relationship, the secondary object detection unit (120) can determine the change in phase (ΔΦ) of the radar signal reflected from the detection point on the chest from the change in distance (Δd) between the detection point on the chest and the point cloud-based object detection device (100).

[0197] In other words, the secondary object detection unit (120) can detect a phase change of a received signal corresponding to each detection point corresponding to the chest among the detection points forming the human body identified as an object, and a distance change of the detection points corresponding to the chest, and output a respiration-related information detection result including information on the phase change and distance change. The object integration detection unit (130) can detect chest movement based on the respiration-related information detection result, generate a respiration signal in which the signal value changes over time, and indicate a respiration cycle, respiration rate, respiration pattern, etc. based on the detection result of the chest movement, and determine the respiration state of the object based on the respiration signal.

[0198] The object integration detection unit (130) can determine the breathing state as one of a normal state, a rapid change in breathing state, and a cessation of breathing state based on the estimation results of the breathing cycle, breathing rate, breathing pattern, etc.

[0199] The secondary object detection unit (120) can extract a micro-Doppler component from the received signal and, based on the micro-Doppler component, detect a phase change of the received signal corresponding to each of the detection points corresponding to the chest among the detection points forming the human body identified as an object, and a change in distance between the detection points corresponding to the chest and the point cloud-based object detection device.

[0200] Referring to the normal state breathing signal (310) of FIG. 3b, the object integration detection unit (130) can determine the breathing state to be normal if the breathing cycle falls within the normal breathing cycle range or is constant, the breathing rate falls within the normal breathing rate range or is constant, and the breathing pattern falls within the normal breathing pattern or is regular.

[0201] Referring to the breathing signal (320) of the breathing sudden change state of FIG. 3b, the object integration detection unit (130) can determine the breathing state as an abnormal breathing sudden change state if, even though breathing is present, the breathing cycle does not fall within the normal breathing cycle range, the breathing cycle is not constant, the breathing rate does not fall within the normal breathing rate range, the breathing rate is not constant, the breathing pattern does not fall within the normal breathing pattern, or the breathing pattern is irregular.

[0202] Referring to the breathing signal (330) of the breathing cessation state in FIG. 3b, the object integration detection unit (130) can determine that the breathing state is a breathing cessation state when breathing suddenly stops, the breathing cycle becomes longer than a predetermined level, the breathing rate becomes slower than a predetermined level, or the breathing pattern becomes a state without breathing.

[0203] Mathematical Equation 7 below is an equation for determining the state of rapid change in breathing, and represents the change in breathing rate (BPM: Breaths Per Minute).

[0204]

[0205] In Equation 7, BPMi is the respiration rate measured at the current time point (i), BPMi-1 is the respiration rate measured at the previous time point (i-1), and ΔBPMi is the change in respiration rate between the current and the previous. K is the count threshold for making a final judgment on a sudden change in respiration. Tbpm is the threshold change in respiration rate suspected of being a sudden change in respiration. If the change in respiration rate (ΔBPMi) suddenly increases, it may indicate that respiration is irregular or has changed rapidly. This may indicate that the respiration state is abnormal (e.g., a state of respiratory arrest, or a sudden change in respiration due to tension or disease).

[0206] The object integration detection unit (130) suspects a sudden change in breathing condition when the change in breathing rate (ΔBPMi) exceeds a threshold change in breathing rate (Tbpm) and increases the number of suspicions (n). The object integration detection unit (130) can finally determine the breathing condition as a sudden change in breathing condition when situations suspected of being a sudden change in breathing occur continuously and the number of suspicions (n) becomes greater than a predetermined count threshold (K).

[0207] The object integration detection unit (130) can determine that the breathing state is a breathing cessation state by using breathing signals (310, 320, 330) expressed by breathing information (breathing cycle, breathing rate, breathing pattern, etc.) obtained from the secondary object detection unit (120), and when the standard deviation (σ) of the sample for breathing signal amplitudes over time is less than the threshold amplitude (Tamp) as shown in Equation 8 below.

[0208]

[0209] In Equation 8, M is the number of breath signals in the population, Ai is the amplitude of the i-th detected breath signal, Aavg is the sample mean, and σ is the sample standard deviation. Tamp is the threshold amplitude of the breath signal determined to be in a stationary state.

[0210] The object integration detection unit (130) can determine the breathing state of the object when it is determined that the object has fallen based on the result of determining whether the object has fallen based on the change in the center of gravity coordinates, and if the object's breathing state is a sudden change in breathing or a cessation of breathing, it can perform an emergency situation notification processing.

[0211] FIGS. 4 and FIGS. 5 are diagrams illustrating the object movement detection function of a point cloud-based object detection device (100) according to embodiments of the present disclosure.

[0212] FIG. 4 illustrates an actual detection situation image (400_REAL) showing a detection area actually detected by an object movement detection unit (210) of a point cloud-based object detection device (100) according to embodiments of the present disclosure, and a detection result screen (400) showing an object movement detection result by an object movement detection unit (210) of a point cloud-based object detection device (100).

[0213] Referring to FIG. 4, a point cloud-based object detection device (100) is installed in a space, and the point cloud-based object detection device (100) detects a detection area in which a person, who is an object (OBJ_REAL), is sitting on a chair in a state of no movement (or a state of almost no movement).

[0214] Referring to the detection result screen (400), if only the micro-Doppler component among the general Doppler component and the micro-Doppler component is extracted from the received signal received by the surroundings (object (OBJ_REAL) and objects in its vicinity) after the transmitted signal transmitted from the point cloud-based object detection device (100) is reflected, the object motion detection unit (210) can detect that the person, who is the object (OBJ_REAL), is in a state of no movement.

[0215] An object image (OBJ) corresponding to the detected object (OBJ_REAL) may be displayed on the detection result screen (400). The object image (OBJ) may have a predefined simple three-dimensional shape. For example, the object image (OBJ) may be displayed as a three-dimensional shape such as a rectangular prism or a cylinder.

[0216] Additionally, on the detection result screen (400), micro-Doppler points (Pmd) corresponding to the detection points where the micro-Doppler component was generated may be displayed.

[0217] Micro-Doppler points (Pmd) are points (detection points) where very fine movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0218] FIG. 5 illustrates an actual detection situation image (500_REAL) showing a detection area actually detected by an object movement detection unit (210) of a point cloud-based object detection device (100) according to embodiments of the present disclosure, and a detection result screen (500) showing an object movement detection result by an object movement detection unit (210) of a point cloud-based object detection device (100).

[0219] Referring to FIG. 5, a point cloud-based object detection device (100) is installed in a certain space, and the object movement detection unit (210) of the point cloud-based object detection device (100) detects a detection area in which a person, who is an object (OBJ_REAL), is sitting on a chair in a moving state.

[0220] Referring to the detection result screen (500), if both the general Doppler component and the micro-Doppler component are extracted from the received signal that is reflected from the surroundings (object (OBJ_REAL) and objects in its vicinity) by the transmission signal transmitted from the point cloud-based object detection device (100), the object motion detection unit (210) can detect that the person, which is the object (OBJ_REAL), is in a state of movement.

[0221] An object image (OBJ) corresponding to the detected object (OBJ_REAL) may be displayed on the detection result screen (500). The object image (OBJ) may have a simple predefined three-dimensional shape. For example, the object image (OBJ) may be displayed as a three-dimensional shape such as a rectangular prism or a cylinder.

[0222] Additionally, the detection result screen (500) may display Doppler points (Pd), which are detection points where a general Doppler component is generated, and micro-Doppler points (Pmd), which are detection points where a micro-Doppler component is generated.

[0223] Doppler points (Pd) may be points (detection points) where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, if an arm or hand moves or a sitting posture changes slightly significantly, points (detection points) with such slightly large movements may be marked as Doppler points (Pd).

[0224] Micro-Doppler points (Pmd) may be points (detection points) where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0225] The object movement detection unit (210) of the point cloud-based object detection device (100) can detect an object (OBJ_REAL) that is stationary or in a state of ultra-low speed movement through the object movement detection function.

[0226] FIGS. 6 and 7 are diagrams illustrating the object movement tracking function of a point cloud-based object detection device (100) according to embodiments of the present disclosure.

[0227] FIG. 6 shows an actual detection situation image (600_REAL_t1) representing a detection area detected at a first time point (t1) by an object movement tracking unit (220) of a point cloud-based object detection device (100) according to embodiments of the present disclosure, and a three-dimensional detection result screen (600_t1) and a planar detection result screen (600_PLN_t1) representing the object movement tracking result at the first time point (t1) by the object movement tracking unit (220) of the point cloud-based object detection device (100).

[0228] FIG. 7 shows an actual detection situation image (600_REAL_t2) representing a detection area detected at a second time point (t2) by an object movement tracking unit (220) of a point cloud-based object detection device (100) according to embodiments of the present disclosure, and a three-dimensional detection result screen (600_t2) and a planar detection result screen (600_PLN_t2) representing the object movement tracking result at the second time point (t2) by an object movement tracking unit (220) of a point cloud-based object detection device (100).

[0229] Referring to FIG. 6, a point cloud-based object detection device (100) is installed in a certain space, and the object movement tracking unit (220) of the point cloud-based object detection device (100) can detect a situation in which a person, who is an object (OBJ_REAL), enters the detection area at a first time point (t1).

[0230] Referring to FIG. 6, when referring to the stereoscopic detection result screen (600_t1) and the planar stereoscopic detection result screen (600_PLN_t1) based on object movement tracking at the first time point (t1), the position information of the object (OBJ_REAL) can be detected by using the general Doppler component and the micro-Doppler component extracted from the received signal that is received by the surroundings (object (OBJ_REAL) and objects in its vicinity) where the transmitted signal transmitted by the point cloud-based object detection device (100) is reflected.

[0231] Referring to FIG. 6, the object image (OBJ_t1) corresponding to the detected object (OBJ_REAL) can be displayed on the stereoscopic detection result screen (600_t1) and the planar stereoscopic detection result screen (600_PLN_t1).

[0232] Referring to FIG. 6, in the stereoscopic detection result screen (600_t1), the object image (OBJ_t1) may have a predefined simple stereoscopic shape. For example, the object image (OBJ) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0233] Referring to FIG. 6, in the plane detection result screen (600_PLN_t1), the object image (OBJ_t1) may have a predefined simple plane shape. For example, the object image (OBJ) may be displayed as a plane shape such as a rectangle or a circle.

[0234] In addition, the stereoscopic detection result screen (600_t1) and the planar stereoscopic detection result screen (600_PLN_t1) may display Doppler points (Pd), which are detection points where a general Doppler component is generated, and micro-Doppler points (Pmd), which are detection points where a micro-Doppler component is generated.

[0235] Doppler points (Pd) may be points (detection points) where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, when an arm or leg moves to walk, the points (detection points) where there is movement of the object may be indicated as Doppler points (Pd) in a situation where there is a change in the object's position and the object's trajectory is detected.

[0236] Micro-Doppler points (Pmd) may be points (detection points) where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0237] Referring to FIG. 6, in the object information (OBJ_INFO_t2) for the corresponding object ID (0) obtained as a result of object movement tracking at the first time point (t1), the (x, y, z) coordinate values ​​(central mass coordinates) of the object (OBJ_REAL) are (-0.938, 2.461, 1.82).

[0238] Referring to FIG. 7, a point cloud-based object detection device (100) is installed in a certain space, and the object movement tracking unit (220) of the point cloud-based object detection device (100) can detect a situation in which a person, who is an object (OBJ_REAL), enters the detection area at a second time point (t2). Here, the second time point (t2) may be a time point after the first time point (t1).

[0239] Referring to FIG. 7, when referring to the stereoscopic detection result screen (600_t2) and the planar stereoscopic detection result screen (600_PLN_t2) based on object movement tracking at the second time point (t2), the position information of the object (OBJ_REAL) can be detected by using the general Doppler component and the micro-Doppler component extracted from the received signal that is received by the transmission signal transmitted from the point cloud-based object detection device (100) being reflected from the surroundings (object (OBJ_REAL) and objects in its vicinity).

[0240] Referring to FIG. 7, the object image (OBJ_t2) corresponding to the object (OBJ_REAL) whose movement (position change) and trajectory were detected can be displayed on the stereoscopic detection result screen (600_t2) and the planar stereoscopic detection result screen (600_PLN_t2).

[0241] Referring to FIG. 7, in the stereoscopic detection result screen (600_t2), the object image (OBJ_t2) may have a predefined simple stereoscopic shape. For example, the object image (OBJ) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0242] Referring to FIG. 7, in the plane detection result screen (600_PLN_t2), the object image (OBJ_t1) may have a predefined simple plane shape. For example, the object image (OBJ) may be displayed as a plane shape such as a rectangle or a circle.

[0243] In addition, the stereoscopic detection result screen (600_t2) and the planar stereoscopic detection result screen (600_PLN_t2) may display Doppler points (Pd), which are detection points where a general Doppler component is generated, and micro-Doppler points (Pmd), which are detection points where a micro-Doppler component is generated.

[0244] Doppler points (Pd) may be points (detection points) where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, when an arm or leg moves to walk, the points (detection points) where there is movement of the object may be indicated as Doppler points (Pd) in a situation where there is a change in the object's position and the object's trajectory is detected.

[0245] Micro-Doppler points (Pmd) may be points (detection points) where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0246] It can be seen that the position where the object image (OBJ_t2) is displayed in the stereoscopic detection result screen (600_t2) and the planar stereoscopic detection result screen (600_PLN_t2) of Fig. 7 has changed compared to the position where the object image (OBJ_t1) is displayed in the stereoscopic detection result screen (600_t1) and the planar stereoscopic detection result screen (600_PLN_t1) of Fig. 6.

[0247] Referring to FIG. 7, in the object information (OBJ_INFO_t2) for the corresponding object ID (0) obtained as a result of object movement tracking at the second time point (t2), the (x, y, z) coordinate values ​​(central coordinates) of the object (OBJ_REAL) are (-0.637, 3.159, 1.971). The (x, y, z) coordinates (central coordinates) of the object (OBJ_REAL) detected at the second time point (t2) are changed compared to the (x, y, z) coordinates (central coordinates) of the object (OBJ_REAL) detected at the first time point (t1).

[0248] The object movement tracking unit (220) of the point cloud-based object detection device (100) can detect information about the trajectory (movement trajectory) of the object (OBJ_REAL) by calculating the change between the (x, y, z) coordinates (center of gravity coordinates) of the object (OBJ_REAL) detected at the first time point (t1) and the (x, y, z) coordinates (center of gravity coordinates) of the object (OBJ_REAL) detected at the second time point (t2).

[0249] FIGS. 8 and 9 are diagrams illustrating the three-dimensional shape recognition function of a point cloud-based object detection device (100) according to embodiments of the present disclosure.

[0250] FIG. 8 shows an actual detection situation image (800_REAL_t1) representing a detection area detected at a first time point (t1) by a three-dimensional shape recognition unit (230) of a point cloud-based object detection device (100) according to embodiments of the present disclosure, and a three-dimensional detection result screen (800_t1) and a planar detection result screen (800_PLN_t1) representing a three-dimensional shape recognition result at a first time point (t1) by a three-dimensional shape recognition unit (230) of a point cloud-based object detection device (100).

[0251] FIG. 9 shows an actual detection situation image (800_REAL_t2) representing a detection area detected at a second time point (t2) by a three-dimensional shape recognition unit (230) of a point cloud-based object detection device (100) according to embodiments of the present disclosure, and a three-dimensional detection result screen (600_t2) and a planar detection result screen (800_PLN_t2) representing a three-dimensional shape recognition result at a second time point (t2) by a three-dimensional shape recognition unit (230) of a point cloud-based object detection device (100).

[0252] Referring to FIG. 8, a point cloud-based object detection device (100) is installed in a designated space, and a three-dimensional shape recognition unit (230) of the point cloud-based object detection device (100) can recognize a three-dimensional shape of a person, which is an object (OBJ_REAL), within the space at a first time point (t1).

[0253] Referring to FIG. 8, when referring to the stereoscopic detection result screen (800_t1) and the planar stereoscopic detection result screen (800_PLN_t1) based on three-dimensional shape recognition at the first time point (t1), the position information of an object (OBJ_REAL) can be detected by using the general Doppler component and the micro-Doppler component extracted from the received signal that is received by the transmission signal transmitted from the point cloud-based object detection device (100) being reflected from the surroundings (object (OBJ_REAL) and objects in its vicinity).

[0254] Referring to FIG. 8, the stereoscopic detection result screen (800_t1) and the planar stereoscopic detection result screen (800_PLN_t1) may display a spatial image (801) representing the space and an object image (OBJ_t1) corresponding to the detected object (OBJ_REAL).

[0255] In the stereoscopic detection result screen (800_t1) and the planar stereoscopic detection result screen (800_PLN_t1), an object image (802) representing an object within the space may be further displayed.

[0256] Referring to FIG. 8, in the stereoscopic detection result screen (800_t1), the object image (OBJ_t1) may have a predefined simple stereoscopic shape. For example, the object image (OBJ_t1) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0257] Referring to FIG. 8, in the plane detection result screen (800_PLN_t1), the object image (OBJ_t1) may have a predefined simple plane shape. For example, the object image (OBJ_t1) may be displayed as a plane shape such as a rectangle or a circle.

[0258] Referring to FIG. 8, in the stereoscopic detection result screen (800_t1), each of the spatial image (801) and the object image (802) may have a predefined simple stereoscopic shape. For example, each of the spatial image (801) and the object image (802) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0259] Referring to FIG. 8, in the plane detection result screen (800_PLN_t1), the space image (801) and the object image (802) may each have a predefined simple plane shape. For example, the space image (801) and the object image (802) may each be displayed as a plane shape such as a square or a circle.

[0260] In addition, the stereoscopic detection result screen (800_t1) and the planar stereoscopic detection result screen (800_PLN_t1) may display Doppler points (Pd), which are detection points where a general Doppler component is generated, and micro-Doppler points (Pmd), which are detection points where a micro-Doppler component is generated.

[0261] Doppler points (Pd) may be points (detection points) where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, when an arm or leg moves to walk, the points (detection points) where there is movement of the object may be indicated as Doppler points (Pd) in a situation where there is a change in the object's position and the object's trajectory is detected.

[0262] Micro-Doppler points (Pmd) may be points (detection points) where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0263] The 3D shape recognition unit (230) can obtain 3D shape recognition information (e.g., height, width, size, etc. of an object) for a single object by grouping (clustering) 3D points (Pd, Pmd), which are detection points, based on 3D coordinates (x, y, z) and velocity information. That is, the object image (OBJ_t1) can be displayed in an area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL).

[0264] Referring to FIG. 8, in the object information (OBJ_INFO_t1) for the corresponding object ID (0) obtained as a result of 3D shape recognition for the object (OBJ_REAL) at the first time point (t1), the (x, y, z) coordinate values ​​(central mass coordinates) of the corresponding object (OBJ_REAL) are (-0.166, 0.251, 1.646).

[0265] Referring to FIG. 8, in the (x, y, z) coordinate values, the x value may be the center value of the x values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL), and the y value may be the center value of the y values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the z values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the height of the object (OBJ_REAL) at the first time point (t1).

[0266] Referring to FIG. 9, a point cloud-based object detection device (100) is installed in a designated space, and a three-dimensional shape recognition unit (230) of the point cloud-based object detection device (100) can recognize a three-dimensional shape of a person, which is an object (OBJ_REAL), within the space at a second time point (t2).

[0267] Referring to FIG. 9, when referring to the stereoscopic detection result screen (800_t2) and the planar stereoscopic detection result screen (800_PLN_t2) based on three-dimensional shape recognition at the second time point (t2), the position information of the object (OBJ_REAL) can be detected by using the general Doppler component and the micro-Doppler component extracted from the received signal that is received by the transmission signal transmitted from the point cloud-based object detection device (100) being reflected from the surroundings (object (OBJ_REAL) and objects in its vicinity).

[0268] Referring to FIG. 9, the stereoscopic detection result screen (800_t2) and the planar stereoscopic detection result screen (800_PLN_t2) may display a spatial image (801) representing the space and an object image (OBJ_t1) corresponding to the detected object (OBJ_REAL).

[0269] In the stereoscopic detection result screen (800_t2) and the planar stereoscopic detection result screen (800_PLN_t2), an object image (802) representing an object within the space may be further displayed.

[0270] Referring to FIG. 9, in the stereoscopic detection result screen (800_t2), the object image (OBJ_t2) may have a predefined simple stereoscopic shape. For example, the object image (OBJ_t2) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0271] Referring to FIG. 9, in the plane detection result screen (800_PLN_t2), the object image (OBJ_t2) may have a predefined simple plane shape. For example, the object image (OBJ_t2) may be displayed as a plane shape such as a rectangle or a circle.

[0272] Referring to FIG. 9, in the stereoscopic detection result screen (800_t2), each of the spatial image (801) and the object image (802) may have a predefined simple stereoscopic shape. For example, each of the spatial image (801) and the object image (802) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0273] Referring to FIG. 9, in the plane detection result screen (800_PLN_t2), the space image (801) and the object image (802) may each have a predefined simple plane shape. For example, the space image (801) and the object image (802) may each be displayed as a plane shape such as a square or a circle.

[0274] In addition, the stereoscopic detection result screen (800_t2) and the planar stereoscopic detection result screen (800_PLN_t2) may display Doppler points (Pd) where a general Doppler component is generated and micro-Doppler points (Pmd) where a micro-Doppler component is generated.

[0275] Doppler points (Pd) may be points (detection points) where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, when an arm or leg moves to walk, the points (detection points) where there is movement of the object may be indicated as Doppler points (Pd) in a situation where there is a change in the object's position and the object's trajectory is detected.

[0276] Micro-Doppler points (Pmd) may be points (detection points) where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0277] The object image (OBJ_t2) can be displayed in the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL).

[0278] Referring to FIG. 9, in the object information (OBJ_INFO_t2) for the corresponding object ID (0) obtained as a result of 3D shape recognition for the object (OBJ_REAL) at the second time point (t2), the (x, y, z) coordinate values ​​(central mass coordinates) of the corresponding object (OBJ_REAL) are (0.073, 0.376, 1.278).

[0279] Referring to FIG. 9, in the (x, y, z) coordinate values, the x value may be the center value of the x values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL), and the y value may be the center value of the y values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the z values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the height of the object (OBJ_REAL) at the first time point (t1).

[0280] Referring to FIGS. 8 and 9, the 3D shape recognition unit (230) can recognize (detect) a change in the shape of the object (OBJ_REAL) by comparing the object information (OBJ_INFO_t1) at the first time point (t1) and the object information (OBJ_INFO_t2) at the second time point (t2). In particular, the 3D shape recognition unit (230) can recognize (detect) a change in the shape and state of the object (OBJ_REAL) by comparing the difference value (a change in height corresponding to the change in the height center value) between the z value (height center value) included in the object information (OBJ_INFO_t1) at the first time point (t1) and the z value (height center value) included in the object information (OBJ_INFO_t2) at the second time point (t2).

[0281] A point cloud-based object detection device (1000) according to embodiments of the present disclosure can provide application functions (e.g., object count measurement function, abnormal situation detection function, object presence detection function, object abnormality detection function, etc.) by utilizing the functions described above (primary object detection function, secondary object detection function (object movement detection function, object movement detection function, 3D shape recognition function), and object integration detection function).

[0282] Hereinafter, application functions of a point cloud-based object detection device (100) according to embodiments of the present disclosure are described. Application functions when a plurality of objects exist in a space where the point cloud-based object detection device (100) is installed are described with reference to FIGS. 10 to 12, and application functions when a single object exists in a space where the point cloud-based object detection device (100) is installed are described with reference to FIGS. 13 to 16.

[0283] FIG. 10 is an additional block diagram of a point cloud-based object detection device (100) according to embodiments of the present disclosure. FIG. 11 is a diagram showing the object count measurement function of the point cloud-based object detection device (100) according to embodiments of the present disclosure. FIG. 12 is a diagram showing the abnormal situation detection function of the point cloud-based object detection device (100) according to embodiments of the present disclosure.

[0284] Referring to FIG. 10, a point cloud-based object detection device (100) according to embodiments of the present disclosure may further include an installation space information management unit (1000) that sets, stores, and updates space information for a space where the point cloud-based object detection device (100) is installed.

[0285] For example, spatial information is information about a space in which a point cloud-based object detection device (100) is installed, and may include at least one of size information of the space, information about objects existing in the space, and characteristic information about the space. The characteristic information about the space may include information indicating the type of space (e.g., information indicating an office, conference room, restroom, living room, room, etc.). Spatial information may also be referred to as a three-dimensional Region of Interest (ROI) for object detection.

[0286] At least one of the primary object detection unit (110), secondary object detection unit (120), and object integration detection unit (130) included in the point cloud-based object detection device (100) can perform the detection operation by referring to spatial information.

[0287] The object integration detection unit (130) can determine the number of objects based on object integration information, assign an object ID to each object, and store the object ID and object integration information in conjunction.

[0288] Referring to FIG. 10, a point cloud-based object detection device (100) according to embodiments of the present disclosure may further include an object count measuring unit (1010) that measures the number of objects in a space where the point cloud-based object detection device (100) is installed, based on the number of object IDs and spatial information.

[0289] Referring to FIG. 10, a point cloud-based object detection device (100) according to embodiments of the present disclosure may further include an abnormal situation detection unit (1020) that detects an abnormal situation in a space where the point cloud-based object detection device (100) is installed, based on abnormal situation judgment information identified based on object integration information associated with an object ID and spatial information. For example, the abnormal situation judgment information mentioned above may include at least one of the position, trajectory, velocity, state, height center value, and height change amount of the height center value of the object for each object ID.

[0290] When an abnormal situation is detected, the abnormal situation detection unit (1020) can perform notification processing (e.g., notification processing such as sound, vibration, etc.) in a preset manner.

[0291] When an abnormal situation is detected, the abnormal situation detection unit (1020) can use a communication module included in the point cloud-based object detection device (100) to send an emergency situation notification message to a recipient with pre-set emergency contact information.

[0292] Referring to FIG. 11, a point cloud-based object detection device (100) according to embodiments of the present disclosure can perform an object count measurement function, which is an example of an application function, by utilizing a basic detection function. Here, the basic detection function may include a primary object detection function, a secondary object detection function (object movement detection function, object movement detection function, 3D shape recognition function), and an object integration detection function.

[0293] FIG. 11 shows a real-world image (1100_REAL) in which a point cloud-based object detection device (100) according to embodiments of the present disclosure performs an object count measurement function, and a three-dimensional execution result screen (1100) and a two-dimensional execution result screen (1100_PLN) showing the results of the execution of the object count measurement function.

[0294] Referring to FIG. 11, a point cloud-based object detection device (100) is installed in a certain space, and the point cloud-based object detection device (100) can measure the number of objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) existing in the space based on the execution result of a basic detection function.

[0295] Referring to FIG. 11, the location information of objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) can be detected by using the general Doppler component and the micro-Doppler component extracted from the received signal, which is received by the transmission signal transmitted from the point cloud-based object detection device (100) being reflected from the surroundings (objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) and objects in the vicinity thereof).

[0296] Referring to FIG. 11, in the stereoscopic execution result screen (1100) and the planar execution result screen (1100_PLN), a spatial image (1101) representing the space and object images (OBJ1, OBJ2, OBJ3) corresponding to the detected objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) may be displayed.

[0297] Referring to FIG. 11, in the stereoscopic execution result screen (1100), each of the object images (OBJ1, OBJ2, OBJ3) may have a predefined simple stereoscopic shape. For example, each of the object images (OBJ1, OBJ2, OBJ3) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0298] Referring to FIG. 11, in the planar execution result screen (1100_PLN), each of the object images (OBJ1, OBJ2, OBJ3) may have a predefined simple planar shape. For example, each of the object images (OBJ1, OBJ2, OBJ3) may be displayed as a planar shape such as a rectangle or a circle.

[0299] Referring to FIG. 11, the stereoscopic execution result screen (1100) and the planar execution result screen (1100_PLN) may display Doppler points (Pd) where a general Doppler component is generated and micro-Doppler points (Pmd) where a micro-Doppler component is generated.

[0300] Doppler points (Pd) may be points (detection points) where at least one large movement (e.g., movement of a part of an object) is detected among the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL). For example, when an arm or leg moves to walk, the points where there is movement of the object in a situation where there is a change in the object's position and the object's trajectory is detected may be indicated as Doppler points (Pd).

[0301] Micro-Doppler points (Pmd) may be points (detection points) where at least one minute movement (e.g., slight shaking, rotation, vibration, small movement of a part of an object, etc.) of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) is detected. For example, if a person blinks their eyes, moves their skin, or moves their glasses, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0302] The object images (OBJ1, OBJ2, OBJ3) can be displayed in the area defined by the detected Doppler points (Pd) and micro-Doppler points (Pmd) for each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL).

[0303] Referring to FIG. 11, a point cloud-based object detection device (100) can generate object information (OBJ_INFO) for objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) as a result of executing a basic detection function and an object count measurement function.

[0304] Object information (OBJ_INFO) may include an object ID for each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) and corresponding object location information (x, y, z) coordinate values ​​(centrality coordinates). In the object location information (x, y, z) coordinate values ​​(centrality coordinates), the x value may be the center of the x values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object, and the y value may be the center of the y values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object. The z value may be the center of the z values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object. The z value may be the center of the height of the object.

[0305] Referring to FIG. 12, a point cloud-based object detection device (100) according to embodiments of the present disclosure can perform an abnormal situation detection function, which is an example of an application function, by utilizing a basic detection function. Here, the basic detection function may include a primary object detection function, a secondary object detection function (object movement detection function, object movement detection function, 3D shape recognition function), and an object integration detection function.

[0306] The point cloud-based object detection device (100) can perform an abnormal situation detection function along with an object count measurement function.

[0307] FIG. 12 shows a real situation image (1200_REAL) in which a point cloud-based object detection device (100) according to embodiments of the present disclosure performs an abnormal situation detection function, and a three-dimensional execution result screen (1200) and a two-dimensional execution result screen (1200_PLN) showing the execution result of the abnormal situation detection function.

[0308] Looking at the actual situation image (1200_REAL), in the space where the point cloud-based object detection device (100) is installed, an abnormal situation occurs in which three people corresponding to the first to third objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL) assault one person corresponding to the fourth object (OBJ4_REAL).

[0309] Referring to FIG. 12, a point cloud-based object detection device (100) can detect an abnormal situation in the space based on the execution result of a basic detection function. In the example of FIG. 12, the abnormal situation is a situation where three people assault one person.

[0310] Referring to FIG. 12, the state of each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) can be detected by using the general Doppler component and the micro-Doppler component extracted from the received signal, which is received by the transmission signal transmitted from the point cloud-based object detection device (100) being reflected from the surroundings (objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) and objects in the vicinity thereof).

[0311] A point cloud-based object detection device (100) monitors a change in height corresponding to a change in the height center value of each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) and a change in position corresponding to a change in position, and can detect that a sudden abnormal situation (e.g., a situation where a person corresponding to the fourth object (OBJ4_REAL) is assaulted, a situation where a person corresponding to the fourth object (OBJ4_REAL) suddenly collapses, etc.) has occurred in relation to the object (e.g., the fourth object (OBJ4_REAL)) when the height center value and / or position of at least one of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) changes rapidly.

[0312] Referring to FIG. 12, in the stereoscopic execution result screen (1200) and the planar execution result screen (1200_PLN), a spatial image (1201) representing the space and object images (OBJ1, OBJ2, OBJ3, OBJ4) corresponding to the detected objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) may be displayed.

[0313] Referring to FIG. 12, in the stereoscopic execution result screen (1100), each of the object images (OBJ1, OBJ2, OBJ3, OBJ4) may have a predefined simple stereoscopic shape. For example, each of the object images (OBJ1, OBJ2, OBJ3, OBJ4) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0314] If the size of at least one of the object images (OBJ1, OBJ2, OBJ3, OBJ4) (e.g., OBJ4) suddenly decreases, it may be detected that an abnormal situation has occurred. In this way, when an abnormal situation is detected, the size of at least one of the object images (OBJ1, OBJ2, OBJ3, OBJ4) (e.g., OBJ4) may be different from the size of the others (e.g., OBJ1, OBJ2, OBJ3).

[0315] Referring to FIG. 12, in the planar execution result screen (1100_PLN), each of the object images (OBJ1, OBJ2, OBJ3, OBJ4) may have a predefined simple planar shape. For example, each of the object images (OBJ1, OBJ2, OBJ3, OBJ4) may be displayed as a planar shape such as a rectangle or a circle.

[0316] Referring to FIG. 12, the stereoscopic execution result screen (1200) and the planar execution result screen (1200_PLN) may display Doppler points (Pd) where a general Doppler component is generated and micro-Doppler points (Pmd) where a micro-Doppler component is generated.

[0317] Doppler points (Pd) may be points (detection points) where at least one large movement (e.g., movement of a part of an object) is detected among the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL). For example, when an arm or leg moves to walk, the points (detection points) where there is movement of the object in a situation where there is a change in the position of the object and the trajectory of the object is detected may be indicated as Doppler points (Pd).

[0318] Micro-Doppler points (Pmd) may be points (detection points) where at least one minute movement (e.g., slight shaking, rotation, vibration, small movement of a part of an object, etc.) of at least one of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) is detected. For example, if a person blinks their eyes, moves their skin, or moves their glasses, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0319] The object images (OBJ1, OBJ2, OBJ3, OBJ4) can be displayed in the area defined by the detected Doppler points (Pd) and micro-Doppler points (Pmd) for each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL).

[0320] Referring to FIG. 12, a point cloud-based object detection device (100) can generate object information (OBJ_INFO) for objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) as a result of executing a basic detection function and an abnormal situation detection function.

[0321] Object information (OBJ_INFO) may include an object ID for each of the objects (OBJ1_REAL, OBJ2_REAL, OBJ3_REAL, OBJ4_REAL) and corresponding object location information (x, y, z) coordinate values ​​(centrality coordinates). In the object location information (x, y, z) coordinate values ​​(centrality coordinates), the x value may be the center value of the x values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object, and the y value may be the center value of the y values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object. The z value may be the center value of the z values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object. The z value may be the center value of the height of the object. In the object information where the object ID is 0, it can be seen that the z value is significantly smaller than the z value of other objects. The abnormal situation detection unit (1020) of the point cloud-based object detection device (100) can recognize that the object with object ID 0 corresponds to the fallen fourth object (OBJ4) and is in a situation where it is being beaten by other people nearby.

[0322] FIG. 13 is an additional block diagram of a point cloud-based object detection device (100) according to embodiments of the present disclosure. FIG. 14 and FIG. 15 are diagrams illustrating an object presence detection function of a point cloud-based object detection device (100) according to embodiments of the present disclosure. FIG. 16 is a diagram illustrating an object anomaly detection function of a point cloud-based object detection device (100) according to embodiments of the present disclosure.

[0323] Referring to FIG. 13, a point cloud-based object detection device (100) according to embodiments of the present disclosure may further include an installation space information management unit (1000) that sets, stores, and updates spatial information for a space where the point cloud-based object detection device (100) is installed. For example, the spatial information may include at least one of size information for a space where the point cloud-based object detection device (100) is installed, object information existing in a space where the point cloud-based object detection device (100) is installed, and characteristic information for a space where the point cloud-based object detection device (100) is installed.

[0324] At least one of the primary object detection unit (110), secondary object detection unit (120), and object integration detection unit (130) included in the point cloud-based object detection device (100) can perform the detection operation by referring to spatial information.

[0325] The object integration detection unit (130) can determine the number of objects based on object integration information, assign an object ID to each object, and store the object ID and object integration information in conjunction.

[0326] Referring to FIG. 13, a point cloud-based object detection device (100) according to embodiments of the present disclosure may further include an object presence detection unit (1310) that detects whether an object exists in a space where the point cloud-based object detection device (100) is installed, based on object integration information and spatial information.

[0327] Referring to FIG. 13, a point cloud-based object detection device (100) according to embodiments of the present disclosure may further include an object abnormality detection unit (1320) that detects whether an object is abnormal based on object abnormality determination information identified based on object integration information when it is determined by an object presence detection unit (1310) that an object exists in the space where the point cloud-based object detection device (100) is installed. For example, the object abnormality determination information may include at least one of the object's location, trajectory, velocity, state, height center value, and height change amount of the height center value.

[0328] If the object abnormality detection unit (1320) determines that the object is in an abnormal state (abnormal situation), it can perform notification processing (e.g., notification processing such as sound, vibration, etc.) in a preset manner.

[0329] When the object abnormality detection unit (1320) determines that the object is in an abnormal state (abnormal situation), it can use a communication module included in the point cloud-based object detection device (100) to send an emergency situation notification message to a recipient with pre-set emergency contact information.

[0330] FIG. 14 shows an actual detection situation image (1400_REAL_t1) representing a detection area detected at a first time point (t1) by an object presence detection unit (1310) of a point cloud-based object detection device (100) according to embodiments of the present disclosure, and a three-dimensional detection result screen (1400_t1) and a planar detection result screen (1400_PLN_t1) representing the object presence detection result at the first time point (t1) by the object presence detection unit (1310).

[0331] FIG. 15 shows an actual detection situation image (1400_REAL_t2) representing a detection area detected at a second time point (t2) by an object presence detection unit (1310) according to embodiments of the present disclosure, and a three-dimensional detection result screen (1400_t2) and a planar detection result screen (1400_PLN_t2) representing the object presence detection result at the second time point (t2) by a three-dimensional shape recognition unit (230) of a point cloud-based object detection device (100).

[0332] Referring to FIG. 14, a point cloud-based object detection device (100) is installed in a designated space (e.g., a restroom, a room, etc.), and the object presence detection unit (1310) of the point cloud-based object detection device (100) can detect whether an object exists within the space at a first time point (t1).

[0333] Referring to FIG. 14, when referring to the stereoscopic detection result screen (1400_t1) and the planar stereoscopic detection result screen (1400_PLN_t1) based on object presence detection at the first time point (t1), the location information of an object can be detected by using the general Doppler component and the micro-Doppler component extracted from the received signal that is received by the surroundings (object (OBJ_REAL) and objects in its vicinity) where the transmitted signal transmitted by the point cloud-based object detection device (100) is reflected.

[0334] Referring to FIG. 14, since there is no object in the space at the first time point (t1), only the space image (1401) representing the space and the object image (1420) of an object (e.g., a toilet) existing in the space are displayed on the stereoscopic detection result screen (800_t1) and the planar stereoscopic detection result screen (800_PLN_t1), and no object image is displayed.

[0335] Therefore, the Doppler points (Pd) where a general Doppler component is generated and the micro-Doppler points (Pmd) where a micro-Doppler component is generated are not displayed on the stereoscopic detection result screen (1400_t1) and the planar stereoscopic detection result screen (1400_PLN_t1).

[0336] Referring to FIG. 14, the object information (OBJ_INFO_t1) at the first time point (t1) does not include any object ID and its corresponding information.

[0337] Referring to FIG. 15, when referring to the stereoscopic detection result screen (1400_t2) and the planar stereoscopic detection result screen (1400_PLN_t2) based on object presence detection at the second time point (t2), the location information of the object (OBJ_REAL) can be detected by using the general Doppler component and the micro-Doppler component extracted from the received signal that is received by the surroundings (object (OBJ_REAL) and objects in its vicinity) where the transmitted signal transmitted by the point cloud-based object detection device (100) is reflected.

[0338] Referring to FIG. 15, the stereoscopic detection result screen (1400_t2) and the planar stereoscopic detection result screen (1400_PLN_t2) display a spatial image (1401) representing the space and an object image (1402) within the space, and an object image (OBJ_t2) corresponding to the detected object (OBJ_REAL) may be displayed.

[0339] Referring to FIG. 15, in the stereoscopic detection result screen (1400_t2), the object image (OBJ_t2) may have a predefined simple stereoscopic shape. For example, the object image (OBJ_t2) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0340] Referring to FIG. 15, in the plane detection result screen (1400_PLN_t2), the object image (OBJ_t2) may have a predefined simple plane shape. For example, the object image (OBJ_t2) may be displayed as a plane shape such as a rectangle or a circle.

[0341] Referring to FIG. 15, in the stereoscopic detection result screen (1400_t2), each of the spatial image (1401) and the object image (1402) may have a predefined simple stereoscopic shape. For example, each of the spatial image (1401) and the object image (1402) may be displayed as a stereoscopic shape such as a rectangular prism or a cylinder.

[0342] Referring to FIG. 15, in the plane detection result screen (1400_PLN_t2), the space image (1401) and the object image (1402) may each have a predefined simple plane shape. For example, the space image (1401) and the object image (1402) may each be displayed as a plane shape such as a square or a circle.

[0343] In addition, the stereoscopic detection result screen (1400_t2) and the planar stereoscopic detection result screen (1400_PLN_t2) may display Doppler points (Pd) where a general Doppler component is generated and micro-Doppler points (Pmd) where a micro-Doppler component is generated.

[0344] Doppler points (Pd) may be points (detection points) where large movements of the object (OBJ_REAL) (e.g., movement of a part of the object) are detected. For example, when an arm or leg moves to walk, the points (detection points) where there is movement of the object in a situation where there is a change in the object's position and the object's trajectory is detected may be indicated as Doppler points (Pd).

[0345] Micro-Doppler points (Pmd) may be points (detection points) where minute movements of an object (OBJ_REAL) (e.g., slight shaking, rotation, vibration, small movement of a part of the object, etc.) are detected. For example, if a person blinks their eyes, their skin moves, or their glasses move, the eyes, the moving skin, or the part of the glasses may be marked as micro-Doppler points (Pmd).

[0346] The object image (OBJ_t2) can be displayed in the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL).

[0347] Referring to FIG. 15, in the object information (OBJ_INFO_t2) for the object (OBJ_REAL) obtained as a result of detecting the presence of the object at the second time point (t2), the (x, y, z) coordinate values ​​(central mass coordinates) of the object (OBJ_REAL) are (0.073, 0.376, 1.278).

[0348] Referring to FIG. 15, in the (x, y, z) coordinate values, the x value may be the center value of the x values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL), and the y value may be the center value of the y values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the z values ​​of the area defined by the Doppler points (Pd) and micro-Doppler points (Pmd) detected for the object (OBJ_REAL). The z value may be the center value of the height of the object (OBJ_REAL) at the first time point (t1).

[0349] Referring to FIGS. 14 and 15, the object presence detection unit (1310) can determine that an object exists at the second time point (t2) because object information (OBJ_INFO_t2) occurred at the second time point (t2).

[0350] When the object presence detection unit (1310) is operating, the object abnormality detection unit (1320) can operate together. The object abnormality detection unit (1320) can determine whether the (x, y, z) coordinate values, which are the location information of an object (OBJ_REAL) included in the object information (OBJ_INFO_t2) which is the object detection result at the second time point (t2) in the object presence detection unit (1310), are included in the normal range.

[0351] In particular, the object abnormality detection unit (1320) can determine whether the object (OBJ_REAL) is in a normal state or an abnormal state by determining whether the z value (height center value) included in the object information (OBJ_INFO_t2) at the second time point (t2) falls within a predefined normal value range.

[0352] The object abnormality detection unit (1320) can determine that the object (OBJ_REAL) is in a normal state if the (x, y, z) coordinate values, which are location information of the object (OBJ_REAL), fall within the normal range.

[0353] The object abnormality detection unit (1320) can determine that the object (OBJ_REAL) is in an abnormal state (abnormal situation) if the (x, y, z) coordinate values, which are location information of the object (OBJ_REAL), are not included in the normal range.

[0354] Referring to FIG. 16, the object abnormality detection unit (1320) can detect, as a result of detecting the presence of an object at a third time point (t3), that an object (OBJ_REA) exists in the space, but the state of the object (OBJ_REAL) is abnormal (e.g., a falling state).

[0355] Referring to FIG. 16, when assuming that at the third time point (t3) following the second time point (t2), a person who is the object (OBJ_REAL) collapses or loses consciousness in the space (e.g., restroom, room, etc.), the object abnormality detection unit (1320) can determine that the state of the object (OBJ_REAL) is abnormal based on the object integration information of the object (OBJ_REAL).

[0356] For example, the object abnormality detection unit (1320) can determine, based on the object integration information of the object (OBJ_REAL), that the height center value (z value or its corresponding value) of the object (OBJ_REAL) at the third time point (t3) has become smaller than the height center value (z value or its corresponding value) of the object (OBJ_REAL) at the second time point (t2), and determine that the state of the object (OBJ_REAL) is an abnormal state, specifically a falling state. In this regard, the spatial image (1401) representing the space and the object image (1402) within the space are continuously displayed on the stereoscopic detection result screen (1400_t3) and the planar stereoscopic detection result screen (1400_PLN_t3) at the third time point (t3), and the object image (OBJ_t2) corresponding to the detected object (OBJ_REAL) may also be continuously displayed.

[0357] However, in the stereoscopic detection result screen (1400_t3), the object image (OBJ_t3) at the third time point (t3) may have a smaller size than the object image (OBJ_t2) at the second time point (t2). This may mean that the object (OBJ_REAL) has fallen.

[0358] Referring to FIG. 16, in the object information (OBJ_INFO_t3) for the object (OBJ_REAL) obtained as a result of detecting the presence of the object at the third time point (t3), the (x, y, z) coordinate values ​​(central mass coordinates) of the object (OBJ_REAL) are (0.107, 0.692, 0.84).

[0359] The object abnormality detection unit (1320) can compare the (x, y, z) coordinate values ​​(0.073, 0.376, 1.278), which are location information of the object (OBJ_REAL) at the second time point (t2), with the coordinate values ​​(0.107, 0.692, 0.84), which are location information of the object (OBJ_REAL) at the third time point (t3), and if there is a significant difference beyond a threshold level, it can determine that the object (OBJ_REAL) is in an abnormal state (abnormal situation).

[0360] That is, when comparing the (x, y, z) coordinate values ​​(0.073, 0.376, 1.278), which are the location information of the object (OBJ_REAL) at the second time point (t2), with the coordinate values ​​(0.107, 0.692, 0.84), which are the location information of the object (OBJ_REAL) at the third time point (t3), the x and y values ​​have increased, and the z value has decreased. This may mean that the height of the area corresponding to the person, which is the object (OBJ_REAL), has decreased and widened.

[0361] Meanwhile, the object abnormality detection unit (1320) may set different object information defining the abnormal state (abnormal situation) of an object according to the space information set in the installation space information management unit (1000). For example, the height center value of an object or the amount of change in height of an object indicating the abnormal state of an object may differ depending on the space information.

[0362] A method for determining a fall condition can be executed by utilizing the center of gravity analysis step (S340), Doppler velocity analysis step (S350), and fall detection step (S370) described with reference to FIG. 3a.

[0363] And, if it is determined that the object has fallen, additional risk situation detection and notification processing regarding breathing status, etc. can be performed through the execution of the additional risk situation detection and notification step (S380).

[0364] FIG. 17 is a block diagram of a point cloud-based object detection device (1700) according to embodiments of the present disclosure.

[0365] Referring to FIG. 17, a point cloud-based object detection device (1700) according to embodiments of the present disclosure may include: a multi-Doppler component extraction unit (1705) that transmits a transmission signal and receives a signal reflected from the surroundings as a reception signal, and extracts a plurality of Doppler components corresponding to different types from the reception signal; a first detection unit (1710) that obtains first detection information including two-dimensional position information of an object existing in the surroundings based on the plurality of Doppler components; a second detection unit (1720) that obtains second detection information including velocity information of the object along with two-dimensional position information based on the plurality of Doppler components; and a third detection unit (1730) that obtains third detection information including a height center value of the object along with the second detection information based on the plurality of Doppler components.

[0366] The first detection unit (1710) is called the first radar unit, the second detection unit (1720) is called the second radar unit (3D radar unit), and the third detection unit (1730) can be called the third radar unit (4D radar unit).

[0367] The multi-Doppler component extraction unit (1705) can extract a Doppler component from a received signal as a general Doppler component among a plurality of Doppler components during a first time period, and extract a Doppler component from a received signal as a micro-Doppler component among a plurality of Doppler components during a second time period. Here, the second time period for extracting the micro-Doppler component can be set to be longer than the first time period for extracting the general Doppler component.

[0368] The third sensing unit (1730) can estimate a height center value from height values ​​detected for an object and recognize the three-dimensional shape of the object based on changes over time for the estimated height center value.

[0369] The third sensing unit (1730) can determine the state of the object based on the recognition result of the object's three-dimensional shape.

[0370] The third detection unit (1730) can determine an abnormal situation of the object by monitoring changes in the object's state over time.

[0371] The point cloud-based object detection device (1700) according to embodiments of the present disclosure may be the point cloud-based object detection device (100) of FIG. 1. The first detection unit (1710) and the second detection unit (1720) correspond to the first object detection unit (110), and the third detection unit (1730) may correspond to the second object detection unit (120).

[0372] A display device according to embodiments of the present disclosure can be described as follows.

[0373] A point cloud-based object detection device according to embodiments of the present disclosure comprises: a transmitting antenna device that transmits a transmission signal; a receiving antenna device that receives a signal reflected from an object located in the vicinity of the transmission signal as a receiving signal; a primary object detection unit that obtains primary object information from the receiving signal, wherein, based on the receiving signal, it obtains primary object information including distance information, azimuth information, and elevation angle information corresponding to each of the detection points, which are points where the transmission signal is reflected from the object, and obtains three-dimensional coordinate information for each of the detection points through the distance information, azimuth information, and elevation angle information for each of the detection points, and further obtains primary object information including three-dimensional coordinate information for each of the detection points; and a secondary object detection unit that, based on the primary object information, generates a three-dimensional point cloud including detection points, clusters the detection points included in the three-dimensional point cloud into object units to recognize the object by distinguishing it from other objects, and obtains secondary object information for the object, wherein it obtains secondary object information including the coordinates of the object's center of gravity based on the three-dimensional coordinate information for each of the detection points in the object. It may include an object integration detection unit that determines whether an object has fallen based on changes in the center of gravity coordinates of the object and detects the determination result as object integration information.

[0374] The first object detection unit extracts a Doppler component as a general Doppler component from a received signal during a first time, which is a predefined general Doppler component extraction time, and extracts a Doppler component as a micro-Doppler component from a received signal during a second time, which is a predefined micro-Doppler component extraction time, and the second time may be set to be longer than the first time.

[0375] The primary object detection unit can extract distance information, azimuth information, and elevation information from the received signal, or extract distance information, azimuth information, and elevation information from the received signal.

[0376] The secondary object detection unit calculates the center of gravity coordinates of the object based on 3D coordinate information of detection points on the object and can detect changes in the center of gravity coordinates of the object over a certain period of time. The object integration detection unit can determine whether the object has fallen by detecting whether the center of gravity of the object has dropped based on changes in the center of gravity coordinates of the object.

[0377] The object integration detection unit compares the change in the z-axis coordinate, which represents a change in the z-axis coordinate among the object's center of gravity coordinates over a certain period of time, with a predetermined fall judgment threshold change value, and if the change in the z-axis coordinate exceeds the fall judgment threshold change value, it can determine that a descent corresponding to a fall has occurred in the object's center of gravity.

[0378] The secondary object detection unit can calculate the ratio of the number of detection points located above a predetermined height out of the total number of detection points forming the object as the upper density of the object.

[0379] The object integration detection unit can additionally determine that the object has fallen if the upper density of the object changes below a predetermined threshold density value for a certain period of time.

[0380] The object integration detection unit can finally determine whether the object has fallen based on the result of determining whether it has fallen based on the change in the center of gravity coordinates and the result of determining whether it has fallen based on the upper density.

[0381] The secondary object detection unit detects the velocity of the object in the z-axis direction, and the object integration detection unit can additionally determine that the object has fallen if the velocity in the z-axis direction exceeds a predefined threshold velocity.

[0382] The secondary object detection unit can detect distance information, azimuth information, and elevation angle information from the received signal, as well as the velocity of the object in the z-axis direction.

[0383] The object integration detection unit can finally determine whether the object has fallen based on the result of determining whether it has fallen based on the change in the center of gravity coordinates and the result of determining whether it has fallen based on the velocity in the z-axis direction.

[0384] The secondary object detection unit can detect a phase change of a received signal corresponding to each of the detection points corresponding to the chest among the detection points forming the human body identified as an object, and a change in distance from the detection points corresponding to the chest, and output a respiration-related information detection result including information on the phase change or distance change.

[0385] The object integration detection unit can detect chest movement based on the detection result of breathing-related information, generate a breathing signal whose signal value changes over time based on the detection result of chest movement, and determine the breathing state of the object based on the breathing signal.

[0386] The secondary object detection unit can extract a micro-Doppler component from the received signal and, based on the micro-Doppler component, detect a phase change of the received signal corresponding to each of the detection points corresponding to the chest among the detection points forming the human body identified as an object, and a change in distance from the detection points corresponding to the chest.

[0387] The object integration detection unit can determine the breathing state of an object if it is determined that the object has fallen based on the result of determining whether the object has fallen according to the change in the center of gravity coordinates, and if the object's breathing state is determined to be in a state of sudden change in breathing or cessation of breathing, it can perform notification processing for an emergency situation.

[0388] The system may further include an installation space information management unit that sets, stores, and updates spatial information for a space where a point cloud-based object detection device is installed. The spatial information may include at least one of size information for a space where a point cloud-based object detection device is installed, object information existing in a space where a point cloud-based object detection device is installed, and characteristic information for a space where a point cloud-based object detection device is installed. At least one of a primary object detection unit, a secondary object detection unit, and an object integration detection unit may operate by referencing the spatial information.

[0389] A point cloud-based object detection device according to embodiments of the present disclosure comprises: a primary object detection unit that receives a signal reflected from an object located in the vicinity as a receiving signal, and obtains primary object information from the receiving signal, wherein, based on the receiving signal, the primary object information includes distance information, azimuth information, and elevation angle information corresponding to each of the detection points, which are points where the transmitting signal is reflected from the object, and obtains three-dimensional coordinate information for each of the detection points through the distance information, azimuth information, and elevation angle information for each of the detection points, and further obtains primary object information including three-dimensional coordinate information for each of the detection points; and a secondary object detection unit that, based on the primary object information, generates a three-dimensional point cloud including detection points, clusters the detection points included in the three-dimensional point cloud into object units to recognize the object by distinguishing it from other objects, and obtains secondary object information for the object, wherein, based on the three-dimensional coordinate information for each of the detection points in the object, the secondary object information includes location information of the object or movement information of the whole or part of the object. It may include an object integration detection unit that detects the state of an object based on secondary object information.

[0390] The object may have one of the characteristics of a general Doppler object having a speed greater than a predefined threshold speed or motion characteristics greater than a predetermined level, one of the characteristics of a micro Doppler object having a speed less than a predefined threshold speed or motion characteristics less than a predetermined level or stationary, one of the characteristics of a 3D Doppler object having distance information, azimuth information and elevation information, and one of the characteristics of a 4D Doppler object having distance information, azimuth information and elevation information, along with additional 3D velocity vector information.

[0391] The secondary object detection unit can calculate the center of gravity coordinates of the object from the three-dimensional coordinate information for each of the detection points in the object, and obtain information about the center of gravity coordinates as secondary object information.

[0392] The object integration detection unit can detect whether an object has fallen based on changes in the center of gravity coordinates of the object identified from secondary object information.

[0393] The secondary object detection unit can extract a micro-Doppler component from a received signal and, based on the extracted micro-Doppler component, detect the movement of the chest of a human body, which is an object, and obtain information about the movement of the chest as secondary object information.

[0394] The object integration detection unit can detect the breathing state of an object based on information regarding the movement of the human chest, which is an object identified from secondary object information.

[0395] According to the embodiments of the present disclosure described above, a point cloud-based object detection device that performs object detection using heterogeneous composite Doppler components can be provided.

[0396] According to embodiments of the present disclosure, a point cloud-based object detection device capable of precisely and rapidly detecting various information, states, or actions (movements) of an object using heterogeneous composite Doppler components can be provided.

[0397] According to embodiments of the present disclosure, a point cloud-based object detection device can be provided that can perform object detection without infringing on privacy or exposing personal information because it is not based on images.

[0398] According to embodiments of the present disclosure, a point cloud-based object detection device capable of providing various application functions using radar-based object detection technology can be provided.

[0399] The foregoing description is merely an illustrative explanation of the technical concept of the present disclosure, and those skilled in the art to which the present disclosure pertains may make various modifications and variations within the scope of the essential characteristics of the present disclosure. Furthermore, the embodiments disclosed in the present disclosure are intended to explain, not limit, the technical concept of the present disclosure, and thus the scope of the technical concept of the present disclosure is not limited by these embodiments.

Claims

Claim 1 A transmitting antenna device that transmits a transmission signal; a receiving antenna device that receives a signal reflected from an object located in the vicinity of the transmission signal as a receiving signal; and a primary object detection unit that obtains primary object information from the receiving signal, wherein, based on the receiving signal, the primary object information includes distance information, azimuth information, and elevation angle information corresponding to each of the detection points, which are points where the transmission signal is reflected from the object, and obtains three-dimensional coordinate information for each of the detection points through the distance information, azimuth information, and elevation angle information for each of the detection points, and further obtains the primary object information including the three-dimensional coordinate information for each of the detection points. A point cloud-based object detection device comprising: a secondary object detection unit that, based on the primary object information, generates a three-dimensional point cloud including the detection points, clusters the detection points included in the three-dimensional point cloud into object units to recognize the object by distinguishing it from other objects, and obtains secondary object information for the object, wherein the secondary object information including the center of gravity coordinates of the object is obtained based on three-dimensional coordinate information for each detection point in the object; and an object integration detection unit that determines whether the object has fallen based on a change in the center of gravity coordinates of the object, detects the determination result as object integration information, determines the number of the object based on the object integration information, assigns an ID to each object, and stores the ID in conjunction with the object integration information, wherein the number of the objects is multiple, and if the z value among the center of gravity coordinates of the first object among the multiple objects is smaller than a certain value than the z value of other objects excluding the first object, the device recognizes it as a beating situation. Claim 2 A point cloud-based object detection device according to claim 1, wherein the primary object detection unit extracts a Doppler component as a general Doppler component from the received signal during a first time, which is a predefined general Doppler component extraction time, and extracts a Doppler component as a micro-Doppler component from the received signal during a second time, which is a predefined micro-Doppler component extraction time, and the second time is set to be longer than the first time. Claim 3 delete Claim 4 A point cloud-based object detection device according to claim 1, wherein the secondary object detection unit calculates the center of gravity coordinates of the object based on three-dimensional coordinate information regarding detection points in the object and detects a change in the center of gravity coordinates of the object over a certain period of time, and the object integration detection unit detects whether the center of gravity of the object descends based on the change in the center of gravity coordinates of the object and determines whether the object has fallen. Claim 5 In claim 4, the object integration detection unit compares a change in the z-axis coordinate, which represents a change in the z-axis coordinate among the center of gravity coordinates of the object over a certain period of time, with a predetermined fall judgment threshold change value, and if the change in the z-axis coordinate exceeds the fall judgment threshold change value, determines that a descent corresponding to a fall has occurred in the center of gravity of the object. This is a point cloud-based object detection device. Claim 6 A point cloud-based object detection device according to claim 1, wherein the secondary object detection unit calculates the ratio of the number of detection points located above a predetermined height to the total number of detection points constituting the object as the upper density of the object, and the object integration detection unit further determines that the object has fallen if the upper density of the object changes to below a predetermined threshold density value for a certain period of time. Claim 7 In claim 6, the object integration detection unit is a point cloud-based object detection device that finally determines whether the object has fallen based on the result of determining whether it has fallen based on the change in the center of gravity coordinates and the result of determining whether it has fallen based on the upper density. Claim 8 A point cloud-based object detection device according to claim 1, wherein the secondary object detection unit detects the velocity of the object in the z-axis direction, and the object integration detection unit additionally determines that the object has fallen when the velocity in the z-axis direction exceeds a predefined threshold speed. Claim 9 delete Claim 10 In claim 8, the object integration detection unit is a point cloud-based object detection device that finally determines whether the object has fallen based on a fall determination result based on a change in the center of gravity coordinates and a fall determination result based on a velocity in the z-axis direction. Claim 11 A point cloud-based object detection device according to claim 1, wherein the secondary object detection unit detects a phase change of a received signal corresponding to each detection point corresponding to the chest among the detection points forming the human body identified as the object, and a change in distance from the detection points corresponding to the chest, and outputs a respiration-related information detection result including information on the phase change or the change in distance, and the object integration detection unit detects chest movement based on the respiration-related information detection result, generates a respiration signal in which the signal value changes over time based on the detection result of the chest movement, and determines the respiration state of the object based on the respiration signal. Claim 12 In claim 11, the secondary object detection unit extracts a micro-Doppler component from the received signal and, based on the micro-Doppler component, detects a phase change of the received signal corresponding to each of the detection points corresponding to the chest among the detection points forming the human body identified as the object, and a change in distance from the detection points corresponding to the chest, a point cloud-based object detection device. Claim 13 In claim 11, the object integration detection unit determines the breathing state of the object when it is determined that the object has fallen based on the result of determining whether the object has fallen according to the change in the center of gravity coordinates, and performs notification processing for an emergency situation when the breathing state of the object is determined to be a state of sudden change in breathing or a state of cessation of breathing, a point cloud-based object detection device. Claim 14 A point cloud-based object detection device according to claim 1, further comprising an installation space information management unit that sets, stores, and updates spatial information for a space in which the point cloud-based object detection device is installed, wherein the spatial information includes at least one of size information for a space in which the point cloud-based object detection device is installed, object information existing in a space in which the point cloud-based object detection device is installed, and characteristic information for a space in which the point cloud-based object detection device is installed, and wherein at least one of the primary object detection unit, the secondary object detection unit, and the object integration detection unit operates by referencing the spatial information. Claim 15 A primary object detection unit that receives a signal reflected from an object located in the vicinity as a receiving signal, and obtains primary object information from the receiving signal, wherein, based on the receiving signal, the primary object information includes distance information, azimuth information, and elevation angle information corresponding to each of the detection points, which are points where the transmitting signal is reflected from the object, and obtains three-dimensional coordinate information for each of the detection points through the distance information, azimuth information, and elevation angle information for each of the detection points, and further obtains the primary object information including the three-dimensional coordinate information for each of the detection points; A point cloud-based object detection device comprising: a secondary object detection unit that, based on the primary object information, generates a three-dimensional point cloud including the detection points, clusters the detection points included in the three-dimensional point cloud into object units to recognize the object by distinguishing it from other objects, and obtains secondary object information for the object, wherein the secondary object information includes location information of the object or movement information of the whole or part of the object based on three-dimensional coordinate information for each detection point in the object; and an object integration detection unit that, based on the secondary object information, detects the state of the object, determines the number of the object based on object integration information, assigns an ID to each object, and stores the ID in conjunction with the object integration information, wherein the number of objects is multiple, and if the z value among the center of gravity coordinates of the first object among the multiple objects is smaller than the z value of other objects excluding the first object, the device recognizes it as a beating situation. Claim 16 delete Claim 17 A point cloud-based object detection device according to claim 15, wherein the secondary object detection unit calculates the center of gravity coordinates of the object from three-dimensional coordinate information for each of the detection points in the object and obtains information regarding the center of gravity coordinates as the secondary object information, and the object integration detection unit detects whether the object has fallen based on the change in the center of gravity coordinates of the object confirmed from the secondary object information. Claim 18 A point cloud-based object detection device according to claim 15, wherein the secondary object detection unit extracts a micro-Doppler component from the received signal and, based on the extracted micro-Doppler component, detects the movement of the chest of the human body, which is the object, and obtains information regarding the movement of the chest as the secondary object information, and the object integration detection unit detects the breathing state of the object based on the information regarding the movement of the chest of the human body, which is the object, confirmed from the secondary object information.

Images