Gesture sensing system and electronic device

Abstract

To provide a gesture sensing system and an electronic device capable of sensing a gesture of an object.SOLUTION: According to an aspect of the present invention, there is provided a gesture detection system including a sensing unit configured to detect a gesture of an object and generate and output input information including a plurality of detection signals, a preprocessing unit configured to detect a peak signal among the plurality of detection signals and convert the input information into correction input information using information on the peak signal, and a classification unit configured to be learned using learning data, receive the correction input information, and classify the gesture. The information on the peak signals may include information on positions of the peak signals and intensities of the peak signals, and the positions of the peak signals and the intensities of the peak signals may be uniformized within a predetermined range and provided to the classifier.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a gesture sensing system and an electronic device, and more particularly to a gesture sensing system and an electronic device that can sense the gesture of an object. [Background technology]

[0002] Millimeter waves (mmWave) are a broadband frequency band between 30 GHz (Gigahertz) and 300 GHz. Because mmWave has strong directionality and is not affected by weather conditions such as rain or fog, it is also used in autonomous driving technology, such as collision avoidance.

[0003] Furthermore, because antennas for transmitting and receiving millimeter waves can be miniaturized, they can also be used in crime prevention sensors for monitoring and surveillance of traffic volume. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Chinese Patent Application Publication No. 115877376 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention aims to provide a gesture sensing system and an electronic device that can sense the gesture of an object. [Means for solving the problem]

[0006] The gesture detection system and electronic device according to the present invention include a sensing unit that senses a gesture of an object, generates input information including a plurality of sensing signals, and outputs the generated input information, a pre-processing unit that detects peak signals from the plurality of sensing signals and converts the input information into corrected input information using information about the peak signals, and a classifying unit that is trained using learning data, receives the corrected input information, and classifies the gesture. The information about the peak signals includes information about positions and intensities of the peak signals, and the positions and intensities of the peak signals are uniformed within a certain range and provided to the classifying unit.

[0007] The pre-processing unit includes a motion detection unit that detects whether the gesture is detected, a signal detection unit that detects the peak signal from the plurality of detection signals, a position correction unit that corrects the position of the peak signal and outputs the corrected position of the peak signal, and an intensity correction unit that corrects the intensity of the peak signal and outputs the corrected intensity of the peak signal.

[0008] The input information further includes a range signal including information about a distance between the sensing unit and the object, and a Doppler signal including information about a velocity of the object. The position of each of the plurality of sensing signals is a position on a range-Doppler map represented by the range signal and the Doppler signal, the sensing unit generates the input information for each frame, and the corrected input information includes information about corrected positions and corrected intensities of the peak signals for all frames.

[0009] The motion detection unit calculates the strength of a differential signal based on the difference in signal strength for the range-Doppler map between two adjacent frames, calculates the average value of the strength of the differential signal for each predetermined frame, and determines whether the gesture is detected based on the average value.

[0010] The motion detection unit calculates a differential signal strength based on the difference in signal strength for the range-Doppler map between two adjacent frames, calculates a normal signal strength by normalizing the differential signal strength, and determines whether the gesture is detected based on the normal signal strength.

[0011] The signal detector selects a sensing signal having an intensity greater than a set threshold from the plurality of sensing signals.

[0012] The signal detection unit selects a first signal, a second signal, a third signal, and a fourth signal from the selected sensing signals for each frame, the first signal being a sensing signal having the greatest intensity among the selected sensing signals, the second signal being a sensing signal having the smallest difference from a center position of the selected sensing signals, the third signal being a sensing signal having the smallest difference in position between a previous frame and a current frame among the selected sensing signals, and the fourth signal being a sensing signal located at a position where Doppler is minimum or maximum among the selected sensing signals.

[0013] The signal detector detects the sensing signal at the smallest position of the range among the first to fourth sensing signals as the peak signal.

[0014] The position correction unit determines whether the current frame is a first frame, which is the frame at the time when the motion detection unit determines that the gesture has started.

[0015] The position corrector calculates a start position according to the position of the detected peak signal if the current frame is the first frame, and calculates a difference between the position of the current frame and the position of the previous frame if the current frame is not the first frame.

[0016] The position corrector does not correct the position of the current frame if the difference is within a reference distance, and corrects the position of the current frame if the difference is greater than the reference distance.

[0017] The intensity corrector corrects the intensity of the peak signal using the learning data, and the learning data includes information on the correlation between the range of the peak signal and the intensity of the peak signal.

[0018] The training data may include information on all of the gestures related to the training subject in one file, or may include information on each gesture related to the training subject individually.

[0019] When the learning data includes information on each gesture related to the learning target, the intensity correction unit compares the intensity of a peak signal for each gesture with the intensity of a peak signal among all frames, selects a gesture among the gestures having the closest peak signal intensity among all frames, and corrects the intensity of the peak signal based on the selected gesture.

[0020] The sensing unit generates the intermediate signal based on a transmission signal and a reception signal, converts the intermediate signal into intermediate signal data by analog-to-digital conversion, and converts the intermediate signal data into the input information by applying a Fourier transform.

[0021] The sensing unit applies the Fourier transform to the intermediate signal data to calculate the range signal, and applies the Fourier transform to the range signal to calculate the Doppler signal.

[0022] The transmitted signal and the received signal are millimeter waves.

[0023] The classifier includes a convolutional neural network and a long-short-term memory, and in a learning process of the classifier, the convolutional neural network receives the learning data and is trained, and in a classification process of the classifier, the long-short-term memory outputs a predicted gesture.

[0024] The electronic device according to the present invention includes a sensing unit that senses a gesture of an object, generates input information including a plurality of sensing signals, and outputs the generated input information, a preprocessing unit that detects a peak signal from the plurality of sensing signals and converts the input information into corrected input information using information about the peak signal, and a classifying unit that is trained using learning data, receives the corrected input information, and classifies the gesture. The information about the peak signal includes information about a position of the peak signal and an intensity of the peak signal, and the position of the peak signal and the intensity of the peak signal are uniformized within a certain range and provided to the classifying unit. [Effects of the Invention]

[0025] According to the present invention, a gesture monitoring system may improve the efficiency of gesture classification by including a preprocessing unit that can correct various information contained in input information within a certain range. Even if highly variable input information is input under various environmental conditions, the accuracy of gesture recognition may be improved by performing a uniform process using the preprocessing unit. In addition, the number of data sets required for training the gesture detection system may be reduced, thereby achieving a more compact gesture detection system. [Brief explanation of the drawings]

[0026] [Figure 1] FIG. 1 is a block diagram of a gesture sensing system according to one embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram of a sensing unit according to an embodiment of the present invention. [Figure 3a] FIG. 2 is an exemplary diagram illustrating range and Doppler according to one embodiment of the present invention. [Figure 3b] FIG. 2 is a diagram illustrating an exemplary range-Doppler map according to one embodiment of the present invention. [Figure 4] FIG. 2 is a block diagram of a preprocessing unit according to an embodiment of the present invention. [Figure 5] 10 is a flowchart illustrating an operation of a preprocessing unit according to an embodiment of the present invention. [Figure 6]10 is a graph showing values ​​used in a motion detection process of a motion detector according to an embodiment of the present invention; [Figure 7] 10 is a flowchart illustrating an operation of a signal detection unit according to an embodiment of the present invention. [Figure 8] 5A and 5B are diagrams illustrating the operation of a position correction unit according to an embodiment of the present invention. [Figure 9] 1 is a graph illustrating the correlation between peak distance and peak intensity according to one embodiment of the present invention. [Figure 10] 10 is a flowchart illustrating an operation of an intensity correction unit according to an embodiment of the present invention. [Figure 11a] 10 is a graph illustrating the correlation between peak distance and peak intensity for the A gesture according to one embodiment of the present invention. [Figure 11b] 10 is a graph illustrating the correlation between peak distance and peak intensity for the B gesture according to one embodiment of the present invention. [Figure 11c] 10 is a graph illustrating the correlation between peak distance and peak intensity for a C gesture according to one embodiment of the present invention. [Figure 11d] 10 is a graph illustrating the correlation between peak distance and peak intensity for the D gesture according to one embodiment of the present invention. [Figure 12a] FIG. 2 is a block diagram of a classifier according to one embodiment of the present invention. [Figure 12b] FIG. 2 is a block diagram of a classifier according to one embodiment of the present invention. [Figure 13] FIG. 10 is a diagram illustrating types of gestures according to an embodiment of the present invention. [Figure 14a] 10 is a graph showing the correlation between peak distance and peak intensity according to a comparative example. [Figure 14b] 1 is a graph illustrating the correlation between peak distance and peak intensity according to one embodiment of the present invention. [Figure 15] 1 is a block diagram of an electronic device according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0027] As used herein, when a component (or region, layer, portion, etc.) is referred to as being "on," "coupled," or "bonded" to another component, it means that it may be directly positioned, coupled, or bonded to the other component, or that a third component may be disposed therebetween.

[0028] The same reference numerals refer to the same elements. In the drawings, the thickness, proportions, and dimensions of the elements are exaggerated for the purpose of effectively explaining the technical content. "And / or" includes all combinations of one or more elements defined by the associated elements.

[0029] Terms such as "first" and "second" are used to describe various components, but the components are not limited to these terms. These terms are used only to distinguish one component from another. For example, a first component may be designated as a "second component" without departing from the scope of the present invention, and similarly, a second component may be designated as a "first component." A singular expression includes a plural expression unless the context clearly dictates otherwise.

[0030] Furthermore, terms such as "under," "below," "on," and "above" are used to describe the relationship between components shown in the drawings. These terms are relative concepts and are described based on the directions shown in the drawings.

[0031] It should be understood that the terms "comprise" or "have" and the like specify the presence of any feature, number, step, operation, component, part, or combination thereof set forth above in the specification, but do not preclude the presence or possible addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0032] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by a person skilled in the art to which the present invention belongs. Furthermore, terms that are the same as those defined in commonly used dictionaries should be interpreted to have a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted in an overly ideal or formal sense unless explicitly defined herein.

[0033] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0034] FIG. 1 is a block diagram of a gesture sensing system according to one embodiment of the present invention.

[0035] Referring to FIG. 1, the gesture sensing system 10 may include a sensing unit 100, a preprocessing unit 200, and a classification unit 300.

[0036] The gesture sensing system 10 may sense the object 20 by utilizing a phase difference between the transmitted signal TX and the received signal RX caused by the physical distance between the gesture sensing system 10 and the object 20 .

[0037] The sensing unit 100 may transmit a transmission signal TX to the object 20 and receive a reception signal RX that is a reflection of the transmission signal TX from the object 20. The sensing unit 100 may generate and output input information 110 for each frame based on the transmission signal TX and the reception signal RX. The input information 110 may include a plurality of digitized sensing signals, a signal related to the distance between the sensing unit 100 and the object 20, and a signal related to the relative velocity of the object 20. The sensing signals of a corresponding frame may be represented on a Doppler-range map RDM (see FIG. 3b) through analysis of the range signal and the Doppler signal. Hereinafter, information related to the distance between the sensing unit 100 and the object 20 will be referred to as range, and information related to the relative velocity of the object 20 will be referred to as Doppler.

[0038] The pre-processing unit 200 may receive the input information 110 from the sensing unit 100, convert the input information 110 into corrected input information 250, and output the corrected input information 250. The corrected input information 250 may include information that is more uniform than the input information 110 because it is information obtained by correcting various information included in the input information 110 within a certain range. Information about some sensing signals (or peak signals) included in the input information 110 may include information about the position and intensity of the peak signals. The corrected input information 250 may include, among information about the peak signals, corrected positions of the peak signals obtained by correcting the positions of the peak signals within a certain range. The pre-processing unit 200 may generate corrected intensities of the peak signals by correcting the intensities of the peak signals within a certain range based on the corrected positions of the peak signals. The pre-processing unit 200 may use learning data 310 in the process of correcting the intensities of the peak signals.

[0039] The classification unit 300 receives the corrected input information 250 from the preprocessing unit 200 and receives the training data 310 from the user. Note that the classification unit 300 may receive the training data 310 from a network instead of from the user. For example, the classification unit 300 may collect information necessary to correct the intensity of peak signals from the network and receive the training data 310 from the network based on the collected information. The classification unit 300 may also receive the training data 310 from the user and from the network during the training process. The classification unit 300 may perform training based on the training data 310. The classification unit 300 receives the corrected input information 250 from the preprocessing unit 200 during the classification process and may represent the gesture classification of the training target as a gesture probability by type. For example, the classification unit 300 represents the similarity between a gesture in each gesture classification and the training target 310 as a gesture probability, and calculates the gesture probability for the training target 310 for each gesture classification type. The classifier 300 may select and output the gesture with the highest probability as the predicted gesture 320 .

[0040] FIG. 2 is a block diagram of a sensing unit according to an embodiment of the present invention.

[0041] Referring to FIG. 2, the sensing unit 100 may include a transmitter 120, a transmitting antenna 130, a receiving antenna 140, a signal mixing unit 150, a receiver 160, and a signal processing unit 170.

[0042] The transmitter 120 may output a millimeter-wave transmit signal TX to the transmit antenna 130 and the signal mixer 150. The millimeter-wave transmit signal TX output from the transmit antenna 130 may be reflected by the object 20 and transmitted to the receive antenna 140 as a millimeter-wave receive signal RX. The signal mixer 150 may combine the transmit signal TX received from the transmitter 120 and the receive signal RX received from the receive antenna 140 to generate and output an intermediate signal IF. The transmit signal TX, the receive signal RX, and the intermediate signal IF may have different frequencies. In one example of the present invention, the transmit signal TX is a frequency modulation signal whose frequency increases linearly from 77 GHz to 81 GHz, and the transmitter 120 may repeatedly output the transmit signal TX in a pulsed form at regular time intervals.

[0043] The intermediate signal IF may include information about the phase difference between the transmitted signal TX and the received signal RX caused by the difference in physical distance between the transmitting antenna 130 and the receiving antenna 140, the strength of the received signal RX, and the like.

[0044] The receiver 160 may receive the intermediate signal IF from the signal mixing unit 150, and may perform filtering and / or analog-to-digital conversion on the intermediate signal IF to generate and output intermediate signal data IFD. For example, the intermediate signal IF may be an analog signal, and the intermediate signal data IFD may be a digital signal.

[0045] The signal processor 170 receives intermediate signal data IFD from the receiver 160, converts the intermediate signal data IFD to generate input information 110, and outputs the input information 110 for each frame. The input information 110 may include a plurality of sensing signals, a range signal, and a Doppler signal.

[0046] Figures 3a and 3b are exemplary diagrams showing range and Doppler in accordance with one embodiment of the present invention, respectively.

[0047] 3a and 3b, the signal processing unit 170 (see FIG. 2) may include a range converter and a Doppler converter. The range converter may perform a Fast Fourier Transform operation on the intermediate signal data IFD to obtain a range signal.

[0048] The range converter may generate a range signal by performing a fast Fourier transform on the intermediate signal data IFD in a first period T1. The first period T1 may refer to a period for sampling one received signal RX. The range signal output from the range converter may be a signal representing the distance between the sensing unit 100 (see FIG. 1) and the object 20 (see FIG. 1). The range converter may calculate the range signal by applying a fast Fourier transform to the intermediate signal data IFD once.

[0049] The Doppler converter may perform a fast Fourier transform operation on the range signal acquired via the range converter at a second period T2 to generate a Doppler signal. The second period T2 may refer to the period of the received signal RX. The Doppler signal may be a signal representing the relative velocity with respect to the object 20 (see FIG. 1). The Doppler converter may calculate the Doppler signal by applying a fast Fourier transform twice to the intermediate signal data IFD.

[0050] The range signal and Doppler signal generated by the range converter and Doppler converter, respectively, may be expressed in the form of a range-Doppler map RDM shown in FIG. 3b. In the range-Doppler map RDM, the vertical axis may represent a range value calculated by analyzing the range signal, and the horizontal axis may represent a Doppler value calculated by analyzing the Doppler signal. The range value may be calculated in m, and the Doppler value may be calculated in m / s. The range-Doppler map RDM may be generated for each frame. The signal processing unit 170 (see FIG. 2) may further include a mapper that maps a sensed signal to the range-Doppler map RDM. A sensed signal corresponding to each position on the range-Doppler map RDM may be mapped. The x-position value (value on the horizontal axis) of each position indicates the Doppler magnitude, and the y-position value (value on the vertical axis) indicates the range magnitude. In FIG. 3b, a relatively bright portion may indicate a relatively strong sensed signal. Here, the sensed signal may be a signal sensed by the sensing unit 100 based on the transmitted signal TX and the received signal RX. As described above, the signal mixer 150 generates the transmission signal TX and the reception signal RX as intermediate signals IF, the receiver 160 further generates intermediate signal data IFD from the intermediate signals IF, and the signal processor 170 generates the input information 110 from the intermediate signal data IFD. Thus, the sensed signal may be included in the input information 110. Then, as described above, the range signal calculated by the range converter and the Doppler signal calculated by the Doppler converter are mapped onto the range-Doppler map RDM shown in FIG. 3b, thereby mapping the sensed signal onto the range-Doppler map RDM.

[0051] FIG. 3b exemplarily illustrates a first sensing signal located at a first position P1, a second sensing signal located at a second position P2, and a third sensing signal located at a third position P3. The first position P1 is a position where the Doppler is 0 m / s and the range is 1 m, the second position P2 is a position where the Doppler is 2 m / s and the range is 2 m, and the third position P3 is a position where the Doppler is 3 m / s and the range is 3 m. In the range-Doppler map RDM, the position of the sensing signal may be defined by the position based on the range and Doppler pair. The position of the sensing signal may indicate information regarding the range and Doppler of the object 20 (see FIG. 1). In other words, the first sensing signal located at the first position P1 may indicate the presence of the object 20 (see FIG. 1) moving at a speed of 0 m / s at a point 1 m away from the sensing unit 100 (see FIG. 1).

[0052] The signal processor 170 (see FIG. 2) processes the detection signals located in the first and second regions AR1 and AR2 in the range-Doppler map RDM. That is, if the Doppler is at a point of 0 m / s or in a predetermined section adjacent to 0 m / s, it is determined that the object 20 is not moving, and the detection signals located in these sections may be ignored without being processed.

[0053] Fig. 4 is a block diagram of a pre-processing unit according to an embodiment of the present invention. Fig. 5 is a flowchart showing the operation of the pre-processing unit according to an embodiment of the present invention. Fig. 6 is a graph showing values ​​used in the motion detection process of the motion detection unit according to an embodiment of the present invention.

[0054] 4, 5, and 6, the pre-processing unit 200 may include a motion detection unit 210, a signal detection unit 220, a position correction unit 230, and an intensity correction unit 240. The motion detection unit 210 receives input information 110 from the sensing unit 100 (or the signal processing unit 170). The motion detection unit 210 may determine whether a gesture of the object 20 (see FIG. 1) has been detected based on the input information 110. In one embodiment of the present invention, the determination of whether a gesture has been detected may be a determination of whether the gesture has started or ended. If it is determined that a gesture has started, the input information 110 may be output, and if the gesture has not started, the input information 110 may not be output (step S100). If it is determined that the gesture has ended, the motion detection unit 210 may provide the corrected input information 250 to the classification unit 300 (see FIG. 1), and if the gesture has not ended, may output further input information 110 (step S800).

[0055] The motion detector 210 may calculate a difference between the signal strength of the range-Doppler map RDM in the current frame (hereinafter referred to as the current signal strength) and the signal strength of the range-Doppler map RDM in the previous frame (hereinafter referred to as the previous signal strength). The motion detector 210 may calculate a differential signal strength Diff-I between two adjacent frames based on the difference between the current signal strength and the previous signal strength. t The differential signal strength Diff-I can be calculated. t may be calculated from each of a plurality of frames. In one embodiment of the present invention, the signal strength of the range-Doppler map RDM in a corresponding frame may represent the average strength of the sensed signals represented in the corresponding range-Doppler map RDM.

[0056] The motion detector 210 calculates the differential signal strength Diff-I for a predetermined number of frames. t Average value Avg-I t For example, the average value Avg-I t is the differential signal strength of the five frames. t The motion detection unit 210 may calculate the average value Avg-It If is greater than a preset reference value, it is determined that a gesture has started, and the average value Avg-I t If the normal signal strength Nor-I is smaller than or equal to the reference value, it can be determined that the operation has ended. t may be used to determine whether a gesture has been detected. t is the differential signal strength Diff-I t Based on the maximum and minimum values ​​of the differential signal strength Diff-I t is the normalized value.

[0057] FIG. 7 is a flowchart showing the operation of the signal detection unit according to one embodiment of the present invention.

[0058] 4, 5, and 7, the signal detection unit 220 may receive input information 110 from the motion detection unit 210 and detect a peak signal from among a plurality of sensing signals included in the input information 110 (step S200). The signal detection unit 220 may output information about the peak signal for each frame. Hereinafter, a peak signal detected in a first frame will be referred to as a first peak signal, and a peak signal detected in a second frame will be referred to as a second peak signal. Herein, the first frame may be a frame at which the motion detection unit 210 determines that a gesture has started, and the second frame may be a frame that appears immediately after the first frame.

[0059] The information about the peak signal includes a peak range PS-R and a peak Doppler PS-D that define the position PS-L of the peak signal located on the range-Doppler map RDM (see FIG. 3b). The information about the peak signal may further include the intensity PS-I of the peak signal.

[0060] The signal detection unit 220 may select sensing signals (hereinafter, referred to as selected sensing signals) whose intensities are greater than a preset threshold from among a plurality of sensing signals included in the input information 110 (step S210).

[0061] The first, second, third, and fourth signals may be further selected from the selected sensing signals for each frame. The first signal may be the sensing signal with the highest intensity among the selected sensing signals (step S221). The second signal may be the sensing signal with the smallest distance from the center position among the selected sensing signals (step S222). The third signal may be the sensing signal that appears in both the previous frame and the current frame and has the smallest distance between its position in the previous frame and its position in the current frame (step S223). The fourth signal may be the sensing signal located above the position of the minimum or maximum Doppler among the selected sensing signals (step S224). Here, the center position of the selected sensing signals may be calculated by multiplying the position of each selected sensing signal by its own intensity to calculate a weighted position, and the average of the weighted positions may be calculated as the center position. The signal with the smallest range among the first to fourth signals may be detected as the peak signal of the corresponding frame (step S230). This peak signal can be located at a peak signal position PS-L on the range-Doppler map RDM (see FIG. 3b), and the peak signal position PS-L can be defined by the peak range PS-R, which is the y-position value (value on the vertical axis), and the peak Doppler PS-D, which is the x-position value (value on the horizontal axis) on the range-Doppler map RDM.

[0062] FIG. 8 is a diagram for explaining the operation of the position correction unit according to one embodiment of the present invention.

[0063] 4, 5, and 8, the position corrector 230 receives information about the detected peak signal from the signal detector 220 and may correct a peak signal position PS-L from the information about the peak signal to generate and output a corrected peak signal position PS-CL. The peak signal position PS-L may be corrected to a certain range through correction, and output as a corrected peak signal position PS-CL. The corrected peak signal position PS-CL may be expressed as a corrected peak range PS-CR and a corrected peak Doppler PS-CD. The corrected peak range PS-CR may be a value obtained by correcting the peak range PS-R, and the corrected peak Doppler PS-CD may be a value obtained by correcting the peak Doppler PS-D. The peak range PS-R may be corrected to a certain range through correction, and output as a corrected peak range PS-CR. The peak Doppler PS-D may be corrected to a certain range through correction, and output as a corrected peak Doppler PS-CD. The position corrector 230 may receive the peak signal strength PS-I from the information about the peak signal and output it as is without correction.

[0064] The position corrector 230 may determine whether the current frame is the first frame (step S300). If the current frame is the first frame, the peak signal detected by the signal detector 220 may be the first peak signal detected from the first frame, and the corrected position PS1-CL (hereinafter referred to as the start position) of the first peak signal may be the same as the position PS1-L of the first peak signal (step S410). The corrected position PS1-CL of the first peak signal may be used as the start position to standardize the start positions of all gestures. If the current frame is not the first frame, the detected peak signal may not be the first peak signal but another peak signal. If the current frame is not the first frame, a difference between the position of the peak signal detected from the current frame (which may be referred to as the position of the current frame) and the position of the peak signal detected from the previous frame (which may be referred to as the position of the previous frame) may be calculated (step S420). For example, the peak signal detected in the second frame may be the second peak signal. If the current frame is the second frame, a difference DD between the position PS2-L of the second peak signal detected in the second frame and the position PS1-CL of the first peak signal detected in the first frame can be calculated.

[0065] The calculated difference DD may be compared with a preset reference distance DL (step S500). If the difference DD is smaller than or equal to the reference distance DL, the corrected position of the current frame (e.g., the corrected position PS2-CL of the second peak signal) may be the same as the position of the current frame (e.g., the position PS2-L of the second peak signal) (step S610). If the difference DD is larger than the reference distance DL, the position of the peak signal of the current frame (the position PS2-L of the second peak signal) may be corrected so that it is within the reference distance DL from the position of the peak signal of the previous frame (the position PS1-CL of the first peak signal) (step S620).

[0066] The intensity correction unit 240 receives the correction position PS-CL of the peak signal and the learning data 310, and may correct the intensity PS-I of the peak signal using the correction position PS-CL of the peak signal and the learning data 310 (step S700). The learning data 310 according to an embodiment of the present invention may include information about the peak signals of all gestures for a learning range.

[0067] The intensity corrector 240 may output corrected information regarding the peak signal of a corresponding frame to the motion detector 210. The corrected information regarding the peak signal may include a corrected peak range PS-CR and a corrected peak Doppler PS-CD that define a corrected position PS-CL of the peak signal, and may also include a corrected intensity PS-CI of the peak signal. The motion detector 210 accumulates information regarding the peak signal of a corresponding frame for each frame, and when it is determined that a gesture has ended, may output corrected information regarding the peak signals from the first frame to the corresponding frame as corrected input information 250.

[0068] 9 is a graph showing the correlation between peak distance and peak intensity according to one embodiment of the present invention. In FIG. 9, a first graph GR1 shows actual peak intensity values ​​as a function of peak distance, and a second graph GR2 is a graph obtained by approximating the first graph GR1 using a model such as regression analysis. The second graph GR2 shows the function of peak distance versus peak intensity.

[0069] 4, 5, and 9, the training data 310 may include a distribution of correlations between peak distances and peak intensities. The horizontal axis of the graph in FIG. 9 represents peak distances, and the vertical axis represents the average peak intensities for all gestures within the training range. Here, the peak distance may represent the peak range for the training data 310, and the peak intensity may represent the intensity of the peak signal for the training data 310. However, the present invention is not limited thereto. The training data 310 may include a distribution of correlations between peak Dopplers and peak signal intensities for the training data 310, and a distribution of correlations between peak signal positions and peak signal intensities for the training data 310. The peak signal intensity PS-I for the input information 110 may be corrected to the corrected peak signal intensity PS-CI according to the second graph GR2 using the corrected peak range PS-CR as a factor.

[0070] Fig. 10 is a flowchart showing the operation of the intensity correction unit according to an embodiment of the present invention. Fig. 11a is a graph showing the correlation between peak distance and peak intensity for an A gesture (gesture A) according to an embodiment of the present invention. Fig. 11b is a graph showing the correlation between peak distance and peak intensity for a B gesture (gesture B) according to an embodiment of the present invention. Fig. 11c is a graph showing the correlation between peak distance and peak intensity for a C gesture (gesture C) according to an embodiment of the present invention. Fig. 11d is a graph showing the correlation between peak distance and peak intensity for a D gesture (gesture D) according to an embodiment of the present invention.

[0071] 4, 10, and 11a-11d, training data 310 according to one embodiment of the present invention may include a distribution of correlations between peak distances and peak intensities for each gesture.

[0072] The intensity corrector 240 may calculate the intensity of the peak signal between all frames (step S710). In one embodiment of the present invention, the intensity of the peak signal between all frames may be calculated as the average intensity of the peak signal between all frames, the accumulated intensity of the peak signal between all frames, or any other value that can represent the value of the peak signal intensity. The calculated intensity of the peak signal between all frames may be compared with the intensity of the peak signal for each gesture (e.g., gestures A through D) (S720), and one gesture that has the closest peak signal intensity between gestures A through D may be selected (S730). The peak signal intensity PS-I may be corrected to the corrected peak signal intensity PS-CI by approximating it to the peak intensity of the selected gesture by factoring the correction peak range PS-CR (S740).

[0073] 4 and 5, after the peak signal strength PS-I is corrected by the strength correction unit 240, the motion detection unit 210 may determine whether the gesture has ended (step S800). If the gesture has ended, information on the corrected peak range PS-CR, corrected peak Doppler PS-CD, and corrected peak signal strength PS-CI for N peak signals obtained during N (N is a natural number greater than or equal to 1) frames may be provided to the classifier 300 (see FIG. 1) as corrected input information 250. If the gesture has not ended, a process of detecting a peak signal in the range-Doppler map RDM for the N+1 frame (step S200) may be further performed.

[0074] 12a and 12b are block diagrams of a classifier according to one embodiment of the present invention.

[0075] 12a, the classifier 300 may include a convolutional neural network (CNN) 330 and a long short-term memory (LSTM) 340. The long short-term memory 340 may include a recurrent neural network (RNN). The classifier 300 may receive corrected input information 250 from the preprocessor 200 (see FIG. 1) and output a predicted gesture 320 of the object 20 (see FIG. 1) based on the corrected input information 250.

[0076] During the training process of the classification unit 300, the convolutional neural network 330 may receive training data 310 and train using the training data 310. The convolutional neural network 330 may generate intermediate training data 311 and provide it to the long-short-term memory 340. The neural network 330 may perform a convolution process on the training data 310, thereby classifying the training data 310 into categories to make it easier to recognize, and generating the intermediate training data 311. The long-short-term memory 340 may receive the intermediate training data 311 and train using the intermediate training data 311.

[0077] During the classification process of the classifier 300, the convolutional neural network 330 receives the corrected input information 250 and converts the corrected input information 250 to generate intermediate corrected input information 251, which is then provided to the long-short-term memory 340. The neural network 330 performs a convolution process on the corrected input information 250, thereby classifying the corrected input information 250 into categories for easier recognition, and generating the intermediate corrected input information 251. The long-short-term memory 340 may output a predicted gesture 320 of the object 20 based on the intermediate corrected input information 251. Here, the long-short-term memory 340 may output a predicted gesture 320 of the object 20 using the intermediate training data 311 and the intermediate corrected input information 251. A gesture 320 that may correspond to the object 20 may be determined based on the intermediate training data 311 and the intermediate corrected input information 251.

[0078] 12b, the classifier 300a may include only the long-short-term memory 340 without the convolutional neural network 330. In this case, the corrected input information 250 and the training data 310 may be provided to the long-short-term memory 340. In the training process of the classifier 300a, the long-short-term memory 340 is trained using the training data 310, and in the classification process of the classifier 300a, the long-short-term memory 340 may output a predicted gesture 320 of the object 20 based on the corrected input information 250.

[0079] FIG. 13 is a diagram showing types of gestures according to an embodiment of the present invention.

[0080] Referring to FIG. 13, the object 20 (see FIG. 1) may perform gestures such as push (approaching the sensing unit 100 (see FIG. 1) with an open hand), pull (moving away from the sensing unit 100 with an open hand), swipe (moving the hand left and right over the sensing unit 100), snap (moving the wrist left and right over the sensing unit 100), grab (clenching the hand with an open hand), and grab push (approaching the sensing unit 100 with a closed hand and an open hand).

[0081] Figure 14a is a graph showing the correlation between peak distance and peak intensity according to a comparative example, and Figure 14b is a graph showing the correlation between peak distance and peak intensity according to an embodiment of the present invention.

[0082] 14a and 14b show the results when the algorithm of the present invention is not applied and when the algorithm of the present invention is applied under the first condition (or typical condition) and the second condition (or variety condition), respectively. The first condition is when a gesture is made at a distance of 30 cm from the sensing unit 100 (see FIG. 1), and the second condition is when a gesture is made at a distance of 20 cm to 60 cm from the sensing unit 100.

[0083] When training is performed under a first condition and verification is performed under a second condition, the correlation between the peak signal range and the peak signal intensity under the first condition may not be similar to the correlation between the peak signal range and the peak signal intensity under the second condition, and the verification result may result in low recognition accuracy. Gesture detection system 10 (see FIG. 1) according to an embodiment of the present invention may correct the peak signal range and the peak signal intensity within a certain range using preprocessing unit 200 (see FIG. 1), and the correlation between the peak signal range and the peak signal intensity under the first condition may be similar to the correlation between the peak signal range and the peak signal intensity under the second condition. Therefore, even if training is performed using only the training data under the first condition without separate training data under the second condition, high recognition accuracy can be obtained in the verification process under the second condition.

[0084] FIG. 15 is a block diagram of an electronic device according to one embodiment of the present invention.

[0085] 15, in an operating system, the electronic device 601 outputs various information through a display module 640. When the processor 610 executes an application stored in the memory 620, the display module 640 provides application information to a user through a display panel 641.

[0086] The processor 610 acquires an external input via the input module 630 or the sensor module 661 and executes an application corresponding to the external input. For example, when a user selects a camera icon displayed on the display panel 641, the processor 610 acquires the user input via the input sensor 661-2 and activates the camera module 671. The processor 610 transmits video data corresponding to the captured image acquired via the camera module 671 to the display module 640. The display module 640 may display an image corresponding to the captured image on the display panel 641.

[0087] As another example, when personal information authentication is performed by the display module 640, the fingerprint sensor 661-1 acquires input fingerprint information as input data. The processor 610 compares the input data acquired through the fingerprint sensor 661-1 with authentication data stored in the memory 620 and executes an application based on the comparison result. The display module 640 may display information executed by the logic of the application on the display panel 641.

[0088] As another example, when a music streaming icon displayed on display module 640 is selected, processor 610 acquires user input via input sensor 661-2 and activates a music streaming application stored in memory 620. When a music play command is input in the music streaming application, processor 610 activates audio output module 663 to provide the user with music information corresponding to the music play command.

[0089] The above is a brief description of the operation of the electronic device 601. The following is a detailed description of the configuration of the electronic device 601. Some of the components of the electronic device 601 described below may be integrated into a single component, or one component may be separated into two or more components.

[0090] 15 , an electronic device 601 may communicate with an external electronic device 602 via a network (e.g., a short-range wireless network or a long-range wireless network). According to one embodiment, the electronic device 601 may include a processor 610, a memory 620, an input module 630, a display module 640, a power module 650, an internal module 660, and an external module 670. According to one embodiment, the electronic device 601 may omit at least one of the above components or may include one or more other components. According to one embodiment, some of the above components (e.g., the sensor module 661, the antenna module 662, or the acoustic output module 663) may be integrated into another component (e.g., the display module 640).

[0091] The processor 610 may execute software to control at least one other component (e.g., a hardware or software component) of the electronic device 601 coupled to the processor 610, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculations, the processor 610 may store instructions or data received from other components (e.g., the input module 630, the sensor module 661, or the communication module 673) in the volatile memory 621, process the instructions or data stored in the volatile memory 621, and store the resulting data in the non-volatile memory 622.

[0092] The processor 610 may include a main processor 611 and an auxiliary processor 612. The main processor 611 may include one or more of a central processing unit (CPU) 611-1 or an application processor (AP). The main processor 611 may further include one or more of a graphics processing unit (GPU) 611-2, a communication processor (CP), and an image signal processor (ISP). The main processor 611 may further include a neural network processing unit (NPU) 611-3. The neural network processing unit is a processor specialized in processing AI models, which may be generated by machine learning. The AI ​​model may include multiple artificial neural network layers. The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more thereof, but is not limited to the above examples. The AI ​​model may include a software structure in addition to or instead of a hardware structure. At least two of the processing units and processors described above may be embodied in an integrated structure (e.g., a single chip) or each may be embodied in an independent structure (e.g., multiple chips).

[0093] The auxiliary processor 612 may include a drive controller 612-1. The drive controller 621-1 may include an interface conversion circuit and a timing control circuit. The drive controller 621-1 receives a video signal from the main processor 611, converts the data format of the video signal to match the interface specifications with the display module 640, and outputs the video data. The drive controller 612-1 may output various control signals necessary to drive the display module 640.

[0094] The auxiliary processor 612 may further include a data conversion circuit 612-2, a gamma correction circuit 612-3, a rendering circuit 612-4, etc. The data conversion circuit 612-2 receives image data from the drive controller 612-1 and may compensate the image data so that an image is displayed at a desired brightness according to the characteristics of the electronic device 601 or a user setting, or may convert the image data to reduce power consumption or compensate for image retention. The gamma correction circuit 612-3 may convert the image data or a gamma reference voltage so that an image displayed on the electronic device 601 has a desired gamma characteristic. The rendering circuit 612-4 receives image data from the drive controller 612-1 and may render the image data taking into account the pixel arrangement of a display panel 641 applied to the electronic device 601, etc. At least one of the data conversion circuit 612-2, the gamma correction circuit 612-3, and the rendering circuit 612-4 may be integrated into another component (e.g., the main processor 611 or the controller 612-1). At least one of the data conversion circuit 612-2, the gamma correction circuit 612-3, and the rendering circuit 612-4 may be integrated into a data driver 643, which will be described later.

[0095] The memory 620 may store various data used by at least one component of the electronic device 601 (e.g., the processor 610 or the sensor module 661) and input or output data for instructions therefor. The memory 620 may include at least one of a volatile memory 621 and a non-volatile memory 622.

[0096] The input module 630 may receive instructions or data for use by components of the electronic device 601 (e.g., the processor 610, the sensor module 661, or the acoustic output module 663) from outside the electronic device 601 (e.g., from a user or an external electronic device 601).

[0097] The input module 630 may include a first input module 631 through which commands or data are input from a user and a second input module 632 through which commands or data are input from the external electronic device 602. The first input module 631 may include a microphone, a mouse, a keyboard, keys (e.g., buttons), or a pen (e.g., a passive pen or an active pen). The second input module 632 may support a specified protocol that may be connected to the external electronic device 602 via a wired or wireless connection. According to one embodiment, the second input module 632 may include a high definition multimedia interface (HDMI), a universal serial bus (USB), an SD card interface, or an audio interface. The second input module 632 may include a connector that may be physically connected to the external electronic device 602, such as an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0098] The display module 640 provides visual information to a user. The display module 640 may include a display panel 641, a scan driver 642, and a data driver 643. The display module 640 may further include a window, a chassis, and a bracket for protecting the display panel 641. The display module 640 may further include a light emitting driver and a voltage generator. The voltage generator may output various voltages (e.g., first and second driving voltages ELVDD and ELVSS, see FIG. 3 ) required to drive the display panel 641.

[0099] The power supply module 650 supplies power to the components of the electronic device 601. The power supply module 650 may include a battery that charges a power supply voltage. The battery may include a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell. The power supply module 650 may include a power management integrated circuit (PMIC). The PMIC supplies optimized power to each of the modules described above and below. The power supply module 650 may include a wireless power transmitting and receiving component electrically connected to the battery. The wireless power transmitting and receiving component may include multiple coil-shaped antenna radiators.

[0100] The electronic device 601 may further include an internal module 660 and an external module 670. The internal module 660 may include a sensor module 661, an antenna module 662, and an acoustic output module 663. The external module 670 may include a camera module 671, a light module 672, and a communication module 673.

[0101] The sensor module 661 may sense input from the user's body or input from the pen of the first input module 631 and generate an electrical signal or data value corresponding to the input. The sensor module 661 may include at least one of a fingerprint sensor 661-1, an input sensor 661-2, and a digitizer 661-3.

[0102] The fingerprint sensor 661-1 may generate a data value corresponding to a user's fingerprint and may include either an optical or capacitive fingerprint sensor.

[0103] The input sensor 661-2 may generate a data value corresponding to coordinate information of an input by the user's body or a pen. The input sensor 661-2 generates a data value representing the amount of change in capacitance due to the input. The input sensor 661-2 may sense an input by a passive pen or may send and receive data to and from an active pen.

[0104] The input sensor 661-2 may measure a biological signal such as blood pressure, water content, or body fat. For example, when a user touches a part of their body to the sensor layer or the sensing panel and remains motionless for a certain period of time, the input sensor 661-2 may sense the biological signal based on the change in the electric field caused by the part of their body and output desired information of the user to the display module 640.

[0105] The digitizer 661-3 may generate data values ​​corresponding to the coordinate information of the input by the pen. The digitizer 661-3 may generate data values ​​based on the amount of electromagnetic change caused by the input. The digitizer 661-3 may sense input by a passive pen or may send and receive data to and from an active pen.

[0106] At least one of the fingerprint sensor 661-1, the input sensor 661-2, and the digitizer 661-3 may be embodied as a sensor layer formed by a continuous process on the display panel 641. The fingerprint sensor 661-1, the input sensor 661-2, and the digitizer 661-3 may be disposed on the upper side of the display panel 641, and one of the fingerprint sensor 661-1, the input sensor 661-2, and the digitizer 661-3, for example, the digitizer 661-3, may be disposed on the lower side of the display panel 641.

[0107] At least two of the fingerprint sensor 661-1, the input sensor 661-2, and the digitizer 661-3 may be formed by the same process to be integrated into one sensing panel. When integrated into one sensing panel, the sensing panel may be disposed between the display panel 641 and a window disposed above the display panel 641. According to one embodiment, the sensing panel may be disposed above the window, and the position of the sensing panel is not particularly limited.

[0108] At least one of the fingerprint sensor 661-1, the input sensor 661-2, and the digitizer 661-3 may be built into the display panel 641. That is, at least one of the fingerprint sensor 661-1, the input sensor 661-2, and the digitizer 661-3 may be formed simultaneously in the process of forming elements (e.g., light-emitting elements, transistors, etc.) included in the display panel 641.

[0109] Additionally, the sensor module 661 may generate an electrical signal or a data value corresponding to an internal or external state of the electronic device 601. The sensor module 661 may further include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0110] The antenna module 662 may include one or more antennas for transmitting or receiving signals or power to or from an external device. According to one embodiment, the communication module 673 may transmit or receive signals to or from an external electronic device via an antenna compatible with a communication method. The antenna pattern of the antenna module 662 may be integrated into one component of the display module 640 (e.g., the display panel 641) or the input sensor 661-2.

[0111] The audio output module 663 is a device for outputting audio signals to the outside of the electronic device 601, and may include, for example, a speaker used for general purposes such as multimedia playback or recording playback, and a receiver used exclusively for receiving telephone calls. In one embodiment, the receiver may be formed integrally with the speaker or separately. The audio output pattern of the audio output module 663 may be integrated into the display module 640.

[0112] The camera module 671 may capture still or video images. According to one embodiment, the camera module 671 may include one or more lenses, an image sensor, or an image signal processor. The camera module 671 may further include an infrared camera that may determine the presence or absence of a user, the user's position, the user's line of sight, etc.

[0113] The light module 672 may provide light. The light module 672 may include a light emitting diode or a xenon lamp. The light module 672 may operate in conjunction with the camera module 671 or independently.

[0114] The communication module 673 may support the establishment of a wired or wireless communication channel between the electronic device 601 and the external electronic device 602 and the communication via the established communication channel. The communication module 673 may include one or both of a wireless communication module, such as a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module, and a wired communication module, such as a local area network (LAN) communication module or a power line communication module. The communication module 673 may communicate with the external electronic device 602 via a short-range communication network, such as Bluetooth, WiFi Direct, or infrared data association (IrDA), or a long-range communication network, such as a cellular network, the Internet, or a computer network (e.g., LAN or WAN). The various types of communication modules 673 described above may be implemented as a single chip or as separate chips.

[0115] The input module 630 , sensor module 661 , camera module 671 , etc. may be utilized in conjunction with the processor 610 to control the operation of the display module 640 .

[0116] The processor 610 outputs commands or data to the display module 640, the audio output module 663, the camera module 671, or the light module 672 based on input data received from the input module 630. For example, the processor 610 may generate video data corresponding to input data applied via a mouse or an active pen and output the video data to the display module 640, or generate command data corresponding to the input data and output the command data to the camera module 671 or the light module 672. If the processor 610 does not receive input data from the input module 630 for a certain period of time, the processor 610 may switch the operation mode of the electronic device 601 to a low power mode or a sleep mode to reduce power consumption of the electronic device 601.

[0117] The processor 610 outputs commands or data to the display module 640, the audio output module 663, the camera module 671, or the light module 672 based on the sensing data received from the sensor module 661. For example, the processor 610 may compare authentication data applied by the fingerprint sensor 661-1 with authentication data stored in the memory 620 and then execute an application based on the comparison result. The processor 610 may execute commands or output corresponding image data to the display module 640 based on the sensing data sensed by the input sensor 661-2 or the digitizer 661-3. If the sensor module 661 includes a temperature sensor, the processor 610 may receive temperature data on the measured temperature from the sensor module 661 and further perform brightness correction on the image data based on the temperature data.

[0118] The processor 610 may receive measurement data regarding the presence or absence of a user, the user's position, the user's line of sight, etc. from the camera module 671. The processor 610 may further perform brightness correction, etc. on the video data based on the measurement data. For example, the processor 610, having determined the presence or absence of a user based on input from the camera module 671, may output brightness-corrected video data to the display module 640 via the data conversion circuit 612-2 or the gamma correction circuit 612-3.

[0119] Some of the components may be connected to each other via a peripheral communication method, such as a bus, a general purpose input / output (GPIO), a serial peripheral interface (SPI), a mobile industry processor interface (MIPI), or an ultrapath interconnect (UPI) link, to exchange signals (e.g., commands or data). The processor 610 may communicate with the display module 640 via a mutually agreed-upon interface, which may use, for example, any one of the communication methods described above, but is not limited to the above.

[0120] The electronic device 601 according to various embodiments disclosed herein may take various forms. The electronic device 601 may include, for example, at least one of a portable communication device (e.g., a smartphone), a computing device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronic device. The electronic device 601 according to embodiments disclosed herein is not limited to the above-mentioned devices.

[0121] Although the present invention has been described above with reference to preferred embodiments, it should be understood by those skilled in the art or those having ordinary knowledge in the art that various modifications and changes can be made to the present invention without departing from the spirit and technical scope of the present invention as set forth in the claims below. Therefore, the technical scope of the present invention should not be limited to the contents of the detailed description of the specification, but should be determined by the claims. [Explanation of symbols]

[0122] 10: Gesture sensing system 20: Object TX: Transmitted signal RX: Received signal 100: Sensing unit 110: Input information 200: Preprocessing unit 210: Motion detection unit 220: Signal detection unit 230: Position correction unit 240: Intensity correction unit 250: Correction input information 300: Classification unit 310: Training data 320: Predicted Gesture RDM: Range-Doppler Map PS-L: Peak signal position PS-R: Peak range PS-D: Peak Doppler PS-I: Peak signal intensity

Claims

1. a sensing unit that senses a gesture of an object and generates and outputs input information including a plurality of sensing signals; a pre-processing unit that detects a peak signal from among the plurality of sensing signals and converts the input information into corrected input information using information about the peak signal; a classification unit that is trained using training data and receives the corrected input information and classifies the gesture; the information about the peak signal includes information about the position of the peak signal and the intensity of the peak signal; The gesture detection system further comprises: a signal processing unit that processes the position and intensity of the peak signal in a uniform manner within a predetermined range;

2. The pre-treatment unit a motion detection unit that detects whether the gesture is detected; a signal detection unit that detects the peak signal from among the plurality of sensing signals; a position correction unit that corrects the position of the peak signal and outputs the corrected position of the peak signal; The gesture sensing system according to claim 1 , further comprising: an intensity corrector that corrects the intensity of the peak signal and outputs the corrected intensity of the peak signal.

3. The input information is a range signal containing information about the distance between the sensing unit and the object; a Doppler signal containing information about the velocity of the object; the position of each of the plurality of sensing signals is a position on a range-Doppler map represented by the range signal and the Doppler signal; the sensing unit generates the input information for each frame; The gesture sensing system of claim 2 , wherein the correction input information includes information regarding a corrected position of the peak signal and a corrected intensity of the peak signal for every frame.

4. The motion detection unit Calculating a differential signal strength based on the difference in signal strength for the range-Doppler map of two adjacent frames; Calculating the average value of the intensity of the difference signal for each predetermined frame; The gesture sensing system according to claim 3 , wherein the average value determines whether the gesture is detected.

5. The motion detection unit Calculating a differential signal strength based on the difference in signal strength for the range-Doppler map of two adjacent frames; Calculating the intensity of a normalized signal by normalizing the intensity of the differential signal; The gesture sensing system according to claim 3 , wherein the detection of the gesture is determined based on the intensity of the normal signal.

6. The signal detection unit The gesture sensing system according to claim 3 , wherein a sensing signal having an intensity greater than a threshold is selected from the plurality of sensing signals.

7. The signal detection unit selecting a first signal, a second signal, a third signal, and a fourth signal from the selected sensing signals for each frame; the first signal is a sensing signal having the greatest intensity among the selected sensing signals, the second signal is a sensing signal having a minimum difference from a center position of the selected sensing signal, among the selected sensing signals; the third signal is the sensing signal having the smallest positional difference between the previous frame and the current frame among the selected sensing signals; The gesture sensing system of claim 6 , wherein the fourth signal is a sensing signal located at a position where Doppler is minimum or maximum among the selected sensing signals.

8. The signal detection unit The gesture sensing system according to claim 7 , wherein the sensing signal at the smallest position of the range among the first to fourth sensing signals is detected as the peak signal.

9. The position correction unit determining whether the current frame is the first frame; The gesture sensing system according to claim 3 , wherein the first frame is a frame at a time when the motion detection unit determines that the gesture has started.

10. The position correction unit If the current frame is the first frame, the position of the detected peak signal is calculated as a start position; The gesture sensing system of claim 9 , further comprising: if the current frame is not the first frame, calculating a difference between the position of the current frame and the position of the previous frame.

11. The position correction unit If the difference is within a reference distance, the position of the current frame is not corrected; The gesture sensing system of claim 10 , further comprising: correcting a position of a peak signal of the current frame if the difference exceeds the reference distance.

12. The intensity correction unit correcting the intensity of the peak signal using the learning data; The gesture sensing system of claim 3 , wherein the training data includes information regarding a correlation between a range of the peak signal and an intensity of the peak signal.

13. The learning data is The gesture sensing system of claim 12 , wherein the gesture sensing system includes information on all the gestures related to the training object in one place.

14. The learning data is The gesture sensing system of claim 12 , further comprising information for each gesture individually related to the training subject.

15. The intensity correction unit comparing the peak signal strength for each gesture with the peak signal strength for all frames; selecting a gesture from among the gestures that has a closest peak signal strength during all the frames; The gesture sensing system of claim 14 , wherein the intensity of the peak signal is corrected based on a selected gesture.

16. The sensing unit generating the intermediate signal from a transmitted signal and a received signal; converting the intermediate signal into intermediate signal data by analog-to-digital conversion; The gesture sensing system of claim 1 , wherein the intermediate signal data is converted into the input information by applying a Fourier transform to the intermediate signal data.

17. The sensing unit applying the Fourier transform to the intermediate signal data to calculate a range signal; The gesture sensing system of claim 16 , wherein the Fourier transform is applied to the range signal to calculate a Doppler signal.

18. 20. The gesture sensing system of claim 17, wherein the transmitted signal and the received signal are millimeter waves.

19. the classifier includes a convolutional neural network and a long-short-term memory; During the training process of the classifier, the convolutional neural network receives the training data and is trained; The gesture sensing system according to claim 1 , wherein the long-term and short-term memories output predicted gestures during the classification process of the classifier.

20. a sensing unit that senses a gesture of an object and generates and outputs input information including a plurality of sensing signals; a pre-processing unit that detects a peak signal from among the plurality of sensing signals and converts the input information into corrected input information using information about the peak signal; a classification unit that is trained using training data and receives the corrected input information and classifies the gesture; the information about the peak signal includes information about the position of the peak signal and the intensity of the peak signal; An electronic device in which the positions and intensities of the peak signals are equalized within a certain range and provided to the classifier.

Images