Gesture segmentation method and device

Abstract

The invention discloses a gesture segmentation method and device, and solves the problem that a continuous micro-motion gesture of a millimeter wave radar is difficult to effectively segment. The gesture segmentation method comprises the following steps: acquiring a radar echo signal of a gesture action to generate Doppler data, and determining a first frame of the Doppler data as a first gesture segmentation boundary; determining that the time interval with the fixed time length is that the sliding window slides on the time axis; in response to the fact that any sliding window of the frame with the Doppler data being 0 contains a frame with the Doppler data not being 0, determining the frame with the Doppler data being 0 as a false alarm frame; and determining that one frame, which is not a false alarm frame and of which the Doppler data on the time axis becomes zero, is a second gesture segmentation boundary. According to the invention, the continuous micro-motion gesture in the millimeter wave radar signal is automatically and accurately segmented, and the practicability and efficiency of the recognition system are improved.

Description

Technical Field

[0001] The present application relates to the field of radar technology, and in particular to a gesture segmentation method and device. Background Art

[0002] Gestures are the most natural and direct way for human communication, and are also an ideal solution for human-computer interaction. At present, wearable sensors and computer vision-based sensors that use gesture recognition technology have been widely used in human-computer interaction fields such as robot control, sign language recognition, virtual environments, and driving assistance. Millimeter-wave radar has become an excellent solution for gesture recognition due to its advantages such as strong privacy, good environmental adaptability, easy embedding, and low cost. With the miniaturization of mobile devices such as smartphones and smart watches, finger input on the screen gradually cannot meet people's requirements for convenience; in addition, when people want to use gesture recognition to control close-range smart devices, micro-gestures are undoubtedly a better choice than large-scale motion gestures. Therefore, micro-gesture recognition based on millimeter-wave radar is one of the current research hotspots and has high practical value.

[0003] Compared with isolated gestures, continuous gestures are more in line with people's usage habits and are more convenient for collecting gesture data. In practical applications, the difficulty in segmenting continuous gestures is a major challenge in the field of gesture recognition. At present, there is a certain research basis for the problem of continuous gesture segmentation. Harbin Institute of Technology proposed a gesture segmentation method based on distance-Doppler joint features. By measuring whether the energy value of the distance-Doppler spectrum of each frame is higher than the set threshold, it detects whether there is a motion gesture in the frame; the University of Science and Technology of China proposed a two-dimensional constant false alarm rate detection method, which distinguishes the background and gesture targets by calculating the pixel peak size corresponding to each frame. However, both methods need to operate in the two dimensions of distance and Doppler, which to a certain extent limits their application effect in gesture recognition systems with high real-time requirements. Summary of the invention

[0004] The embodiments of the present application provide a gesture segmentation method and device, which solve the problem that continuous micro-motion gestures of millimeter-wave radar are difficult to effectively segment.

[0005] In a first aspect, an embodiment of the present application provides a gesture segmentation method, comprising the steps of:

[0006] Acquire a radar echo signal of the gesture action to generate Doppler data, and determine a first frame of the Doppler data as a first gesture segmentation boundary;

[0007] Determine a time interval of a fixed time length for the sliding window to slide on the time axis; the time interval of the sliding window is greater than or equal to 2 frames and less than or equal to the minimum value of the pause time between two gestures;

[0008] In response to any sliding window containing a frame with Doppler data of 0, which contains frames with Doppler data not being 0, determining that the frame with Doppler data of 0 is a false alarm frame;

[0009] A frame on the time axis where the Doppler data becomes 0 and is not a false alarm frame is determined as a second gesture segmentation boundary.

[0010] There is also an embodiment of determining whether the gesture action of each frame has changed, which specifically includes the steps of:

[0011] Calculate the Doppler value of the second-order binary matrix composed of frame numbers and Doppler values;

[0012] If the Doppler value corresponding to the frame number in the second-order binary matrix is ​​not 0 and the output value of the binary matrix is ​​1, then the gesture is moving;

[0013] or,

[0014] If the Doppler value corresponding to the frame number in the second-order binary matrix is ​​0 and the output value of the binary matrix is ​​0, the gesture is paused.

[0015] An embodiment is also included, wherein the time interval is equal to a minimum value of the pause time between two gestures.

[0016] There is also an embodiment of determining whether the gesture action of each frame has changed, which specifically includes the steps of:

[0017] Doppler data to detect hand gestures;

[0018] If the Doppler data is 0, the gesture stops.

[0019] There is also an embodiment, in which the generation of time-varying Doppler data based on the radar echo signal of the gesture action further includes the steps of:

[0020] Collect millimeter-wave radar echo signals and perform filtering processing;

[0021] Fast Fourier transform the echo signal frame by frame to obtain the energy value of the signal at different distances;

[0022] Each frame of data is superimposed along the distance dimension and then short-time Fourier transformed to obtain the Doppler value of each frame of the gesture action;

[0023] The Doppler values ​​of each frame are spliced ​​frame by frame to obtain the initial Doppler data of the gesture action changing over time;

[0024] OS-CFAR detects and normalizes the initial Doppler data of the hand gesture that changes with time, and obtains the Doppler data of the hand gesture that changes with time.

[0025] An embodiment is also included, wherein the filtering process uses an MTI filter to suppress static clutter.

[0026] In one embodiment, the OS-CFAR detection is used to reduce false alarms, which are information that a target is judged to exist when there is no target in reality.

[0027] In the second aspect, the embodiments of the present application provide a gesture segmentation device for implementing the gesture segmentation method described in any one of the embodiments of the first aspect, comprising: an acquisition module for acquiring a radar echo signal of a gesture action; a processing module for converting the radar echo signal of the gesture action into Doppler data that varies with time; a segmentation module for determining whether the Doppler data that varies with time for the gesture action has changed; and also for determining a gesture segmentation boundary of the Doppler data that varies with time for the gesture action.

[0028] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the first aspect.

[0029] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the first aspect when executing the computer program.

[0030] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0031] This application breaks through the technical bottleneck that millimeter-wave radar continuous micro-gestures are difficult to effectively segment. By combining the sliding window with the Doppler feature boundary detection method, it realizes the automatic and accurate segmentation of continuous micro-gestures in millimeter-wave radar signals, greatly improving the practicality and efficiency of the recognition system; by optimizing the boundary positioning algorithm through the sliding window mechanism, the accuracy of gesture segmentation is improved, and while maintaining high sensitivity, the possibility of system false alarms is reduced, and the stability and reliability in practical applications are enhanced; with scalability, this application is not only suitable for existing millimeter-wave radar equipment, but can also be easily integrated into different types of smart devices, promoting the widespread application and development of micro-gesture interaction technology in various fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0033] Figure 1 This is a flow chart of a gesture segmentation method according to an embodiment of the present application;

[0034] Figure 2 This is another flow chart of a gesture segmentation method according to an embodiment of the present application;

[0035] Figure 3 This is a Doppler feature extraction flow chart of an embodiment of the present application;

[0036] Figure 4 This is a schematic diagram of the OS-CFAR principle of an embodiment of the present application;

[0037] Figure 5 This is a schematic diagram after OS-CFAR detection according to an embodiment of the present application;

[0038] Figure 6 A schematic diagram of the overall process of micro-gesture segmentation according to an embodiment of the present application;

[0039] Figure 7 This is a schematic diagram of a gesture segmentation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0041] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0042] Figure 1 This is a flow chart of a gesture segmentation method according to an embodiment of the present application, which specifically includes steps 21 to 24:

[0043] Step 21: Acquire a radar echo signal of the gesture action to generate Doppler data, and determine the first frame of the Doppler data as a first gesture segmentation boundary;

[0044] Generate Doppler data that changes with time based on the radar echo signal of the gesture action, wherein the first frame is the start time, and determine the first time as the first gesture segmentation boundary;

[0045] When the pulse wave emitted by the radar encounters a moving target, it will be reflected back. The frequency of the reflected echo signal is different from the frequency of the transmitted signal. This phenomenon is called the Doppler effect.

[0046] The Doppler effect is caused by the motion of the target relative to the radar. If the target moves towards the radar, the frequency of the echo signal will be higher; if the target moves away from the radar, the frequency of the echo signal will be lower.

[0047] In this embodiment, the movement of the continuous micro-motion gesture relative to the radar generates a radar echo, and Doppler data of the continuous micro-motion gesture changing with time is obtained through a series of processing.

[0048] The specific method and principle of generating time-varying Doppler data from radar echo signals are described in Figure 2 The detailed description is given in .

[0049] In the acquired Doppler data varying with time, the first frame is the starting time and also the gesture segmentation boundary of the gesture action.

[0050] Step 22: Determine a time interval of a fixed time length for the sliding window to slide on the time axis; the time interval of the sliding window is greater than or equal to 2 frames and less than or equal to the minimum value of the pause time between two gestures;

[0051] Determining a change in the gesture action on the time axis starting from the first frame according to the Doppler data;

[0052] An embodiment is also included, in which a time interval of a fixed length is used as a sliding window to slide on a time axis to determine that a gesture action has changed.

[0053] Specifically, a time interval of a fixed time length is determined as a sliding window, and the sliding window is moved from the start time to determine whether the gesture action of each frame on the time axis has changed;

[0054] In step 21, the time-Doppler diagram of the continuous micro-gesture is obtained. In this continuous time period, the time series is dynamically scanned frame by frame, and a sliding window mechanism is introduced. The sliding window is a set fixed time interval.

[0055] The maximum time length of the sliding window is one-fifteenth of the time axis of the Doppler data.

[0056] Usually, a gesture lasts for 0.3s to 2s, and the pause time between consecutive gestures is T p 0.5s~2.5s. Assume that the length of each frame is T f The length of the sliding window is l (in frames). In order to ensure that p The inner sliding window can detect the boundary, l should satisfy

[0057] T f l≤T P Formula 1

[0058] At the same time, in order to suppress false alarms, the length of the sliding window should be no less than 2 frames. Combined with the above formula, l should satisfy

[0059] 2≤l≤T p / T f Formula 2

[0060] Step 23, in response to any sliding window containing frames whose Doppler data is not 0, determining that the frame whose Doppler data is 0 is a false alarm frame;

[0061] The false alarm frame refers to a frame in which the Doppler data is 0, which is caused by the slow gesture movement rather than the pause of the gesture.

[0062] The sliding window slides backward on the time axis of the acquired Doppler data from the start time, dynamically scanning frame by frame on the time series to determine whether the gesture action of each frame on the time axis has changed.

[0063] For example, the sliding window slides on the time axis, and the Doppler data of the leftmost frame of the sliding window (i.e., the side close to the starting point of the time axis) is used to determine the gesture segmentation boundary. If the Doppler data of the leftmost frame of the sliding window is not 0, it is considered that the frame is not a gesture segmentation boundary. The sliding window slides one frame to the right, and if the Doppler data of the leftmost frame after sliding is 0, it is considered that the frame is suspected to be a gesture segmentation boundary.

[0064] It should be noted that, since the length of the sliding window is less than or equal to the minimum value of the pause time between two gestures, the time interval of a gesture pause may contain more than one situation where the Doppler data of all frames in the time interval of the sliding window are 0. Therefore, the fact that the Doppler data of all frames in the time interval of a sliding window are 0 does not mean that the leftmost frame of the sliding window is the gesture segmentation boundary.

[0065] However, if any sliding window in which a frame with Doppler data of 0 exists contains frames with Doppler data not being 0, it means that the frame with Doppler data of 0 may be due to the slow gesture, which causes the radar recognition to believe that the gesture is in a pause state when it is made, and therefore the Doppler data is 0.

[0066] For example, when setting radar parameters, T f Set to 20ms, T p Taking the minimum value of 500ms, the range of the sliding window length l is [2, 25]. In this range, the larger the sliding window length, the better the effect of suppressing false alarms.

[0067] If the gesture does not change during the time interval when the sliding window moves, the last frame of this time interval is the suspected gesture segmentation boundary;

[0068] The sliding window is used as a fixed time interval to scan the acquired time axis frame by frame. If the gesture action is determined to have not changed during the time interval scanned by the sliding window, the last frame in this time interval is the suspected gesture segmentation boundary.

[0069] For example, let the length of the sliding window be l, and x(i) indicates whether there is an object in the i-th frame.

[0070]

[0071] If the starting frame is t0, when none of the l frames in the window contains a moving target, that is, , then the last frame in the window, that is, the t0+1th frame, is determined to be the suspected gesture segmentation boundary.

[0072] It should be noted that the motion target is the gesture action, and determining whether there is a motion target is to determine whether the gesture action has changed. If there is no motion target, it means that the gesture action has not changed.

[0073] For another example, the total time of Doppler data acquired is 30 seconds, the sliding window is set to 1 second, and the sliding window moves continuously for 2.5 seconds from the start time, scanning frame by frame. If the gesture action does not change within the 2.5 second time interval, the last frame within the 2.5 second time interval is the suspected gesture segmentation boundary.

[0074] It can be seen that as the sliding window scans backward along the time axis, there is more than one suspected gesture segmentation boundary.

[0075] Determine whether the gesture action changes in each frame:

[0076] In one embodiment, the method specifically comprises the steps of:

[0077] Step 231-1, calculating the Doppler value of the second-order binary matrix composed of the frame number and the Doppler value;

[0078] Step 231-2: If the Doppler value corresponding to the frame number in the second-order binary matrix is ​​not 0 and the output value of the binary matrix is ​​1, the gesture action changes; or, if the Doppler value corresponding to the frame number in the second-order binary matrix is ​​0 and the output value of the binary matrix is ​​0, the gesture action does not change.

[0079] Another embodiment is also included, which specifically includes the steps of:

[0080] Step 232-1, step detecting Doppler data of gesture action;

[0081] Step 232-2: If the Doppler data is 0, the gesture stops.

[0082] Step 24: Determine a frame on the time axis where the Doppler data becomes 0 and is not a false alarm as the second gesture segmentation boundary.

[0083] When the Doppler data of a certain frame on the time axis becomes 0, it means that the Doppler data of the previous frame is not 0, the gesture is in motion, and in the current frame, the gesture stops and the Doppler data is 0.

[0084] In response to the gesture action changing from motion to pause, and any gesture segmentation boundary existing on the previous timeline, a frame in which the gesture action changes is determined as a second gesture segmentation boundary.

[0085] Any one of the gesture segmentation boundaries includes a first gesture segmentation boundary or a second gesture segmentation boundary.

[0086] Any two adjacent gestures split the boundary and divide a gesture action.

[0087] If there is a time interval in which the gesture action changes between the suspected gesture segmentation boundary and the first gesture segmentation boundary or the second gesture segmentation boundary of the previous time interval, the suspected gesture segmentation boundary is the second gesture segmentation boundary.

[0088] As the sliding window moves, a time interval in which the next gesture action does not change will appear on the timeline, and its last frame is also the suspected gesture segmentation boundary.

[0089] After the first gesture segmentation boundary is determined, there is a time interval between the suspected gesture segmentation boundary and the first gesture segmentation boundary where the gesture action changes, and the suspected gesture segmentation boundary is the second gesture segmentation boundary.

[0090] Therefore, there must be a time interval between gesture segmentation boundaries where the gesture action changes.

[0091] In this way, the boundaries of continuous gestures are confirmed and gesture segmentation is performed.

[0092] Figure 2 This is a flow chart of another gesture segmentation method according to an embodiment of the present application.

[0093] A gesture segmentation method, in which a radar echo signal based on a gesture action generates Doppler data that varies with time, further includes steps 12 to 16:

[0094] Steps 12 to 16 are Figure 1 In step 21, the radar echo signal based on the gesture action generates a specific expansion of the Doppler data that varies with time.

[0095] Step 12: Collect millimeter wave radar echo signals and perform filtering processing;

[0096] When collecting gestures, static clutter will inevitably appear. If not processed, it will affect the subsequent gesture segmentation and gesture recognition.

[0097] This embodiment uses an MTI filter to suppress static clutter, which has a simple structure and is physically easy to implement. The single delay line canceller is one of the most commonly used MTI filters. Assume that the delay of the single delay line canceller is T, and the radar echo signal x(t) and the reference signal x(tT) pass through a mixer to achieve phase detection. The output signal phase y(t) of the single delay canceller is proportional to the phase difference between the two input signals. The input signal is x(t), and the output signal is x(t)-x(tT), that is, the delay difference between the input signal and the input signal. Static background noise can be eliminated by a single delay line canceller.

[0098] Its frequency response can be expressed as

[0099]

[0100] The information contained in the initial radar echo signal is relatively redundant and contains more information about non-moving targets. By suppressing static clutter, the information in the radar signal is more concentrated on the frame where the moving gesture target is located, and the boundaries between continuous gestures are more obvious, which is helpful for subsequent boundary detection.

[0101] Step 13, fast Fourier transform the echo signal frame by frame to obtain the energy value of the signal at different distances;

[0102] The horizontal axis of the original signal collected in each frame is the sampling point, and the vertical axis is the chirp dimension. After performing a fast Fourier transform on the sampling point dimension, the energy value of the signal at different distances can be obtained.

[0103] By superimposing along the distance dimension, one-dimensional data can be obtained, which represents the energy value of the micro-gesture target.

[0104] Step 14, each frame of data is superimposed along the distance dimension and short-time Fourier transformed to obtain the Doppler value of each frame of the gesture action;

[0105] Then, the one-dimensional data is subjected to short-time Fourier transform to obtain the Doppler characteristic value of each frame of gesture action.

[0106] As a commonly used time-frequency analysis method, short-time Fourier transform can process non-stationary signals compared with Fourier transform. For non-stationary signals, if they are divided into multiple small parts in the time domain, each part can be approximately regarded as a stationary process. Short-time Fourier transform uses this principle to perform Fourier transform on each small process, thereby obtaining the time domain position corresponding to each frequency component.

[0107] Step 15, splicing the Doppler values ​​of each frame frame by frame to obtain the initial Doppler data of the gesture action changing over time;

[0108] Finally, frame-by-frame splicing is performed, that is, the Doppler values ​​of each frame of gesture action are spliced ​​together to obtain a set of Doppler-time characteristics of the echo signal.

[0109] That is, the Doppler initial data of the gesture movement changing over time.

[0110] Step 16: OS-CFAR detects and normalizes the initial Doppler data of the hand gesture over time, and obtains the Doppler data of the hand gesture over time.

[0111] The main purpose of normalization is to facilitate data processing, speed up the convergence of the program during runtime, and ensure the comparability of data.

[0112] OS-CFAR detection is used to reduce false alarms.

[0113] A false alarm is information that is judged to be a target when there is no target in fact.

[0114] The specific principle of OS-CFAR detection is Figure 4 In the description.

[0115] Figure 3 This is a flow chart of micro-gesture Doppler feature extraction according to an embodiment of the present application.

[0116] Figure 3 The leftmost side represents the original signal collected in each frame. The horizontal axis is the sampling point and the vertical axis is the chirp dimension.

[0117] Chirp refers to a signal whose frequency changes (increases or decreases) over time.

[0118] After fast Fourier transforming the sampling point dimension, the energy value of the signal at different distances can be obtained. By superimposing along the distance dimension, one-dimensional data can be obtained, which represents the energy value of the micro-gesture target. Then, the one-dimensional data is short-time Fourier transformed to obtain the Doppler characteristic value of each frame target.

[0119] Finally, frame-by-frame splicing is performed, that is, the Doppler values ​​of the target in each frame are spliced ​​together to obtain a set of Doppler-time characteristics of the echo signal.

[0120] Figure 4 This is a schematic diagram of the OS-CFAR principle of an embodiment of the present application.

[0121] CFAR, also known as constant false alarm detection, is a commonly used target detection and tracking algorithm, widely used in fields such as millimeter wave radar.

[0122] Its main function is to detect the target signal in the background noise while ensuring that the false detection probability remains unchanged.

[0123] The basic idea of ​​the CFAR algorithm is to establish a detection window around each radar measured data point, calculate the mean and variance of the signal power in the window, and divide the window into several sub-windows. Then, according to the expected false detection probability and the statistical characteristics of the background noise, the threshold of each sub-window is calculated to determine whether there is a target signal in the window.

[0124] OS-CFAR is a type of CFAR algorithm, which is suitable for multi-target detection.

[0125] In this embodiment, when extracting micro-motion gesture features, the palm and fingers may be recognized as multiple targets.

[0126] like Figure 4 As shown, the sample data in the reference window are sorted from large to small, and the Kth value is selected as the threshold. OS-CAFR includes detection units, protection units, and reference units. The energy of the target may leak. If the energy of the target leaks into the protection unit, it will not affect the detection result. There are 2n reference units in total, which are distributed on both sides of the protection unit.

[0127] When data is input, it is necessary to sort all the data in the reference unit according to the power size, and select the Kth value as the threshold value as the estimation Z of the background clutter power, that is, Z = Xk. Using μ to represent the noise power, the probability density function of Z under a uniform background can be obtained as

[0128]

[0129] Let γ represent the threshold coefficient, and the false alarm probability P of OS-CFAR detector under uniform clutter background is fa for

[0130]

[0131] Among them, the probability density function of Γ distribution is

[0132] f(y)=β -a y a-1 exp(-y / β) / Γ(a),y≥0,a≥0,β≥0 Formula 6

[0133] Among them, α and β are two constants, and Γ(α) is the most common Γ function.

[0134] Figure 5 This is a schematic diagram after OS-CFAR detection according to an embodiment of the present application.

[0135] like Figure 5As shown, suppose the binary moment C obtained after OS-CFAR detection is M×N The matrix is, the rows of the matrix represent the Doppler dimension, and the columns of the matrix represent the frames. M represents the number of points in the Doppler dimension, and N represents the number of frames.

[0136] C(i, j)=1 indicates that there is a target with Doppler value j in the i-th frame, and C(i, j)=0 indicates that there is no target with Doppler value j in the i-th frame.

[0137] The next step is to determine whether there is a moving target in the i-th frame.

[0138] It should be noted that the motion target is the gesture action, and determining whether there is a motion target is to determine whether the gesture action has changed.

[0139] The first method for determining whether the gesture action of each frame has changed is to calculate the Doppler value of a second-order binary matrix composed of frame numbers and Doppler values;

[0140] If the Doppler value corresponding to the frame number in the second-order binary matrix is ​​not zero 1, and the output value of the binary matrix is ​​1, the gesture action changes;

[0141] If the Doppler value corresponding to the frame number in the second-order binary matrix is ​​0 and the binary matrix output value is 0, the gesture action does not change.

[0142] That is to say, if there is no target at all Doppler points in the i-th frame, the gesture action in the i-th frame must not have changed.

[0143] The second method for determining whether the gesture action of each frame has changed is to detect the Doppler data of the gesture action;

[0144] If the Doppler data is 0, the gesture stops, otherwise the gesture moves;

[0145] The Doppler data being 0 indicates that the target is stationary or that the target is moving radially perpendicular to the radar wave.

[0146] If the target Doppler data detected in the i-th frame is 0, then the gesture action in the i-th frame has not changed.

[0147] Next, the boundaries between continuous gestures are detected. To determine that a frame is a gesture segmentation boundary, the following two conditions must be met: ① There is no moving target in the frame, that is, the gesture action has not changed; ② There is a moving target at the boundary between the frame and the previous frame, that is, the gesture action has changed between the frame and the previous boundary. Condition ① indicates that the boundary can only exist in frames without moving targets, and condition ② indicates that there must be moving gestures between two adjacent boundaries. In order to suppress the false alarms that may be caused by CFAR detection, a sliding window mechanism is introduced to dynamically scan frame by frame in the time series.

[0148] The sliding window is a time interval used to suppress false alarms that may be caused by OS-CFAR.

[0149] For example, when there is no moving target in a certain frame, the output value is 0. There are moving targets before and after this frame, which may not be the gesture segmentation boundary and may be a false alarm.

[0150] Figure 6 The figure is a flow chart of a gesture segmentation method according to an embodiment of the present application.

[0151] First, the micro-gesture signal is collected by millimeter-wave radar, and the original gesture signal is pre-processed by a single delay line canceller to effectively filter out the background static clutter interference. Figure 2 Step 12.

[0152] Subsequently, short-time Fourier transform is used on the preprocessed clean signal to extract the Doppler feature parameters of the micro-gesture.

[0153] Next, the extracted Doppler features are tested with the help of OS-CFAR technology to obtain the test results. Figure 2 13 to 16 in the

[0154] The OS-CFAR technology principle is specifically developed in Figure 4 Detailed description in.

[0155] Based on the detection results, it is determined whether each frame of data contains a valid micro-gesture motion target. The specific development of OS-CFAR detection results and judgment methods is in Figure 5 Detailed description in.

[0156] Finally, the sliding window mechanism is introduced, combined with the custom boundary detection criteria, and each frame of data is scanned and analyzed frame by frame along the time axis to locate the start and end boundaries of each micro-gesture, thereby realizing the automatic segmentation of continuous micro-gestures. Figure 1 Steps 22 to 24 in the above procedure.

[0157] Figure 7 This is a schematic diagram of a gesture segmentation device according to an embodiment of the present application.

[0158] The application embodiment further provides a gesture segmentation device, which is used to implement the gesture segmentation method described in any one of the embodiments, comprising:

[0159] The acquisition module 31 is used to obtain the radar echo signal of the gesture action;

[0160] A processing module 32, used to convert the radar echo signal of the gesture action into Doppler data that varies with time;

[0161] The segmentation module 33 is used to determine whether the Doppler data of the gesture action changing with time has changed; and is also used to determine the gesture segmentation boundary of the Doppler data of the gesture action changing with time.

[0162] The conversion process includes radar echo signal filtering, signal Doppler feature extraction and OS-CFAR detection.

[0163] The present invention proposes a gesture segmentation method and device based on Doppler features, which only needs to perform boundary detection in the Doppler dimension to improve the real-time performance of gesture segmentation. Through the steps of radar echo signal preprocessing, micro-gesture feature extraction, continuous gesture segmentation boundary detection, etc., the automatic processing of continuous gesture segmentation is realized, which saves time and labor costs, and provides strong support for the rapid construction of micro-gesture standard data sets and the formation of micro-gesture recognition systems.

[0164] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] Therefore, the present application also proposes a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any embodiment of the present application is implemented.

[0166] Furthermore, the present application also proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present application when executing the computer program.

[0167] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0170] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0171] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0172] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A gesture segmentation method, characterized in that: Includes steps: Acquire a radar echo signal of the gesture action to generate Doppler data, and determine a first frame of the Doppler data as a first gesture segmentation boundary; Determine a time interval of a fixed time length for the sliding window to slide on the time axis; the time interval of the sliding window is greater than or equal to 2 frames and less than or equal to the minimum value of the pause time between two gestures; In response to any sliding window containing a frame with Doppler data of 0, which contains frames with Doppler data not being 0, determining that the frame with Doppler data of 0 is a false alarm frame; A frame on the time axis where the Doppler data becomes 0 and is not a false alarm frame is determined as a second gesture segmentation boundary.

2. The gesture segmentation method according to claim 1, characterized in that: Determine whether the gesture action of each frame has changed, including the following steps: Calculate the Doppler value of the second-order binary matrix composed of frame numbers and Doppler values; If the Doppler value corresponding to the frame number in the second-order binary matrix is ​​not 0 and the output value of the binary matrix is ​​1, then the gesture is moving; or, If the Doppler value corresponding to the frame number in the second-order binary matrix is ​​0 and the output value of the binary matrix is ​​0, the gesture is paused.

3. The gesture segmentation method according to claim 1, characterized in that: The time interval is equal to the minimum value of the pause time between two gestures.

4. The gesture segmentation method according to claim 1, characterized in that: Determine whether the gesture action of each frame has changed, including the following steps: Doppler data to detect hand gestures; If the Doppler data is 0, the gesture stops.

5. The gesture segmentation method according to claim 1, characterized in that: Generating time-varying Doppler data based on the radar echo signal of the gesture action also includes the following steps: Collect millimeter-wave radar echo signals and perform filtering processing; Fast Fourier transform the echo signal frame by frame to obtain the energy value of the signal at different distances; Each frame of data is superimposed along the distance dimension and then short-time Fourier transformed to obtain the Doppler value of each frame of the gesture action; The Doppler values ​​of each frame are spliced ​​frame by frame to obtain the initial Doppler data of the gesture action changing over time; OS-CFAR detects and normalizes the initial Doppler data of the hand gesture that changes with time, and obtains the Doppler data of the hand gesture that changes with time.

6. The gesture segmentation method according to claim 5, characterized in that: The filtering process uses an MTI filter to suppress static clutter.

7. The gesture segmentation method according to claim 5, characterized in that: The OS-CFAR detection is used to reduce false alarms; The false alarm is information that is judged to be a target when there is actually no target.

8. A gesture segmentation device, characterized in that: The method for implementing the gesture segmentation method according to any one of claims 1 to 7 comprises: An acquisition module is used to obtain radar echo signals of gesture actions; A processing module, used for converting the radar echo signal of the gesture action into Doppler data varying with time; The segmentation module is used to determine whether the gesture action has changed according to the Doppler data of the gesture action changing with time; and is also used to determine the gesture segmentation boundary of the Doppler data of the gesture action changing with time.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images