Radar-based action recognition method, home device, and storage medium

Abstract

The application discloses a radar-based action recognition method, a household device and a storage medium, and relates to the technical field of smart home. The method comprises the following steps: collecting a plurality of radar point cloud data based on a radar unit; wherein each radar point cloud data comprises at least one detection point, different radar point cloud data correspond to different detection time, the detection points in the same radar point cloud data correspond to the same detection time, the detection points have target attribute information detected based on the radar unit emitting and receiving radar signals, and the target attribute information comprises at least two of detection time information, distance information and speed information; generating a target feature map according to the target attribute information of the detection points in the plurality of radar point cloud data; wherein the target feature map is used for describing the correlation feature between two target attribute information; and identifying a target action performed by a user on the household device according to at least a distance-speed quantization feature map in the target feature map.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and more specifically, to a radar-based motion recognition method, a home appliance, and a computer-readable storage medium. Background Technology

[0002] With the advancement of electronic information technology, smart home devices such as smart lighting, smart curtains, and smart appliances are being used more and more widely. Smart home devices can interact with users, for example, by receiving user commands to automatically execute related functions. The interaction methods between smart home devices and users typically include touch interaction, voice interaction, or contactless action interaction (such as gestures, kicking, etc.). Traditional touch and voice interactions suffer from limitations in operation, susceptibility to interference, and privacy risks. Contactless action interaction has significant advantages over traditional touch and voice interactions, making it a research hotspot in the field of smart home user interaction methods.

[0003] In related technologies, deep learning or feature extraction methods can be used to recognize user actions to achieve contactless interaction between home appliances and users. However, deep learning methods have high computational complexity, requiring high-performance chips and large training samples, resulting in high costs. Traditional feature extraction methods often rely on angle measurement functions, requiring multi-antenna architectures, which lead to high hardware costs and computational demands, and also have limited recognition accuracy. Therefore, how to achieve accurate user action recognition at low cost has become an urgent problem to be solved. Summary of the Invention

[0004] One objective of this application is to provide a new technical solution for action recognition.

[0005] According to a first aspect of this application, a radar-based action recognition method is provided, applied to a home appliance, wherein the home appliance is equipped with a radar unit; the method includes:

[0006] Multiple radar point cloud data are collected based on the radar unit; wherein, each radar point cloud data includes at least one detection point, different radar point cloud data correspond to different detection times, and the detection points in the same radar point cloud data correspond to the same detection time. The detection point has target attribute information detected based on the radar unit transmitting and receiving radar signals, and the target attribute information includes detection time information, distance information, and speed information.

[0007] A target feature map is generated based on the target attribute information of the detection points in multiple radar point cloud data; wherein, the target feature map is used to describe the correlation features between two target attribute information, and the target feature map includes a distance-velocity quantization feature map that characterizes the correlation features between velocity information and distance information;

[0008] Based at least on the distance-velocity quantization feature map in the target feature map, the target action performed by the user on the home device can be identified.

[0009] Optionally, generating a target feature map based on the target attribute information of the detection points in the plurality of radar point cloud data includes:

[0010] The target attribute information is quantized based on a preset attribute quantization interval to obtain quantized attribute information; wherein, different target attribute information corresponds to different preset attribute quantization intervals.

[0011] Determine the target attribute pair containing the first quantization attribute and the second quantization attribute from the quantization attribute information;

[0012] A two-dimensional attribute feature map is generated based on the target attribute pairs in multiple radar point cloud data; wherein, the two-dimensional attribute feature map includes multiple location points in a two-dimensional coordinate system, each location point corresponds to at least one detection point, the first coordinate of the two-dimensional coordinate system is the coordinate determined based on the first quantization attribute, and the second coordinate of the two-dimensional coordinate system is the coordinate determined based on the second quantization attribute;

[0013] The target feature map is determined based on the two-dimensional attribute feature map.

[0014] Optionally, the detection point also has signal strength information characterizing the strength of the received radar signal;

[0015] The step of generating a two-dimensional attribute feature map based on the target attribute pairs in multiple radar point cloud data includes:

[0016] For each detection point, the position point corresponding to the detection point in the two-dimensional coordinate system is determined based on the first quantization attribute and the second quantization attribute of the detection point;

[0017] If multiple detection points correspond to the same location point, then the detection point with the highest radar signal strength among the multiple detection points shall be taken as the detection point corresponding to the location point.

[0018] The two-dimensional attribute feature map is generated based on the location point.

[0019] Optionally, determining the target feature map based on the two-dimensional attribute feature map includes: performing binarization and / or denoising processing on the position points in the two-dimensional attribute feature map to generate the target feature map.

[0020] Optionally, the radar unit has a transmission channel and a receiving channel; the step of identifying the target action performed by the user on the home appliance based at least on the distance-velocity quantization feature map in the target feature map includes: determining the target graphic feature type corresponding to the target action; detecting whether there is a target graphic in the target feature map corresponding to the target graphic feature type; and determining that the user has performed the target action on the home appliance if the target graphic exists in the target feature map.

[0021] Optionally, the target feature map further includes a distance-time quantization feature map characterizing the correlation between the detection time information and the distance information. The first coordinate of the distance-time quantization feature map is the detection time coordinate corresponding to the detection time information, and the second coordinate of the distance-time quantization feature map is the distance coordinate corresponding to the distance information. The first coordinate and the second coordinate are perpendicular. The target graphic feature type includes a first feature type corresponding to the distance-time quantization feature map. The first feature type characterizes a graphic feature with a minimum distance value.

[0022] The step of detecting whether a target graphic corresponding to the target graphic feature type exists in the target feature map includes:

[0023] Obtain the first position point corresponding to the minimum distance in the distance-time quantization feature map;

[0024] Multiple sets of first location points are obtained based on the first detection time coordinates of the first location point; wherein, the multiple sets of first location points include a first point set and a second point set, the first point set includes location points whose detection time is less than the first detection time coordinates, and the second point set includes location points whose detection time is greater than the first detection time coordinates;

[0025] Linear fitting is performed on the position points in each of the first set of position points to determine the first fitting parameters of the linear fitting equation corresponding to each of the first set of position points; wherein, the first fitting parameters include at least one of the first linear slope, the first linear intercept, or the first linear fit goodness of fit.

[0026] If the first fitting parameter satisfies the preset first linear fitting condition, then it is determined that there is a target graphic corresponding to the first feature type in the distance-time quantization feature map.

[0027] Optionally, the velocity information includes positive velocity values ​​and negative velocity values. The positive velocity value indicates that the target object detected by the radar unit is moving away from the radar unit, and the negative velocity value indicates that the target object is moving towards the radar unit. The target feature map further includes a velocity time quantization feature map that represents the correlation between the detection time information and the velocity information. The first coordinate of the velocity time quantization feature map is the detection time coordinate corresponding to the detection time information, and the second coordinate is the velocity coordinate corresponding to the velocity information. The first coordinate and the second coordinate are perpendicular. The target graphic feature type includes a second feature type corresponding to the velocity time quantization feature map. The second feature type represents a graphic feature with a velocity direction inflection point.

[0028] The step of detecting whether a target graphic corresponding to the target graphic feature type exists in the target feature map includes:

[0029] Among multiple location points in the velocity instantaneous quantization feature map, a second location point, a third location point, and a fourth location point located within a continuous detection time window are obtained; wherein, the second location point is the negative velocity peak location point in the continuous detection time window, the third location point is the positive velocity peak location point in the continuous detection time window, and the fourth location point is located between the second location point and the third location point and is the location point where the velocity information changes from a negative velocity value to a positive velocity value;

[0030] Based on the second detection time coordinates of the second location point, the third detection time coordinates of the third location point, and the fourth detection time coordinates of the fourth location point, a plurality of second location point sets are obtained; wherein, the plurality of second location point sets include a third point set and a fourth point set, the third point set includes location points whose detection time is greater than the second detection time coordinates and less than the fourth detection time coordinates, and the fourth point set includes location points whose detection time is greater than the fourth detection time coordinates and less than the third detection time coordinates;

[0031] A linear fit is performed on the position points in each of the second set of position points to determine the second fitting parameters of the linear fitting equation corresponding to each of the second set of position points; wherein, the second fitting parameters include at least one of the second linear slope, the second linear intercept, or the second linear fit goodness of fit.

[0032] If the second fitting parameter satisfies the preset second linear fitting condition, and the second velocity value at the second position point and the third velocity value at the third position satisfy the preset velocity threshold condition, then it is determined that there is a target graphic corresponding to the second feature type in the velocity moment quantization feature map.

[0033] Optionally, the velocity information includes positive velocity values ​​and negative velocity values. The positive velocity value indicates that the target object detected by the radar unit is moving away from the radar unit, and the negative velocity value indicates that the target object is moving towards the radar unit. The first coordinate of the range-velocity quantization feature map is the velocity coordinate corresponding to the velocity information, and the second coordinate of the range-velocity quantization feature map is the distance coordinate corresponding to the distance information. The first coordinate and the second coordinate are perpendicular. The target graphic feature type includes a third feature type corresponding to the range-velocity quantization feature map. The third feature type indicates the presence of a diamond-shaped feature composed of a distance peak and a velocity peak.

[0034] The step of detecting whether a target graphic corresponding to the target graphic feature type exists in the target feature map includes:

[0035] A fifth, sixth, seventh, and eighth position point are obtained from multiple position points in the distance-velocity quantization feature map; wherein, the fifth position point is the position point with the smallest distance among the multiple position points, the sixth position point is the position point with the largest distance among the multiple position points, the seventh position point is the position point with the largest velocity among the multiple position points, and the eighth position point is the position point with the smallest velocity among the multiple position points.

[0036] Construct a target quadrilateral based on the fifth, seventh, sixth, and eighth position points, and determine the side length and diagonal length of each side of the target quadrilateral; wherein, the order of the four vertices of the quadrilateral connected in sequence is the fifth, seventh, sixth, and eighth position points;

[0037] If the side length satisfies the preset side length condition and the diagonal length satisfies the preset diagonal length condition, then it is determined that there is a target graphic corresponding to the third feature type in the distance-velocity quantization feature map.

[0038] According to a second aspect of this application, a home appliance is provided, the home appliance including a radar unit and a control unit, the control unit including a memory and a processor, the memory for storing computer instructions, and the processor for calling the computer instructions from the memory to perform the method as described in any one of the first aspects.

[0039] According to a third aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the first aspects.

[0040] One beneficial effect of this application is that it acquires multiple radar point cloud data based on radar units; generates a target feature map describing the correlation between two target attribute information based on the target attribute information of the detection points in the multiple radar point cloud data; and identifies the target actions performed by the user on home devices based on the target feature map. Compared with action recognition methods in related technologies, it significantly reduces hardware costs and computational complexity, is suitable for contactless interaction scenarios of smart home devices, and solves the problems of high cost and high computing power requirements of traditional methods, thereby improving the accuracy of target action recognition by using low-cost radar units.

[0041] Other features and advantages of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present application and, together with their description, serve to explain the principles of the present application.

[0043] Figure 1 This is a schematic diagram of the structure of a home appliance according to an embodiment of this application;

[0044] Figure 2 This is a schematic flowchart of a radar-based action recognition method according to an embodiment of this application;

[0045] Figure 3A This is a schematic diagram of a distance-time quantization feature map according to an embodiment of this application;

[0046] Figure 3B This is a schematic diagram of a velocity time-quantization feature map according to an embodiment of this application;

[0047] Figure 3C This is a schematic diagram of a distance velocity quantization feature map according to an embodiment of this application;

[0048] Figure 4 This is a flowchart illustrating a method for acquiring radar point cloud data based on radar unit acquisition according to an embodiment of this application. Detailed Implementation

[0049] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0050] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0051] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0052] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0053] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0054] <Hardware Implementation>

[0055] Figure 1 This is a schematic diagram of the structure of a home appliance according to an embodiment of this application.

[0056] like Figure 1 As shown, the home appliance 1000 can be any smart home device such as smart lighting, smart curtains, smart appliances, smart sockets, smart security equipment, and smart bathroom equipment. The home appliance 1000 may include a radar unit 1100 and a control unit 1200. The control unit 1200 may include a processor 1210 and a memory 1220. The processor may include a central processing unit (CPU), a microcontroller unit (MCU), etc.; the memory may include random access memory (RAM), read-only memory (ROM), or non-temporary storage such as a hard disk.

[0057] For example, the radar unit can collect radar point cloud data and transmit it to the control unit. The control unit can perform action recognition based on the radar point cloud data to identify the target action performed by the user on the home device. The control unit can also control the home device to perform corresponding functions based on the identified target action, such as controlling the on / off of smart lights, controlling the raising and lowering of smart curtains, controlling the start / stop of smart home appliances, and screen operation, thereby realizing contactless control of smart home devices.

[0058] This radar unit can have a transmit channel and a receive channel, used for transmitting and receiving radar signals respectively. For example, the radar unit can be a 1T1R (one transmit, one receive) architecture radar, meaning it can have only one transmit channel and one receive channel. Adopting a 1T1R architecture can reduce the hardware cost of home appliances. As another example, the radar unit can also be a 1T2R (or 1T4R) architecture radar, meaning it can have one transmit channel and multiple receive channels. Yet another example is that the radar unit can also be a 2T2R (or 2T4R, or 4T4R) architecture radar, meaning it can have multiple transmit channels and multiple receive channels.

[0059] This radar unit can transmit radar signals (referred to as transmitted signals) into the sensing area via a transmission channel and receive radar signals reflected by a target object within the sensing area (referred to as echo signals). Processing the transmitted and echo signals yields radar point cloud data. This radar point cloud data can include multiple detection points, each a data product of the radar detecting a target object. Each detection point is generated based on the radar unit's detection of a target object within the sensing area through transmitted and received radar signals. For example, if the received radar signal strength exceeds a preset signal strength threshold, the detection point and its attribute information can be determined based on the transmitted and received radar information. The attribute information of this detection point reflects information related to the detected target object, such as the distance between the target object and the radar unit, and the target object's motion state information (e.g., speed).

[0060] In the embodiments applied in this application, the memory of the home device 1000 can be used to store instructions that, when executed by the processor, can implement the radar-based motion recognition method provided in the embodiments of this application.

[0061] In the above description, those skilled in the art can design instructions based on the scheme disclosed in this application. How the instructions are executed by the processor can be found in descriptions in related technologies, and will not be repeated here.

[0062] <Method Implementation>

[0063] Figure 2 This is a flowchart illustrating a radar-based action recognition method according to an embodiment of this application. This method can be applied to, for example... Figure 1 The home appliance shown can be equipped with a radar unit.

[0064] according to Figure 2 As shown, the radar-based action recognition method in this embodiment may include the following steps S2100~S2300.

[0065] Step S2100: Multiple radar point cloud data are collected based on radar unit data.

[0066] Each radar point cloud data includes at least one detection point. Different radar point cloud data correspond to different detection times, while detection points in the same radar point cloud data correspond to the same detection time.

[0067] For example, each detection point may have target attribute information detected based on radar signals emitted and received by the radar unit. This target attribute information may include at least two of the following: detection time information, distance information, and velocity information. The detection time information may correspond to the detection time of the radar point cloud data, the time of radar signal emission, or the time of radar signal reception. The distance information may characterize the distance between the target object detected by the radar unit and the radar unit. The velocity information may characterize the rate of change of the distance between the target object and the radar unit per unit time. For example, the velocity information includes positive velocity values ​​and negative velocity values. A positive velocity value indicates that the target object detected by the radar unit is moving away from the radar unit, and a negative velocity value indicates that the target object is moving towards the radar unit. In some examples, the target attribute information may include detection time information and distance information, or it may include detection time information and velocity information, or it may include distance information and velocity information, or it may include all three pieces of information: detection time information, distance information, and velocity information.

[0068] Optionally, the detection point may also have signal strength information characterizing the strength of the received radar signal. This signal strength information may include one or more of the following: signal-to-noise ratio (SNR), signal-to-interference-plus-noise ratio (SINR), signal-to-clutter ratio (SCR), and received signal strength indicator (RSSI).

[0069] For example, the radar unit can periodically transmit and receive radar signals to detect multiple radar point cloud data. A radar point cloud data can also be called a radar point cloud data frame or a radar point cloud data frame. Different radar point cloud data can correspond to different detection times, and multiple radar point cloud data can be arranged in the order of detection times. For example, radar signal detection (i.e., a process of transmitting and receiving radar signals and performing analysis and detection) can be performed once every preset period, thereby obtaining one radar point cloud data every preset period. The preset period can be any pre-set time period, such as 1 millisecond, 5 milliseconds, 10 milliseconds, or 100 milliseconds. Each radar point cloud data includes one or more detection points, and one or more detection points in the same radar point cloud data correspond to the same detection time. The target attribute information of each detection point in the radar point cloud data at a certain detection time can be determined based on the radar point cloud data at that detection time, or it can be determined based on multiple (multiple frames) of radar point cloud data corresponding to that detection time and previous historical detection times. This embodiment does not limit this.

[0070] Step S2200: Generate a target feature map based on the target attribute information of the detection points in multiple radar point cloud data.

[0071] The target feature map can be used to describe the correlation between two target attribute information. For example, the two target attribute information are a first attribute and a second attribute, and the correlation feature can characterize the trend of the first attribute changing linearly or non-linearly with the second attribute. This correlation feature can be obtained by fitting the target attribute information of detection points in multiple radar point cloud data. For example, the multiple radar point cloud data used to generate the target feature map can be radar point cloud data collected within a preset time period, which can be preset by the user, such as 3 seconds, 5 seconds, or 10 seconds.

[0072] In this embodiment, the target feature map may include at least one of the following: a distance-time quantized feature map (also known as a distance-time cumulative map) characterizing the correlation between detection time information and distance information; a velocity-time quantized feature map (also known as a velocity-time cumulative map) characterizing the correlation between detection time information and velocity information; or a distance-velocity quantized feature map (also known as a distance-velocity cumulative map) characterizing the correlation between velocity information and distance information. For example, the target feature map may include any one, any two, or all three of the following: a distance-time quantized feature map, a velocity-time quantized feature map, or a distance-velocity quantized feature map.

[0073] For example, the target feature map can be a two-dimensional feature map (also known as a two-dimensional cumulative map). The two-dimensional feature map can include multiple location points in a two-dimensional coordinate system. The two-dimensional coordinate system can have a first coordinate (e.g., the horizontal coordinate) and a second coordinate (e.g., the vertical coordinate). The first coordinate and the second coordinate can be determined based on different target attribute information. Based on the two-dimensional feature map, the correlation features between two target attribute information can be described.

[0074] The aforementioned target feature map can be generated based on at least two of the target attribute information (detection time information, distance information, and velocity information) of the detection points in multiple radar point cloud data, for example, generating... Figures 3A to 3C The three types of target feature maps are shown.

[0075] Figure 3A This is a schematic diagram of a distance-time quantization feature map according to an embodiment of this application, as shown below. Figure 3A As shown, the detection time information can be used as the first coordinate (horizontal coordinate) and the distance information as the second coordinate (vertical coordinate). Multiple location points in the distance-time quantization feature map can be determined based on the detection time information and distance information of the detection points in multiple radar point cloud data.

[0076] Figure 3B This is a schematic diagram of a velocity time-quantization feature map according to an embodiment of this application, such as... Figure 3B As shown, the detection time information can be used as the first coordinate (horizontal coordinate) and the velocity information as the second coordinate (vertical coordinate). Multiple location points in the velocity-time quantization feature map can be determined based on the detection time information and velocity information of the detection points in multiple radar point cloud data.

[0077] Figure 3C This is a schematic diagram of a distance-velocity quantization feature map according to an embodiment of this application, as shown below. Figure 3C As shown, velocity information can be used as the first coordinate (horizontal axis) and distance information as the second coordinate (vertical axis). Multiple location points in the distance-velocity quantization feature map can be determined based on the velocity and distance information of the detection points in multiple radar point cloud data.

[0078] Step S2300: Identify the target action performed by the user on the home device based on the target feature map.

[0079] In this step, the target graphic feature type corresponding to the target action can be determined; the existence of a target graphic corresponding to the target graphic feature type in the target feature map can be detected; if the target graphic exists in the target feature map, it can be determined that the user has performed the target action on the home device.

[0080] In some examples, the target action performed by the user on the home device can be identified at least based on the distance-velocity quantization feature map in the target feature map.

[0081] For example, the target graphic feature type can be a pre-defined geometric feature, such as a V-shape, Z-shape, rhombus, quadrilateral, or W-shape. The specific method for detecting whether a target graphic corresponding to a target graphic feature type exists in the target feature map can be achieved by fitting multiple line segments to multiple position points in the target feature map. For example, one or more target vertices can be determined from multiple position points in the target feature map based on preset rules. The position points are then grouped based on the one or more target vertices to obtain multiple position point sets. Multiple line segments (e.g., two or more) are then fitted based on the grouped position point sets. The existence of a target graphic corresponding to the target graphic feature type in the target feature map is then determined based on the one or more target vertices and the multiple line segments.

[0082] For example, the target order can be determined based on candidate attribute information of some or all of the location points constituting the target feature map, and the candidate attribute information can be any of the aforementioned target attribute information. For instance, the candidate attribute information of the target vertex can be the maximum or minimum value of the candidate attribute information among all the location points constituting the target feature map; or, for another example, the candidate attribute information of the target vertex can be the maximum or minimum value of the candidate attribute information among multiple location points within a partial feature map region of the target feature map, and this partial feature map region can be a continuous feature map region.

[0083] This method can convert dynamic motion features into static graphic features, thereby enabling accurate identification of the target actions performed by users on home appliances.

[0084] For example, the method for target action recognition based on the above three types of target feature maps may include at least one of the following:

[0085] For the distance-time quantization feature map, the horizontal axis represents time and the vertical axis represents distance. If the target action is a specific "forward and backward" waving gesture, the distance first decreases and then increases. The feature map has a "V" shape, with the vertex corresponding to the closest time.

[0086] For the velocity moment quantification feature map, the horizontal axis is time and the vertical axis is velocity. If the target action is a specific "forward and backward" waving gesture, the velocity will change from "increasing-decreasing-increasing in the opposite direction-decreasing". The feature map has an inverted Z-shaped feature, with two peaks corresponding to the maximum velocity and the turning point corresponding to the moment when the velocity direction changes.

[0087] For the distance-velocity quantization feature map, the horizontal axis represents velocity and the vertical axis represents distance. If the target action is a specific "forward and backward" waving gesture, then the distance and velocity are symmetrically related. The feature map shows a "diamond" feature with four vertices corresponding to key motion states.

[0088] It is evident that the three types of feature maps possess uniqueness and stability under specific target actions, unaffected by environmental interference or differences in initial parameters; only their size scales, while the core contour remains unchanged, facilitating low-complexity detection. Gesture detection can be achieved by detecting these three types of feature maps. Similarly, left and right hand gestures will produce "W", "W", and "W". Shape features such as "rhombus" and "diamond" can be used to identify kicks. Similarly, kicks produce similar features, and similar methods can be used to identify kicks.

[0089] Based on the above, this application acquires multiple radar point cloud data using radar units; generates a target feature map describing the correlation between two target attribute information based on the target attribute information of the detection points in the multiple radar point cloud data; and identifies the target actions performed by the user on home devices based on the target feature map. Compared with action recognition in related technologies, the technical solution of this application eliminates the dependence on deep learning models and multi-antenna angle measurement architectures, requiring only a single-transmitter, single-receiver radar, significantly reducing hardware costs and computational complexity. It is suitable for contactless interaction scenarios of smart home devices, solving the problems of high cost and high computing power requirements of traditional methods, thereby improving the accuracy of target action recognition by using low-cost radar units.

[0090] In one embodiment of this application, step S2200 may include the following sub-steps:

[0091] Step S2201: Quantize the target attribute information based on the preset attribute quantization interval to obtain quantized attribute information.

[0092] Different target attribute information corresponds to different preset attribute quantization intervals.

[0093] The aforementioned preset attribute quantization interval can be pre-set by the user based on the target action to be identified. For example, based on the distance and speed of gestures and kicks, the preset attribute quantization interval for distance information can be set to 0.1 meters. For instance, for a distance range of 0-5m, a preset attribute quantization interval of 0.1m with 50 grids is used, quantizing the distance information into distance quantization information from 1 to 50. The preset attribute quantization interval for speed information can be set to 0.2 meters per second. For instance, for a speed range of -5 to 5m / s, a preset attribute quantization interval of 0.2m / s with 50 grids is used, quantizing the speed information into speed quantization information from 1 to 50. The preset attribute quantization interval for detection time information can be set to 100 milliseconds. For instance, based on a sliding window for detection points within a continuous 5-second time range, a preset attribute quantization interval of 100 milliseconds with 50 grids is used, quantizing the detection time information into detection time quantization information from 1 to 50. If the frame rate is high, detection points detected in several consecutive frames can be merged into one frame, controlling the time interval to 100 milliseconds per frame. Thus, the target feature map constructed based on quantized attribute information can be a 50×50 grid pattern; for example, it can be constructed as follows: Figure 3A The distance-time quantization feature map shown (50×50 grid) is as follows: Figure 3B The velocity-time quantization feature map shown (50×50 grid) is as follows: Figure 3C At least one of the distance-velocity quantization feature maps (50×50 grid) shown.

[0094] Step S2202: Determine the target attribute pair containing the first quantization attribute and the second quantization attribute from the quantization attribute information.

[0095] For example, any two of the above-mentioned distance quantization information, velocity quantization information, and detection time quantization information can be combined to obtain a target attribute pair, or the above three quantization information can be combined in pairs to obtain three sets of target attribute pairs. For example, the first quantization attribute of the first target attribute pair is the detection time quantization information, and the second quantization attribute is the distance quantization information; the first quantization attribute of the second target attribute pair is the detection time quantization information, and the second quantization attribute is the velocity quantization information; the first quantization attribute of the third target attribute pair is the velocity quantization information, and the second quantization attribute is the distance quantization information.

[0096] Step S2203: Generate a two-dimensional attribute feature map based on target attribute pairs in multiple radar point cloud data.

[0097] The two-dimensional attribute feature map includes multiple location points in a two-dimensional coordinate system. Each location point corresponds to at least one detection point. The first coordinate of the two-dimensional coordinate system is the coordinate determined based on the first quantization attribute, and the second coordinate of the two-dimensional coordinate system is the coordinate determined based on the second quantization attribute.

[0098] For example, the detection points also possess signal strength information characterizing the received radar signal strength. In step S2203, for each detection point, the position point corresponding to the detection point in the two-dimensional coordinate system can be determined based on the first quantization attribute and the second quantization attribute. If multiple detection points correspond to the same position point, the detection point with the highest radar signal strength among the multiple detection points is taken as the detection point corresponding to the position point. A two-dimensional attribute feature map is generated based on the aforementioned position point. Optionally, the value of the aforementioned position point can be the signal strength information.

[0099] By using signal strength information as a filtering criterion, when generating a two-dimensional feature map, only the detection point with the highest signal strength is retained when multiple detection points are mapped to the same grid position. This maximum value criterion effectively suppresses noise and weak reflection signals, enhances the salience of the target signal in the feature map, improves the signal-to-noise ratio of the feature map, provides a cleaner and more reliable data foundation for subsequent action recognition, and improves recognition accuracy.

[0100] Step S2204: Determine the target feature map based on the two-dimensional attribute feature map.

[0101] In some examples, a two-dimensional attribute feature map can be used as the target feature map.

[0102] In other examples, the location points in the two-dimensional attribute feature map can be binarized and / or denoised to generate the target feature map.

[0103] For example, a two-dimensional attribute feature map can be converted from a grayscale image to a black-and-white image through binarization, where the value of each point in the two-dimensional attribute feature map is set to 0 or 1, thereby distinguishing the target from the background. For instance, a minimum signal strength threshold can be preset. If the signal strength information at any point is greater than or equal to the minimum signal strength threshold, the value of that point is 1; conversely, if the signal strength information at any point is less than the minimum signal strength threshold, the value of that point is 0. The binarized two-dimensional attribute feature map can then be used as the target feature map.

[0104] For example, the two-dimensional attribute feature map can also be denoised. For instance, morphological opening operations can be used to process the feature map, removing noise and restoring the graphic contour to obtain the target feature map. For example, a 3×3 structuring element morphological opening operation can be used, employing a combination of erosion and dilation operations to update the two-dimensional attribute feature map, and the updated two-dimensional attribute feature map can be used as the target feature map.

[0105] Using this method, the two-dimensional attribute feature map can be binarized and subjected to 3×3 structuring element morphological opening operations to separate the target from the background, remove noise, and repair the graphic outline, thereby obtaining the target feature map.

[0106] In this way, the two-dimensional attribute feature map is purified through binarization and / or denoising, further separating the target from the background, removing noise points, and repairing the contour. This processing step can eliminate isolated noise points, smooth the target contour, and make the geometric features (such as V-shapes, rhombuses, etc.) in the feature map clearer and more continuous, reducing the risk of misjudgment by subsequent geometric detection algorithms and improving the robustness and accuracy of action recognition.

[0107] Based on the methods described in steps S2201 to S2204, the target attribute information is quantized using a preset attribute quantization interval. Continuous attributes are discretized into grid coordinates, and a two-dimensional attribute feature map is generated based on the discretized target attribute pairs, further yielding the target feature map. This quantization process simplifies data storage and computation, giving the feature map a regular grid structure, facilitating subsequent geometric feature detection, while reducing algorithm implementation complexity and improving the system's deployment feasibility on resource-constrained smart home devices.

[0108] To make the specific implementation of this embodiment clearer to those skilled in the art, the construction methods of the above three types of target feature maps will be described below.

[0109] In one embodiment of this application, a distance-time quantization feature map can be constructed based on the following steps S3101 to S3105:

[0110] Step S3101, distance quantization. The distance information of each detection point in the radar point cloud data at the current detection time is quantized in units of 0.1m. For example, if the distance information of detection point A is rng, the distance index is round(rng / 0.1)+1. For example, when the distance information rng is 3 meters, the quantized distance index is 31.

[0111] Step S3102: Assign values ​​from columns 2 to 50 in the distance time quantization feature map to columns 1 to 49 in sequence. That is, assign the value of column 2 to column 1, the value of column 3 to column 2, and so on.

[0112] Step S3103: Fill the quantized distance index of each detection point in the latest radar point cloud data into column 50. For each detection point, the information placed at the position corresponding to the calculated distance index value can be signal strength information (such as SNR). For example, if the calculated distance index of detection point A is 31, then the SNR value of A is placed at the 31st index position in column 50. Optionally, to reduce storage space, the SNR can be quantized in 1dB units, ranging from 5 to 40dB, for a total of 36 possible values. Values ​​less than 5dB are taken as 5dB, and values ​​greater than 40dB are taken as 40dB.

[0113] Step S3104: If there are multiple detection points at a certain index location, then the maximum value SNR criterion is used to set the detection points, that is, the one with the largest SNR is retained.

[0114] Step S3105: Repeat steps S3101 to S3104 continuously. From the beginning to 50 frames later, the distance time quantization feature map completes the information storage of the entire 50 frames, thereby constructing the distance time quantization feature map.

[0115] In one embodiment of this application, a velocity time-quantization feature map can be constructed based on the following steps S3201 to S3205.

[0116] Step S3201, Velocity Quantization. The velocity information of each detection point in the radar point cloud data at the current detection time is quantized in units of 0.2 m / s. For example, if the velocity information of detection point A is vlc, the velocity index is round(vlc / 0.2) + 25. Due to the existence of negative velocities, a velocity offset of 25 is added. For instance, when vlc = 3 m / s, the quantized velocity index is round(3 / 0.2) + 25 = 40.

[0117] Step S3202: Assign values ​​from columns 2 to 50 in the velocity time quantization feature map to columns 1 to 49 in sequence. That is, assign the value of column 2 to column 1, the value of column 3 to column 2, and so on.

[0118] Step S3203: Fill the quantized velocity information of each detection point in the latest radar point cloud data into column 50. For each detection point, the information placed at the position corresponding to the calculated velocity index value can be the SNR (Speed ​​Ratio). For example, if the calculated velocity index of detection point A is 40, then the SNR value of A is placed at the 40th index position in column 50. Optionally, to reduce storage space, the SNR can be quantized in 1dB units, ranging from 5 to 40dB, for a total of 36 possible values. Values ​​less than 5dB are taken as 5dB, and values ​​greater than 40dB are taken as 40dB.

[0119] Step S3204: If there are multiple detection points at a certain index position, the setting is made according to the maximum value SNR criterion, that is, the detection point with the largest SNR is retained.

[0120] Step S3205: Repeat steps S3201 to S3204 continuously. From the beginning to 50 frames later, the velocity time-quantization feature map completes the information storage of the entire 50 frames, thereby constructing the velocity time-quantization feature map.

[0121] In one embodiment of this application, a distance-velocity quantization feature map can be constructed based on the following steps S3301 to S3305.

[0122] Step S3301, distance quantization and velocity quantization. The distance and velocity information of each detection point in the radar point cloud data at the current detection time are quantized in units of 0.1m and 0.2m / s, respectively. For example, the distance information of detection point A is rng and the velocity information is vlc. The distance quantization index is IdxRng = round(rng / 0.1) + 1, and the velocity quantization index is IdxVlc = round(vlc / 0.2) + 25.

[0123] Step S3302: For detection point A, fill in the SNR of the detection point at the (IdxRng, IdxVlc) positions in the distance-velocity quantization feature map. To reduce storage space, the SNR can be quantized in 1dB units, ranging from 5 to 40dB, for a total of 36 possible values. Values ​​less than 5dB are taken as 5dB, and values ​​greater than 40dB are taken as 40dB.

[0124] In step S3303, if there is new point cloud data at the position (IdxRng, IdxVlc), the quantized SNR of the new point cloud is compared with the SNR stored at the current position (IdxRng, IdxVlc), and the maximum value is retained.

[0125] Step S3304: For each updated frame, the SNR at each position (IdxRng, IdxVlc) is attenuated according to an attenuation factor of 'a', i.e., SNR = SNR × a, where 'a' can be 0.96. For example, if the SNR at position (IdxRng, IdxVlc) is 20, and no new point cloud data fills this position, after 50 frames, the SNR value at this position will be round(20 × 0.96). 50 = 3; After 100 frames, the SNR value at this position is round(20 × 0.96). 100 )=0.

[0126] Step S3305: Repeat steps S3301 to S3304 continuously. From the beginning to 50 frames later, the distance velocity quantization feature map completes the information storage of the entire 50 frames, thereby constructing the distance velocity quantization feature map.

[0127] Thus, based on the above embodiments, three types of feature maps can be constructed respectively: distance-time quantization feature map, velocity-time quantization feature map, or distance-velocity quantization feature map. It should be noted that the feature map constructed based on this embodiment can be the two-dimensional attribute feature map in the aforementioned embodiments. This two-dimensional attribute feature map can be directly used as the target feature map, or it can be further preprocessed (e.g., binarization and / or denoising) to obtain the final target feature map.

[0128] In this embodiment of the application, the detection can be performed on three types of target feature maps: distance-time quantization feature map, velocity-time quantization feature map, or distance-velocity quantization feature map, to determine whether there is a target graphic corresponding to the target graphic feature type in the target feature map. The detection methods for the three types of target feature maps are described below.

[0129] In one embodiment, for the distance-time quantization feature map, the first coordinate of the distance-time quantization feature map is the detection time coordinate corresponding to the detection time information, and the second coordinate of the distance-time quantization feature map is the distance coordinate corresponding to the distance information. The first coordinate and the second coordinate are perpendicular. The target graphic feature type to be detected includes a first feature type corresponding to the distance-time quantization feature map. The first feature type represents a geometric graphic feature with a minimum distance value, such as a V-shaped feature. The specific method for detecting whether there is a target graphic corresponding to the target graphic feature type in the target feature map can be based on the following steps S4101 to S4104 to detect whether there is a target graphic corresponding to the first feature type in the distance-time quantization feature map.

[0130] Step S4101: Obtain the first position point corresponding to the minimum distance in the distance-time quantization feature map.

[0131] This first location point can be called the V-shaped vertex. For example, suppose the set of valid location points in the distance-time quantization feature map is S_V, containing n location points. Traverse all distance coordinate values ​​d_i in the set S_V, and by comparing them one by one, select the minimum distance value d_min = min{d_i | i=1, 2, ..., n}, whose corresponding detection time coordinate value is t_min. Thus, the V-shaped vertex P_v = (t_min, d_min) is obtained. The vertex is the core marker of the "V"-shaped feature, and its positioning accuracy directly affects the accuracy of subsequent feature detection. This step uses a simple traversal comparison operation, with low computational cost, and can quickly complete the positioning of the V-shaped vertex.

[0132] Step S4102: Obtain a set of multiple first position points based on the first detection time coordinates of the first position point.

[0133] The multiple sets of first location points include a first point set and a second point set. The first point set includes location points whose coordinates are less than the first detection time, and the second point set includes location points whose coordinates are greater than the first detection time.

[0134] This step can be used to partition the point set. For example, using the time coordinate t_min of the V-shaped vertex P_v as the dividing point, the set of position points S_V is divided into a left half point set S_V1 and a right half point set S_V2. S_V1 = {(t_i, d_i)|t_i<t_min,i=1,2,...,n1},S_V2 = {(t_i,d_i) | t_i> Let t_min, i=1,2,...,n2}, where n1 is the number of points in S_V1, n2 is the number of points in S_V2, and n1 + n2 ≤ n. This partitioning is based on the fact that the "V"-shaped feature consists of two straight line segments on either side of the vertex; after partitioning, features can be extracted from each of the two straight line segments separately.

[0135] Step S4103: Perform linear fitting on the position points in each first position point set to determine the first fitting parameters of the linear fitting equation corresponding to each first position point set.

[0136] The first fitting parameter includes at least one of the following: first linear slope, first linear intercept, or first linear fit goodness of fit.

[0137] For example, this step can involve linear fitting and slope calculation. For instance, the least squares method can be used to linearly fit S_V1 and S_V2 respectively, yielding the left fitted line L1 and the right fitted line L2. The core operation of the least squares method is solving a system of linear equations, involving only basic addition, subtraction, multiplication, division, and summation operations. The specific fitting process is as follows: For S_V1, assuming the fitted line equation is d = k1×t + b1, the slope k1 and intercept b1 are obtained by minimizing the sum of squared errors Σ(d_i - (k1×t_i + b1))²; similarly, fitting S_V2 yields the line equation d = k2×t + b2, and the slope k2 and intercept b2 are obtained. Based on the physical meaning of the "V"-shaped characteristic, L1 corresponds to the process of the target approaching the radar, where the distance decreases as time increases, therefore k1 should be negative; L2 corresponds to the process of the target moving away from the radar, where the distance increases as time increases, therefore k2 should be positive.

[0138] Step S4104: If the first fitting parameters satisfy the preset first linear fitting condition, then it is determined that there is a target graphic corresponding to the first feature type in the distance time quantization feature map.

[0139] Conversely, if the first fitting parameters do not meet the pre-set first linear fitting condition, then it is determined that there is no target graphic corresponding to the first feature type in the distance time quantization feature map.

[0140] For example, the first linear fitting condition may include a first linear slope threshold condition and / or a first linear fit goodness-of-fit threshold condition. For instance, the first linear slope may be greater than a pre-set first linear slope threshold, and the first linear fit goodness-of-fit may be greater than a pre-set first linear fit goodness-of-fit threshold. The first linear slope threshold k_th and the first linear fit goodness-of-fit threshold R_th can be pre-set. The value of k_th can range from 0.3 to 3 m / s, based on the slope range corresponding to the speed of common user movements, ensuring that only straight line segments matching the user's movement speed are considered valid features. The value of R_th can range from 0.8 to 0.9. The goodness-of-fit R² measures the degree of fit between the fitted straight line and the target point set. The closer R² is to 1, the closer the point set is to a straight line, and the more regular the graphic contour. The specific judgment logic is as follows: if k1∈-k_th (ensuring that the left line segment has sufficient slope and excluding flat point sets), k2∈k_th (ensuring that the right line segment has sufficient slope), and the goodness of fit R² of L1 and L2 are both ≥ R_th (ensuring that the point sets on both sides are linearly distributed), then it is determined that there is a "V"-shaped feature (i.e., the first feature type) in the distance time quantization feature map; otherwise, it is determined that there is no "V"-shaped feature.

[0141] In this way, the "approach-away" action is identified by finding the minimum distance point, performing linear fitting, and determining parameters. This method transforms gesture recognition into simple geometric feature detection and linear fitting operations, with extremely low computational cost, making it suitable for real-time operation on low-computing-power chips. It enables accurate recognition of common gestures such as "waving" on low-cost hardware.

[0142] In another embodiment, the first coordinate of the aforementioned velocity time quantization feature map is the detection time coordinate corresponding to the detection time information, and the second coordinate of the velocity time quantization feature map is the velocity coordinate corresponding to the velocity information. The first and second coordinates are perpendicular. The velocity information includes positive velocity values ​​and negative velocity values. The positive velocity value indicates that the target object detected by the radar unit is moving away from the radar unit, and the negative velocity value indicates that the target object is moving towards the radar unit. The target graphic feature type includes a second feature type corresponding to the velocity time quantization feature map. The second feature type represents a graphic feature with a velocity direction inflection point, such as an inverted Z-shaped feature. The specific method for detecting whether there is a target graphic corresponding to the target graphic feature type in the target feature map can be based on the following steps S4201 to S4204 to detect whether there is a target graphic corresponding to the second feature type in the velocity time quantization feature map.

[0143] Step S4201: Obtain the second, third, and fourth position points located within the continuous detection time window from multiple position points of the velocity time quantization feature map.

[0144] The second position point is the negative velocity peak position point within the continuous detection time window, that is, the position point with the largest absolute value among multiple position points with negative velocity information; the third position point is the positive velocity peak position point within the continuous detection time window, that is, the position point with the largest absolute value among multiple position points with positive velocity information; the fourth position point is located between the second and third position points and is the position point where the velocity information changes from a negative velocity value to a positive velocity value. The velocity information at the fourth position point can be the smallest positive velocity value or the smallest negative velocity value in absolute terms.

[0145] The second position point, the third position point, and the fourth position point can be referred to as velocity feature points, and the continuous detection time window can be a sliding window. Exemplarily, assume that the set of valid position points in the velocity moment quantization feature map is S_M. A sliding window can be used to traverse the target point set S_M to filter out velocity peak points and turning zero points. The duration of the sliding window can be a preset duration, such as 1 s, to ensure that the velocity change characteristics can be accurately captured. The specific filtering process can include: moving the sliding window along the time axis in sequence, calculating the velocity extreme values within each window, filtering out a negative velocity peak point P_m1 (the maximum velocity of the target approaching the radar) and a positive velocity peak point P_m2 (the maximum velocity of the target moving away from the radar), and there is a velocity turning zero point P_m0 (the transition point where the velocity changes from negative to positive, and the detection time information of the three position points satisfies the chronological order t_m1 < t_m0 < t_m2), initially determining that there is a core contour with an inverted Z-shaped feature in the image, that is, P_m1 and P_m0 correspond to the first broken line segment of the inverted Z shape, and P_m0 and P_m2 correspond to the second broken line segment of the inverted Z shape. Among them, the negative velocity peak point P_m1 can be referred to as the second position point, the positive velocity peak point P_m2 can be referred to as the third position point, and the velocity turning zero point P_m0 can be referred to as the fourth peak point.

[0146] Step S4202: Obtain multiple second position point sets according to the second detection time coordinates of the second position points, the third detection time coordinates of the third position points, and the fourth detection time coordinates of the fourth position points.

[0147] Among them, the multiple second position point sets include a third point set and a fourth point set. The third point set includes position points whose detection time is greater than the second detection time coordinate and less than the fourth detection time coordinate, and the fourth point set includes position points whose detection time is greater than the fourth detection time coordinate and less than the third detection time coordinate.

[0148] Step S4203: Perform linear fitting on the position points in each second position point set to determine the second fitting parameters of the linear fitting equations corresponding to each second position point set.

[0149] Among them, the second fitting parameters can include at least one of a second linear slope, a second linear intercept, or a second linear goodness of fit.

[0150] For example, the least squares method can be used to linearly fit the third point set S_M1 = {(t_j, v_j) | t_m1 ≤ t_j ≤ t_m0} between P_m1 and P_m0, and the fourth point set S_M2 = {(t_j, v_j) | t_m0 ≤ t_j ≤ t_m2} between P_m0 and P_m2, respectively, to obtain fitted lines L3 and L4. Based on the physical meaning of the inverted Z-shaped feature, S_M1 corresponds to the process of the target object decelerating from approaching the radar to coming to a standstill; as time increases, the velocity approaches 0 from a negative peak, therefore the slope of L3 should be positive. S_M2 corresponds to the process of the target accelerating away from the radar from a standstill; as time increases, the velocity increases from 0 to a positive peak, therefore the slope of L4 should be positive. Linear fitting verifies the regularity of the broken line segments and eliminates interference from irregular point sets.

[0151] Step S4204: If the second fitting parameters satisfy the preset second linear fitting conditions, and the second velocity value at the second position point and the third velocity value at the third position satisfy the preset velocity threshold conditions, then it is determined that there is a target graphic corresponding to the second feature type in the velocity moment quantization feature map.

[0152] Conversely, if the second fitting parameters do not meet the pre-set second linear fitting conditions, or if the second velocity value at the second position point and the third velocity value at the third position do not meet the pre-set velocity threshold conditions, then it is determined that there is a target graphic corresponding to the second feature type in the velocity moment quantization feature map.

[0153] For example, the second linear fitting condition may include a second linear fitting goodness-of-fit threshold condition, that is, the second linear fitting goodness-of-fit in the second fitting parameters is greater than the second linear fitting goodness-of-fit threshold. This velocity threshold condition can be that both the second velocity value and the third velocity value are greater than a preset velocity threshold. For example, a velocity threshold Δv_th and a second linear fitting goodness-of-fit threshold R_th (consistent with the R_th value for "V"-shaped detection, which is 0.8-0.9) can be preset. The velocity threshold Δv_th ranges from 0.2 to 0.5 m / s to ensure that the velocity difference between the peak and the zero inflection point is sufficiently large, avoiding misjudgments due to small velocity fluctuations. The specific judgment logic can be as follows: if |P_m1 - P_m0| ≥ Δv_th (the velocity difference between the negative peak and the zero point), |P_m2 - P_m0| ≥ Δv_th (the velocity difference between the positive peak and the zero point), and the goodness-of-fit R² of L3 and L4 are both ≥ R_th (ensuring that the two line segments are linearly distributed), then it is determined that there is an inverted Z-shaped feature (i.e., the second feature type) in the velocity moment quantization feature map; otherwise, it is determined that there is no inverted Z-shaped feature. It should be noted that the second linear fit goodness-of-fit threshold can be the same as or different from the first linear fit goodness-of-fit threshold.

[0154] In this way, actions are identified by detecting the inflection point of speed from negative to positive, piecewise linear fitting, and speed threshold judgment. This method can accurately capture the features of speed direction changes, has good recognition ability for actions involving speed reversal such as "approaching and then moving away", and requires only simple linear operations, making it computationally efficient and suitable for real-time gesture recognition applications.

[0155] In another embodiment, the first coordinate of the aforementioned distance-velocity quantization feature map is the velocity coordinate corresponding to the velocity information, and the second coordinate of the distance-velocity quantization feature map is the distance coordinate corresponding to the distance information; the first coordinate and the second coordinate are perpendicular. The target graphic feature type may include a third feature type corresponding to the distance-velocity quantization feature map, the third feature type representing a diamond-shaped feature composed of a distance peak and a velocity peak. The specific method for detecting whether a target graphic corresponding to a target graphic feature type exists in the target feature map can be based on the following steps S4301 to S4303 to detect whether a target graphic corresponding to the third feature type exists in the distance-velocity quantization feature map.

[0156] Step S4301: Obtain the fifth, sixth, seventh, and eighth position points from multiple position points in the distance-velocity quantization feature map.

[0157] Among them, the fifth position point is the position point with the smallest distance among the multiple position points, the sixth position point is the position point with the largest distance among the multiple position points, the seventh position point is the position point with the largest speed among the multiple position points, and the eighth position point is the position point with the smallest speed among the multiple position points.

[0158] The fifth, sixth, seventh, and eighth position points can be referred to as the vertices of the rhombus feature. For example, the core of the rhombus feature consists of four feature vertices with explicit physical meaning. Assuming the set of valid position points in the distance-velocity quantization feature map is S_D, the following explains the definition and determination method of the four vertices in the position point set S_D.

[0159] The method for determining the fifth position point (i.e. the minimum distance point P_d1) may include: traversing all distance coordinate values ​​d_k, filtering out the minimum value d_min1 = min{d_k}, the corresponding velocity coordinate value is v_d1, and obtaining P_d1 = (d_min1, v_d1).

[0160] The method for determining the sixth position point (i.e. the point with the maximum distance P_d2) may include: traversing all distance coordinate values ​​d_k, filtering out the maximum value d_max2 = max{d_k}, the corresponding velocity coordinate value is v_d2, and thus obtaining P_d2 = (d_max2, v_d2).

[0161] The method for determining the seventh position point (i.e. the point of maximum positive velocity P_v1) may include: traversing all velocity coordinate values ​​v_k, filtering out the maximum positive value v_max1 = max{v_k | v_k>0}, the corresponding distance coordinate value is d_v1, and obtaining P_v1 = (d_v1, v_max1).

[0162] The method for determining the eighth position point (i.e. the negative maximum velocity point P_v2) may include: traversing all velocity coordinate values ​​v_k, filtering out the negative maximum value (i.e. the negative value with the largest absolute value) v_max2 = max{v_k | v_k<0}, the corresponding distance coordinate value is d_v2, and thus P_v2 = (d_v2, v_max2).

[0163] The four vertices mentioned above are the core components of the rhombus feature, and their positioning accuracy directly determines the accuracy of rhombus detection.

[0164] Step S4302: Construct the target quadrilateral based on the fifth, seventh, sixth, and eighth position points, and determine the side length and diagonal length of each side of the target quadrilateral.

[0165] The order of the four vertices of the quadrilateral connected in sequence is the fifth position point, the seventh position point, the sixth position point, and the eighth position point.

[0166] For example, four feature vertices can be connected in the order of "P_d1→P_v1→P_d2→P_v2→P_d1" to construct a closed quadrilateral. Then, the key length parameters of the quadrilateral, such as the side lengths and diagonal lengths of the target quadrilateral, are calculated according to the distance formula between two points.

[0167] For example, the lengths of two pairs of opposite sides can be calculated, such as calculating the distance L1 between P_d1 and P_v1, and the distance L2 between P_d2 and P_v2 (the first pair of opposite sides), and calculating the distance L3 between P_v1 and P_d2, and the distance L4 between P_v2 and P_d1 (the second pair of opposite sides). The lengths of the diagonals can also be calculated, such as calculating the distance D1 between P_d1 and P_d2 (the diagonal in the distance direction), and calculating the distance D2 between P_v1 and P_v2 (the diagonal in the velocity direction). It should be noted that the length between two points can be calculated based on the following formula: L = sqrt[(x2 - x1)² + (y2 - y1)²], where (x1, y1) are the coordinates of the first position point, (x2, y2) are the coordinates of the second position point, and L is the length between the first and second position points. This formula has a small computational load and can quickly complete the length calculation.

[0168] Step S4303: If the side length meets the preset side length condition and the diagonal length meets the preset diagonal length condition, then it is determined that there is a target graphic corresponding to the third feature type in the distance velocity quantization feature map.

[0169] Conversely, if the side length does not meet the preset side length condition or the diagonal length does not meet the preset diagonal length condition, then it is determined that there is no target graphic corresponding to the third feature type in the distance velocity quantization feature map.

[0170] For example, the preset side length condition may include the difference between opposite side lengths being less than or equal to a preset length difference threshold, and the preset diagonal length condition may include the difference between the lengths of the two diagonals being less than or equal to a preset length difference threshold. The length difference threshold Δl_th can be preset, and its value range can be 0.1–0.3 m. This value range is based on the symmetry of the rhombus feature, ensuring that the opposite sides and diagonals of the quadrilateral are approximately equal in length. The core geometric feature of a rhombus is that "two pairs of opposite sides are equal and two diagonals are equal". Therefore, the specific judgment logic is as follows: if the difference in length of the first pair of opposite sides |L1 - L2| ≤ Δl_th, the difference in length of the second pair of opposite sides |L3 - L4| ≤ Δl_th (ensuring that the two pairs of opposite sides are basically equal), and the difference in length of the two diagonals |D1 - D2| ≤ Δl_th (ensuring that the two diagonals are basically equal), then the closed quadrilateral is determined to be a rhombus, that is, the distance-velocity quantification feature map contains a rhombus feature (i.e., the third feature type); otherwise, it is determined to have no rhombus feature.

[0171] In this way, periodic actions are identified by finding the extreme points of distance and velocity, constructing quadrilaterals, and judging geometric conditions. This method utilizes the symmetry of the distance-velocity graph, which has high recognition accuracy for periodic actions such as "waving hands forward and backward." Moreover, the detection algorithm only involves simple geometric calculations (judging side length and diagonal length), resulting in low computational complexity, making it suitable for deployment on resource-constrained smart home devices.

[0172] In one embodiment of this application, the radar unit in the aforementioned home appliance can transmit and receive radar signals in any available radar frequency band, such as one or more of the following: 5.8GHz, 10GHz, 24GHz, 60GHz, 5~9GHz Ultra Wide Band (UWB) radar band, Synchronous Link Positioning (SLP) radar band. This radar unit exhibits strong environmental adaptability, good penetration, small size, and low power consumption, which is beneficial for achieving accurate user action recognition at low cost.

[0173] Figure 4This is a flowchart illustrating a method for acquiring radar point cloud data based on radar unit acquisition according to an embodiment of this application.

[0174] For example, if the radar unit uses radars in the 5.8GHz, 10GHz, 24GHz, or 60GHz frequency bands, radar point cloud data can be obtained based on the following process: A linear frequency modulated (LFM) signal is transmitted, and the echo is received after reflection from the target. The LFM signal is then mixed to obtain an intermediate frequency (IF) signal. The IF signal is preprocessed with windowing and subjected to 2DFFT to obtain a range-velocity two-dimensional spectrum. Noise is detected and filtered using a constant false alarm rate (CFAR) algorithm, retaining the moving target point cloud data, thus obtaining the radar point cloud data.

[0175] like Figure 4 As shown, for radars in the 5.8GHz, 10GHz, 24GHz, and 60GHz frequency bands, the process of acquiring radar point cloud data may include the following steps S41 to S45.

[0176] In step S41, the radar unit receives the echo signal and mixes the echo signal of each chirp with the transmitted signal to obtain the intermediate frequency signal.

[0177] Step S42: Sample the intermediate frequency signal and perform Fast Fourier Transform (FFT) processing to obtain the distance dimension FFT signal, which is called the distance spectrum.

[0178] Step S43: Accumulate multiple distance spectra, perform velocity-dimensional FFT according to distance, and obtain the distance-velocity spectrum.

[0179] Step S44: Calculate the absolute value of the distance-velocity spectrum and use the constant false alarm rate (CFAR) detection algorithm to detect the detection points (filtering out noise).

[0180] Step S45: Obtain one or more detection points from the radar point cloud data. Each detection point has target attribute information and signal strength information (e.g., SNR). The target attribute information may include at least two of the following: detection time information, distance information, and velocity information.

[0181] For example, if the radar unit uses a 5-9 GHz UWB radar or a star-flash SLP radar, radar point cloud data can be obtained based on the following process: The acquired echo signal is the Channel Impulse Response (CIR). FFT is performed on the CIR of multiple frames according to the taps to obtain the range-velocity two-dimensional spectrum. Noise is filtered by CFAR detection, and the moving target point cloud is preserved, thus obtaining the radar point cloud data.

[0182] like Figure 4 As shown, for UWB / SLP radar, the process of acquiring radar point cloud data may include the following steps S46, S47, S44 and S45.

[0183] In step S46, the radar unit receives the echo signal and performs autocorrelation with the transmitted signal to obtain the channel impulse response (CIR).

[0184] Step S47: Accumulate multiple channel impulse responses, perform velocity-dimensional FFT according to distance, and obtain the range-velocity spectrum.

[0185] Step S44: Calculate the absolute value of the distance-velocity spectrum and use the constant false alarm rate (CFAR) detection algorithm to detect the detection points (filtering out noise).

[0186] Step S45: Obtain one or more detection points from the radar point cloud data. Each detection point has target attribute information and signal strength information (e.g., SNR). The target attribute information may include at least two of the following: detection time information, distance information, and velocity information.

[0187] It should be noted that the radar units mentioned above can all adopt a 1T1R architecture. Based on the 1T1R architecture, angle measurement calculation can be avoided, which greatly reduces the computational complexity. The CFAR algorithm is simple to calculate, highly adaptive, and can quickly filter clutter, ensuring the purity of the point cloud, thus laying the foundation for feature extraction and action recognition in the aforementioned embodiments of this application.

[0188] It should also be noted that the radar point cloud data can also be obtained using methods from related technologies, and this disclosure does not limit this approach.

[0189] This application also provides a home appliance device, which includes a radar unit and a control unit. The control unit includes a memory and a processor. The memory is used to store computer instructions, and the processor is used to retrieve the computer instructions from the memory to perform some or all of the steps of any of the methods provided in the above embodiments.

[0190] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the methods in the foregoing embodiments of this application. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto; it may also be a temporary storage medium.

[0191] This application also provides a computer program product, which may include a computer program that, when executed by a processor, can implement any of the methods described in the foregoing embodiments of this application.

[0192] This application may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement any of the methods in the foregoing embodiments of this application.

[0193] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media may include, for example, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), compact disc-read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any combination thereof. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0194] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0195] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(e.g., Smalltalk, C++, etc.) and conventional procedural programming languages ​​(e.g., the "C" language or similar programming languages). The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, may execute computer-readable program instructions to implement various aspects of the embodiments of this disclosure by utilizing state information from the computer-readable program instructions.

[0196] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0197] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0198] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0199] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It should be noted that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are all equivalent.

[0200] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.

Claims

1. A radar-based action recognition method, characterized in that, Applied to home appliances, the home appliances being equipped with radar units; the method includes: Multiple radar point cloud data are collected based on the radar unit; wherein, each radar point cloud data includes at least one detection point, different radar point cloud data correspond to different detection times, and the detection points in the same radar point cloud data correspond to the same detection time. The detection point has target attribute information detected based on the radar unit transmitting and receiving radar signals, and the target attribute information includes detection time information, distance information, and speed information. A target feature map is generated based on the target attribute information of the detection points in multiple radar point cloud data. The target feature map describes the correlation between two sets of target attribute information. The target feature map includes a distance-velocity quantization feature map representing the correlation between velocity information and distance information. The velocity information includes positive and negative velocity values. The positive velocity value indicates that the target object detected by the radar unit is moving away from the radar unit, and the negative velocity value indicates that the target object is moving closer to the radar unit. The first coordinate of the distance-velocity quantization feature map corresponds to the velocity coordinate of the velocity information, and the second coordinate of the distance-velocity quantization feature map corresponds to the distance coordinate of the distance information. The first coordinate and the second coordinate are perpendicular. Determine the target graphic feature type corresponding to the target action to be detected; the target graphic feature type includes a third feature type corresponding to the distance-velocity quantization feature map, the third feature type representing a diamond-shaped feature composed of distance peaks and velocity peaks; wherein, the target graphic feature type includes a third feature type corresponding to the distance-velocity quantization feature map, the third feature type representing a diamond-shaped feature composed of distance peaks and velocity peaks; Detect whether a target graphic corresponding to the target graphic feature type exists in the target feature map; If the target graphic exists in the target feature map, it is determined that the user performed the target action on the home appliance. The step of detecting whether a target graphic corresponding to the target graphic feature type exists in the target feature map includes: The fifth, sixth, seventh, and eighth position points are obtained from multiple position points in the distance-velocity quantization feature map; wherein, the fifth position point is the position point with the smallest distance among the multiple position points, the sixth position point is the position point with the largest distance among the multiple position points, the seventh position point is the position point with the largest velocity among the multiple position points, and the eighth position point is the position point with the smallest velocity among the multiple position points. Construct a target quadrilateral based on the fifth, seventh, sixth, and eighth position points, and determine the side length and diagonal length of each side of the target quadrilateral; wherein, the order of the four vertices of the quadrilateral connected in sequence is the fifth, seventh, sixth, and eighth position points; If the side length satisfies the preset side length condition and the diagonal length satisfies the preset diagonal length condition, then it is determined that there is a target graphic corresponding to the third feature type in the distance velocity quantization feature map.

2. The method according to claim 1, characterized in that, The step of generating a target feature map based on the target attribute information of the detection points in multiple radar point cloud data includes: The target attribute information is quantized based on a preset attribute quantization interval to obtain quantized attribute information; wherein, different target attribute information corresponds to different preset attribute quantization intervals. Determine the target attribute pair containing the first quantization attribute and the second quantization attribute from the quantization attribute information; A two-dimensional attribute feature map is generated based on the target attribute pairs in multiple radar point cloud data; wherein, the two-dimensional attribute feature map includes multiple location points in a two-dimensional coordinate system, each location point corresponds to at least one detection point, the first coordinate of the two-dimensional coordinate system is the coordinate determined based on the first quantization attribute, and the second coordinate of the two-dimensional coordinate system is the coordinate determined based on the second quantization attribute; The target feature map is determined based on the two-dimensional attribute feature map.

3. The method according to claim 2, characterized in that, The detection point also has signal strength information that characterizes the strength of the received radar signal; The step of generating a two-dimensional attribute feature map based on the target attribute pairs in multiple radar point cloud data includes: For each detection point, the position point corresponding to the detection point in the two-dimensional coordinate system is determined based on the first quantization attribute and the second quantization attribute of the detection point; If multiple detection points correspond to the same location point, then the detection point with the highest radar signal strength among the multiple detection points shall be taken as the detection point corresponding to the location point. The two-dimensional attribute feature map is generated based on the location point.

4. The method according to claim 2, characterized in that, Determining the target feature map based on the two-dimensional attribute feature map includes: The location points in the two-dimensional attribute feature map are binarized and / or denoised to generate the target feature map.

5. The method according to any one of claims 1 to 4, characterized in that, The radar unit has a transmit channel and a receive channel.

6. The method according to claim 5, characterized in that, The target feature map further includes a distance-time quantized feature map that characterizes the correlation between the detection time information and the distance information. The first coordinate of the distance-time quantized feature map is the detection time coordinate corresponding to the detection time information, and the second coordinate of the distance-time quantized feature map is the distance coordinate corresponding to the distance information. The first coordinate and the second coordinate are perpendicular. The target graphic feature type includes a first feature type corresponding to the distance-time quantized feature map. The first feature type characterizes a graphic feature with a minimum distance value. The step of detecting whether a target graphic corresponding to the target graphic feature type exists in the target feature map further includes: Obtain the first position point corresponding to the minimum distance in the distance-time quantization feature map; Multiple sets of first location points are obtained based on the first detection time coordinates of the first location point; wherein, the multiple sets of first location points include a first point set and a second point set, the first point set includes location points whose detection time is less than the first detection time coordinates, and the second point set includes location points whose detection time is greater than the first detection time coordinates; Linear fitting is performed on the position points in each of the first set of position points to determine the first fitting parameters of the linear fitting equation corresponding to each of the first set of position points; wherein, the first fitting parameters include at least one of the first linear slope, the first linear intercept, or the first linear fit goodness of fit. If the first fitting parameter satisfies the preset first linear fitting condition, then it is determined that there is a target graphic corresponding to the first feature type in the distance-time quantization feature map.

7. The method according to claim 5, characterized in that, The velocity information includes positive velocity values ​​and negative velocity values. The positive velocity value indicates that the target object detected by the radar unit is moving away from the radar unit, and the negative velocity value indicates that the target object is moving towards the radar unit. The target feature map also includes a velocity time quantization feature map that represents the correlation between the detection time information and the velocity information. The first coordinate of the velocity time quantization feature map is the detection time coordinate corresponding to the detection time information, and the second coordinate is the velocity coordinate corresponding to the velocity information. The first coordinate and the second coordinate are perpendicular. The target graphic feature type includes a second feature type corresponding to the velocity instantaneous quantized feature map, and the second feature type represents the graphic feature where there is a velocity direction inflection point; The step of detecting whether a target graphic corresponding to the target graphic feature type exists in the target feature map further includes: Among multiple location points in the velocity instantaneous quantization feature map, a second location point, a third location point, and a fourth location point located within a continuous detection time window are obtained; wherein, the second location point is the negative velocity peak location point in the continuous detection time window, the third location point is the positive velocity peak location point in the continuous detection time window, and the fourth location point is located between the second location point and the third location point and is the location point where the velocity information changes from a negative velocity value to a positive velocity value; Based on the second detection time coordinates of the second location point, the third detection time coordinates of the third location point, and the fourth detection time coordinates of the fourth location point, a plurality of second location point sets are obtained; wherein, the plurality of second location point sets include a third point set and a fourth point set, the third point set includes location points whose detection time is greater than the second detection time coordinates and less than the fourth detection time coordinates, and the fourth point set includes location points whose detection time is greater than the fourth detection time coordinates and less than the third detection time coordinates; A linear fit is performed on the position points in each of the second set of position points to determine the second fitting parameters of the linear fitting equation corresponding to each of the second set of position points; wherein, the second fitting parameters include at least one of the second linear slope, the second linear intercept, or the second linear fit goodness of fit. If the second fitting parameter satisfies the preset second linear fitting condition, and the second velocity value at the second position point and the third velocity value at the third position satisfy the preset velocity threshold condition, then it is determined that there is a target graphic corresponding to the second feature type in the velocity moment quantization feature map.

8. A household appliance, characterized in that, The home appliance includes a radar unit and a control unit, the control unit including a memory and a processor, the memory for storing computer instructions, and the processor for retrieving the computer instructions from the memory to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images