State recognition method, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a state recognition method, electronic equipment and a storage medium, relates to the technical field of terminal equipment, can recognize and effectively distinguish motion tracks of a human body and other objects, can recognize a static position of the human body, and solves the problem that a millimeter wave radar is difficult to detect a static target. The method comprises the steps that a radar sensor obtains object motion position information; the processor judges and determines whether the end point of the motion trail is the static position of the target human body according to the motion trail of the object, the motion trail of the object is obtained by the radar sensor or the processor through the motion position information of the object, and the object comprises the target human body.

Description

Technical Field

[0001] This invention relates to the field of terminal technology, specifically to a status recognition method, an electronic device, and a storage medium. Background Technology

[0002] Millimeter-wave radar utilizes its strong penetration and anti-jamming capabilities to detect the trajectory of objects in complex environments. However, traditional millimeter-wave radar cannot accurately identify stationary targets.

[0003] Related technologies often employ more complex hardware or more advanced radar technology to improve the accuracy of recognizing stationary human figures, such as using higher-precision radar sensors. However, this leads to increased hardware costs, and the increased sensor accuracy also results in receiving more environmental noise or interference signals. Therefore, how to achieve radar recognition of stationary human figures without changing costs is a pressing problem that needs to be solved. Summary of the Invention

[0004] This application discloses a state recognition method, electronic device, and storage medium. It acquires the motion position information of an object through millimeter-wave radar, and determines whether the endpoint of the motion trajectory is the stationary position of the target human body based on the motion trajectory of the object, thereby realizing the recognition of the stationary state of the human body by millimeter-wave radar.

[0005] This application discloses a state recognition method applied to an electronic device. The electronic device is equipped with a radar sensor that acquires object position information in the environment by detecting object signals, and a processor for performing state recognition. The method includes:

[0006] The radar sensor obtains information about the object's position.

[0007] The processor determines whether the endpoint of the motion trajectory is the stationary position of the target human body based on the motion trajectory of the object. The motion trajectory of the object is obtained by the radar sensor or the processor through the motion position information of the object, and the object includes the target human body.

[0008] In the above technical solution, the motion position information of an object is obtained by a radar sensor, the motion trajectory of the object is obtained based on the motion position information, and the motion trajectory of the object is used to determine whether the motion trajectory of the object is the motion trajectory of a human body, and if the motion trajectory is the motion trajectory of a human body, whether the endpoint of the motion trajectory is the stationary position of the human body, thereby enabling the radar to identify stationary human bodies.

[0009] Furthermore, since any type of radar can acquire the position information of a moving object, the radar in the above scheme can be any type of radar, thus eliminating the need to modify existing radars and reducing equipment costs.

[0010] In one possible implementation, the processor determines whether the endpoint of the motion trajectory is the stationary position of the target human body based on the motion trajectory of the object, including:

[0011] The processor obtains the motion trajectory of the target human body from the motion trajectory of the object;

[0012] The processor determines whether the endpoint of the target human body's motion trajectory is the target human body's stationary position based on the target human body's motion trajectory.

[0013] In the above technical solution, the processor separates the trajectory of the target human body from the trajectory of the object's motion, thus processing only the trajectory of the human body without processing the trajectory of the object's motion. This reduces the amount of computation and minimizes the impact of the object's motion trajectory on the results, thereby improving the reliability of static state recognition.

[0014] In one possible implementation, the processor determines whether the endpoint of the target human body's motion trajectory is the target human body's stationary position based on the target human body's motion trajectory, including:

[0015] The processor determines the endpoint of the target human body's movement trajectory as the target human body's stationary position based on the fact that the endpoint of the target human body's movement trajectory is within the permanent position area. The permanent position area is an area where the total number of position points in the pre-obtained human body movement trajectory exceeds a preset number. The position points include the starting point and the ending point of the human body movement trajectory.

[0016] In the above technical solution, the processor constructs a human body's permanent location region using historical data, and then identifies the static position of the target human body through the human body's permanent location region. The method is simple, has low complexity, and can reduce the processor's computational load.

[0017] In one possible implementation, the processor obtains the motion trajectory of the target human body from the motion trajectory of the object, including:

[0018] The processor identifies the motion trajectory of the target human body from the motion trajectory of the object based on the length of the motion trajectory, wherein the length of the motion trajectory of the target human body is between a pre-set first threshold and a second threshold.

[0019] In the above technical solution, the length threshold used to identify whether the motion trajectory is a human motion trajectory can be set in advance, and different length thresholds can be set according to different environmental scenarios, thereby improving the accuracy of the judgment and the flexibility of the solution.

[0020] In one possible implementation, the method by which the processor obtains the resident location region includes:

[0021] The processor clusters the start and end points of historical human movement trajectories to obtain the human body's permanent location region, which is the region in the historical human movement trajectory where the total number of start and end points exceeds the preset number.

[0022] In the above technical solution, historical trajectory data is analyzed by clustering algorithm to generate permanent location areas. The processing method is simple and can reduce the amount of computation of the processor. Furthermore, the starting and ending points of historical human movement trajectories can reflect human behavior habits. Estimating permanent areas through historical behavior habits can improve the accuracy of determining permanent location areas of the human body.

[0023] In one possible implementation, the method further includes:

[0024] The processor increments the number of location points included in the resident location area by 1; or

[0025] If the number of location points in the permanent location area is 0, and the radar sensor continuously detects a signal in the permanent location area, the processor will increment the number of location points in the permanent location area by 1.

[0026] In the above technical solution, when the processor detects that a human body has entered and remained in the permanent location area, or that the human body has initially remained in the permanent location area, it increments the number of stationary human body position points contained in the permanent location area by 1 to indicate the validity of the permanent location area. The more position points contained in the permanent location area, the higher the validity of the permanent location area.

[0027] In one possible implementation, the method further includes:

[0028] When the processor detects that the starting point of any human movement trajectory is in the permanent location area, and the ending point of any human movement trajectory is outside the permanent location area, the number of location points in the permanent location area is decremented by 1.

[0029] In the above technical solution, when the processor detects that a human body has left the permanent location area, it decrements the number of stationary human body location points contained in the permanent location area by 1 to indicate the validity of the permanent location area. The fewer the location points contained in the permanent location area, the lower the validity of the permanent location area.

[0030] This can be understood as follows: after determining the permanent location area, the accuracy of that area can be verified by subsequently detecting human movement trajectories. If the number of location points in a certain permanent location area is 0 for an extended period, it indicates that the permanent location area may be inaccurate, and the area can be updated. Conversely, if the number of location points in a certain permanent location area is relatively high, it indicates that determining the stationary position of a human body through that area is quite accurate.

[0031] In one possible implementation, the method further includes:

[0032] If, after a preset period of time, the number of location points within the permanent location area is less than a preset number of points, the processor updates the permanent location area based on the movement trajectory of the target human body within the preset time period.

[0033] In the above technical solution, when the number of location points in the permanent location area is less than the preset number of points, it indicates that the permanent location area can no longer represent the actual permanent location area of ​​the human body and needs to be updated. For example, the scene layout in the area detected by the radar may have changed, or the human body's movement habits may have changed. This allows the permanent location area to adapt to various changes and further improves the accuracy of recognizing the human body in a static state.

[0034] In one possible implementation, the processor determines whether the endpoint of the motion trajectory is the stationary position of the target human body based on the motion trajectory of the object, including:

[0035] The processor uses a pre-set neural network model to identify the motion trajectory of the object and whether the endpoint of the object's motion trajectory is the stationary position of the target human body. The neural network model is trained using motion trajectories with labels, and the labels are used to indicate whether the trajectory is a human body trajectory and whether the endpoint of the motion trajectory is a stationary state.

[0036] In the above technical solution, the motion trajectory of an object is classified and identified by using a neural network model. Since the neural network model itself has the characteristic of high accuracy, the use of a neural network model in this solution can improve the accuracy of recognizing the static state of the human body.

[0037] In one possible implementation, the processor uses a pre-set neural network model to identify the motion trajectory of the object, and identifies whether the endpoint of the object's motion trajectory is the stationary position of the target human body, including:

[0038] The processor inputs the motion trajectory of the object into the neural network model and obtains the recognition result output by the neural network model. The recognition result includes the confidence level that the trajectory type is a human trajectory and the confidence level that the endpoint is a stationary state.

[0039] The processor determines whether the endpoint of the object's trajectory is the stationary position of the target human body based on the confidence level of the human body trajectory and the confidence level of the endpoint being stationary.

[0040] Specifically, if the confidence level of the trajectory type being a human trajectory is greater than a third threshold and the confidence level of the endpoint being a stationary state is greater than a fourth threshold, then the endpoint of the motion trajectory is determined to be the stationary position of the target human body.

[0041] In the above technical solution, the processor outputs the confidence level of the human trajectory and the confidence level of the endpoint being stationary through a neural network model. By combining the dual judgment of trajectory type and endpoint stationary state, the reliability of human stationary state recognition can be improved, especially under conditions of weak signals or significant interference. The third and fourth thresholds used to determine whether the endpoint of the motion trajectory is the stationary position of the target human body can be set in advance, thereby improving the flexibility of the solution.

[0042] In one possible implementation, the neural network model includes a feature extraction module and a recognition module. The feature extraction module is used to extract trajectory features of the object's motion trajectory, the trajectory features including temporal features and motion features. The recognition module is used to identify whether the endpoint of the object's motion trajectory is a stationary position of a human body based on the temporal features and motion features extracted by the feature extraction module.

[0043] In the above technical solution, the features of motion trajectory in time and space are extracted by a neural network model. By combining temporal features and motion features, the accuracy of identifying the static state of the human body can be improved.

[0044] This application discloses an electronic device, including:

[0045] Radar sensors are used to detect object signals and obtain information about the position of objects in the environment.

[0046] The radar sensor is configured to obtain information about the position of an object's motion.

[0047] The processor is used for status recognition;

[0048] The processor is configured to determine whether the endpoint of the motion trajectory is the stationary position of the target human body based on the motion trajectory of the object, wherein the motion trajectory of the object is obtained by the radar sensor or the processor through the motion position information of the object, and the object includes the target human body.

[0049] This application discloses an air conditioner, including:

[0050] Indoor unit;

[0051] The radar module, installed in the indoor unit, is used to detect object signals and obtain object location information in the environment;

[0052] The radar module is configured to obtain the position information of the moving object;

[0053] The processor is used for status recognition;

[0054] The processor is configured to determine whether the endpoint of the motion trajectory is the stationary position of the target human body based on the motion trajectory of the object, wherein the motion trajectory of the object is obtained by the radar sensor or the processor through the motion position information of the object, and the object includes the target human body.

[0055] This application discloses a computer-readable storage medium for storing a computer program, the computer program including instructions for implementing the above-described processor.

[0056] This application discloses a computer program product, which includes computer program code. When the computer program code is run on a computer, it enables the computer to implement the method executed by the processor. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a schematic diagram of the structure of the electronic device involved in the embodiments of this application;

[0059] Figure 2 This is an application scenario diagram of the state recognition method provided in the embodiments of this application;

[0060] Figure 3 A schematic flowchart illustrating the state recognition method provided in the embodiments of this application;

[0061] Figure 4 A schematic flowchart illustrating trajectory construction provided in the embodiments of this application;

[0062] Figure 5 A schematic diagram illustrating trajectory matching based on the distance between trajectory points, provided as an embodiment of this application;

[0063] Figure 6 A schematic flowchart illustrating the identification of a stationary human body based on a frequently visited location area, provided for embodiments of this application;

[0064] Figure 7 A schematic flowchart illustrating the clustering construction of human body resident location regions provided in this application embodiment;

[0065] Figure 8 This is a schematic diagram illustrating the recognition of a stationary human body in a room setting, as provided in an embodiment of this application.

[0066] Figure 9 A schematic flowchart illustrating the identification of a static human body based on a neural network model, provided for embodiments of this application;

[0067] Figure 10 This is a schematic diagram of a recurrent neural network model provided in an embodiment of this application;

[0068] Figure 11 A schematic block diagram of a processor provided for an embodiment of this application;

[0069] Figure 12 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0071] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. For example, "first instruction" and "second instruction" are used to distinguish different user instructions and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0072] It should be noted that, in this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in this application should not be construed as having a greater advantage than other embodiments or designs. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0073] Furthermore, "at least one" refers to one or more, while "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0074] Furthermore, the terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0075] Millimeter-wave radar (mmWave Radar) is a radar system that uses the millimeter-wave frequency band (typically 30 GHz to 300 GHz) for detection and measurement. Millimeter waves belong to the high-frequency part of the electromagnetic spectrum, with wavelengths between 1 millimeter and 10 millimeters. Due to its high resolution and strong anti-jamming capabilities, millimeter-wave radar can be used to accurately detect the position and velocity of objects in complex environments.

[0076] Millimeter-wave radar emits high-frequency electromagnetic wave signals (usually continuous waves or modulated signals). These electromagnetic waves travel at the speed of light and are reflected when they encounter objects. After the emitted electromagnetic waves encounter an object, some of the signal is reflected back. The receiver of the millimeter-wave radar receives these reflected wave signals and determines various parameters of the object by analyzing these signals.

[0077] On one hand, millimeter-wave radar uses the propagation time of electromagnetic waves to measure the distance between a target and the radar. The basic principle is the time difference between the transmitted wave reaching the target object and being reflected back. As an example, the distance can be calculated using the following formula:

[0078]

[0079] Where d is the distance between the object and the radar, and c is the speed of light (approximately 3 × 10⁻⁶). 8(m / s), where t is the time difference between transmitting the signal and receiving the reflected signal. Since electromagnetic waves travel at the speed of light, millimeter-wave radar can accurately measure the distance to a target through an extremely short time difference.

[0080] On the other hand, when a target object moves relative to the radar, the frequency of the reflected wave changes. Specifically, if the target approaches the radar, the frequency of the reflected wave increases; if the target moves away from the radar, the frequency of the reflected wave decreases. By measuring the frequency change, the radar can calculate the relative speed of the target.

[0081] As an example, radar calculations are performed using the Doppler effect formula, which is as follows:

[0082]

[0083] Where is the velocity of the target, Δf is the Doppler frequency shift (i.e., the change in frequency), c is the speed of light, and f0 is the frequency of the emitted signal.

[0084] On the other hand, millimeter-wave radar can perform angle measurements using antenna arrays. After receiving reflected waves, the radar antenna array sends signals to different antennas with varying phases and intensities. By processing the phase differences between these signals, the azimuth angle of an object can be calculated. The specific calculation formulas will not be elaborated here. This process of angle measurement is also known as beamforming.

[0085] In practical applications, millimeter-wave radar faces certain challenges in identifying stationary objects (including stationary human bodies).

[0086] As mentioned above, millimeter-wave radar primarily relies on the Doppler effect to measure the relative velocity of objects, that is, determining the object's velocity by detecting changes in the frequency of reflected waves. Therefore, if an object is stationary (or nearly stationary) relative to the radar, the frequency of the reflected wave will not change significantly, meaning the Doppler shift may not be detected, making it difficult for the radar to detect a stationary object.

[0087] Furthermore, the signal reflection characteristics of millimeter-wave radar depend on the shape, material, and surface properties of the object. The human body is an irregular shape with a complex surface, resulting in relatively dispersed reflections of millimeter-wave signals. This irregular reflection leads to a weaker signal received by the radar, especially when the human body is stationary, where the signal may be insufficient for effective target identification. Moreover, millimeter-wave radar typically measures objects using phase difference, which is more effective when dealing with moving targets. However, for stationary targets, the phase change of the reflected signal is small, limiting the accuracy of angle measurement and preventing millimeter-wave radar from accurately determining the specific location of a stationary human body.

[0088] To address the aforementioned issues, related technologies often employ high-sensitivity receivers and multi-sensor fusion to enable millimeter-wave radar to identify stationary human figures. However, increasing sensitivity typically requires higher-quality hardware, including more advanced antennas and receiver modules, which increases system complexity and cost. Furthermore, high-sensitivity devices necessitate more complex signal processing algorithms, further increasing the processor's workload.

[0089] To address the aforementioned issues, related technologies often employ high-sensitivity receivers and multi-sensor fusion to enable millimeter-wave radar to identify stationary human figures. However, increasing sensitivity typically requires higher-quality hardware, including more advanced antennas and receiver modules, which increases system complexity and cost. Furthermore, high-sensitivity devices necessitate more complex signal processing algorithms, further increasing the processor's workload.

[0090] Therefore, embodiments of this application disclose a state recognition method, an electronic device, and a storage medium. In this application, the motion position information of an object is obtained through a radar sensor, the motion trajectory of the object is obtained based on the motion position information, and the motion trajectory is used to determine whether the motion trajectory of the object is the motion trajectory of a human body, and if the motion trajectory is the motion trajectory of a human body, whether the endpoint of the motion trajectory is the stationary position of the human body, thereby enabling radar to identify a stationary human body.

[0091] Furthermore, since any type of radar can acquire the position information of a moving object, the radar in the above scheme can be any type of radar, thus eliminating the need to modify existing radars and reducing equipment costs.

[0092] To implement the method described above, the electronic device involved in the embodiments of this application will first be introduced.

[0093] like Figure 1 The diagram shown is a structural schematic of an electronic device 100 involved in an embodiment of this application. The electronic device may include, but is not limited to, a processor 101 and a radar sensor 102.

[0094] The radar sensor 102 can be a millimeter-wave radar sensor or other types of radar sensors, such as micrometer-wave radar sensors, etc., and is not limited in this embodiment. The radar sensor 102 and the processor 101 can be electrically connected, allowing communication between them. For example, the radar sensor 102 can collect the motion position information of moving objects and communicate with the processor 101 based on this information. As an example, the radar sensor 102 can send the collected motion position information to the processor 101, or it can obtain the motion trajectory of each object based on the collected motion position information and then send the trajectory to the processor 101.

[0095] The processor 101 can identify whether the moving object is a human body based on information sent by the radar sensor, and whether the endpoint of the movement trajectory is the stationary position of the human body. The processor 101 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and lines. It executes various functions and processes data by running or executing stored software programs and / or modules, and by calling stored data. Optionally, the processor 101 may include one or more processing units; the processor 101 may integrate application processors and modem processors, etc.

[0096] Those skilled in the art will understand that Figure 2 The architecture shown does not constitute a limitation on the electronic device. The architecture of the electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0097] As an example, the electronic device can be an air conditioner. To facilitate understanding of the application of the aforementioned state recognition method in an air conditioner, the following uses an air conditioner as an example to introduce an application scenario involved in the embodiments of this application.

[0098] like Figure 2 The figure shown is an application scenario diagram 200 of the state recognition method provided in the embodiment of this application. The application scenario diagram includes an air conditioner 201, a radar sensor 202, an air outlet 203, and a human body 204.

[0099] In some possible embodiments, the air conditioner 201 can be a split air conditioner, a window air conditioner, a central air conditioner, or a portable air conditioner. This application does not specifically limit the embodiments.

[0100] In some possible embodiments, the radar sensor 202 transmits and receives millimeter-wave signals reflected from the human body 204 to measure the distance between the radar sensor 202 and the human body 204. When the radar sensor 202 detects the location of the human body, the air conditioner 201 can automatically adjust the airflow direction of the air outlet 203 to direct the cool or warm air towards the area where the person is located, thereby improving comfort.

[0101] In some possible embodiments, the air conditioner 201 can also be configured to avoid direct airflow. The millimeter-wave radar identifies the position of the human body 204 and intelligently adjusts the airflow angle of the air outlet 203 to avoid the position of the human body 204, providing a more comfortable experience.

[0102] In some possible embodiments, when the millimeter-wave radar sensor 202 detects that there is no human body 204 in the room, the air conditioner 201 can automatically enter standby mode or turn off, thereby saving energy; when the millimeter-wave radar sensor 202 detects that there is a human body 204 in the room, it automatically resumes normal operation.

[0103] In some possible embodiments, by detecting a sleeping human body 204, the millimeter-wave radar sensor 202 can automatically adjust the temperature according to the human body's state. For example, as the depth of sleep changes, the temperature setting of the air conditioner 201 can be automatically lowered or raised to provide a more comfortable sleep environment.

[0104] In some possible embodiments, the millimeter-wave radar sensor 202 can monitor the distribution of human bodies 204 in different areas of the room, and combined with the multi-vent design of the air conditioner 201, it can achieve precise regional temperature control.

[0105] In the application scenarios of the aforementioned state detection method, the radar sensor 202 accurately locates the human body 204, avoiding the phenomenon of some areas being too cold or too hot due to the fixed airflow direction of traditional air conditioners 201. This results in a more uniform indoor temperature and allows for personalized airflow modes based on the location of different people in the room. Compared to the traditional timed on / off mode, the millimeter-wave radar sensor 202 can intelligently adjust the operating status of the air conditioner 201 based on the actual presence of the human body 204, avoiding unnecessary energy waste, especially in places like offices and conference rooms. Furthermore, for human bodies 204 that remain stationary for extended periods (such as sleeping or working), the radar sensor 202 can continuously monitor them, preventing misjudgments about the air conditioner needing to be turned off due to insufficient movement, thus improving the level of intelligence.

[0106] Next, the state recognition method involved in the embodiments of this application will be introduced. This state recognition method can be applied to, for example... Figure 1 In the electronic devices shown, or, it can also be applied to, such as Figure 2 The air conditioner 201 is shown in the application scenario.

[0107] like Figure 3 The diagram shown is a schematic flowchart 300 of a state recognition method provided in an embodiment of this application. The method includes the following steps:

[0108] S301, the radar sensor obtains information about the position of the moving object.

[0109] In some possible embodiments, the radar sensor periodically sends high-frequency electromagnetic wave signals to a preset area that it can monitor. Based on the electromagnetic wave signals reflected by objects and the propagation time of the electromagnetic waves, the radar sensor can measure the distance between itself and various objects in the preset area. The size of the preset area can be determined according to the scanning range of the radar sensor; the larger the scanning range, the larger the preset area.

[0110] In some possible embodiments, the radar sensor determines the relative position of an object in the horizontal scanning direction and obtains the azimuth angle of the object relative to the radar sensor by transmitting and receiving electromagnetic wave signals and analyzing the horizontal deflection of the beam using an antenna array.

[0111] In the embodiments of this application, the azimuth angle can be the angle between the object and the center line of the radar in the horizontal scanning direction. Of course, the radar can also measure the angle between the object and the center line of the radar in the vertical scanning direction, or it can simultaneously measure the angles between the object and the center lines of the horizontal and vertical directions. This is not limited here.

[0112] In some possible embodiments, the radar sensor can detect the angle between the beam and the corresponding centerline in the vertical scanning direction by transmitting and receiving electromagnetic wave signals and using an antenna array to determine the height or vertical position of the object and obtain the downward angle of the object relative to the radar sensor. This downward angle can be understood as the angle between the object and the corresponding centerline in the vertical scanning direction of the radar.

[0113] For example, if a radar sensor detects a moving object at a certain moment with an azimuth angle of α, a downward angle of β, and a distance of x, then the radar sensor obtains the motion position information corresponding to the moving object as (xcosαcosβ, xsinαcosβ). Alternatively, the motion position information can also be expressed as (x, α, β), without any limitation.

[0114] The radar sensor can collect the motion position information of moving objects in a preset area that the radar can monitor, according to a preset sampling frequency. This sampling frequency can be set according to actual usage requirements.

[0115] S302, the processor determines whether the endpoint of the motion trajectory is the stationary position of the target human body based on the motion trajectory of the object, wherein the motion trajectory of the object is obtained by the radar sensor or the processor through the motion position information of the object, and the object includes the target human body.

[0116] In some possible embodiments, the radar sensor can send the motion position information of the object to the processor at the aforementioned sampling frequency. The processor can construct the corresponding motion trajectory based on the acquired multiple motion position information, and then identify the motion trajectory.

[0117] In some possible embodiments, after the radar sensor acquires the motion position information of the object according to the aforementioned sampling frequency, the radar sensor may also process the acquired motion position information to obtain the corresponding motion trajectory according to its own processor, and then send the motion trajectory to the processor.

[0118] After acquiring the object's motion trajectory, the processor determines whether the endpoint of the acquired trajectory is the stationary position of the target human body. It's important to understand that the target human body is not a specific individual, but rather a general term.

[0119] In this embodiment of the application, the processor may determine whether the endpoint of the acquired motion trajectory is the stationary position of the target human body in ways including but not limited to the following two methods:

[0120] In the first method, the processor determines whether the endpoint of the acquired motion trajectory is the stationary position of the target human body based on a preset permanent location area.

[0121] In this application embodiment, the permanent location area is one of the preset areas monitored by the radar sensor. It can be understood as an area where the human body frequently resides or arrives, or an area where the probability of the human body residing is greater than or equal to a probability threshold, or an area where the number of times the human body residing is greater than or equal to a number threshold, or an area where the frequency of the human body residing is greater than or equal to a frequency domain threshold.

[0122] In the first example, the permanent location area can be determined based on the positional layout of various objects within a preset area monitored by the radar sensor. For instance, the preset area monitored by the radar sensor might be the area encompassed by a room, such as a living room or bedroom, containing various furniture and appliances. Taking the living room as an example, the living room includes a sofa, coffee table, and TV cabinet. However, a person cannot reside in the area where the coffee table and TV cabinet are located. Therefore, the area corresponding to the sofa can be set as the permanent location area.

[0123] In the second example, the permanent location area can be determined based on historical human movement trajectories. For instance, clustering the movement location information in historical human movement trajectories can identify which locations are target locations reached by the human body more than or equal to a certain number of times. Then, the permanent location area can be determined based on all the identified target locations.

[0124] In this case, since the historical human movement trajectory may include multiple movement location information, such as all the intermediate locations the human passes through from the starting point to the end point, and the intermediate locations may not be the areas where the human wants to stay, in order to more accurately determine the permanent location area, the starting point and the end point in the historical human movement trajectory can be clustered to obtain the permanent location area.

[0125] In the second method, the processor determines whether the endpoint of the acquired motion trajectory is the stationary position of the target human body based on a preset neural network model.

[0126] As an example, sample data can be pre-acquired, which may include historical human motion trajectories, historical object motion trajectories, and label information corresponding to each motion trajectory. The label information is used to indicate whether the corresponding motion trajectory is a human motion trajectory or an object motion trajectory, and whether the endpoint of the corresponding motion trajectory is a stationary position of the human body. The pre-trained neural network model is trained using the sample data to obtain a pre-trained neural network model.

[0127] As a first example, the input of the pre-trained neural network model is a motion trajectory, and the output is the probability that the type of the motion trajectory is a human body and the probability that the endpoint of the motion trajectory is a stationary position of the human body. If the probability that the type of the motion trajectory output by the pre-trained neural network model is a human body is greater than or equal to a first preset threshold, then the motion trajectory can be considered to be a human body motion trajectory; if the probability that the endpoint of the motion trajectory output by the pre-trained neural network model is a stationary position of the human body is greater than or equal to a second preset threshold, then the endpoint of the motion trajectory can be considered to be a stationary position of the human body.

[0128] Of course, the input and output of a pre-trained neural network model can also be in other forms.

[0129] As a second example, the input to the pre-trained neural network model can be the trajectory features of the motion trajectory, such as the temporal features of the trajectory or the spatial location features of the trajectory. The output of the pre-trained neural network model can also be only one. For example, the output of the pre-trained neural network model can be the probability that the endpoint of the motion trajectory is a stationary position of the human body. If the probability that the endpoint of the pre-trained neural network model is a stationary position of the human body is greater than or equal to a second preset threshold, then the endpoint of the motion trajectory can be considered to be a stationary position of the human body. If the probability that the endpoint of the pre-trained neural network model is a stationary position of the human body is less than the second preset threshold, then the endpoint of the motion trajectory can be considered not to be a stationary position of the human body.

[0130] It should be noted that, in this case, the reason why the probability that the endpoint of the pre-trained neural network model output is the static position of the human body is less than the second preset threshold may be because the motion trajectory input to the pre-trained neural network model is not the motion trajectory of the human body itself, or it may be that the motion trajectory input to the pre-trained neural network model is the motion trajectory of the human body, but the endpoint is not the static position of the human body.

[0131] Unlike the first example, in the first example, the pre-trained neural network model outputs two probability values: the probability that the type of the motion trajectory is a human body and the probability that the endpoint of the motion trajectory is a stationary position of the human body. Based on the output results, it can be determined whether the reason why the endpoint of the motion trajectory is not a stationary position of the human body is that the trajectory type itself is not a human motion trajectory, or because the motion trajectory is a human motion trajectory but the endpoint is not a stationary position of the human body.

[0132] As an example, the neural network model can be a deep neural network (DNN), a convolutional neural network (CNN), or a generative adversarial network (GAN), etc., without limitation.

[0133] In some possible embodiments, after the processor acquires the motion trajectory of the object, the processor uses a pre-trained neural network model to determine whether the endpoint of the trajectory is a stationary position of the human body.

[0134] Additionally, it should be noted that when the sensor acquires a large amount of information about the object's movement position, there may also be a large number of movement trajectories, which would result in a large computational load for the processor. In this case, before determining whether the endpoint of the trajectory is the stationary position of the human body, the processor can filter the acquired movement trajectories to identify the target human body's movement trajectory, and then only determine whether the endpoint of the trajectory is the stationary position of the human body based on the target human body's movement trajectory.

[0135] As an example, the processor can calculate the length of each motion trajectory and determine whether the corresponding trajectory is a human motion trajectory based on the length. For instance, if the length of a motion trajectory is greater than or equal to a length threshold, the processor determines that the motion trajectory is the target human motion trajectory; otherwise, the processor confirms that the motion trajectory is not the target human motion trajectory.

[0136] The methods for determining the length of a motion trajectory may include, but are not limited to:

[0137] 1. Determine the length of the motion trajectory based on the number of location points included in the trajectory.

[0138] For example, the length of the motion trajectory can be the number of all the location points included in the motion trajectory. For instance, if the number of location points included in the motion trajectory is greater than 10, the processor determines that the motion trajectory is a human motion trajectory.

[0139] 2. Determine the length of the motion trajectory based on the distance between the starting and ending points.

[0140] For example, the location information of the starting point and the location information of the ending point can be obtained. According to Euclidean theorem, the distance between the starting point and the ending point can be obtained, which is the length of the motion trajectory.

[0141] Of course, the length of the trajectory can also be determined in other ways, which will not be listed here.

[0142] In the above technical solution, the motion position information of an object is obtained by a radar sensor, the motion trajectory of the object is obtained based on the motion position information, and the motion trajectory of the object is used to determine whether the motion trajectory of the object is the motion trajectory of a human body, and if the motion trajectory is the motion trajectory of a human body, whether the endpoint of the motion trajectory is the stationary position of the human body, thereby enabling the radar to identify stationary human bodies.

[0143] Furthermore, since any type of radar can acquire the position information of a moving object, the radar in the above scheme can be any type of radar, thus eliminating the need to modify existing radars and reducing equipment costs.

[0144] The above embodiments describe the steps of the state recognition method. The execution process of each step involved in the state recognition method will be explained below.

[0145] The following embodiments further illustrate the execution process of trajectory construction involved in the above embodiments. The trajectory construction can be understood as constructing the content of the motion trajectory based on the motion position information obtained by the radar sensor.

[0146] like Figure 4 The diagram shown is a schematic flowchart 400 for trajectory construction provided in an embodiment of this application. The flowchart includes:

[0147] S401, the radar sensor acquires the motion and position information of an object.

[0148] In the following description, we will use a millimeter-wave radar sensor as an example, and the preset area monitored by the millimeter-wave radar sensor is a room.

[0149] In some possible embodiments, a millimeter-wave radar sensor periodically transmits electromagnetic wave signals into a room. Based on the electromagnetic wave signals reflected from objects within the room, the millimeter-wave radar sensor can obtain the distance, top angle, and azimuth of the target object. The position of the target object relative to the radar sensor can then be calculated based on the distance, top angle, and azimuth.

[0150] For example, if the millimeter-wave radar sensor measures the azimuth angle of the robot vacuum cleaner in the room as α, the downward angle as β, and the distance as l, then the coordinates of the robot vacuum cleaner relative to the radar sensor are (x, y) = (lcosαcosβ, lsinαcosβ).

[0151] The method by which the millimeter-wave radar sensor acquires the motion position information of the object is similar to that in step S301, and will not be described again here.

[0152] S402, trajectory initialization.

[0153] In some possible embodiments, for an initial time t=0, the millimeter-wave radar sensor detects multiple points P0, P1, P2, P3, ..., P n They are treated as independent trajectories, with each point representing the starting position of a motion trajectory.

[0154] S403, construct the distance matrix.

[0155] In some possible embodiments, for a set of target position data output by the radar sensor at the current time t, the distance between the new monitoring point and the endpoint of each existing trajectory is first calculated:

[0156]

[0157] Each element D[i,j] of the distance matrix D represents the distance between the i-th new location point and the j-th existing trajectory point. The matrix has dimensions m×n, where m is the number of new location points and n is the number of existing trajectories.

[0158] S404, Global Match.

[0159] In some possible implementations, the goal of the global nearest neighbor algorithm is to find a set of one-to-one matches in the distance matrix that minimizes the total distance. This requires the use of the Hungarian algorithm, which is used to solve the optimal matching problem. Assuming the size of matrix D is m×n, if m≠n, dummy columns or rows need to be added to make the matrix a square matrix. These newly added rows / columns can be assigned a large distance value, indicating that matching these dummy points is extremely costly and therefore will not be selected in the final match.

[0160] Next, for the distance matrix after supplementing it into a square matrix, perform the following steps: (1) Subtract the minimum value of each row so that at least one element in each row is 0; (2) Subtract the minimum value of each column so that at least one element in each column is 0; (3) After the row and column subtraction operations, the zero elements in the matrix can represent potential optimal matches. By finding independent zero elements, an initial matching scheme can be constructed. Independent zero elements refer to at most one zero element selected in each row and each column. If n pairs of independent zero elements can be found, the optimal match is found and the algorithm ends; (4) If n pairs of independent zero elements cannot be found, the matrix needs to be adjusted: cover all zero elements with the fewest horizontal lines. Find the minimum value that is not covered and update the matrix with it: subtract the minimum value from the elements that are not covered, and add the minimum value to the elements that are covered twice. Repeat the above steps until n pairs of independent zero elements are found. The output of the Hungarian algorithm is the optimal matching scheme, that is, the new position points P1, P2, ..., P m With existing trajectories T1, T2, ..., T n The matching relationship between them. Each location point is assigned to a trajectory to minimize the total distance.

[0161] For example, let the distance matrix be... This represents the distances between three new locations P1, P2, P3 and three existing trajectories T1, T2, T3. The minimum value in the first row is 4, so subtracting 4 from the entire row gives (4, 8, 0). The minimum value in the second row is 5, so subtracting 5 from the entire row gives (0, 5, 1). The minimum value in the third row is 3, so subtracting 3 from the entire row gives (6, 4, 0). This results in a new matrix. The minimum value in column 1 is 0, so subtracting 0 from the entire column gives (4, 0, 6); the minimum value in column 2 is 4, so subtracting 4 from the entire column gives (4, 1, 0); the minimum value in column 3 is 0, so subtracting 0 from the entire column gives (0, 1, 0), resulting in the updated matrix. In this matrix, we can find three independent pairs of 0 elements: P1 corresponds to T3, P2 corresponds to T1, and P3 corresponds to T2. This is the optimal match, which minimizes the total cost.

[0162] S405, New and Old Track Management.

[0163] In some possible implementations, for a new location point that cannot be matched with an existing trajectory, the location point is initialized as a new trajectory. If a trajectory fails to match any new point for a long time, it may mean that the target has disappeared, and the trajectory is terminated.

[0164] The above embodiments use a global nearest neighbor algorithm to match newly detected location points of the radar sensor with existing trajectories, ensuring that the trajectory association in multi-target tracking is globally optimal and avoiding the mismatch problem in simple nearest neighbor algorithms. Especially when multiple targets are close to each other, the global nearest neighbor algorithm can effectively avoid erroneous associations.

[0165] Additionally, it should be noted that a radar sensor can detect the motion position information of an object at various times and then send the detected motion position information to a processor. The processor then processes the motion position information as described above to obtain the motion trajectory. Since the radar sensor itself also has a processor, after detecting the motion position information of an object at various times, the radar sensor can also process the motion position information through its own processor to obtain the motion trajectory and then directly send the motion trajectory to the processor. Those skilled in the art can set this according to actual usage requirements, and no restrictions are imposed here.

[0166] The following example illustrates how radar sensors and processors construct trajectories, based on the trajectory construction method described above.

[0167] like Figure 5 The diagram shown is a schematic diagram 500 illustrating trajectory matching based on the distance between trajectory points according to an embodiment of this application.

[0168] In some possible embodiments, the radar sensor supports trajectory tracking methods. At time t, the millimeter-wave radar simultaneously detects two signals, namely... and At time t+1, the millimeter-wave radar simultaneously detected two signals, namely... and Where a and b are the ID tags assigned by the radar sensor to the detected targets, and the radar sensor assigns a unique ID tag to each target. These IDs are used to distinguish the trajectories of multiple targets, and the radar sensor constructs a trajectory for signals with the same ID tag in chronological order. Therefore, the trajectories of target a and target b from time t to t+n are as follows:

[0169] Target A:

[0170] Target b:

[0171] Next, the radar sensor transmits the motion trajectories of target a and target b to the processor in the electronic device for further processing, in order to identify whether the endpoint of the trajectory is a stationary position of the human body.

[0172] In the above embodiments, x and y can be the angle and distance of the target relative to the radar sensor, or the horizontal and vertical coordinates of the target relative to the radar sensor. This application does not impose specific limitations on these embodiments.

[0173] In one possible embodiment, the radar sensor transmits the detected target signals to the processor, and the target signals output by the radar sensor do not contain markers. The processor then constructs trajectories from these target signals. For the four sets of signals detected by the radar sensor at adjacent times a and b [(x a ,y a ),(x' a ,y' a ),(x b ,y b ),(x' b ,y' b The processor calculates the two sets of signals collected at time b according to the distance calculation formula. b ,y b ),(x' b ,y' b The two sets of signals collected at time a [(x)] and a a ,y a ),(x' a ,y' a The distance between [] is used to obtain the distance matrix D, and the processor constructs the trajectory based on the distance matrix D.

[0174] For example, let the distance matrix be... Represents the location point (x) a ,y a ) and location point (x b ,y b The distance between (x) and (x) is 1, and the location point (x) a ,y a) and location point (x b ',y b The distance between ') is 3, and the location point (x) a ',y a ') and location point (x b ,y b The distance between (x) and (x) is 3, and the location point (x) a ',y a ') and location point (x b ',y b The distance between (x, y) and (x, y) is 4. The processor selects distances 1 and 4 to minimize the total distance cost. Therefore, the processor will move the position point (x) to the desired location. a ,y a ) and location point (x b ,y b Match the location point (x) a ',y a ') and location point (x b ',y b If the total distance cost is minimized when matching '), then the processor will select the location point (x) b ,y b Add to location point (x) a ,y a In the trajectory where ) is located, the position point (x) b ',y b Add to location point (x) a ',y a The trajectory where ') is located.

[0175] By providing specific examples of trajectory matching, the principles of trajectory construction algorithms can be more clearly understood. Next, we will introduce how to use resident location regions for human stationary state recognition.

[0176] like Figure 6 The diagram shown is a schematic flowchart 600 for identifying a person's stationary state based on their usual location area, according to an embodiment of this application. The flowchart includes:

[0177] S601, the millimeter-wave radar outputs the position of an object's motion.

[0178] In some possible embodiments, a millimeter-wave radar sensor periodically transmits electromagnetic wave signals into a room. Based on the electromagnetic wave signals reflected from objects within the room, the millimeter-wave radar sensor can obtain the distance, top angle, and azimuth of the target object. The position of the target object relative to the radar sensor can then be calculated based on the distance, top angle, and azimuth.

[0179] For example, if the radar sensor measures the azimuth angle of the robot vacuum cleaner in the room as α, the downward angle as β, and the distance as l, then the coordinates of the robot vacuum cleaner relative to the radar sensor are (x, y) = (lcosαcosβ, lsinαcosβ).

[0180] The S602 processor constructs a trajectory based on the motion position detected by radar.

[0181] In some possible embodiments, the processor uses a global nearest neighbor algorithm for trajectory construction. For each moment, the processor matches the object's motion position transmitted by the radar sensor with an existing trajectory or establishes a new trajectory. Regarding trajectory construction methods and... Figure 4 The methods involved are similar and will not be repeated here.

[0182] S603, the processor determines whether the length of the motion trajectory is between a preset first threshold and a second threshold.

[0183] In some possible embodiments, the length of the motion trajectory is the number of trajectory position points included in the trajectory, and the number of trajectory position points included in each trajectory indicates the duration of the event in the radar field of view. Human motion trajectories often generate more trajectory points than static or small-range moving objects. Therefore, by setting a first threshold and a second threshold, if the length of the motion trajectory is greater than or equal to the first threshold and less than or equal to the second threshold, the processor determines that the motion trajectory is a human motion trajectory; otherwise, the processor determines that the motion trajectory is the motion trajectory of another object.

[0184] For example, the first threshold can be set to 15 and the second threshold to 50. If the processor obtains that trajectory A contains 5 location points and trajectory B contains 30 location points at the current moment, then trajectory B is likely the trajectory of the human body.

[0185] In some possible embodiments, if the number of location points included in the trajectory of the human body at the current moment is less than a first threshold, the processor temporarily considers the trajectory to be the trajectory of another object. After a period of time, the radar receives the subsequent multiple location points of the human body and transmits them to the processor. The processor updates the trajectory of the human body and finds that the length of the human body's trajectory is between a first preset value and a second preset value. At this time, the processor determines that the updated trajectory is the human body's trajectory.

[0186] S604, the processor determines whether the endpoint of the human movement trajectory is located within the human's usual position area.

[0187] In some possible embodiments, if there is no permanent human body location area, it indicates that the processor is identifying a stationary human body for the first time, and there is no historical trajectory data. In this case, the permanent human body location area needs to be initialized: the processor continuously constructs human movement trajectories until the total number of start and end points of the human movement trajectories reaches a preset number. The start point of the human movement trajectory is the first point where the human body enters the radar sensor detection area or the point where the human body's state changes from stationary to moving.

[0188] The endpoint of a human motion trajectory is the last point where the human leaves the radar sensor's detection area or the point where the human transitions from motion to stillness. Therefore, the start and end points of a human motion trajectory likely include the human's still state. The processor extracts the start and end points of the human motion trajectory and performs clustering. The resulting clusters represent areas where the human is frequently still; these areas are the human stillness position regions. When the endpoint of a human motion trajectory is located within this human stillness position region, the endpoint of the trajectory can be considered a human stillness position point, within acceptable error margins.

[0189] In some possible embodiments, if the processor determines that the target motion trajectory is a human motion trajectory, it needs to further determine whether the endpoint of the human motion trajectory is a stationary position of the human body. The determination is based on whether the endpoint of the human motion trajectory is located within the area of ​​the human body's usual stationary position. If the endpoint of the human motion trajectory is located within the area of ​​the human body's usual stationary position, the processor determines that the endpoint of the human motion trajectory is a stationary position of the human body.

[0190] S605, the processor increments the number of stationary human position points included in the area where the human body is usually located by 1.

[0191] In some possible embodiments, if the processor detects that the endpoint of the human motion trajectory is located in the human stationary position area, the processor determines that the endpoint of the human motion trajectory is the human stationary position. At this time, the number of human stationary position points included in the human stationary position area is incremented by 1 to indicate the validity of the human stationary position area.

[0192] For example, when a human body enters the human body's usual location area from outside the radar's detection range, the number of human body stationary location points included in the human body's usual location area is increased by 1.

[0193] In some possible embodiments, when the processor and radar sensor are activated, the number of stationary human body positions included in the human body's permanent location area is 0. If the radar sensor detects that there is a continuous output of motion signals in the human body's permanent location area, it indicates that the human body's permanent location area already includes stationary human bodies before the processor and radar sensor are activated.

[0194] For example, if a person is in a place where people usually reside, such as a sofa, and uses a remote control to start the air conditioner, the person is still stationary. The radar sensor can detect that there is a signal generated in the area where the person usually resides, but there is no moving object. In this case, the number of stationary people included in the area where the person usually resides should still be increased by 1.

[0195] S606, the processor decrements the number of stationary human position points included in the area of ​​the human's permanent location by 1.

[0196] In some possible embodiments, if the starting point of the human movement trajectory is located within the human's usual location area and the ending point of the human movement trajectory is located outside the human's usual location area, then the surface indicates that a human has left the human's usual location area. In this case, the processor needs to decrement the number of stationary human bodies included in the human's usual location area by 1 to indicate the validity of the human's usual location area.

[0197] S607, the processor determines whether the number of static human position points included in the human body's permanent location area is less than the preset number of points.

[0198] In some possible embodiments, after the radar sensor collects the movement position points of an object over a preset period of time, the processor determines whether there is a region where a person is typically stationary that contains fewer stationary human position points than a preset number. If the number of stationary human position points in a region is less than the preset number, it indicates that the region may no longer be suitable for determining whether the endpoint of a person's movement trajectory is a stationary position due to changes in room layout or changes in human movement habits. The preset number of points can be flexibly set according to the environment, and the preset time can also be flexibly set according to the behavioral habits of people in the room.

[0199] For example, the preset number of points is 5, and the preset time is set to 3 days. That is, after 3 days, if the number of stationary points included in the human body's permanent location area is less than 10, it indicates that the human body's permanent location area can no longer accurately represent the area where the human body frequently stays.

[0200] S608, the processor extracts the start and end points of the collected human motion trajectory.

[0201] In some possible embodiments, if the number of stationary human locations included in the human's permanent location region is less than a preset value, then the human's permanent location region can no longer accurately represent the area where the human frequently stays, and the processor needs to update the human's permanent location region. The processor extracts the start and end points of the human movement trajectory collected within a preset time period for subsequent clustering and updating of the human's permanent location region. For the start point of the human movement trajectory, this position is the first point where the human enters the radar sensor detection area or the point where the human's state changes from stationary to moving; for the end point of the human movement trajectory, this position is the last point where the human leaves the radar sensor detection area or the point where the human's state changes from moving to stationary. Therefore, the start and end points of the human movement trajectory likely include the human's stationary state.

[0202] S609, the processor clusters the start and end points to obtain new human habitation location regions.

[0203] In some possible embodiments, the processor extracts the start and end points of human motion trajectories and performs clustering. The clustered regions represent areas where the human body is frequently stationary, which are the human body's stationary position regions. When the end point of a human motion trajectory is located within this human body stationary position region, the end point of the human motion trajectory can be considered as the human body's stationary position point, provided that the error is acceptable.

[0204] The above embodiments describe a method for identifying a stationary state of a human body based on the human body's usual location area. By constructing the human body's usual location area to identify the stationary state of the human body, the processor's throughput is reduced, and the speed of millimeter-wave radar in identifying the stationary state of the human body is improved.

[0205] The following section will introduce how to use K-means clustering to construct the human body's resident location regions, based on the above embodiments.

[0206] like Figure 7 The diagram shown is a schematic flowchart 700 for constructing human resident location regions using clustering, as provided in an embodiment of this application. The flowchart uses the K-means clustering algorithm as an example. The flowchart includes:

[0207] S701, initialize K centroids.

[0208] In some possible implementations, K data points (start and end points) are randomly selected as initial centroids. The choice of the value of K can be made using the elbow method or the profile coefficient method.

[0209] Elbow method: Calculate the total sum of squared distances (i.e., the sum of squared errors within a cluster) under different K values, and select the K value where the error change trend begins to slow down.

[0210] Silhouette coefficient: Evaluate the clustering effect under different K values ​​and select the K value that maximizes the silhouette coefficient.

[0211] S702, calculate the distance from each start point and end point to each centroid, and assign it to the cluster to which the nearest centroid belongs.

[0212] For example, if the distances of data point C to the three centroids a, b, and c are 1, 2, and 3 respectively, then data point C belongs to centroid c.

[0213] S703, update the centroid of each cluster to the average value of all points within that cluster.

[0214] In some possible embodiments, after classifying each data point into its own cluster, the average coordinates of the data points in each cluster are calculated to obtain a new centroid.

[0215] S704 determines whether the centroid of each cluster has converged.

[0216] In some possible implementations, convergence is achieved if the updated centroid is in the same position as the original centroid. If the centroid does not converge, it is recalculated.

[0217] S705, based on the distribution of points within the cluster, the region where the human body is permanently located is obtained.

[0218] In some possible embodiments, the boundaries of the region are represented by polygons, circles or other geometric shapes, depending on the distribution of points within the cluster, to obtain the human body's permanent location region.

[0219] The above embodiments illustrate how to construct a human's permanent location region based on the start and end points of the human movement trajectory. By using the K-means clustering algorithm to process the start and end point data, rather than the complete trajectory path, computational costs are reduced and the efficiency of data analysis is improved.

[0220] Next, we will use a room setting to explain in detail how to identify a person in a static state based on their usual location.

[0221] like Figure 8 The diagram shown is a schematic diagram of human static state recognition in a room scene provided in an embodiment of this application.

[0222] In some possible embodiments, such as Figure 8As shown in Figure a, the processor receives the object signal detected by the radar sensor and constructs four trajectories: A, B, C, and D, with corresponding trajectory lengths of 7, 5, 2, and 4, respectively. The trajectory length represents the number of position points included in the trajectory. The processor determines that trajectories with lengths between a first length threshold and a second length threshold are human motion trajectories. For example, if the first length threshold is set to 3 and the second length threshold is set to 10, the processor determines trajectories A, B, and D as human motion trajectories.

[0223] In some possible embodiments, such as Figure 8 As shown in Figure a, the historical permanent location region a is obtained by clustering the start and end points of historical human movement trajectories. The processor determines that the end point of human movement trajectory B is outside permanent location region a, meaning trajectory B indicates the human entered permanent location region a, and the number of location points in permanent location region a is incremented by 1. The processor determines that the start point of human movement trajectory D is within permanent location region a and the end point is outside permanent location region a, meaning trajectory D indicates the human left permanent location region a, and the number of location points in permanent location region a is decremented by 1. After a period of time, if the processor detects that the number of location points in permanent location region a is 0, and the preset number of points is 5, then the processor needs to update permanent location region a.

[0224] In some possible embodiments, such as Figure 8 As shown in b, the processor extracts the starting and ending points of the human motion trajectories A, B, and D collected during the aforementioned time period.

[0225] In some possible embodiments, such as Figure 8 As shown in c, the processor clusters the starting and ending points of the human motion trajectories A, B, and D extracted in the above embodiment to obtain the updated permanent location regions b and c. The clustering method has been introduced in previous embodiments and will not be repeated here.

[0226] The above examples provide a clearer understanding of the update process for the resident location area. Next, we will introduce a method for recognizing the stationary state of the human body using a neural network model.

[0227] like Figure 9 The diagram shown is a schematic flowchart 900 for recognizing the static state of a human body based on a neural network model, according to an embodiment of this application. The flowchart includes:

[0228] S901, a millimeter-wave radar, acquires the position of moving objects.

[0229] In some possible embodiments, a millimeter-wave radar sensor periodically transmits electromagnetic wave signals into a room. Based on the electromagnetic wave signals reflected from objects within the room, the millimeter-wave radar sensor can obtain the distance, top angle, and azimuth of the target object. The position of the target object relative to the radar sensor can then be calculated based on the distance, top angle, and azimuth.

[0230] For example, if the radar sensor measures the azimuth angle of the robot vacuum cleaner in the room as α, the downward angle as β, and the distance as l, then the coordinates of the robot vacuum cleaner relative to the radar sensor are (x, y) = (lcosαcosβ, lsinαcosβ).

[0231] The S902 millimeter-wave radar constructs trajectories based on the moving position of objects.

[0232] In some possible embodiments, millimeter-wave radar uses a global nearest neighbor algorithm for trajectory construction. For each moment, the radar sensor detects the object's motion position, and the millimeter-wave radar matches this position with an existing trajectory or establishes a new trajectory. Regarding trajectory construction methods and... Figure 4 The methods involved are similar and will not be repeated here.

[0233] S903, the processor inputs the motion trajectory of the target object into a pre-trained neural network model.

[0234] In some possible embodiments, the neural network model is a recurrent neural network model, because it is able to effectively process time series data and extract temporal features from it.

[0235] In some possible embodiments, the neural network model is trained using labeled motion trajectories, the labels indicating whether the trajectory is a human body trajectory and whether the endpoint of the motion trajectory is a stationary state.

[0236] For example, the label of motion trajectory A is (0, 1), indicating that trajectory A is a non-human motion trajectory and the trajectory endpoint is a stationary position; the label of motion trajectory B is (1, 0), indicating that trajectory B is a human motion trajectory and the trajectory endpoint is not a stationary position.

[0237] In some possible implementations, when the number of sample trajectories used for training is large, the trained neural network model is able to classify motion trajectories.

[0238] S904, the processor outputs the confidence level of the human trajectory and the confidence level of the endpoint stationary state.

[0239] In some possible embodiments, when the confidence value of the human body trajectory is greater than a third preset value and the confidence value of the endpoint being stationary is greater than a fourth threshold, the processor determines that the endpoint of the target object's motion trajectory is the human body's stationary position.

[0240] For example, if the third threshold = the fourth threshold = 0.8, and for a motion trajectory, the confidence level of the human trajectory output by the neural network model is 0.9, then the processor determines that the motion trajectory is a human motion trajectory. If the confidence level of the endpoint of the motion trajectory being stationary output by the neural network model is 0.81, then the processor determines that the endpoint of the motion trajectory is a stationary position.

[0241] The above embodiments use a neural network model to identify the static state of a human body. This model can automatically learn complex patterns and features in motion trajectories, especially nonlinear and complex temporal characteristics. Compared to traditional rule-matching methods, neural networks can more accurately identify minute changes in the human body's trajectory, thus more accurately determining the static state.

[0242] The recurrent neural network models involved in the following embodiments will be described in detail.

[0243] like Figure 10 The diagram shown is a schematic diagram 1000 of a recurrent neural network model provided in an embodiment of this application.

[0244] Recurrent Neural Networks (RNNs) are a class of neural network models used to process sequential data. Unlike traditional feedforward neural networks, RNNs introduce a recurrent structure into time series data, allowing them to store past information and apply it to current computations. This makes RNNs particularly suitable for tasks requiring contextual information, such as time series analysis, language modeling, and speech recognition. Through recurrent connections in hidden layers, RNNs store previous input information in hidden states and combine it with the current input to compute the output. Theoretically, it can capture dependencies of arbitrary length within a sequence. Since RNNs use the same parameters at each time step, they are well-suited for handling variable-length sequential data.

[0245] The basic structure of an RNN is as follows: The input sequence is set to x1, x2, x3, ..., x... T That is, a sequence of length T, where at each time step t, the RNN receives the current input x. t and the previous hidden state h t-1 Calculate the current hidden state h t and output o t :

[0246] h t =f(W hh h t-1 +W xh x t +b h (4)

[0247] o t =g(W ho h t +b o (5)

[0248] Among them W hh W xh W ho Let b be the weight matrix. h b o As a bias, f and g are non-linear activation functions.

[0249] W hh : The weight matrix from the input layer to the hidden layer. It represents the weights of the current input x. t This is converted into a representation in the hidden state. The size of this matrix depends on the dimension of the input vector and the size of the hidden layer.

[0250] W xh The weight matrix from hidden layer to hidden layer. It connects the hidden state h from the previous time step. t-1 and the hidden state h at the current moment t This ensures that past information can be passed through time steps. This is also the key to RNNs capturing time-series dependencies.

[0251] W ho : The weight matrix from the hidden layer to the output layer. It represents the weights of the hidden state h. t Converted to network output o t This maps the representations in the hidden layer to the final output space.

[0252] b h : The bias term of the hidden layer. It is added to the linear combination of the input and the hidden layer to adjust the activation value of the hidden state. The bias ensures that the neuron can still be activated even if all the inputs are zero.

[0253] b o : The bias term of the output layer. It acts on the linear combination passed from the hidden layer to the output layer, making certain corrections to the output.

[0254] Activation functions introduce non-linearity, allowing the network to approximate more complex functions while ensuring that the input values ​​remain within a certain range to avoid infinite growth or shrinkage of the output. Commonly used activation functions include tanh, ReLU, and sigmoid.

[0255] In some possible embodiments, the training process of a recurrent neural network (RNN) first inputs a series of labeled two-dimensional coordinate sequences X. X contains two labels: a trajectory category label M1 and a stationary state label M2. A stationary state label of 1 indicates that the last position point of the trajectory sequence is a stationary position point, and a stationary state label of 0 indicates that the last position point of the trajectory sequence is a non-stationary position point. A trajectory category label of 1 indicates that the trajectory is a human trajectory, and a trajectory category label of 0 indicates that the trajectory is not a human trajectory. The RNN then extracts the hidden features h of the trajectory sequence. T The system outputs the human trajectory confidence score Y1 and the endpoint stationary confidence score Y2 through two fully connected layers. The loss function for the human trajectory confidence score Y1 can be a binary cross-entropy loss.

[0256] L1=-(M1log(y1)+(1-M1)log(1-Y1)) (6)

[0257] The loss function for the endpoint rest confidence Y2 can be the binary cross-entropy loss:

[0258] L2=-(M2log(Y2)+(1-M2)log(1-Y2)) (7)

[0259] Weighting the two loss components, we obtain the total loss function:

[0260] L=αL1+βL2 (8)

[0261] α and β are weighting coefficients used to balance the losses of the two parts.

[0262] Next, the gradient is calculated using the backpropagation algorithm, and the model parameters, including the weights and biases of the RNN layers and fully connected layers, are updated using an optimization algorithm (such as Adam or SGD). These steps are repeated until the loss converges or the set number of training epochs is reached.

[0263] In some possible embodiments, for the output process of an RNN, the input two-dimensional coordinates X are given at time t. t = (x, y), the time series is processed by a recurrent layer, and the hidden state h of each step is output. t By selecting the hidden state h of the last time step T As a time-series feature of the motion trajectory, h is input to the human trajectory confidence output layer. T After passing through a fully connected layer, the Sigmoid activation function outputs a human trajectory confidence score, indicating whether the motion trajectory is a human trajectory; the input h is at the endpoint static confidence output layer. T After passing through a fully connected layer, the Sigmoid activation function outputs the endpoint stationary confidence score, indicating whether the endpoint of the motion trajectory is a stationary position.

[0264] The above embodiment uses a recurrent neural network model to extract the temporal and motion features of the motion trajectory. The output of the last hidden layer is used as the input to two fully connected layers, which output the confidence score of the human trajectory and the confidence score of the endpoint being stationary, respectively. By using a neural network model to analyze motion trajectory data, the accuracy of radar sensors in recognizing stationary human states can be improved to some extent.

[0265] To implement the methods described above, the processor requires certain functional modules. The following description of the apparatus embodiments is similar to the description of the method embodiments described above, and has similar beneficial effects. For technical details not disclosed in the apparatus embodiments of this application, please refer to the description of the method embodiments of this application for understanding. Figure 11 The diagram shown is a schematic block diagram of a processor 1100 provided in an embodiment of this application. The processor 1100 may include a feature extraction module 1110 and a recognition module 1120.

[0266] The feature extraction module 1110 is used to extract the trajectory features of the motion trajectory of the object, the trajectory features including temporal features and motion features.

[0267] In some possible embodiments, the feature extraction module 1110 can process temporal data such as motion trajectories, learning the temporal dependencies between trajectory points by progressively reading the trajectory coordinates at each time point. At each time step, the feature extraction module 1110 saves the previous trajectory information through a state vector, and updates the model's state by combining it with the current trajectory points, thereby capturing the temporal dynamics of the trajectory.

[0268] The identification module 1120 is used to identify whether the endpoint of the motion trajectory of the object is the stationary position of the human body based on the temporal features and motion features extracted by the feature extraction module.

[0269] In some possible embodiments, the identification module 1120 outputs the confidence level of the human trajectory and the confidence level of the endpoint stationary position of the object's motion trajectory based on the temporal features and motion features extracted by the feature extraction module. When the confidence level of the human trajectory is greater than a third preset value and the confidence level of the endpoint stationary position is greater than a fourth threshold, the identification module 1120 determines that the endpoint of the target object's motion trajectory is the human stationary position.

[0270] The coordinated operation of the aforementioned functional modules enables the processor to recognize a still human body.

[0271] like Figure 12 The diagram shown is a schematic block diagram of an electronic device 1200 provided in an embodiment of this application. The electronic device 1200 may include a processor 1210, a memory 1220, a bus 1230, and a device interface 1240.

[0272] The processor 1210 calls the executable program code stored in the memory 1220 to execute any of the air conditioner control methods disclosed in the embodiments of this application.

[0273] The memory 1220 stores executable program code, which is executed by the processor 1210 to implement any of the air conditioner control methods disclosed in the embodiments of this application.

[0274] Bus 1230 is used to transfer program code stored in memory 1220 to processor 1210 for execution.

[0275] Device interface 1240 is connected to bus 1230 to enable the processor 1100 and memory 1220 to connect with other devices.

[0276] Optionally, the memory 1220 may include read-only memory and random access memory, and provide instructions and data to the processor 1210. A portion of the memory 1220 may also include non-volatile random access memory. For example, the memory 1220 may also store device type information. The processor 1210 can be used to execute instructions stored in the memory, and when the processor executes the instructions, the processor 1210 can perform the various steps and / or processes corresponding to the terminal device in the above method embodiments.

[0277] It should be understood that, in the embodiments of this application, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0278] It should be noted that, Figure 12 The electronic device 1200 shown may also include components not shown, such as a power supply, which will not be described in detail in this embodiment.

[0279] This embodiment discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute any of the air conditioner control methods disclosed in this application embodiment.

[0280] This application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any of the air conditioner control methods disclosed in this application.

[0281] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or as a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor executes the instructions in the memory, combining them with its hardware to complete the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0282] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0283] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0284] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0285] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0286] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0287] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0288] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A state recognition method applied to an electronic device, the electronic device being provided with a radar sensor for acquiring object position information in an environment by detecting an object signal, and a processor for performing state recognition, characterized by, The method comprises: The radar sensor obtains object motion position information; The processor determines whether the end point of the motion trajectory of the object is the stationary position of the target human body according to the motion trajectory of the object, wherein the motion trajectory of the object is obtained by the radar sensor or the processor through the motion position information of the object, and the object includes the target human body.

2. The method of claim 1, wherein, The processor determines whether the end point of the motion trajectory of the object is the stationary position of the target human body according to the motion trajectory of the object, comprising: The processor obtains the motion trajectory of the target human body from the motion trajectory of the object; The processor determines whether the end point of the motion trajectory of the target human body is the stationary position of the target human body according to the motion trajectory of the target human body.

3. The method of claim 2, wherein, The processor determines whether the end point of the motion trajectory of the target human body is the stationary position of the target human body according to the motion trajectory of the target human body, comprising: The processor determines that the end point of the motion trajectory of the target human body is the stationary position of the target human body according to the fact that the end point of the motion trajectory of the target human body is in the resident position area, wherein the resident position area is an area in which the total number of position points in the pre-obtained human motion trajectory exceeds a preset number, and the position points include the start point in the human motion trajectory and the end point in the human motion trajectory.

4. The method of claim 3, wherein, The processor obtains the motion trajectory of the target human body from the motion trajectory of the object, comprising: The processor identifies the motion trajectory of the target human body from the motion trajectory of the object according to the length of the motion trajectory, and the length of the motion trajectory of the target human body is between a first threshold value and a second threshold value.

5. The method of claim 4, wherein, The processor obtains the resident position area by the method comprising: The processor clusters the start points and the end points in the historical human motion trajectory to obtain the human resident position area, and the human resident position area is an area in which the total number of the start points and the end points in the historical human motion trajectory exceeds the preset number.

6. The method of any of claim 3, wherein, The method further comprises: The processor adds 1 to the number of position points included in the resident position area; or When the number of position points in the resident position area is 0, if the radar sensor continuously detects a signal in the resident position area, the processor adds 1 to the number of position points in the resident position area.

7. The method of claim 3, wherein, The method further comprises: When the processor detects that the start point of any human motion trajectory is in the resident position area and the end point of the any human motion trajectory is located outside the resident position area, the processor subtracts 1 from the number of position points in the resident position area.

8. The method according to any of claims 6 or 7, characterized in that, The method further comprises: After a preset time period, if the number of position points in the resident position area is less than a preset number, the processor updates the resident position area according to the motion trajectory of the target human body in the preset time period.

9. The method of claim 1, wherein The processor determines whether the end point of the motion trajectory of the object is the stationary position of the target human body according to the motion trajectory of the object, comprising: The processor uses a pre-set neural network model to identify the motion trajectory of the object, and identifies whether the end point of the motion trajectory of the object is a stationary position of a target human body. The neural network model is obtained by training a motion trajectory with a label, and the label is used to indicate whether the trajectory is a human body trajectory and whether the end point of the motion trajectory is a stationary state.

10. The method of claim 9, wherein, The processor uses a pre-set neural network model to identify the motion trajectory of the object, and identifies whether the end point of the motion trajectory of the object is a stationary position of a target human body. The neural network model is obtained by training a motion trajectory with a label, and the label is used to indicate whether the trajectory is a human body trajectory and whether the end point of the motion trajectory is a stationary state. The processor inputs the motion trajectory of the object into the neural network model to obtain an identification result output by the neural network model, and the identification result includes a confidence degree of the trajectory type being a human body trajectory and a confidence degree of the end point being a stationary state. The processor determines whether the end point of the motion trajectory of the object is a stationary position of a target human body according to the confidence degree of the human body trajectory and the confidence degree of the end point being stationary. In a case where the confidence degree of the trajectory type being a human body trajectory is greater than a third threshold value and the confidence degree of the end point being a stationary state is greater than a fourth threshold value, it is determined that the end point of the motion trajectory is a stationary position of a target human body.

11. The method according to claim 9 or 10, characterized in that, The neural network model includes a feature extraction module and an identification module. The feature extraction module is used to extract trajectory features of the motion trajectory of the object, and the trajectory features include time sequence features and motion features. The identification module is used to identify whether the end point of the motion trajectory of the object is a stationary position of a human body according to the time sequence features and the motion features extracted by the feature extraction module.

12. An electronic device, comprising: It includes: A radar sensor is configured to obtain object motion position information. The radar sensor is configured to obtain object motion position information. A processor is configured to perform state recognition. The processor is configured to determine whether the end point of the motion trajectory of the object is a stationary position of a target human body according to the motion trajectory of the object, wherein the motion trajectory of the object is obtained by the radar sensor or the processor through the object motion position information, and the object includes a target human body.

13. An air conditioner characterized by comprising: It includes: An indoor unit; A radar sensor is configured to obtain object motion position information. The radar sensor is configured to obtain object motion position information. A processor is configured to perform state recognition. The processor is configured to determine whether the end point of the motion trajectory of the object is a stationary position of a target human body according to the motion trajectory of the object, wherein the motion trajectory of the object is obtained by the radar sensor or the processor through the object motion position information, and the object includes a target human body.

14. A computer-readable storage medium, characterized in that, A computer program is stored, and the computer program includes instructions for implementing a method performed by a processor as claimed in any one of claims 1 to 11.

15. A computer program product comprising computer program code in said computer program product, characterised in that, When the computer program code runs on a computer, the computer implements a method performed by a processor as claimed in any one of claims 1 to 11.

Images