Kitchen appliance control method, apparatus, system, and storage medium

Abstract

The application relates to a kitchen equipment control method, device, system and storage medium. The method comprises the following steps: using the multipath signals periodically fed back by at least one millimeter wave radar in a kitchen area to extract multipath echo features; identifying a user behavior state according to the multipath echo features corresponding to different time stamps of each millimeter wave radar in a preset sliding window; and outputting a target control instruction matched with the user behavior state to a target kitchen equipment. The above kitchen equipment control process does not need to rely on a camera, a microphone, an infrared sensor or Wi-Fi channel state information (CSI) for human body sensing and behavior identification, avoids the risk of privacy leakage, and the sensing accuracy of the millimeter wave radar is higher than that of the infrared sensor and the Wi-Fi channel state information, thereby solving the problem that the prior art cannot realize high-privacy kitchen equipment linkage control.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and in particular to a method, apparatus, system and storage medium for controlling kitchen equipment. Background Technology

[0002] With the development of smart homes, the kitchen, as a frequently used functional space in the home, is experiencing a growing demand for intelligent features. Existing smart kitchen systems mostly rely on cameras, microphones, infrared sensors, or Wi-Fi Channel Status Information (CSI) for human perception and behavior recognition. However, cameras pose serious privacy risks, microphones are susceptible to environmental noise interference and also involve voice privacy, infrared sensors have short detection ranges and are easily affected by heat sources, and while Wi-Fi CSI has wall-penetrating capabilities, its low resolution makes it difficult to achieve precise behavior recognition.

[0003] Therefore, there is an urgent need for a method that can both fully utilize the physical characteristics of the complex kitchen environment to enhance perception capabilities and achieve highly private kitchen equipment linkage control. Summary of the Invention

[0004] This application provides a kitchen equipment control method, device, system, and storage medium to address the problem of how to fully utilize the physical characteristics of the complex kitchen environment to enhance sensing capabilities while achieving highly private kitchen equipment linkage control.

[0005] In a first aspect, this application provides a method for controlling kitchen equipment, the method comprising: Acquire multipath signals periodically provided by at least one millimeter-wave radar within the kitchen area; Based on the multipath signal, determine the multipath echo characteristics; Based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each millimeter-wave radar, the user behavior status in the kitchen area is determined. Based on the user's behavior status and the current operating status of each kitchen device in the kitchen area, the corresponding target control command is output to the target kitchen device.

[0006] Optionally, determining the multipath echo characteristics based on the multipath signal includes: The multipath signal is preprocessed to obtain the multipath echo signal; The multipath echo signal is subjected to range Doppler transformation to obtain a range Doppler image; Based on the analysis and processing results of the distance Doppler image and the multipath echo signal, at least two of the following are determined for at least one echo path: time delay, echo angle, Doppler frequency shift, amplitude attenuation factor, phase change rate, micro-Doppler modulation period, and signal-to-noise ratio. The multipath echo characteristics include at least two of the following for each echo path: time delay, echo angle, Doppler frequency shift, amplitude attenuation factor, phase change rate, micro-Doppler modulation period, and signal-to-noise ratio.

[0007] Optionally, determining the user behavior status within the kitchen area based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window for each of the millimeter-wave radars includes: Based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each millimeter-wave radar, the user's presence status and user movement trajectory are determined. When the user exists, the user's motion trajectory and the multipath echo features corresponding to different timestamps of each millimeter-wave radar within the preset sliding window are input into the preset neural network model, and the user's behavior state is determined based on the output of the preset neural network model.

[0008] Optionally, the user's presence status and motion trajectory are determined based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each of the millimeter-wave radars, including: Based on the multipath echo characteristics of the millimeter-wave radar corresponding to the target timestamp within a preset sliding window, the time delay, echo angle, and preset kitchen map corresponding to different echo paths are used to infer the position of each reflecting object in the echo path, wherein the target timestamp is any timestamp within the preset sliding window; The user presence status corresponding to each echo path is determined based on the micro-Doppler modulation period and / or phase change rate corresponding to each echo path. The echo path in which the user exists is taken as the target path, and the motion speed corresponding to the human features in the target path is determined according to the Doppler frequency shift and / or phase change rate corresponding to the target path. Based on the position and speed of the human body features in the target path, the millimeter-wave radar determines the target state vector corresponding to the target timestamp within a preset sliding window; The user's motion trajectory is determined based on the target state vector corresponding to different timestamps within a preset sliding window of each millimeter-wave radar.

[0009] Optionally, the user's motion trajectory is determined based on the target state vector corresponding to different timestamps within a preset sliding window for each of the millimeter-wave radars, including: Based on the signal-to-noise ratio and path length of the target path corresponding to different timestamps within a preset sliding window, the weight values ​​of the target state vectors corresponding to different timestamps of each millimeter-wave radar are determined. Based on the weight values ​​of the target state vectors corresponding to different timestamps of each millimeter-wave radar, the effective state vectors corresponding to each timestamp within the preset sliding window are determined. The user's motion trajectory is determined based on the curve fitting results of the effective state vectors corresponding to each timestamp within the preset sliding window.

[0010] Optionally, determining the weight values ​​of the target state vectors of each millimeter-wave radar at different timestamps based on the signal-to-noise ratio and path length of the target path corresponding to different timestamps within a preset sliding window includes: The range exponential decay term is determined based on the path length of the target path corresponding to the target timestamp of the millimeter-wave radar. The path signal score corresponding to the millimeter-wave radar is determined by multiplying the range exponential attenuation term and the signal-to-noise ratio of the target path corresponding to the millimeter-wave radar. The weight value of the millimeter-wave radar in the target state vector corresponding to the target timestamp is determined based on the ratio between the path signal score corresponding to the millimeter-wave radar and the sum of the path signal scores corresponding to all millimeter-wave radars.

[0011] Optionally, based on the weight values ​​of the target state vectors corresponding to different timestamps of each of the millimeter-wave radars, the effective state vectors corresponding to each timestamp within the preset sliding window are determined, including: Within the preset sliding window, each millimeter-wave radar selects the target state vector with the largest weight value from the target state vectors corresponding to the target timestamp as the effective state vector corresponding to the target timestamp; or, Based on the weight values ​​of the target state vectors corresponding to the target timestamps of each millimeter-wave radar, the target state vectors corresponding to the target timestamps of each millimeter-wave radar are weighted and summed to determine the effective state vector corresponding to the target timestamp.

[0012] Secondly, this application provides a kitchen equipment control device, the kitchen equipment control device comprising: A receiving module is used to acquire multipath signals periodically provided by at least one millimeter-wave radar in the kitchen area; The feature extraction module is used to determine the multipath echo features based on the multipath signal; The behavior recognition module is used to determine the user behavior status in the kitchen area based on the multipath echo characteristics corresponding to different timestamps of each millimeter-wave radar within a preset sliding window. The control module is used to output corresponding target control commands to the target kitchen equipment based on the user's behavior status and the current operating status of each kitchen equipment in the kitchen area.

[0013] Thirdly, this application provides a kitchen equipment control system, which includes at least one millimeter-wave radar, multiple kitchen devices, and a kitchen equipment control device as described above, wherein the kitchen equipment control device is communicatively connected to each of the millimeter-wave radars and the kitchen devices.

[0014] Fourthly, this application also provides a computer storage medium storing computer-executable instructions for executing the above-described kitchen equipment control method.

[0015] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application utilizes the multipath signal periodically fed back by at least one millimeter-wave radar in the kitchen area to extract multipath echo features. Based on the multipath echo features corresponding to different timestamps within a preset sliding window of each millimeter-wave radar, the user's behavior state is identified, and then a target control command matching the user's behavior state is output to the target kitchen equipment. The above-mentioned kitchen equipment control process does not rely on cameras, microphones, infrared sensors, or Wi-Fi channel status information (CSI) for human perception and behavior recognition, avoiding the risk of privacy leakage. Moreover, the perception accuracy of millimeter-wave radar is higher than that of infrared sensors and Wi-Fi channel status information, thereby solving the problem that the prior art cannot achieve high-privacy kitchen equipment linkage control. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0019] Figure 1An application environment diagram of a kitchen equipment control method provided in this application embodiment; Figure 2 A schematic flowchart illustrating a kitchen equipment control method provided in an embodiment of this application; Figure 3 A structural block diagram of a kitchen equipment control device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the internal structure of a kitchen equipment control system provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0022] Figure 1 This is a diagram illustrating the application environment of a kitchen equipment control method in one embodiment. (Refer to...) Figure 1 The kitchen equipment control method is applied to the control system of kitchen equipment 120. The control system of kitchen equipment 120 includes at least one millimeter-wave radar 110, multiple kitchen equipment 120, and a kitchen equipment control device 130. The kitchen equipment control device 130 is communicatively connected to each of the millimeter-wave radar 110 and the kitchen equipment 120.

[0023] The kitchen equipment control device 130 can be set up independently of the kitchen equipment 120, such as a control panel, touch panel, or central control device. Alternatively, the kitchen equipment control device 130 can be integrated into the kitchen equipment 120, which can specifically include a smart range hood, smart gas stove, kitchen air conditioner, kitchen curtains, ventilation equipment, smart gas valve, smart faucet, and smart lighting fixtures.

[0024] At least one millimeter-wave radar is installed in the kitchen area. In this embodiment, three millimeter-wave radars are installed in the kitchen area: millimeter-wave radar A, millimeter-wave radar B, and millimeter-wave radar C. Millimeter-wave radar A is located at the center below the wall cabinet, 1.8 meters above the ground, with a main view covering the worktop and sink. Millimeter-wave radar B is located on the side wall next to the stove, 1.2 meters above the ground, with a main view covering the stove area, used to capture cooking actions. Millimeter-wave radar C is located at the bottom of the ceiling above the sink, 2.1 meters above the ground, looking down at the sink area. Each millimeter-wave radar adopts the FMCW (Frequency Modulated Continuous Wave) system, with a transmission frequency range of 24.0–24.25 GHz, a frequency modulation slope of 100 MHz / μs, a sampling rate of not less than 100 kSPS, an ADC resolution of 12 bits, achieving a range resolution of approximately 15 cm and a velocity resolution of approximately 0.2 m / s, possessing high time-range-velocity resolution capability. All radars achieve nanosecond-level time alignment through hardware-level synchronization pulses (such as GPS / PTP protocol). Three metal corner markers (such as stainless steel corner marks) are preset in the kitchen at known locations. Each radar establishes a local coordinate system by detecting their reflected signals and solves the transformation matrix by the least squares method to achieve a globally unified coordinate system.

[0025] In one embodiment, Figure 2 This is a flowchart illustrating a kitchen equipment control method in one embodiment, with reference to... Figure 2 A method for controlling kitchen equipment is provided. This embodiment mainly applies this method to the above-mentioned... Figure 1 Taking the kitchen equipment control device 130 as an example, the specific steps of this kitchen equipment control method include the following: Step S210: Acquire multipath signals periodically provided by at least one millimeter-wave radar within the kitchen area.

[0026] Specifically, the millimeter-wave radars in the kitchen area periodically feed back the multipath signals they receive. Multipath signals refer to the collection of all signals that, after being emitted from the transmitter of the millimeter-wave radar, propagate through multiple different paths and ultimately reach the receiver. In other words, multipath signals include direct wave signals, first-reflection wave signals, second-reflection wave signals, and edge-diffracted wave signals. Because the different millimeter-wave radars are located in different positions, meaning different emitted signals are reflected back to different receivers via different paths, the multipath signals provided by each millimeter-wave radar are also different.

[0027] Step S220: Determine the multipath echo characteristics based on the multipath signal.

[0028] Specifically, since multipath signals contain reflected signals corresponding to multiple echo paths, feature analysis and extraction based on multipath signals can yield multipath echo features. Multipath echo features include signal features corresponding to at least one echo path. The echo path can be a direct wave path, a primary reflection wave path, a secondary reflection wave path, etc., and the signal features can be time delay, echo angle, path length, etc.

[0029] Based on the multipath echo characteristics combined with a pre-set kitchen map of the kitchen area, the path of the echo within the kitchen area can be deduced.

[0030] Step S230: Determine the user behavior status in the kitchen area based on the multipath echo characteristics corresponding to different timestamps of each millimeter-wave radar within a preset sliding window.

[0031] Specifically, since millimeter-wave radar provides multipath signals at regular intervals, and the kitchen equipment control device analyzes and processes the multipath signals provided by each millimeter-wave radar at regular intervals, the corresponding multipath echo features can be obtained. These multipath echo features are timestamped and correlated with the millimeter-wave radar signals. By sliding a preset sliding window to select the multipath echo features corresponding to each millimeter-wave radar at multiple timestamps, a time feature sequence is selected. This time feature sequence includes the multipath echo features corresponding to each millimeter-wave radar at multiple timestamps. For each timestamp within the preset sliding window, there are multiple corresponding multipath echo features, each originating from a multipath signal provided by a millimeter-wave radar.

[0032] The preset sliding window length n is the selected time period. That is, according to the preset sliding window, the multipath echo features of each millimeter-wave radar at different timestamps within the most recent time period are selected. The preset sliding window length can be customized to meet the user identification sensitivity in combination with the actual application scenario.

[0033] For example, the time feature sequences selected according to the preset sliding window are shown in Table 1 below:

[0034] Table 1 Based on the multipath echo characteristics of each millimeter-wave radar at each time point in the time feature sequence, the multipath signal of each millimeter-wave radar at each time point can be deduced to pass through the kitchen area. This allows for the analysis of user behavior status within the kitchen area. User behavior status represents the user's behavior posture within the kitchen area, and can include actions such as cooking, washing vegetables, chopping vegetables, lifting the pot lid, washing hands, leaving the kitchen, and entering the kitchen.

[0035] Step S240: Based on the user's behavior status and the current operating status of each kitchen device in the kitchen area, output the corresponding target control command to the target kitchen device.

[0036] Specifically, based on the user's behavior state and the current operating status of each kitchen device within the kitchen area, the corresponding target control command and target kitchen device are determined, and the target control command is output to the target kitchen device. This enables automatic linkage control of each kitchen device according to the user's behavior within the kitchen area. The target kitchen device includes at least one kitchen device. That is, the user's behavior state and the current operating status of each kitchen device within the kitchen area are used to indicate the triggering conditions. There is a mapping relationship between the triggering conditions, the target control command, and the target kitchen device, as shown in Table 2.

[0037] Table 2 The aforementioned kitchen equipment control process does not rely on cameras, microphones, infrared sensors, or Wi-Fi channel status information (CSI) for human perception and behavior recognition, thus avoiding the risk of privacy leakage. Millimeter-wave radar, due to its high resolution, strong penetration, low power consumption, and non-contact perception capabilities, has a higher perception accuracy than infrared sensors and Wi-Fi channel status information. This can solve the problem of how to fully utilize the physical characteristics of the complex kitchen environment to improve perception capabilities while achieving highly private kitchen equipment linkage control.

[0038] In one embodiment, determining the multipath echo characteristics based on the multipath signal includes: The multipath signal is preprocessed to obtain the multipath echo signal; The multipath echo signal is subjected to range Doppler transformation to obtain a range Doppler image; Based on the analysis and processing results of the distance Doppler image and the multipath echo signal, at least two of the following are determined for at least one echo path: time delay, echo angle, Doppler frequency shift, amplitude attenuation factor, phase change rate, micro-Doppler modulation period, and signal-to-noise ratio. The multipath echo characteristics include at least two of the following for each echo path: time delay, echo angle, Doppler frequency shift, amplitude attenuation factor, phase change rate, micro-Doppler modulation period, and signal-to-noise ratio.

[0039] Specifically, the multipath signal undergoes preprocessing, including mixing, digital down-conversion, windowing, zero-fill interpolation, and clutter suppression. Mixing combines the multipath signal with the transmitted signal to obtain the intermediate frequency (IF) signal. Digital down-conversion down-converts the IF signal to baseband. Windowing uses a Hanning window to reduce spectral leakage. Zero-fill interpolation improves the range resolution to 5 cm. Clutter suppression removes fixed background clutter using an adaptive filter. After this preprocessing, the multipath signal is converted into a multipath echo signal.

[0040] The multipath echo signal is processed by range-Doppler transformation to obtain the range-Doppler map (RD map). Based on the analysis results of the range-Doppler map, the signal characteristics corresponding to each echo path in the multipath echo signal can be determined. The multipath echo characteristics include the signal characteristics corresponding to each echo path, which can be time delay, echo angle, Doppler frequency shift, amplitude attenuation factor, phase change rate, micro-Doppler modulation period, or signal-to-noise ratio, etc.

[0041] The process of determining the time delay is as follows: find the peak point with the strongest energy in the distance Doppler map. Distance index Convert to physical distance d, and then calculate the round-trip time delay based on the speed of light c. In the case of multiple echo paths in a multipath signal, there are multiple peak points in the distance Doppler image. That is, each peak point corresponds to the time delay of an independent echo path. The time delay directly corresponds to the path length of the signal propagation and is used to distinguish the spatial location of different reflection sources (such as sink vs. stove).

[0042] The process of determining the echo angle (i.e., the angle of arrival) is as follows: A covariance matrix R is constructed using the intermediate frequency signals received by multiple receiving antennas. ,in For intermediate frequency signals; perform eigenvalue decomposition on the covariance matrix to separate M eigenvalues ​​and their corresponding eigenvectors, the M eigenvalues ​​being as follows ( M represents the number of receiving antennas, and the characteristic value is greater than the noise power ( The eigenvectors of the plane wave are partitioned into the signal subspace, resulting in an eigenvector with K eigenvalues, where K is the number of signal sources, i.e., the number of echo paths. The signal subspace contains information about all multipath reflected signals and is conjugate with the steering vector (array manifold vector) of the receiving array. The steering vector describes the plane wave at an angle. When incident on the receiving antenna array, the phase difference sequence received by each array element; eigenvectors with eigenvalues ​​less than or equal to the noise power are partitioned into the noise subspace. Noise subspace include The eigenvectors of each eigenvalue, the noise subspace is orthogonal to the signal subspace, contains no target signal information, and only includes environmental noise; based on the noise subspace and guide vector Constructing the spatial spectral function , make the angle exist Calculate the spectral value at each angle for continuous variation within a range (e.g., step size of 0.1°). When the assumed angle Corresponding guide vector When the steering vector aligns with the true direction of signal arrival, it will lie within the signal subspace, thus being orthogonal to the noise subspace (inner product = 0); conversely, if the angles do not match, non-zero components will exist; plot the spectral values. The curve changes with angle, and the angle corresponding to the sharp peak on the curve is taken as the echo angle (AoA) of the multipath signal. If K obvious peaks are detected, it means that there are K echo paths, each corresponding to K different echo angles.

[0043] The process of determining the Doppler frequency shift is as follows: find the frequency index corresponding to the energy peak in the distance-Doppler map. And convert the frequency index into the corresponding frequency value. This frequency value is used as the Doppler frequency shift. It is used to reflect the radial velocity of the target in order to distinguish stationary objects (multipath background) from moving human bodies.

[0044] The process of determining the amplitude attenuation factor is as follows: find the peak point with the strongest energy in the distance Doppler plot. And obtain the complex amplitude magnitude of the echo path corresponding to the peak point. Subtract radar transmit power, antenna gain, and free-space path loss (with known time delay) from the complex amplitude magnitude. The compensated modulus can be obtained by calculating the influence of receiver noise floor (which can be calculated) and the compensated modulus. The ratio of the compensated modulus to the reference modulus is used as the amplitude attenuation factor. , ,in Using the reference modulus, the magnitude of the amplitude attenuation factor mainly depends on the material of the reflective surface. For example, a metal sink / stove reflects a stronger signal and has a larger amplitude attenuation factor, while a wooden cabinet reflects a weaker signal and has a smaller amplitude attenuation factor. Therefore, the amplitude attenuation factor is a key parameter for distinguishing the object type of the reflective object.

[0045] The process of determining the phase change rate is as follows: extract the instantaneous phase from the multipath echo signal. Because of the existence of phase If the phase is blurred, phase unwrapping is required first; this involves the phase of the time series. Perform difference or numerical differentiation operations to calculate its rate of change over time. Then the phase change rate is obtained. The phase change rate is used to map to the equivalent micro-motion velocity or displacement amplitude. Extremely high phase change rates usually correspond to rapid micro-movements of the human body, such as waving hands or breathing fluctuations, while the phase change rate of stationary objects is close to zero, that is, it is only affected by environmental noise.

[0046] The process of determining the micro-Doppler period is as follows: A short-time Fourier transform is performed on the multipath echo signal to generate a high-resolution time-frequency plot; the time-frequency plot is projected along the time axis, or the amplitude sequence at a specific frequency is extracted, and a fast Fourier transform (FFT) is performed to find the peak representing the action frequency. The repetition period of this peak is then identified, thus obtaining the micro-Doppler period. For example, the micro-Doppler cycle for the action of chopping vegetables is 0.5s to 1s. The micro-Doppler cycle is a key feature for identifying user behavior and is used to clearly distinguish specific kitchen actions such as standing still, walking slowly, chopping vegetables, and stirring.

[0047] In one embodiment, determining the user behavior status within the kitchen area based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window for each of the millimeter-wave radars includes: Based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each millimeter-wave radar, the user's presence status and user movement trajectory are determined. When the user exists, the user's motion trajectory and the multipath echo features corresponding to different timestamps of each millimeter-wave radar within the preset sliding window are input into the preset neural network model, and the user's behavior state is determined based on the output of the preset neural network model.

[0048] Specifically, by utilizing the multipath echo characteristics corresponding to each millimeter-wave radar under continuous timestamps within a preset sliding window, the presence status and movement trajectory of users within the kitchen area during the corresponding time period of the preset sliding window are analyzed. The user presence status is either present or absent. If the user presence status is absent, the movement trajectory is empty, indicating that there is no one in the kitchen area during the corresponding time period of the preset sliding window, and no one is moving. If the user presence status is present, it indicates that there is someone in the kitchen area during the corresponding time period of the preset sliding window, and the corresponding user movement trajectory is either a static movement trajectory or a dynamic movement trajectory. A static movement trajectory indicates that the user's position has not moved during the corresponding time period of the preset sliding window, such as standing while washing vegetables or standing while cooking, but the user's body characteristics show subtle movements, such as hand or head movements. A dynamic movement trajectory indicates that the user's position has moved, such as the user walking.

[0049] When the user is present, the user's movement trajectory and the multipath echo features corresponding to different timestamps of each millimeter-wave radar within the preset sliding window are input into the preset neural network model. The preset neural network model uses the user's movement trajectory and the multipath echo features corresponding to different millimeter-wave radars at continuous timestamps to accurately identify the user's behavior status in the kitchen area. That is, the output of the preset neural network model is used as the user's behavior status.

[0050] The pre-defined neural network model is trained based on a real kitchen behavior dataset, which includes echo features and user movement trajectories corresponding to different kitchen behaviors, such as chopping vegetables, washing dishes, washing vegetables, opening cabinet doors, cooking, and lifting pot lids. The pre-defined neural network model performs user behavior recognition on the input multipath echo features and user movement trajectories, and then outputs corresponding behavior labels, which represent the user behavior state.

[0051] For example, a preset neural network model, based on the latest coordinates and amplitude attenuation factor of the user's movement trajectory, can identify whether the user's current location is a wooden cabinet area or a metal sink area, and then combine the micro-Doppler modulation period and phase change rate to identify the user's micro-movements.

[0052] The pre-defined neural network model is also used to analyze the signal abrupt changes of the same millimeter-wave radar (such as a target radar) at two adjacent timestamps based on the multipath echo characteristics. These signal abrupt changes specifically include path abrupt changes, micro-Doppler modulation period abrupt changes, phase abrupt changes, and time delay abrupt changes. The user's behavioral state is identified based on these signal abrupt changes and the multipath echo characteristics corresponding to the current timestamp. For example, when the hand is away from the cookware, the target radar mainly receives the reflected signal from the "hand-wall / ceiling," resulting in a relatively long multipath path and stable phase changes. When the hand touches the cookware, due to the formation of a new coupling between the hand and the cookware, or the hand's close contact with the cookware causing a change in the reflective surface, the strong multipath signal originally pointing to the wall may suddenly weaken or disappear, replaced by a new multipath component pointing to the cookware surface. If a high-frequency micro-vibration micro-Doppler modulation period abrupt change or a drastic phase change is detected, and the amplitude attenuation factor matches human characteristics, then the user's behavioral state is determined to be that their hand is touching and operating the cookware.

[0053] For example, when the pot lid is closed, the target radar mainly receives the reflection from the "top of the pot lid," resulting in a fixed multipath path and extremely low micro-Doppler modulation period and phase change rate (stationary). When a hand grasps the handle of the pot lid, causing a vertical upward displacement, the target radar detects a step change in the time delay τ of this multipath path (corresponding to a sudden change in time delay). Simultaneously, the phase change rate... If a significant upward pulse appears (corresponding to a phase change), and the change in height corresponding to the change in time delay τ is greater than the change threshold (e.g., 5cm), and the duration T matches the "lifting lid" action characteristics (i.e., T<1s), the user behavior state is immediately determined to be "opening the pot lid". Once it is determined that "opening the pot lid" and "someone is present", the range hood acceleration command (corresponding to the target control command) is immediately triggered.

[0054] The multi-radar trajectory information recognition method can make full use of the spatial location and micro-motion details contained in the multipath echo. The multipath echo of multiple radars can fill the signal blind spot caused by the user's body blocking the single radar, and avoid incomplete extraction of behavioral features due to the lack of local signals. After combining the user's motion trajectory constraints, the neural network can more accurately distinguish different kitchen behaviors with similar action features. For example, it can clearly distinguish between the two behaviors of chopping vegetables and kneading dough while standing still, which can improve the recognition accuracy of kitchen behaviors.

[0055] In one embodiment, determining the user's presence status and motion trajectory based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window for each of the millimeter-wave radars includes: Based on the multipath echo characteristics of the millimeter-wave radar corresponding to the target timestamp within a preset sliding window, the time delay, echo angle, and preset kitchen map corresponding to different echo paths are used to infer the position of each reflecting object in the echo path, wherein the target timestamp is any timestamp within the preset sliding window; The user presence status corresponding to each echo path is determined based on the micro-Doppler modulation period and / or phase change rate corresponding to each echo path. The echo path in which the user exists is taken as the target path, and the motion speed corresponding to the human features in the target path is determined according to the Doppler frequency shift and / or phase change rate corresponding to the target path. Based on the position and speed of the human body features in the target path, the millimeter-wave radar determines the target state vector corresponding to the target timestamp within a preset sliding window; The user's motion trajectory is determined based on the target state vector corresponding to different timestamps within a preset sliding window of each millimeter-wave radar.

[0056] Specifically, for a single millimeter-wave radar at a certain timestamp within a preset sliding window, the multipath echo characteristics include signal characteristics corresponding to multiple echo paths. The signal characteristics corresponding to each echo path are mapped onto a preset kitchen map. For example, the path length L and echo angle corresponding to the time delay of the echo path are mapped. Mapped onto a preset kitchen map, the positions of each reflecting object in the echo path can be inferred using a triangulation / reflection model.

[0057] For example, the incident direction of the reflected wave can be determined according to the echo angle. A straight ray is emitted starting from the receiving end O of the millimeter-wave radar according to the echo angle, and the intersection point of the straight ray with the preset kitchen map is taken as the first reflection point P3. The object to which the first reflection point belongs is taken as the first candidate reflection object, and the backtracking distance d1=∣OP3∣ is calculated. If 2*d1<L, it indicates that the optical path backtracking is not completed and multiple reflections exist, so the deduction continues; if 2*d1=L, it indicates that there is only a single reflection, and P3 is the only reflection point. Taking 2*d1<L as an example, the reverse reasoning continues: the normal direction of the contact surface is obtained at the intersection point P3, and the propagation direction of the signal before contacting point P3 is deduced according to the reflection geometric rule that the incident angle equals the reflection angle. A straight ray is emitted continuously according to the propagation direction, the intersection point of the straight ray with the preset kitchen map is taken as the second reflection point P2, and the backtracking distances d2=∣P3P2∣ and d3=∣P2O∣ are calculated, where d3 represents the distance from the second reflection point P2 to the transmitting end O. The total backtracking distance D=d1+d2+d3 is calculated and compared with the path length again. If D<L, the backtracking for other reflection points continues by following the method of determining the reflection direction, obtaining intersection points and finding reflection points, so as to find the third reflection point P1 accordingly. The backtracking distances d4=∣P2P1∣ and d5=|P1O| are calculated, where d5 represents the distance from the third reflection point P1 to the transmitting end O. The total backtracking distance D=d1+d2+d4+d5 is calculated. If D=L, it indicates that the optical path backtracking is completed, thereby finding each reflection point P1, P2, P3 on the echo path and the reflection objects to which each reflection point belongs in sequence along the path, so as to determine that the echo path is transmitting end O—P1 (e.g., pot lid)—P2 (e.g., wall surface)—P3 (e.g., human hand)—receiving end O.

[0058] The phase change rate is used to reflect the micron-level displacement velocity, and the micro-Doppler modulation period is used to reflect the periodic action frequency. If the phase change rate is close to 0 and / or the micro-Doppler modulation period is less than the period threshold, it indicates that the reflection objects in the echo path are stationary objects; if the phase change rate is greater than the change rate threshold and / or the micro-Doppler modulation period is greater than the period threshold, it indicates that there is a moving target such as a human hand among the reflection objects in the echo path. In this way, whether each reflection object in the echo path is a human body feature can be determined, thereby determining the user presence state corresponding to each echo path. The user presence state corresponding to the echo path having a human body feature as a reflection object is presence, that is, the presence of a user is detected, and the echo path with the user presence state of presence is taken as the target path.

[0059] According to the Doppler frequency shift and / or the phase change rate corresponding to the target path, the movement velocities of the human body feature in different directions are decomposed and calculated. The movement velocities include the movement velocities in three XYZ axial directions, so as to obtain the velocity components ( , , Since the locations of the reflecting objects in each echo path are known, it can be determined that the locations of the reflecting objects representing human features in the target path are those of human features. Based on the position and velocity components of human features in the target path, a corresponding target state vector is constructed. The target state vector represents the user's running state in the kitchen space at a certain timestamp. There is a correspondence between the target state vector and the millimeter-wave radar, the target timestamp, and the target path. That is, under each timestamp in the preset sliding window, each target path corresponding to each millimeter-wave radar corresponds to a target state vector used to represent the user's movement state.

[0060] Based on the target state vector corresponding to the target path of each millimeter-wave radar under the continuous timestamps within the preset sliding window, users in the kitchen area can be tracked, that is, multi-radar collaborative tracking can be achieved.

[0061] In one embodiment, determining the user's motion trajectory based on the target state vector corresponding to different timestamps within a preset sliding window for each of the millimeter-wave radars includes: Based on the signal-to-noise ratio and path length of the target path corresponding to different timestamps within a preset sliding window, the weight values ​​of the target state vectors corresponding to different timestamps of each millimeter-wave radar are determined. Based on the weight values ​​of the target state vectors corresponding to different timestamps of each millimeter-wave radar, the effective state vectors corresponding to each timestamp within the preset sliding window are determined. The user's motion trajectory is determined based on the curve fitting results of the effective state vectors corresponding to each timestamp within the preset sliding window.

[0062] Specifically, the signal-to-noise ratio (SNR) represents the ratio between the target signal power and the background noise power in the echo path, while the path length reflects the degree of signal transmission loss. A longer path length indicates greater signal transmission loss, and vice versa. Therefore, combining the SNR and path length to determine the weight values ​​of the target state vectors corresponding to each millimeter-wave radar at different timestamps is essentially determining the weight values ​​of the target state vectors corresponding to each millimeter-wave radar. However, this requires determining the weight values ​​of the target state vectors for each timestamp within a preset sliding window. The weight values ​​indicate the reliability of the target state vectors; that is, the lower the weight value, the lower the reliability of the target state vectors.

[0063] Based on the weight values ​​of each target state vector, the effective state vector is determined from multiple target state vectors corresponding to each time stamp within a preset sliding window. The effective state vector corresponding to each time stamp is then subjected to curve fitting according to a time series to obtain the user's motion trajectory. Even if a millimeter-wave radar can only detect weak signals of human features, the system can still track the user's motion trajectory by combining the sensing results (target state vectors) from other millimeter-wave radars. For example, even if a person is standing behind a stove, the system can still draw their complete movement path from the "sink" to "behind the stove."

[0064] This technical solution, which uses multiple radar weights to filter effective state vectors and then fits them to generate trajectories, can effectively filter out low-reliability abnormal state points caused by obstruction and signal attenuation compared to single-radar trajectory generation solutions. This reduces the impact of local signal interference on the overall trajectory accuracy. At the same time, the time segmentation processing method with a preset sliding window can dynamically update the effective state vectors as the user moves, promptly capturing sudden changes in the user's movement direction and speed. This avoids the loss of trajectory details caused by overly smooth fitting under a large time window, and ensures the accuracy of trajectory reconstruction even when the user makes rapid turns, starts, stops, or other changes in speed and direction.

[0065] In one embodiment, determining the weight value of the target state vector corresponding to different timestamps for each millimeter-wave radar based on the signal-to-noise ratio and path length of the target path corresponding to different timestamps within a preset sliding window includes: The range exponential decay term is determined based on the path length of the target path corresponding to the target timestamp of the millimeter-wave radar. The path signal score corresponding to the millimeter-wave radar is determined by multiplying the range exponential attenuation term and the signal-to-noise ratio of the target path corresponding to the millimeter-wave radar. The weight value of the millimeter-wave radar in the target state vector corresponding to the target timestamp is determined based on the ratio between the path signal score corresponding to the millimeter-wave radar and the sum of the path signal scores corresponding to all millimeter-wave radars.

[0066] Specifically, the distance exponential decay term is ,in Let be the path length of the target path corresponding to the i-th millimeter-wave radar, and λ be the attenuation coefficient. The path signal score is... ,in The signal-to-noise ratio of the target path corresponding to the i-th millimeter-wave radar is given by the formula for calculating the weight value of the target state vector corresponding to the target timestamp: ; in, This represents the sum of path signal scores for j millimeter-wave radars at the same timestamp. Based on the above ratio, the weight value of the target state vector of each millimeter-wave radar at a certain timestamp is determined.

[0067] For example, when a person is behind a stove, millimeter-wave radar A only receives a weak direct wave signal with a signal-to-noise ratio (SNR) of <-15dB; millimeter-wave radar B captures a clear micro-Doppler signal through the reflection path of the stove with a SNR of >-5dB. Based on the above formula, the weight value of the target state vector corresponding to the millimeter-wave radar is automatically increased to 0.7, while the weight value of the target state vector corresponding to millimeter-wave radar A is reduced to 0.3. Even if millimeter-wave radar A loses the target user, it can still accurately determine the target state vector corresponding to the user and the user's movement trajectory using the high-weight data of millimeter-wave radar B.

[0068] In one embodiment, determining the effective state vector corresponding to each time point within the preset sliding window based on the weight values ​​of the target state vectors corresponding to different timestamps of each millimeter-wave radar includes: Within the preset sliding window, each millimeter-wave radar selects the target state vector with the largest weight value from the target state vectors corresponding to the target timestamp as the effective state vector corresponding to the target timestamp; or, Based on the weight values ​​of the target state vectors corresponding to the target timestamps of each millimeter-wave radar, the target state vectors corresponding to the target timestamps of each millimeter-wave radar are weighted and summed to determine the effective state vector corresponding to the target timestamp.

[0069] Specifically, the target state vector with the largest weight value is selected as the effective state vector at the same timestamp to ensure the accuracy of user motion state recognition.

[0070] Alternatively, by weighting the target state vectors according to their respective weight values ​​at the same timestamp, the sensing results from various millimeter-wave radars can be fused. Target state vectors with lower weight values ​​contribute less to the effective state vector, while those with higher weight values ​​contribute more. The fused state vector is the effective state vector at that timestamp. This approach effectively offsets the target state vector bias caused by obstruction, multipath reflection, and self-detection noise from a single millimeter-wave radar. Compared to directly using all radar data without weighting, this reduces the error in target state estimation. Furthermore, the weighted fusion method retains the detection information from multiple radars, further improving the robustness of target state estimation in complex scenarios compared to retaining only the highest weight value from a single radar. In scenarios with multiple dynamically interfering targets, it can improve the accuracy of the effective state vector, providing a stable and reliable data foundation for subsequent target tracking and behavior recognition tasks.

[0071] Based on the above method, the detection accuracy of user presence can be improved in obstructed and low-light scenarios. Traditional radar relies solely on line-of-sight (LOS). In a kitchen, if a person is obstructed by a tall stove, refrigerator, or cabinet, the LOS path is cut off, and traditional radar will determine "no one is present," leading to missed detections. The above method treats metal / hard surfaces such as sinks, stoves, and walls as signal relay stations. Even if a person is in an obstructed area (such as behind a stove), the micro-motion signals they generate can be reflected back to the radar via a multipath path of "radar → obstruction → person" or "radar → person → obstruction." The radar can receive signals after secondary or multiple reflections, thus detecting the presence and micro-motions of a person even when the line of sight is completely blocked, completely eliminating detection blind spots in kitchen corners and obstructed areas.

[0072] Metallic materials (such as stainless steel sinks and stovetops) have high reflectivity to millimeter waves. When a human body approaches these highly reflective surfaces, the multipath signal not only carries information about the human body but also significantly enhances the echo intensity by utilizing the strong reflectivity of the metal surface. Based on the amplitude attenuation factor and phase change rate extracted from the multipath signal, even if the direct wave is weak, the strongly reflected multipath wave (such as the signal reflected by the sink) can still provide clear micro-Doppler features (such as using the micro-Doppler modulation period to reflect breathing fluctuations). In low light conditions (where visual assistance is not possible), the system can stably detect stationary or slightly moving human bodies solely based on the enhancement of physical echo intensity, without relying on image brightness.

[0073] Traditional radar often misidentifies dynamic environmental disturbances (such as swaying curtains, running pets, or airflow disturbances) as human bodies, or makes false alarms due to obstruction from a single viewpoint. Based on the method described above, a pre-set neural network model is used to identify only signals that conform to a specific geometric path (e.g., pointing towards a sink area) and possess specific human movement frequencies (e.g., breathing at 0.2Hz, chopping vegetables at 1-3Hz) as valid targets. If an interference source (e.g., a swaying plastic bag) generates clutter, but its time delay does not conform to any known human path model, or its phase change rate does not possess human characteristics, the pre-set neural network model will automatically filter it out. Through multi-parameter joint decision-making, false alarms caused by environmental noise are significantly reduced, improving detection accuracy.

[0074] Traditional single-radar systems, when encountering obstruction, cannot recover the lost target signal, leading to detection interruption. Based on the aforementioned method, three millimeter-wave radar nodes (A, B, and C) are deployed. When millimeter-wave radar A is obstructed, millimeter-wave radar B or C may capture the same target through different multipath paths (e.g., millimeter-wave radar B reflects from the side). Using an improved EKF algorithm, weights are dynamically assigned based on the signal-to-noise ratio (SNR) of each target path. If millimeter-wave radar A detects a direct wave (high SNR), it is assigned a high weight; if the signal from millimeter-wave radar A is lost, but millimeter-wave radar B detects strong multipath reflection (high SNR), the system automatically switches to trusting the data from millimeter-wave radar B. When a person is behind a stove (NLOS area), millimeter-wave radar A can only receive a weak direct wave, but millimeter-wave radar B can capture clear micro-motion features through the stove's reflection path. The system dynamically assigns higher confidence weights to different paths through a path weight allocation mechanism, achieving continuous tracking of the obstructed area and realizing seamless tracking. Even if some radars fail or are blocked, they can still maintain continuous perception of the human body's presence, greatly enhancing the system's robustness.

[0075] Figure 2 This is a flowchart illustrating a kitchen equipment control method in one embodiment. It should be understood that, although... Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0076] In one embodiment, such as Figure 3 As shown, a kitchen equipment control device 130 is provided, comprising: The receiving module 310 is used to acquire multipath signals periodically provided by at least one millimeter-wave radar in the kitchen area; The feature extraction module 320 is used to determine the multipath echo features based on the multipath signal; The behavior recognition module 330 is used to determine the user behavior status in the kitchen area based on the multipath echo characteristics corresponding to different timestamps of each millimeter-wave radar within a preset sliding window. The control module 340 is used to output corresponding target control commands to the target kitchen equipment based on the user's behavior status and the current operating status of each kitchen equipment in the kitchen area.

[0077] In one embodiment, the feature extraction module 320 is further configured to: The multipath signal is preprocessed to obtain the multipath echo signal; The multipath echo signal is subjected to range Doppler transformation to obtain a range Doppler image; Based on the analysis and processing results of the distance Doppler image and the multipath echo signal, at least two of the following are determined for at least one echo path: time delay, echo angle, Doppler frequency shift, amplitude attenuation factor, phase change rate, micro-Doppler modulation period, and signal-to-noise ratio. The multipath echo characteristics include at least two of the following for each echo path: time delay, echo angle, Doppler frequency shift, amplitude attenuation factor, phase change rate, micro-Doppler modulation period, and signal-to-noise ratio.

[0078] In one embodiment, the behavior recognition module 330 is further configured to: Based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each millimeter-wave radar, the user's presence status and user movement trajectory are determined. When the user exists, the user's motion trajectory and the multipath echo features corresponding to different timestamps of each millimeter-wave radar within the preset sliding window are input into the preset neural network model, and the user's behavior state is determined based on the output of the preset neural network model.

[0079] In one embodiment, the behavior recognition module 330 is further configured to: Based on the multipath echo characteristics of the millimeter-wave radar corresponding to the target timestamp within a preset sliding window, the time delay, echo angle, and preset kitchen map corresponding to different echo paths are used to infer the position of each reflecting object in the echo path, wherein the target timestamp is any timestamp within the preset sliding window; The user presence status corresponding to each echo path is determined based on the micro-Doppler modulation period and / or phase change rate corresponding to each echo path. The echo path in which the user exists is taken as the target path, and the motion speed corresponding to the human features in the target path is determined according to the Doppler frequency shift and / or phase change rate corresponding to the target path. Based on the position and speed of the human body features in the target path, the millimeter-wave radar determines the target state vector corresponding to the target timestamp within a preset sliding window; The user's motion trajectory is determined based on the target state vector corresponding to different timestamps within a preset sliding window of each millimeter-wave radar.

[0080] In one embodiment, the behavior recognition module 330 is further configured to: Based on the signal-to-noise ratio and path length of the target path corresponding to different timestamps within a preset sliding window, the weight values ​​of the target state vectors corresponding to different timestamps of each millimeter-wave radar are determined. Based on the weight values ​​of the target state vectors corresponding to different timestamps of each millimeter-wave radar, the effective state vectors corresponding to each timestamp within the preset sliding window are determined. The user's motion trajectory is determined based on the curve fitting results of the effective state vectors corresponding to each timestamp within the preset sliding window.

[0081] In one embodiment, the behavior recognition module 330 is further configured to: The range exponential decay term is determined based on the path length of the target path corresponding to the target timestamp of the millimeter-wave radar. The path signal score corresponding to the millimeter-wave radar is determined by multiplying the range exponential attenuation term and the signal-to-noise ratio of the target path corresponding to the millimeter-wave radar. The weight value of the millimeter-wave radar in the target state vector corresponding to the target timestamp is determined based on the ratio between the path signal score corresponding to the millimeter-wave radar and the sum of the path signal scores corresponding to all millimeter-wave radars.

[0082] In one embodiment, the behavior recognition module 330 is further configured to: Within the preset sliding window, each millimeter-wave radar selects the target state vector with the largest weight value from the target state vectors corresponding to the target timestamp as the effective state vector corresponding to the target timestamp; or, Based on the weight values ​​of the target state vectors corresponding to the target timestamps of each millimeter-wave radar, the target state vectors corresponding to the target timestamps of each millimeter-wave radar are weighted and summed to determine the effective state vector corresponding to the target timestamp.

[0083] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of a device, can operate in environments such as... Figure 1 The hardware environment shown can be implemented either through software or through hardware.

[0084] like Figure 4 As shown, this application provides a kitchen equipment control system, including a processor 711, a communication interface 712, a memory 713, and a communication bus 714. The processor 711, the communication interface 712, and the memory 713 communicate with each other through the communication bus 714. The memory 713 is used to store computer programs. When the processor 711 executes the program stored in the memory 713, it implements the kitchen equipment control method provided in any of the aforementioned method embodiments.

[0085] The memory and processor in the aforementioned electronic devices communicate with each other via a communication bus and a communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0086] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0087] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0088] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the kitchen equipment control system to which the present application is applied. A specific kitchen equipment control system may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0089] According to another aspect of the embodiments of this application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a kitchen equipment control system reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the kitchen equipment control system to perform the steps of any of the above embodiments.

[0090] In one embodiment, the kitchen equipment control device provided in this application can be implemented as a computer program, and the computer program can be implemented as follows: Figure 4 The kitchen equipment control system shown operates on this system. The memory of the kitchen equipment control system can store the various program modules that make up the kitchen equipment control device, for example, Figure 3 The diagram shows a receiving module 310, a feature extraction module 320, a behavior recognition module 330, and a control module 340. The computer program comprised of these modules causes the processor to execute the kitchen equipment control methods of the various embodiments of this application described in this specification.

[0091] Figure 4 The kitchen equipment control system shown can be controlled via, for example... Figure 3The receiving module 310 in the kitchen equipment control device shown acquires multipath signals periodically provided by at least one millimeter-wave radar within the kitchen area. The kitchen equipment control system can use the feature extraction module 320 to determine multipath echo characteristics based on the multipath signals. The kitchen equipment control system can use the behavior recognition module 330 to determine the user's behavior status within the kitchen area based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each of the millimeter-wave radars. The kitchen equipment control system can use the control module 340 to output corresponding target control commands to the target kitchen equipment based on the user's behavior status and the current operating status of each kitchen device within the kitchen area.

[0092] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the kitchen equipment control method provided in any of the foregoing method embodiments.

[0093] Optionally, in embodiments of this application, the computer-readable medium is configured to store program code for the processor to perform the following steps: Acquire multipath signals periodically provided by at least one millimeter-wave radar within the kitchen area; Based on the multipath signal, determine the multipath echo characteristics; Based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each millimeter-wave radar, the user behavior status in the kitchen area is determined. Based on the user's behavior status and the current operating status of each kitchen device in the kitchen area, the corresponding target control command is output to the target kitchen device.

[0094] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0095] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0096] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0097] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0098] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0099] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0100] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a USB flash drive, mobile hard drive, ROM, RAM, magnetic disk, or optical disk, or other media capable of storing program code, including several instructions to cause a kitchen equipment control system (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0103] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also mean including the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that alternatives or substitutions may be used.

[0104] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for controlling kitchen equipment, characterized in that, The method includes: Acquire multipath signals periodically provided by at least one millimeter-wave radar within the kitchen area; Based on the multipath signal, determine the multipath echo characteristics; Based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each millimeter-wave radar, the user behavior status in the kitchen area is determined. Based on the user's behavior status and the current operating status of each kitchen device in the kitchen area, the corresponding target control command is output to the target kitchen device.

2. The method according to claim 1, characterized in that, The step of determining the multipath echo characteristics based on the multipath signal includes: The multipath signal is preprocessed to obtain the multipath echo signal; The multipath echo signal is subjected to range Doppler transformation to obtain a range Doppler image; Based on the analysis and processing results of the distance Doppler image and the multipath echo signal, at least two of the following are determined for at least one echo path: time delay, echo angle, Doppler frequency shift, amplitude attenuation factor, phase change rate, micro-Doppler modulation period, and signal-to-noise ratio. The multipath echo characteristics include at least two of the following for each echo path: time delay, echo angle, Doppler frequency shift, amplitude attenuation factor, phase change rate, micro-Doppler modulation period, and signal-to-noise ratio.

3. The method according to claim 2, characterized in that, The step of determining the user behavior status in the kitchen area based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each millimeter-wave radar includes: Based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each millimeter-wave radar, the user's presence status and user movement trajectory are determined. When the user exists, the user's motion trajectory and the multipath echo features corresponding to different timestamps of each millimeter-wave radar within the preset sliding window are input into the preset neural network model, and the user's behavior state is determined based on the output of the preset neural network model.

4. The method according to claim 3, characterized in that, Based on the multipath echo characteristics corresponding to different timestamps within a preset sliding window of each millimeter-wave radar, the user's presence status and motion trajectory are determined, including: Based on the multipath echo characteristics of the millimeter-wave radar corresponding to the target timestamp within a preset sliding window, the time delay, echo angle, and preset kitchen map corresponding to different echo paths are used to infer the position of each reflecting object in the echo path, wherein the target timestamp is any timestamp within the preset sliding window; The user presence status corresponding to each echo path is determined based on the micro-Doppler modulation period and / or phase change rate corresponding to each echo path. The echo path in which the user exists is taken as the target path, and the motion speed corresponding to the human features in the target path is determined according to the Doppler frequency shift and / or phase change rate corresponding to the target path. Based on the position and speed of the human body features in the target path, the millimeter-wave radar determines the target state vector corresponding to the target timestamp within a preset sliding window; The user's motion trajectory is determined based on the target state vector corresponding to different timestamps within a preset sliding window of each millimeter-wave radar.

5. The method according to claim 4, characterized in that, Based on the target state vectors corresponding to different timestamps within a preset sliding window for each of the millimeter-wave radars, the user's motion trajectory is determined, including: Based on the signal-to-noise ratio and path length of the target path corresponding to different timestamps within a preset sliding window, the weight values ​​of the target state vectors corresponding to different timestamps of each millimeter-wave radar are determined. Based on the weight values ​​of the target state vectors corresponding to different timestamps of each millimeter-wave radar, the effective state vectors corresponding to each timestamp within the preset sliding window are determined. The user's motion trajectory is determined based on the curve fitting results of the effective state vectors corresponding to each timestamp within the preset sliding window.

6. The method according to claim 5, characterized in that, The step of determining the weight values ​​of the target state vectors of each millimeter-wave radar at different timestamps based on the signal-to-noise ratio and path length of the target path at different timestamps within a preset sliding window includes: The range exponential attenuation term is determined based on the path length of the target path corresponding to the target timestamp of the millimeter-wave radar. The path signal score corresponding to the millimeter-wave radar is determined by multiplying the distance exponential attenuation term and the signal-to-noise ratio of the target path corresponding to the millimeter-wave radar. The weight value of the millimeter-wave radar in the target state vector corresponding to the target timestamp is determined based on the ratio between the path signal score corresponding to the millimeter-wave radar and the sum of the path signal scores corresponding to all millimeter-wave radars.

7. The method according to claim 5, characterized in that, Based on the weight values ​​of the target state vectors corresponding to different timestamps of each millimeter-wave radar, the effective state vectors corresponding to each timestamp within the preset sliding window are determined, including: Within the preset sliding window, each millimeter-wave radar selects the target state vector with the largest weight value from the target state vectors corresponding to the target timestamp as the effective state vector corresponding to the target timestamp; or, Based on the weight values ​​of the target state vectors corresponding to the target timestamps of each millimeter-wave radar, the target state vectors corresponding to the target timestamps of each millimeter-wave radar are weighted and summed to determine the effective state vector corresponding to the target timestamp.

8. A kitchen equipment control device, characterized in that, The kitchen equipment control device includes: A receiving module is used to acquire multipath signals periodically provided by at least one millimeter-wave radar in the kitchen area; The feature extraction module is used to determine the multipath echo features based on the multipath signal; The behavior recognition module is used to determine the user behavior status in the kitchen area based on the multipath echo characteristics corresponding to different timestamps of each millimeter-wave radar within a preset sliding window. The control module is used to output corresponding target control commands to the target kitchen equipment based on the user's behavior status and the current operating status of each kitchen equipment in the kitchen area.

9. A kitchen equipment control system, characterized in that, The kitchen equipment control system includes at least one millimeter-wave radar, multiple kitchen devices, and a kitchen equipment control device as described in claim 8, wherein the kitchen equipment control device is communicatively connected to each of the millimeter-wave radars and the kitchen devices.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images