A toilet control method and control system based on visual detection

Abstract

The application discloses a toilet control method and control system based on visual detection, which is started after confirming the existence of a user through a millimeter wave radar and infrared thermal imaging. A monocular camera module collects images, and a multi-beam ranging radar collects three-dimensional point clouds to generate a posture geometry graph. A support vector machine outputs the user's orientation state. According to the user distance start behavior prediction, the trunk pitch angle change rate, the center of mass vertical velocity, the double-hand approach distance and the change rate are calculated to form a feature matrix. A pre-trained classifier is used to calculate the likelihood probability, the maximum likelihood category is selected, and the confidence is recorded. When the confidence is insufficient, a safety bottom-up strategy is adopted, otherwise, a pre-action is executed to control the toilet cover and / or seat to flip to an appropriate angle. Compared with the prior art, the technical scheme of the application improves the toilet control ability to predict different postures or behaviors of different categories of users and adaptability.

Description

Technical Field

[0001] This invention relates to the field of toilet technology, and in particular to a toilet control method and control system based on vision detection. Background Technology

[0002] Toilet fixtures are common bathroom products used in daily life. They typically consist of a toilet body that is fixed to the ground at the bottom, and a toilet seat and a toilet lid that are rotatably connected to the top of the toilet body. The toilet lid directly covers the toilet seat to prevent dust and splashing of water or excrement during flushing.

[0003] However, different users have different needs when actually using toilets. For example, male users may want to flip up both the toilet seat and lid simultaneously before urinating, directing urine directly into the toilet bowl. Men may need to flip the toilet seat down to defecate, while women may need to sit on it. Even elderly or child users require different methods for controlling the lowering of the toilet seat and lid. Furthermore, if users directly hold and control the toilet seat and lid with their hands, the potentially contaminated surfaces can soil their hands. Therefore, current technology aims to automate the raising and lowering of the toilet seat and lid.

[0004] Existing technologies address this by using high-definition cameras to recognize users' faces or gestures to determine their intentions for lifting and lowering control, or by using infrared ranging sensors to apply fixed control methods when a fixed distance is reached. However, such control schemes may raise privacy concerns, have blind spots leading to malfunctions, or rely too heavily on image texture data, resulting in inoperability. Furthermore, existing technologies lack the ability to predict different postures or behaviors of different user groups and are not adapted for specific populations. Summary of the Invention

[0005] The main objective of this invention is to propose a toilet control method and control system based on vision detection, which aims to improve the toilet control system's ability to predict and adapt to different postures or behaviors of different types of users.

[0006] To achieve the above objectives, this invention proposes a toilet control method based on vision detection, comprising the following steps: Step S1: The millimeter-wave radar installed in the bathroom continuously monitors the heartbeat and breathing of the user in the bathroom, and the infrared thermal imaging sensor detects the temperature gradient field of the user in the bathroom. After joint confirmation, the signal is activated. Step S2: A monocular camera mold installed on the toilet surface collects image data from the user, and at the same time, a multi-beam ranging radar array collects three-dimensional point cloud data of the user's body surface. Step S3: Map the 3D point cloud data onto the original image data, establish a dark-color correlation matrix, extract the human body region in the original image by masking, remove background texture and interference, and generate the user's pose geometry. Step S4: Based on the posture geometry, calculate the user's axial moment of inertia and radial gradient variance, construct a support vector machine classifier, and output the user's orientation towards the toilet. Step S5: Based on the horizontal distance between the user and the toilet, initiate the behavior prediction process; Step S6: Calculate the rate of change of the user's torso pitch angle, the vertical velocity of the user's body center of mass, and the approach distance and rate of change of the user's hands, and combine the three feature quantities into a feature matrix. Step S7: Use a pre-trained model as a classifier and train it for different user behaviors. During online inference, calculate the likelihood probability of each model given the observation sequence. Calculate the likelihood value using the front-to-back algorithm. Select the category corresponding to the maximum likelihood and record the confidence level. Then, switch to the safety fallback strategy or switch to the pre-action execution. Step S8: Classify different user behaviors and control the toilet seat and / or toilet seat ring to rotate to the appropriate angle.

[0007] Preferably, in step S3, a complex outline of the user's body is generated using radar point clouds, a point cloud dataset is set, and all point clouds are projected onto a horizontal plane to generate a binarized occupancy grid map, thus completing the geometric constraints.

[0008] Preferably, in step S4, the user's body is rotated to a standard coordinate system by calculating the centroid and principal axis of inertia of the grid diagram, and the user's orientation toward the toilet is determined based on the elongation change in a specific direction; the distribution variance of the point cloud in the front-back direction is extracted, the time derivative of the distribution variance is calculated, and the positional change of the user and the toilet is determined by combining the data changes.

[0009] Preferably, in step S5, a sampling window length time is set, and multiple frames of images and radar point clouds are continuously acquired. Each frame is input into a lightweight pose estimation network to extract key points for the head, shoulder midpoint, left shoulder, right shoulder, left hip, right hip, left wrist, and right wrist. Simultaneously, the hip midpoint and shoulder midpoint are calculated. Preferably, in step S6, calculating the user's torso pitch angle change rate requires calculating the user's torso direction vector using X and Y components, as shown in the formula: ; The instantaneous pitch angle is calculated using the following formula: ; when When the value is 'h', it indicates that the user's torso is vertical; a negative value indicates leaning forward, and a positive value indicates leaning backward. Since leaning backward is extremely rare in actual user applications, the center difference method is used, taking one frame of data before and after the current frame to calculate the angular velocity. ; The inter-frame interval is approximately 0.033 seconds, followed by a first-order low-pass filter to eliminate jitter, using the following formula: .

[0010] Preferably, in step S6, calculating the vertical velocity of the user's center of mass requires calculating the vertical velocity of the user's center of mass. First, estimate the instantaneous centroid height of the user. The vertical weighted average of radar point clouds is used, and the point cloud set is set as follows: The reflection intensity at each point is The value range is 0-255, then ; In actual detection, when the number of radar point cloud points n < 10, it means that the user is far from the toilet, or the reflection during the actual detection process is weak. In this case, a degraded image estimation method can be used. ; Here, represents the ordinate of the lower edge of the human detection frame, and 0.53 is the proportionality constant of the average centroid height of an adult human body. In actual detection, this needs to be corrected to 0.55-0.58 for child users. Then, the vertical velocity of the user's centroid is calculated. If an outlier occurs during the calculation process, it will be handled according to... Then let .

[0011] Preferably, in step S6, the proximity distance between the user's hands is calculated. When considering its rate of change, the world coordinates of the toilet center are defined as follows: The height of the toilet seat surface from the ground is =0.42m, the position of the toilet seat center on the Z-axis =0.15m; The Euclidean distance from the left and right wrists to the center of the toilet seat is calculated separately: ; ; The average distance between the hands is calculated as follows: ; The rate of change of distance is calculated as follows: ; Similarly, a first-order low-pass filter is applied to the distance change rate.

[0012] Preferably, in step S6, for each frame within the sampling window The user's torso pitch angle change rate, the user's body center of gravity vertical velocity, and the user's hand approach distance and change rate are combined into a feature vector: ; And form the feature matrix: .

[0013] The present invention also proposes a control system applying the aforementioned vision-based toilet control method, characterized in that it includes: Millimeter-wave radar is used to continuously transmit frequency-modulated continuous waves, collect echo signals, and extract the micro-Doppler spectrum of users in the bathroom to detect the presence of periodic micro-movement sidebands of 0.8-2Hz, and to determine whether there are micro-movement signals in the chest cavity caused by heartbeat or breathing. Infrared thermal imaging sensors are used to detect temperature gradient fields of 28-37℃ to determine whether there are users inside the bathroom; A monocular camera module, installed on the toilet surface, is used to collect image data from the user; Multibeam ranging radar array, used to collect three-dimensional point cloud data of the user's body surface; The controller is configured to activate the monocular camera module and multi-beam ranging radar array when the millimeter-wave radar detects periodic micro-movement sidebands and the infrared thermal imaging sensor detects a temperature gradient field; map the 3D point cloud data to the image data, establish a depth-color correlation matrix, and extract the human body region in the image data by masking to generate the user's pose geometry; based on the pose geometry, calculate the user's axial moment of inertia and radial gradient variance; construct a support vector machine classifier, input the axial moment of inertia and radial gradient variance, and output whether the user is facing the toilet or facing away from the toilet.

[0014] The technical solution of this invention has the following advantages over the prior art: This invention's technical solution double-confirms the presence of a real user in the bathroom by detecting the micro-movements caused by the user's heartbeat and breathing, combined with infrared thermal imaging to detect the user's temperature gradient field. By introducing a multi-beam ranging radar array, the user's three-dimensional point cloud data is used to create an image and establish geometric constraints. When the user is facing away from the toilet or the camera image is unavailable, the radar point cloud is used to generate a posture geometry map to accurately determine the user's orientation. Simultaneously, it integrates three features: the user's torso pitch angle change rate, the vertical velocity of the center of mass, and the change in the distance between the user's hands and the toilet seat. A model is used to predict multiple types of behavior, and under controllable confidence conditions, adaptive control of the toilet seat and / or toilet lid can be achieved before the user even touches the toilet. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0016] Figure 1 This is a flowchart of the toilet control method based on vision detection according to the present invention.

[0017] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] This invention proposes a toilet control system and control method based on visual detection.

[0020] Please see Figure 1 The toilet control system based on vision detection in this embodiment of the invention uses a millimeter-wave radar installed in the bathroom to continuously transmit frequency-modulated continuous waves, collect echo signals and extract the micro-Doppler spectrum of the user in the bathroom, and detect whether there is a periodic micro-movement sideband of 0.8-2Hz, which corresponds to the micro-movement signal of the chest cavity caused by human heartbeat or breathing.

[0021] Meanwhile, this embodiment of the invention also uses an infrared thermal imaging sensor to detect a temperature gradient field of 28-37°C, which corresponds to the normal temperature range of the human body, to further confirm whether there is a user inside the bathroom. In practical applications, the visual detection-based toilet control system of this invention will only be activated and operational when both the millimeter-wave radar and the infrared thermal imaging sensor simultaneously confirm the presence of a real user in the bathroom.

[0022] To determine whether a user inside the bathroom is facing the toilet, has their back to the toilet, or is passing by the toilet sideways, and to provide corresponding controls based on the user's gender during subsequent operations.

[0023] The technical solution of this invention, upon responding to a preset trigger signal, acquires image data of the user's body through a monocular camera module installed on the toilet surface, and simultaneously acquires 3D point cloud data of the user's body surface through a multi-beam ranging radar array. After data acquisition, the 3D point cloud data is mapped onto the original image data, a dark-color correlation matrix is ​​established, and the human body region in the original image data is masked to remove background textures and interference, generating a user's posture geometry map. Based on the posture geometry map, the user's axial moment of inertia and radial gradient variance are calculated. A support vector machine classifier is constructed, taking the user's axial moment of inertia and radial gradient variance as input, and outputting the user's state as facing or away from the toilet.

[0024] This invention's technical solution, by introducing multi-beam ranging radar, can be applied to real-world toilet usage scenarios. For example, when a user approaches the toilet from behind, existing cameras cannot capture images of the user's face. This invention utilizes radar point clouds to fill in visual blind spots, avoiding the conventional approach of relying solely on camera computing power.

[0025] Specifically, in the technical solution of the present invention, the center of the toilet body is taken as the origin, the horizontal direction to the right is the X-axis, the vertical direction is the Y-axis, and the Y-axis is perpendicular to the plane of the toilet body. In addition, the user's forward and backward movement direction is the Z-axis.

[0026] Since the camera in this invention is mounted on the toilet surface, this mounting method results in visual limitations during actual shooting. When the user faces the camera with their back to the toilet surface, the RGB image will be invalid.

[0027] The technical solution of this invention generates a complex contour map of the user's body using radar point clouds, wherein the point cloud dataset is set as follows: ,in By projecting all point clouds onto the XZ plane, i.e., the horizontal plane, a binarized occupancy grid map is generated. Thus, the geometric constraints of the technical solution of the present invention are completed. Therefore, the technical solution of the present invention is determined only by the shape geometry and does not rely on texture data.

[0028] Based on real-life experience, in actual toilet usage scenarios, the probability of a user facing sideways while using the toilet is extremely low. People usually use the toilet facing the tank, such as when men urinate, or facing away from the tank, such as when sitting on the toilet seat.

[0029] The technical solution of this invention calculates the centroid of the grid diagram. and the direction of the principal axis of inertia The calculation formula is: , where μ is the central moment, which can rotate the user's body to the standard coordinate system.

[0030] Then rotate the coordinate axis to the principal axis direction and calculate the elongation rate E along the principal axis direction. When the user is facing the toilet, the user's shoulder width is greater than the user's body thickness, and the elongation rate E < 1. When the user is facing away from the toilet, the user's body is often leaning forward, which causes the user's body thickness data point to suddenly become longer, and the E value changes in the opposite direction.

[0031] To prevent accidental triggering by users passing by in the restroom, the technical solution of this invention also needs to determine whether the user intends to sit on the toilet. This technical solution utilizes the micro-Doppler effect of radar, extracting the variance of the point cloud P in the Z direction. Five frames of data were continuously collected by the radar module, and calculations were performed. time derivative .

[0032] If a user needs to sit down to use the toilet provided by this invention, the time derivative... The waveform will show an increase followed by a decrease. The height distribution will change drastically when the user squats down, and the waveform will show the opposite when the user suddenly gets up from the toilet.

[0033] Those skilled in the art will understand that existing technologies typically use high-definition cameras to recognize users' faces or gestures, or use infrared ranging for control. The present invention, based on a low-resolution data acquisition method and without texture features, determines whether the user is facing the toilet by relying solely on the user's axial moment of inertia. In practical applications, it is not necessary to clearly see the user's face to determine whether the user is facing forward or backward. Furthermore, the present invention introduces a multi-beam radar array, which, in practical applications with toilets, can resolve privacy concerns related to cameras or failure scenarios caused by the user facing away from the toilet.

[0034] In subsequent work, the present invention continuously detects the horizontal distance between the user and the toilet using radar. If the distance between the user and the toilet is greater than or equal to 0.6m and less than or equal to 1.2m, the system of the present invention will initiate the corresponding behavior prediction process.

[0035] Specifically, the sampling window length of the technical solution of this invention is set to 1.2 seconds, during which 36 frames of images and 24 frames of radar point clouds are continuously acquired. Then, each frame of image is processed by inputting it into a lightweight pose estimation network to extract the two-dimensional coordinates of 7 key points, including the top of the head. midpoint of the shoulder left shoulder right shoulder left hip right hip , left wrist , right wrist Key points that are obscured during the detection process are completed using linear interpolation based on the coordinates of the previous frame. For example, when both hands are extended behind the back, the wrists are not visible.

[0036] At the same time, the coordinates of auxiliary points, such as the midpoint of the hip, are also calculated. midpoint of the shoulder .

[0037] Then, the micro-image coordinates are converted into world coordinates O-XYZ with the center of the toilet tank's rear wall as the origin, using the camera's intrinsic and extrinsic parameter calibration results. The X-axis represents the horizontal left-right direction (positive when facing the toilet directly), the Y-axis is vertically upward, and the Z-axis represents the horizontal front-back direction (negative for the back of the toilet, positive for the front). After this conversion, the three-dimensional coordinates of each key point can be obtained. .

[0038] Then, the rate of change of the user's torso pitch angle is calculated, where the user's torso direction vector is calculated using only the X and Y components, as shown in the formula: When calculating the instantaneous pitch angle, the formula is used. ,when When the value is positive, it indicates that the user's torso is vertical; a negative value indicates leaning forward, and a positive value indicates leaning backward.

[0039] Since backward tilting is extremely rare in actual user applications, the center difference method is used to calculate the angular velocity by taking one frame of data before and after the tilting motion. The inter-frame interval is approximately 0.033 seconds, and then... Perform a first-order low-pass filter to eliminate jitter, the specific formula is as follows: .

[0040] After calculating the rate of change of the user's torso pitch angle, it is necessary to calculate the user's vertical velocity of center of mass. First, estimate the instantaneous centroid height of the user. Since the technical solution of this invention uses a vertical weighted average of radar point clouds, the point cloud set is set as follows: The reflection intensity at each point is The value range is 0-255, then .

[0041] In actual detection, when the number of radar point cloud points n < 10, it means that the user is far from the toilet, or the reflection during the actual detection process is weak. In this case, a degraded image estimation method can be used. .

[0042] in To select the lower edge ordinate of the human body detection frame, 0.53 is the average centroid height proportionality constant for an adult human body. In actual detection, it needs to be corrected to 0.55-0.58 for child users.

[0043] Then calculate the vertical velocity of the user's center of mass. If an outlier occurs during the calculation process, it will be handled according to... Then let .

[0044] The technical solution of this invention calculates the proximity distance between a user's hands. When considering its rate of change, the world coordinates of the toilet center are defined as follows: The height of the toilet seat surface from the ground is =0.42m, the position of the toilet seat center on the Z-axis =0.15m.

[0045] The Euclidean distance from the left and right wrists to the center of the toilet seat is calculated separately:

[0046] .

[0047] The average distance between the hands is calculated as follows: ; The rate of change of distance is calculated as follows: Similarly, a first-order low-pass filter is applied to the distance change rate.

[0048] For each frame within the sampling window The above three features are combined into a feature vector: ; This forms the characteristic matrix: .

[0049] The technical solution of this invention uses a pre-trained Hidden Markov Model (HMM) as a classifier. An HMM is pre-trained for each of the following five categories of behavior: category C1 is adult males standing to urinate, category C2 is adult women sitting to urinate, category C3 is adult males sitting to urinate, category C4 is the elderly sitting, and category C5 is children sitting.

[0050] The HMM parameters for each category are: ,in The initial state distribution, The state transition probability matrix is ​​used, and in this embodiment of the invention, four hidden states are selected. To observe the probability distribution, we assume that each dimension of the eigenvector is independent and follows a Gaussian distribution, i.e.: .

[0051] During online inference, the likelihood probability of the observed sequence F under each model is calculated:

[0052] The likelihood value was calculated using both the pre- and post-calculus algorithms, and the category corresponding to the maximum likelihood was selected as follows: Simultaneously record the confidence level. If conf < 0.6, the system determines it as low confidence and switches to a safety fallback strategy; otherwise, it switches to pre-action.

[0053] According to output category Before the user touches the toilet, the drive motor is controlled to perform a specified function. Category C1 controls the toilet seat and lid to flip upwards when the user is 1.0m away from the toilet, facilitating standing urination for male users. Categories C2 and C3 activate when the user is 0.6m away from the toilet and their torso is noticeably leaning forward, controlling the toilet seat to flip upwards and the toilet seat to flip downwards, for male or female users to sit on the toilet seat. Category C4 activates in advance when the user is 0.8m away from the toilet, causing the toilet seat to flip upwards and the toilet seat to flip downwards in a soft-close mode. Category C5 activates when a child user is 0.6m away and their torso is noticeably leaning forward, causing the toilet seat to flip upwards and the toilet seat to flip downwards upon contact with the child lock.

[0054] When conf < 0.6, the system will not perform any pre-action and will remain in standby mode. After the user actually contacts the toilet seat, the system will then specify the corresponding actions of the toilet seat and toilet lid according to the contact classification structure, thereby ensuring that in the worst case, no erroneous actions will be performed that could cause user discomfort.

[0055] Furthermore, during user operation, the system of this invention continuously monitors the distributed pressure sensor array installed on the toilet seat and toilet body. When the pressure sensor reading is greater than or equal to 5 kPa, it can be determined that the user has sat down on the toilet seat. The system verifies the prediction result based on the contact width between the user and the toilet seat surface, the peak pressure, and the pressure rise time. If the verification result matches the prediction, the current action is maintained. If they do not match, for example, if the prediction is that the user will urinate standing up, but the actual need is to sit down, the position of the toilet seat and / or toilet lid is immediately adjusted to accommodate the user's new need.

[0056] This invention's technical solution embodiment uses a scenario of an adult woman using a toilet normally. At time 0.00s, radar detects that the female user is 1.2m away from the toilet, triggering behavior prediction. At time 0.10s, the posture geometry shows that the torso begins to lean forward. = -0.62 rad / s. At time 0.20s, the center of gravity of the female user began to decrease. = -0.28m / s. At time 0.35s, the female user began to move her hands backward and downward. The depth decreased from 0.72m to 0.45m. = -0.42m / s. At time 0.50s, the feature matrix F input HMM has the highest C2 similarity with a confidence level of 0.89. The system of this invention drives the toilet seat to flip upwards and the toilet seat to flip downwards. At time 0.85s, the pressure sensor located between the toilet seat and the toilet body is triggered, ensuring that the female user sits down, consistent with the system's prediction. The system then enters a standby state for flushing. Therefore, in actual use, the user does not need to perform manual operation; the system controls the toilet seat and toilet seat to move to the appropriate positions accordingly.

[0057] Preferably, in the final stroke stage of the lifting and lowering action of the toilet seat or toilet lid to the closing position, the motor controller dynamically selects the corresponding slow-descent strategy parameters based on the detected user category label. The parameters that need to be dynamically adjusted include the slow-descent starting angle and the slow-descent speed.

[0058] Specifically, for women using the toilet while seated, a quiet operation strategy is employed. The slow-closing start angle is 10° from the fully closed position, and the slow-closing speed is 15° / s, minimizing closing noise. For men urinating, the slow-closing start angle is 5° from the fully closed position, and the slow-closing speed is 40° / s, prioritizing quick reset after the male user leaves. For elderly or mobility-impaired users, the invention limits the speed of the toilet seat or lid to 45° / s, with a slow-closing start angle of 15° and a slow-closing speed of 20° / s. It also increases the sensitivity of the resistance torque threshold by 50%.

[0059] During the control process of this invention, the motor drive controller samples the torque feedback value of the drive motor at a frequency of not less than 100Hz. When the detected torque value exceeds the safety threshold corresponding to the current strategy, it is determined that an obstacle has been encountered. In response, the motor drive controller immediately performs the following actions: stops the current descent motion, reverses the drive by 5-10° to release pressure, and provides a prompt via voice or buzzer. Simultaneously, the time of the obstacle encounter is recorded, including the time of occurrence, the angle of the obstruction position, and the current torque value.

[0060] The control system of this invention generates a record vector after each use, containing orientation tags, gender confidence levels, dwell time, time period, intervention flags, and obstruction time markers. It then uses offline or lightweight online clustering algorithms to create and update profiles for different users. Specifically, if a user profile experiences manual intervention contradicting the system's decision a preset number of times consecutively, the system automatically corrects the profile's decision parameters. If a user profile accumulates more than a preset number of obstruction events within a set time, the system automatically lowers the default descent speed for subsequent uses or advances the descent start angle to achieve long-term safe adaptation.

[0061] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made under the concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A toilet control method based on vision detection, characterized in that, Includes the following steps: Step S1: The millimeter-wave radar installed in the bathroom continuously monitors the heartbeat and breathing of the user in the bathroom, and the infrared thermal imaging sensor detects the temperature gradient field of the user in the bathroom. After joint confirmation, the signal is activated. Step S2: A monocular camera mold installed on the toilet surface collects image data from the user, and at the same time, a multi-beam ranging radar array collects three-dimensional point cloud data of the user's body surface. Step S3: Map the 3D point cloud data onto the original image data, establish a dark-color correlation matrix, extract the human body region in the original image by masking, remove background texture and interference, and generate the user's pose geometry. Step S4: Based on the posture geometry, calculate the user's axial moment of inertia and radial gradient variance, construct a support vector machine classifier, and output the user's orientation towards the toilet. Step S5: Based on the horizontal distance between the user and the toilet, initiate the behavior prediction process; Step S6: Calculate the rate of change of the user's torso pitch angle, the vertical velocity of the user's body center of mass, and the approach distance and rate of change of the user's hands, and combine the three feature quantities into a feature matrix. Step S7: Use a pre-trained model as a classifier and train it for different user behaviors. During online inference, calculate the likelihood probability of each model given the observation sequence. Calculate the likelihood value using the front-to-back algorithm. Select the category corresponding to the maximum likelihood and record the confidence level. Then, switch to the safety fallback strategy or switch to the pre-action execution. Step S8: Classify different user behaviors and control the toilet seat and / or toilet seat ring to rotate to the appropriate angle.

2. The toilet control method based on vision detection as described in claim 1, characterized in that, In step S3, a complex outline of the user's body is generated using radar point clouds, a point cloud dataset is set, and all point clouds are projected onto a horizontal plane to generate a binarized occupancy grid map, thus completing the geometric constraints.

3. The toilet control method based on vision detection as described in claim 1, characterized in that, In step S4, the user's body is rotated to a standard coordinate system by calculating the centroid and principal axis of inertia of the grid diagram, and the user's orientation toward the toilet is determined based on the elongation change in a specific direction; the distribution variance of the point cloud in the front and back directions is extracted, the time derivative of the distribution variance is calculated, and the positional change of the user and the toilet is determined by combining the data changes.

4. The toilet control method based on vision detection as described in claim 1, characterized in that, In step S5, the sampling window length time is set, and multiple frames of images and multiple frames of radar point clouds are continuously acquired. Each frame of image is input into a lightweight pose estimation network to extract key points of the head, shoulder midpoint, left shoulder, right shoulder, left hip, right hip, left wrist, and right wrist. At the same time, the hip midpoint and shoulder midpoint are calculated as an auxiliary calculation.

5. The toilet control method based on vision detection as described in claim 1, characterized in that, In step S6, calculating the rate of change of the user's torso pitch angle requires calculating the user's torso direction vector, using the X and Y components, as shown in the formula: ; The instantaneous pitch angle is calculated using the following formula: ； when When the value is 'h', it indicates that the user's torso is vertical; a negative value indicates leaning forward, and a positive value indicates leaning backward. Since leaning backward is extremely rare in actual user applications, the center difference method is used, taking one frame of data before and after the current frame to calculate the angular velocity. ； The inter-frame interval is approximately 0.033 seconds, followed by a first-order low-pass filter to eliminate jitter, using the following formula: 。 6. The toilet control method based on vision detection as described in claim 1, characterized in that, In step S6, calculating the vertical velocity of the user's center of gravity requires calculating the vertical velocity of the user's center of gravity. First, estimate the instantaneous centroid height of the user. The vertical weighted average of radar point clouds is used, and the point cloud set is set as follows: The reflection intensity at each point is The value range is 0-255, then ; In actual detection, when the number of radar point cloud points n < 10, it means that the user is far from the toilet, or the reflection during the actual detection process is weak. In this case, a degraded image estimation method can be used. ； in To select the lower edge ordinate of the human detection frame, 0.53 is used as the proportionality constant for the average centroid height of an adult human body. In actual detection, this needs to be corrected to 0.55-0.58 for child users. Then, the vertical velocity of the user's centroid is calculated. If an outlier occurs during the calculation process, it will be handled according to... Then let .

7. The toilet control method based on vision detection as described in claim 1, characterized in that, In step S6, the proximity distance between the user's hands is calculated. When considering its rate of change, the world coordinates of the toilet center are defined as follows: The height of the toilet seat surface from the ground is =0.42m, the position of the toilet seat center on the Z-axis =0.15m; The Euclidean distance from the left and right wrists to the center of the toilet seat is calculated separately: ； ； The average distance between the hands is calculated as follows: ; The rate of change of distance is calculated as follows: ; Similarly, a first-order low-pass filter is applied to the distance change rate.

8. The toilet control method based on vision detection as described in claim 1, characterized in that, In step S6, for each frame within the sampling window, the user's torso pitch angle change rate, the user's body center of mass vertical velocity, the user's hand approach distance, and the change rate feature are combined into a feature vector: ; And form the feature matrix: .

9. A control system applying the vision-based toilet control method as described in claim 1, characterized in that, include: Millimeter-wave radar is used to continuously transmit frequency-modulated continuous waves, collect echo signals, and extract the micro-Doppler spectrum of users in the bathroom to detect the presence of periodic micro-movement sidebands of 0.8-2Hz, and to determine whether there are micro-movement signals in the chest cavity caused by heartbeat or breathing. Infrared thermal imaging sensors are used to detect temperature gradient fields of 28-37℃ to determine whether there are users inside the bathroom; A monocular camera module, installed on the toilet surface, is used to collect image data from the user; Multibeam ranging radar array, used to collect three-dimensional point cloud data of the user's body surface; The controller is configured to activate the monocular camera module and the multi-beam ranging radar array when the millimeter-wave radar detects periodic micro-movement sidebands and the infrared thermal imaging sensor detects a temperature gradient field. The 3D point cloud data is mapped onto the image data to establish a depth-color correlation matrix. The human body region in the image data is extracted by masking to generate the user's pose geometry. Based on the pose geometry, the user's axial moment of inertia and radial gradient variance are calculated. A support vector machine classifier is constructed, which takes the axial moment of inertia and radial gradient variance as input and outputs whether the user is facing the toilet or facing away from the toilet.

Images