Self-adaptive control system based on clothes state recognition

Abstract

The invention discloses a self-adaptive control system based on clothes state recognition, which belongs to the technical field of intelligent control and comprises a sensing module, an analysis module and a decision control module. The sensing module is used for collecting visual sensing data in an airing area, performing feature association extraction in combination with scene auxiliary data, and generating a scene feature set; the analysis module is used for carrying out single-piece separation on the scene feature set, generating analysis subsets, synthesizing visual features, thermal imaging features and environment time sequence features, calculating a multi-dimensional drying probability, judging a drying state, analyzing and identifying a user intention, constructing a collaborative relation graph containing clothes states, intentions, user states and environment nodes, and sending the collaborative relation graph to a cloud server; carrying out evolution prediction; wherein the user intention comprises a clear operation intention, and an intention guide set is constructed for the clear operation intention; and the decision control module is used for reading the intention-oriented set in the analysis module and carrying out intention-oriented control based on the target intention, so that the intelligent level is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to an adaptive control system based on clothing state recognition, belonging to the technical field of intelligent control. BACKGROUND

[0002] With the continuous progress of science and technology, non-contact control technologies such as gesture recognition and voice interaction have gradually penetrated into the traditional home field. The intelligent clothesline is an integrated product that emerges as the times require in this background. Through built-in electronic control, remote operation, environmental monitoring and other intelligent functions, the traditional clothesline is upgraded to provide users with a more convenient and comfortable drying experience, becoming a common choice in modern homes.

[0003] The Chinese patent with the granted announcement CN116027686B discloses a method and device for controlling an intelligent clothesline and an intelligent clothesline. The method comprises: determining to start an outdoor drying mode or an indoor drying mode according to the light intensity of the area where the intelligent clothesline is located; in the case of starting the outdoor drying mode, controlling the opening and closing device to open and close according to the environmental humidity; in the case of starting the indoor drying mode, controlling the opening and closing device to open and close according to the environmental humidity and the environmental wind speed. The light intensity of the area where the intelligent clothesline is located is determined to judge the current drying scene, and the outdoor drying mode or the indoor drying mode is started according to the drying scene.

[0004] Although the prior art starts the corresponding drying mode for drying according to different drying scenes, improves the drying effect of the intelligent clothesline, and improves the user experience, the recognition of the scene is mostly limited to the detection of basic environmental parameters, lacks deep semantic understanding of the drying scene, and is difficult to accurately judge the state of the core object, such as whether the clothes are in the process of being dried or have reached the dry state ready for collection. At the same time, it is also difficult to effectively distinguish the user's behavior intention, for example, whether the user is simply passing by the clothesline or preparing for the operation of drying. This lack of cognitive level limits the further intelligent and humanized upgrading of the function of the intelligent clothesline. SUMMARY

[0005] In view of the deficiencies of the prior art, the purpose of the present application is to provide an adaptive control system based on clothing state recognition. The scene feature set is generated by collecting visual perception data, the clothing drying state is accurately determined through single-piece separation and multi-feature fusion, the user's intention is recognized through audio-video-behavior time alignment, and targeted control is realized combined with evolution prediction, solving the problems of insufficient deep semantic understanding of the scene and inaccurate state and intention judgment.

[0006] To achieve the above purpose, the present application provides the following technical solutions:

[0007] The adaptive control system based on clothing state recognition comprises a perception module, an analysis module and a decision control module.

[0008] The perception module is used to collect visual perception data in the drying area, extract feature correlation in combination with scene auxiliary data, and generate a scene feature set;

[0009] The analysis module is used to separate the scene feature set, generate an analysis subset of single clothes, integrate visual features, thermal imaging features and environmental time sequence features, calculate a multi-dimensional drying probability, determine a drying state, analyze and identify user intentions through audio and behavior time sequence alignment, construct a collaborative relationship graph containing clothes state, intention, user state and environmental nodes, and perform evolution prediction; wherein the user intention includes an explicit operation intention, and an intention guidance set is constructed for the explicit operation intention;

[0010] The decision control module is used to read the intention guidance set in the analysis module and perform intention guidance control based on the target intention.

[0011] Specifically, the feature correlation extraction step includes:

[0012] Based on the configured camera, the clothes image in the drying area is obtained, including the original color image and the infrared thermal image;

[0013] The camera is configured with a dual-mode triggering mechanism, including timed sampling and event triggering, and the event triggering includes clothes amount triggering, offset triggering and temperature mutation triggering;

[0014] Gaussian filtering algorithm is used to remove environmental noise in the clothes image, and scale-invariant feature transformation algorithm is used to align the clothes images at different sampling times;

[0015] Based on the denoised and aligned color image, the texture feature and color saturation feature of the clothes are extracted to form a visual feature map of the clothes;

[0016] Based on the denoised and aligned infrared thermal image, an adaptive threshold segmentation algorithm is used to extract the temperature profile of the clothes, and a temperature distribution matrix is generated through temperature quantization of the pixel points;

[0017] The visual feature map and the temperature distribution matrix are integrated to construct a clothes feature set.

[0018] Specifically, the feature correlation extraction step further includes:

[0019] Obtain the environmental time sequence data of the drying area, including temperature and humidity, wind speed, and light intensity curve, calculate the environmental change rate through a sliding window, and adjust the sampling frequency in combination with the change threshold;

[0020] Based on the audio sensor and the camera, the camera image is analyzed in real time, and user behavior is captured, including audio signals and human image frames;

[0021] Obtain subsets of audio features and behavioral features to generate an interaction feature set;

[0022] Get and quantify environmental warnings for a future preset time period, get the current day / night time and environmental attributes, convert them into label features, and generate scene auxiliary features in combination with the environmental warnings;

[0023] A scene feature set is constructed based on time synchronization and regional identification.

[0024] Specifically, the parsing module includes a state determination unit and an association reasoning unit;

[0025] The state determination unit is used to separate clothing from the scene feature set, calculate the probability of collecting a single piece of clothing by combining visual features, thermal imaging features, and environmental time series features, determine the dry state, and form a clothing state list.

[0026] The association reasoning unit is used to construct a user behavior sequence, determine the user's intent by combining the clothing status list, generate intent tags, perform environmental trend analysis on the scene auxiliary features and the environmental time series data, and perform evolution prediction by combining clothing status and user intent to generate scene evolution trend tags.

[0027] Specifically, the steps for determining the drying state include:

[0028] The visual outline of the clothing is extracted from the visual feature map using the Canny edge detection algorithm.

[0029] Based on the spatial mask of the visual contour, the overlapping regions in the temperature distribution matrix are targeted and thresholded to extract the temperature contour.

[0030] Calculate the coordinate intersection rate. Only when the coordinate intersection rate is not less than the confidence threshold is it determined that the visual contour and the temperature contour belong to the same garment, and pixel-level fusion is performed to generate a composite contour.

[0031] The coordinate range of the region identifier is divided into mutually exclusive parts to obtain the outline center, and the reference region identifier is obtained by comparing them one by one.

[0032] Obtain the region area threshold and outline area to classify clothing categories;

[0033] For a single garment that has been categorized, corresponding exclusive features are extracted based on the scene feature set to generate a parsed subset for that garment.

[0034] Specifically, the steps for determining the drying state also include:

[0035] Based on the aforementioned parsed subset, texture features and color saturation features are weighted and fused to generate drying visual features, which are then corrected by combining scene auxiliary features.

[0036] Using a visual judgment model, with the aforementioned drying visual features as input, a visual dryness probability is generated;

[0037] Obtain the average surface temperature of a single garment and the average ambient temperature, and calculate the average temperature difference.

[0038] Set the imaging temperature difference threshold and adjust it based on scene auxiliary features;

[0039] Construct a temperature feature classification system and generate thermal imaging dryness probability based on the currently calculated average temperature difference;

[0040] Using a time-series determination model, the probability of an environment being dry is generated by taking the hanging time of a single garment, environmental time-series data, and environmental early warning as inputs.

[0041] Based on scene-assisted features, the weights of the drying probabilities of each dimension are configured, the collection probability is calculated, and the drying status is generated using a probability mapping table, thereby generating a clothing status list for each garment.

[0042] Specifically, the steps for determining user intent include:

[0043] For the subsets of audio features and the subsets of behavioral features, a synchronization sequence is constructed through temporal alignment;

[0044] By utilizing a long-short dependency capture mechanism, sequence features are extracted, and user behavior sequences are generated based on time.

[0045] Pre-defined scene intents are used to generate an initial candidate intent set.

[0046] Based on prior probability estimation, and combined with historical interaction datasets, base probabilities are configured for various candidate intents.

[0047] Set the target region and match it with the region identifiers that exist in the parsed subset;

[0048] Configure scene correction factors and correct the base probabilities of various candidate intentions;

[0049] Extract key signals from a subset of audio features, and perform intent enhancement on the audio features based on the key signals;

[0050] Using an intent classification model and user behavior sequences, probability distributions of three types of candidate intents are generated. The highest probability is taken as the intent confidence level, and combined with a secondary confidence threshold, the user intent label is determined.

[0051] Specifically, the steps of evolution prediction include:

[0052] Set up evolution nodes, including clothing state nodes, intent nodes, user state nodes, and environment nodes, and construct a collaborative relationship graph by combining directed causal edges;

[0053] Based on the operational intent, an evolution prediction is made to obtain the predicted collection time. For the remaining clothes after the predicted collection, the drying optimization requirements are generated through the collaborative relationship graph.

[0054] Based on environmental changes, the system predicts evolution and targets non-dry clothing. Using scene-assisted features, it obtains environmental warnings for a preset time period in the future, calculates the probability of non-dry clothing getting wet in the rain, and makes corresponding adjustments.

[0055] Evolutionary prediction is performed based on the intention to dry clothes. Scenarios with clear operational intentions and target intentions of preparing to dry clothes are selected. The drying curve of damp clothes in the current environment is simulated to generate the amount of water evaporation per unit time and generate decision-making schemes.

[0056] Based on scenario evolution, key time nodes and corresponding associated parameters are obtained, and a predictive evolution table is generated by combining the collaborative relationship diagram.

[0057] Specifically, the steps of intent-oriented control include:

[0058] If the target's intention is to collect, obtain their height and the range of the drying pole's height.

[0059] Set a safety redundancy height, calculate the target height of the drying rack, and adjust it according to the range of drying rack heights;

[0060] The target height of the drying rack is converted into a pulse signal to generate an adjustment command;

[0061] The actual height of the drying rack is collected in real time, and the absolute deviation is calculated by combining it with the target height of the drying rack.

[0062] Once the absolute deviation exceeds the deviation threshold, a secondary fine-tuning is performed using a PID control algorithm.

[0063] Set a lighting activation threshold, combine the environmental time series data with the scene auxiliary features, perform auxiliary lighting linkage, and generate supplementary lighting commands;

[0064] Real-time monitoring of target clothing, combined with composite contours to calculate contour similarity;

[0065] Once the contour similarity is less than the contour determination threshold within the disappearance determination interval, it is determined that the clothing has been removed, a state update instruction is generated, and the scene feature set is updated.

[0066] Specifically, the steps of intent-oriented control also include:

[0067] If the target's intention is to prepare for drying clothes, calculate the target's hanging distance based on the current environmental parameters and the influence of clothes drying.

[0068] Based on the aforementioned subset of behavioral features, the user's movement path is analyzed to confirm that the user is approaching the target area. Simultaneously, the infrared sensing device in the target area is activated to collect sensing data in real time.

[0069] Once the user's handheld object enters the preset sensing range, the command to lower the drying rack is triggered.

[0070] When the clothesline is lowered to the target height, a descent stop signal is triggered, and the images of the clothes in the target area are monitored in real time, comparing the images before and during the hanging process.

[0071] If a new composite outline is detected, the garment hanging is considered complete; otherwise, monitoring continues.

[0072] Once the hanging is confirmed, a collection association instruction is generated, the scene feature set is updated, and a new parsing subset for each garment is generated.

[0073] The beneficial effects of this invention are:

[0074] By associating visual perception data of the drying area with scene-aided data, a complete scene feature set is generated, avoiding the limitations of traditional single-parameter detection. A dedicated analytical subset is generated through single-item separation, and multi-dimensional drying probabilities are calculated by integrating visual, thermal imaging, and environmental temporal features to accurately determine the drying status of clothing, distinguishing between clothes drying and those awaiting collection. Simultaneously, precise analysis and recognition of user intent are achieved through audio and behavioral temporal alignment. A collaborative relationship graph is constructed by combining clothing status, environment, and other nodes, and evolutionary prediction is completed, effectively distinguishing between user actions such as passing by or preparing to dry clothes. Based on the identified explicit operational intent, an intent-guided set is constructed to achieve targeted intent-guided control. Through a deep understanding of the status of core objects in the scene and user intent, the smart clothes rack is upgraded from basic environmental detection to a personalized service that proactively adapts to needs, improving ease of use and intelligence, and fully meeting users' needs for precise and humanized services in the drying scene. Attached Figure Description

[0075] Fig. 1 This is a structural diagram of an adaptive control system based on clothing status recognition.

[0076] Fig. 2 This is a flowchart for determining the drying state in this invention;

[0077] Fig. 3 This is a flowchart illustrating the process of determining user intent in this invention;

[0078] Fig. 4 This is a flowchart of the intent-guided control in this invention. Detailed Implementation

[0079] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0080] refer to Figs. 1 to 4 As shown in the figure, this embodiment introduces an adaptive control system based on clothing status recognition, including: a sensing module, a parsing module, and a decision control module;

[0081] The perception module uses the distributed sensor array mounted on the smart clothes rack to collect visual perception data in the drying area, including clothing images, environmental time-series data, and behavioral interaction data. Combined with scene auxiliary data, it performs feature association extraction and associates and binds scene auxiliary features with multi-dimensional drying features to form a scene feature set to provide preliminary information.

[0082] Specifically, the steps for feature association extraction include:

[0083] Based on a wide-angle camera deployed on the clothes rack crossbar, images of clothes in the drying area are acquired, including raw color images and infrared thermal images; the camera's monitoring range covers the clothes rack and the area where clothes are hanging in the drying area.

[0084] The camera is configured with a dual-mode triggering mechanism, including timed sampling and event triggering. Timed sampling acquires clothing images at a preset sampling frequency. Event triggering includes clothing quantity triggering, offset triggering, and temperature change triggering. Clothing quantity triggering occurs when the number of clothes in the drying area changes; offset triggering occurs when the offset of the hanging position of the clothes in the drying area exceeds a preset offset threshold; and temperature change triggering occurs when the rate of change of the temperature distribution in the clothing area of ​​the drying area exceeds a preset change threshold. Once an event is triggered, the sampling frequency is increased to a preset high-frequency state, and then returns to the normal state after a period of time. The sampling frequency and related thresholds are set by those skilled in the art. The sampling frequency includes a normal state and a high-frequency state, with the normal state corresponding to timed sampling and the high-frequency state corresponding to event triggering.

[0085] The Gaussian filtering algorithm is used to remove environmental noise in the clothing images, such as dust and light reflection interference. Since the clothes may be slightly displaced due to wind or slight touch during the drying process, the scale-invariant feature transformation algorithm is used to align the clothing images at different sampling times.

[0086] Based on the denoised and aligned color image, the texture features and color saturation features of the clothing are extracted to form a visual feature map of the clothing.

[0087] Based on denoised and aligned infrared thermal imaging, an adaptive threshold segmentation algorithm is used to extract the temperature contour of clothing. The temperature value of each pixel within the contour is quantified to integrate the discrete temperature information and generate a temperature distribution matrix. The matrix corresponds to different parts of the clothing in coordinate form, presenting the temperature differences in different areas of the clothing. The visual feature map and the temperature distribution matrix are integrated to construct a clothing feature set.

[0088] Using the sampling frequency under normal conditions, environmental time-series data of the drying area are obtained, including temperature and humidity, wind speed, and light intensity curves. The environmental change rate is calculated through a sliding window. Once the environmental change rate exceeds the preset change threshold, the sampling frequency is increased to a high frequency state until the environmental change rate does not exceed the change threshold.

[0089] Based on an audio sensor and a camera, the YOLO target detection algorithm is used to analyze camera images in real time. Once a user target is detected in the clothes drying area, user behavior is immediately captured, including audio signals and human image frames. For audio signals, the Mel frequency cepstral coefficient feature extraction algorithm is used to extract voice keywords, such as "open," "lower," and "collect clothes." The Mel spectrum feature recognition algorithm is used to capture interactive audio, such as the collision sound of clothes and hangers, and folding sounds, forming an audio feature subset. For human image frames, the MediaPipe skeletal keypoint detection algorithm is used to extract the user's posture features in the clothes drying area. The user's movement path in the clothes drying area is tracked based on the coordinates of the human body center point in continuous frames. The YOLO lightweight model is used to identify the type of object held by the user, forming a behavioral feature subset. The audio feature subset and the behavioral feature subset are combined to construct an interaction feature set. The cameras mentioned in this application are all cameras equipped with a dual-mode triggering mechanism in the clothes drying rack.

[0090] By configuring the smart clothes rack to connect to the network, the system obtains weather forecasts issued by the local meteorological department, extracts environmental warnings for the future preset time period, including no warning, mild warning, and severe warning. For example, a sunny weather forecast corresponds to no warning, strong winds correspond to a mild warning, and rainfall corresponds to a severe warning. The system also quantifies environmental warnings.

[0091] Determine the current day / night time period, including daytime and nighttime, and simultaneously acquire environmental attributes entered by the user, such as orientation and whether the balcony is enclosed. Convert the environmental attributes into binary label features, combine them with the quantified environmental warning, and generate scene auxiliary features.

[0092] Based on time synchronization, the system integrates clothing feature sets, environmental time-series data, interaction feature sets, and scene auxiliary features. At the same time, it associates clothing features, environmental time-series data, and environmental attributes through area identifiers to form a scene feature set. Among them, the area identifier is the position code assigned to each hanging point on the horizontal bar of the smart clothes rack.

[0093] The parsing module is used to separate scene feature sets into individual items, generating parsing subsets for each garment. It integrates visual features, thermal imaging features, and environmental temporal features to calculate multidimensional drying probabilities. It then uses scene auxiliary features to fuse these multidimensional drying probabilities, generating collection probabilities and determining the drying status. This enables accurate quantification and labeling of the drying status of each garment. By aligning audio and behavioral temporal sequences, it constructs user behavior sequences to analyze and identify user intentions. It builds a collaborative relationship graph containing garment status, intentions, user status, and environmental nodes, and performs evolutionary predictions based on operational intentions, environmental changes, and drying intentions. This ensures that the drying service is proactive, efficient, and meets actual needs.

[0094] The parsing module includes a state determination unit and an associative reasoning unit;

[0095] The status determination unit is used to separate clothing from the scene feature set by using the area identifier, decompose it into multiple parsing subsets, each parsing subset corresponds to a single piece of clothing, and calculate the collection probability of each piece of clothing by combining visual features, thermal imaging features, and environmental time series features, thereby determining the dry status and forming a clothing status list.

[0096] The associative reasoning unit is used to construct user behavior sequences using temporal attention, determine user intent by combining clothing status list, thereby generating intent tags, and perform environmental trend analysis on scene auxiliary features and environmental temporal data. Based on the causal temporal changes of clothing status, user intent, and environmental changes, evolution prediction is performed to generate scene evolution trend tags.

[0097] Specifically, the steps for determining the drying state include:

[0098] Using the Canny edge detection algorithm, based on the boundary differences of texture and color saturation, the visual contours of all clothes are extracted from the visual feature map. Since the clothes in the drying area are all physically separated entities, there are gaps even when they are hung adjacently. Therefore, multiple visual contours are extracted, and each visual contour corresponds to the physical boundary of a piece of clothing. This initially achieves the differentiation of individual pieces of clothing from a visual perspective. Subsequent visual contours are based on any one of the multiple extracted visual contours.

[0099] Based on the spatial mask of the visual contour, the region in the temperature distribution matrix that completely overlaps with the coordinate range of the visual contour is targeted and locked. The temperature difference between the clothing and the background is used to perform threshold segmentation on the locked region, extract the temperature contour that only covers the corresponding clothing, and bind it to the corresponding visual contour.

[0100] The bound visual contour and temperature contour are superimposed on coordinates, and the coordinate intersection rate is calculated. Only when the coordinate intersection rate is not less than the confidence threshold is it confirmed that the visual contour and temperature contour belong to the same garment, thus avoiding the mis-extraction of the visual contour or temperature contour, such as misjudging the shadow of the drying rod as clothing.

[0101] The confirmed visual contour and temperature contour are fused at the pixel level to generate a composite contour. The outer boundary is based on the visual contour, and the internal features integrate the temperature distribution information of the temperature contour.

[0102] Set the coordinate range of all area markers in the drying area to be non-overlapping and non-intersecting. The coordinate range of each area marker only covers its corresponding single hanging point. This is to divide the coordinate range of the area markers into mutually exclusive parts, obtain the center coordinates of the composite contour, and use them as the contour center. Then, compare the contour center with the mutually exclusive coordinate ranges of all area markers one by one. Since the coordinate ranges are mutually exclusive, the contour center can only fall within the coordinate range of one area marker, thus obtaining the reference area marker.

[0103] Obtain the area threshold of the baseline area identifier and the outline area of ​​the composite outline, and classify the clothing category based on the area adaptability; if the outline area is not greater than the area area threshold, the composite outline is determined to be a single piece of regular clothing corresponding to the area identifier; if the outline area is greater than the area area threshold, it indicates that the clothing is a large item, occupying multiple adjacent hanging points. Traverse all area identifiers, extract all area identifiers that overlap with the composite outline, generate a joint area identifier, including the baseline area identifier and the associated area identifier, and determine it to be a single piece of oversized clothing.

[0104] For a single garment that has been categorized, specific features are extracted based on the scene feature set, including the visual feature map and temperature distribution matrix of the single garment. Combined with the visual contour, temperature contour, and composite contour, a baseline region identifier and an associated region identifier are bound to generate a parsed subset of the single garment. The baseline region and the associated region are collectively referred to as the relevant region.

[0105] Based on the parsed subset, texture and color saturation features of individual garments are extracted. Through feature weighted fusion, drying visual features are generated based on the initial feature weights. The initial feature weights are then modified by combining day and night time and environmental attributes from the scene auxiliary features to correct the drying visual features. For example, during the day, the south-facing drying area has strong sunlight, and the color saturation is easily distorted, so its weight needs to be reduced. At night, when there is no sunlight, the weight of texture features is increased. The initial feature weights are set based on historical data statistics.

[0106] A visual determination model is constructed using a CNN convolutional neural network model and a historical visual determination dataset. The model takes the corrected visual features of clothes drying as input and generates the visual dryness probability of the current subset of data. The historical visual determination dataset includes images of clothes drying in the past and their visual dryness probabilities.

[0107] Based on the temperature distribution of a single garment, the average surface temperature is obtained, and combined with the average environmental temperature in the time series data of the relevant area identifier in the parsed subset, the average temperature difference between the average surface temperature and the average environmental temperature is calculated.

[0108] Based on the physical characteristics of undried clothes evaporating and absorbing heat, and having a temperature lower than the environment, an imaging temperature difference threshold is set by combining historical data, and the imaging temperature difference threshold is adjusted based on scene auxiliary features. For example, if the ventilation in the closed area is weak, the evaporation and heat absorption effect of undried clothes is weak, and the imaging temperature difference threshold is corrected upward; if the ventilation in the open area is strong, the imaging temperature difference threshold is corrected downward.

[0109] By utilizing the logistic regression classification algorithm and historical imaging judgment dataset, a nonlinear mapping management of average temperature difference and thermal imaging dryness probability is constructed, thereby generating a temperature feature classification system. Based on the currently calculated average temperature difference, the thermal imaging dryness probability is generated. The historical imaging judgment dataset contains data on the average temperature difference between clothing and the environment under different clothing drying conditions in historical drying scenarios, including damp, slightly damp, and dry conditions, thermal imaging temperature distribution characteristics, and manually labeled corresponding thermal imaging dryness probability labels. This provides training samples for the logistic regression classification algorithm to learn the nonlinear relationship between average temperature difference and thermal imaging dryness probability.

[0110] Using an LSTM temporal neural network model and a historical environmental assessment dataset, a temporal assessment model is constructed to obtain the hanging time of a single garment. Combined with the environmental time-series data and environmental warnings of the relevant area corresponding to each garment, a temporal assessment input is constructed to generate the environmental dryness probability of the current subset. The historical environmental assessment dataset includes the hanging time of garments under historical drying conditions, environmental time-series data, environmental warnings, and environmental dryness probabilities. The temporal assessment model predicts the moisture evaporation rate of garments by simulating the garment drying curve, thereby predicting the environmental dryness probability.

[0111] Based on scene-assisted features, the weights of the drying probabilities of each dimension are configured, and the collection probability is calculated through weighted fusion. Combined with a preset probability mapping table, the collection probability is mapped to the corresponding drying state, including damp, slightly damp, and dry, and a clothing status list for each garment is generated, including the relevant area identifier, drying status, collection probability, and drying probability of each dimension of each garment. This achieves accurate quantification and labeling of the drying status of each garment. The probability mapping table is set based on common drying standards.

[0112] Specifically, the steps for determining user intent include:

[0113] For the subsets of audio features and the subsets of behavioral features, a time-series alignment algorithm is used to bind audio features and behavioral features with the same timestamp to construct a synchronization sequence;

[0114] By utilizing the long-short dependency capture mechanism, the features in the synchronization sequence are weighted and aggregated to extract sequence features. Based on the time order of each sequence feature, a user behavior sequence is generated, such as entering the drying area → walking towards a certain area marker → holding an empty clothes basket → staring at the clothes in the corresponding area → sending relevant keywords, in order to capture the coherence and logic of user behavior and avoid misjudgment caused by isolated analysis of a single action.

[0115] Based on the core requirements of the clothes-drying scenario, prior settings are made for scenario intents to generate an initial candidate intent set, including the intent to prepare to dry clothes, the intent to prepare to collect clothes, and the intent with no action. Simultaneously, based on prior probability estimation, the frequency of occurrence of each type of intent in different scenarios in the historical interaction dataset is statistically analyzed to configure a base probability for each type of candidate intent. If no historical data is available, it is initialized with equal probability for all three types of candidate intents. ;

[0116] The target area is identified by the region identifier corresponding to the user's movement path endpoint and gaze direction, and is matched with the relevant region identifiers existing in the parsing subset. If the region identifier of the target area exists in the parsing subset, the target area is determined to be successfully matched. At this time, the clothing status is combined with the clothing status list to generate the clothing dry status of the target area. If the region identifier of the target area is not in the parsing subset, the target area is determined to be unmatched, and it is marked as no clothing.

[0117] For successful matches only, a scene correction factor is configured based on scene auxiliary features, and the base probability of various candidate intentions is corrected to achieve scene adaptation of the base probability. For example, the scene correction factor for preparing to collect intentions in the evening is 1.2.

[0118] Key signals are extracted from the subset of audio features, and the audio features are enhanced based on the key signals. Specifically, if the sound of clothes being hung up, the clothesline being opened, or the sound of clothes colliding with the clothes hanger is detected, the probability of the intention to hang clothes is increased incrementally. If the sound of clothes being taken down, taken off, or clothes being folded is detected, the probability of the intention to take down clothes is increased incrementally. If unrelated audio is detected, such as irrelevant dialogue or environmental noise, no enhancement is performed.

[0119] Based on a multi-layer Transformer encoder architecture, an intent classification model is constructed, including a fusion layer, a mining layer, and a classification output layer. The fusion layer is used to receive user behavior sequences, the dryness status of a single garment, and scene auxiliary features, and performs dimensional unification and fusion. The mining layer extracts deep correlations between features through a multi-head attention mechanism. The classification output layer uses the Softmax activation function to output the probability distribution of the three types of intents. The historical interaction dataset is used as the training sample, and the Adam optimizer and cross-entropy loss function are used to train the intent classification model.

[0120] Using validated user behavior sequences as input, an intent classification model is used to generate probability distributions for three types of candidate intents, with the highest probability being taken as the intent confidence level. , The corresponding candidate intent is used as the target intent;

[0121] Set a second-level confidence threshold , ,and ;like Furthermore, if the target intent is to prepare to hang clothes to dry or to prepare to collect them, then the user intent tag is determined to be a clear operational intent. The user intent tag is then bound to the relevant area identifier of the target clothing to generate an intent guidance set, which includes the target intent, scene auxiliary features, relevant area identifiers, target area, and clothing drying status. Among these, the user intent tag is used to characterize the level of intent certainty.

[0122] like If the target intent is to prepare to hang clothes to dry or to prepare to collect them, then the user intent tag is determined to indicate that there is an operational intent.

[0123] like If the target intent is no operational intent, then the user intent label is determined to be a non-operational intent.

[0124] Specifically, the steps of evolution prediction include:

[0125] Evolutionary nodes are set up, including clothing status nodes, intent nodes, user status nodes, and environment nodes. Clothing status nodes are set through a clothing status list, intent nodes are set through target intents, environment nodes are set through environmental time-series data and scene auxiliary features, and user status nodes are set through user intent tags. Causal association mining is used, based on the physical laws of the drying scene and user behavior logic, to establish directed causal edges between nodes. The weight of the edge represents the association strength. For example, if the user's intention to collect the clothes is followed by the drying status and the environment remains unchanged, a collaborative relationship graph is constructed using a GNN graph neural network to provide causal relationships for evolutionary prediction and avoid isolated predictions that are detached from scene logic.

[0126] Evolutionary prediction based on operational intent is used to anticipate the timing of user collection actions. This includes: filtering out scenarios with clear operational intent and the target intent being preparation for collection, while confirming that the corresponding target clothing is dry, which meets the prerequisite for collection; and verifying the existence of the association logic in the collaborative relationship graph. For example, if the weight of the causal edge of preparing for collection - drying clothing - no sudden change in environment exceeds the preset association threshold, then the association logic is valid. Based on the environmental time series data of the target area and the time interval of the user behavior sequence, an attention-based LSTM model is used to predict the probability distribution of the user performing the collection operation within a set time window. The time point corresponding to the maximum probability is taken as the predicted collection time. For the remaining clothing after the predicted collection, it is not dry at this time. The collaborative relationship graph is used to match the environmental nodes corresponding to the area identifiers of each remaining clothing area, analyze the matching degree between the dryness state of each remaining clothing and the corresponding area environmental data, and filter out the area with the highest matching degree to generate the drying optimization requirements, including the clothing area identifiers that need to be adjusted and the target optimization area identifiers, such as adjusting the clothing in area identifier 6 to area identifier 8.

[0127] Based on environmental changes, the system predicts evolution and provides environmental warnings for non-dry clothing within a preset time period using scene-assisted features. It also verifies risk associations using a collaborative relationship graph. If the causal edge weights of rainfall warning, non-dry clothing, and open drying area exceed the association threshold, then the risk association is established. Logistic regression is used to calculate the probability of non-dry clothing getting wet in the rain. High-risk clothing is selected by using the risk threshold and then adjusted in conjunction with user intent tags, such as executing a rain shelter operation.

[0128] Evolutionary predictions are made based on the intention to dry clothes. Scenarios with clear operational intentions and target intentions of preparing to dry clothes are selected. Based on a subset of behavioral features, it is confirmed that the item held by the user is wet clothes. The association logic is verified using a collaborative relationship graph. Using an LSTM time series model that incorporates Fick's law, the drying curve of wet clothes in the current environment is simulated to generate the amount of water evaporated per unit time as a drying rate indicator. Based on the prediction results of the drying rate indicator, combined with the association logic between environmental nodes and clothing state nodes in the collaborative relationship graph, a decision scheme to improve drying efficiency is generated.

[0129] Based on scenario evolution, key time nodes and corresponding associated parameters are obtained, such as predicted collection time, predicted rainfall time, and predicted drying time. Combined with the collaborative relationship diagram, a predicted evolution table is generated.

[0130] The decision control module receives the target intent from the intent-guided set, performs intent-guided control for clear operation intents, obtains the target height of the drying pole for the intent to collect clothes, generates adjustment instructions, and performs supplementary lighting operations in conjunction with auxiliary lighting, obtains the target hanging spacing for the intent to hang clothes, generates a lowering instruction, uses contour detection to confirm the hanging status of the clothes, and updates the scene feature set.

[0131] Specifically, the steps of intent-oriented control include:

[0132] Read the intent-guided set, including the target intent, scene auxiliary features, relevant area identifiers, target area, and clothing dryness status;

[0133] If the target intent is to collect, based on the user's personalized data and environmental attributes in the scene auxiliary features, the height and the range of the drying rack height are obtained, and a safety redundancy height is set to avoid the user from raising their arms or bending over excessively. Based on the sum of the height and the safety redundancy height, the target height of the drying rack is calculated, and the target height of the drying rack is compared with the range of the drying rack height. If the target height of the drying rack exceeds the range of the drying rack height, the closest value within the range of the drying rack height is used as the target height of the drying rack, such as the upper or lower limit of the range of the drying rack height. Otherwise, the original target height of the drying rack is maintained. Among them, the user's personalized data is the height data and operating habits entered by the user through the APP. If there is no pre-entered data, the default parameters are used. The default parameters are set by the technical staff.

[0134] Based on position servo control, the target height of the drying rack is converted into a pulse signal that the drying rack control device can recognize, such as the rotation angle of the drying rack lifting motor, thereby generating adjustment commands, including control codes, target height of the drying rack, and lifting speed;

[0135] The motor that drives the clothesline lifting mechanism performs adjustment actions. At the same time, the encoder built into the motor collects the actual height of the clothesline in real time and calculates the absolute deviation between the actual height of the clothesline and the target height of the clothesline for deviation verification.

[0136] If the absolute deviation is not greater than the preset deviation threshold, the adjustment is considered complete; if the absolute deviation is greater than the deviation threshold, a second fine-tuning is performed using the PID control algorithm until the deviation requirement is met.

[0137] The system acquires the current light intensity from environmental time-series data and the day / night time period from scene auxiliary features, and sets a lighting activation threshold to determine whether to activate auxiliary lighting linkage. Auxiliary lighting linkage is triggered only when the day / night time period is night or the current light intensity is less than the lighting activation threshold, indicating insufficient light. This triggers a supplementary lighting command, including supplementary lights for relevant areas and supplementary light brightness. The supplementary light brightness is set based on the current light intensity and supplementary light compensation.

[0138] After the supplemental lighting is activated, the light intensity of the relevant area is collected in real time to ensure that the light intensity after supplemental lighting is greater than the lighting activation threshold, and the supplemental lighting is maintained until the clothes are collected.

[0139] Once clothing collection is complete, the camera monitors the clothing outline in the relevant area in real time. The composite outline of the target clothing is used as a comparison benchmark to calculate the outline similarity. A disappearance judgment interval is set. If the outline similarity is less than the preset outline judgment threshold within the disappearance judgment interval, the clothing is determined to have been removed. The collection is then confirmed to be complete, a status update instruction is generated, the dry status corresponding to the relevant area is marked as empty, and the collection completion timestamp is recorded simultaneously to update the scene feature set.

[0140] If the target intention is to prepare to dry clothes, obtain the current environmental parameters from the environmental time series data, including the current wind speed, the current light intensity, and the drying direction, the spacing between drying poles, and the spacing between hanging points from the scene auxiliary features; among them, the spacing between drying poles only exists when there are two or more drying poles, and the spacing between hanging points is determined based on the layout of the drying hardware;

[0141] Based on the influence of current environmental parameters and clothing drying, the target hanging spacing is calculated using a multiple of the hanging point spacing, and then fed back to the control panel. If the current environmental parameters are less than the preset environmental threshold, the drying conditions are poor, and the target hanging spacing needs to be increased. If the current environmental parameters are not less than the environmental threshold, the drying conditions are good, and the hanging point spacing is used as the target hanging spacing to save drying space.

[0142] Based on a subset of behavioral features, the user's movement path is analyzed to confirm that the user is approaching the target area. At the same time, the infrared sensor in the target area is activated to collect sensor data in real time. When the user's hand-held object, such as wet clothing, is detected and enters the preset sensing range, a command to slowly lower the drying rack is triggered. The lowering height is dynamically adjusted according to the real-time distance between the user and the drying rack. The preset sensing range is a distance range set by the technicians, such as 30-50cm in front of the target area.

[0143] When the clothesline descends to the target height, a descent stop signal is triggered, and the user is informed via voice prompt that the clothes can be hung. After the clothesline stops descending, the image of the clothes in the target area is monitored in real time and compared with the image before and during the hanging process. If a new composite outline is detected, the clothes are determined to be hung. If no new composite outline is detected, the monitoring continues.

[0144] Once the hanging is confirmed, a collection association instruction is generated, which binds the new clothing to the target area identifier, synchronously associates the collected initial visual feature map and temperature distribution matrix of the new clothing, updates the scene feature set, and generates a new parsing subset for a single piece of clothing.

[0145] Working principle and effects:

[0146] By collecting multi-dimensional data of the drying area and associating it with scene auxiliary data to generate a scene feature set, and then separating it into individual items to obtain an analytical subset, the drying status of clothes is determined by integrating visual, thermal imaging and environmental temporal features. User intent is identified through audio-video-behavioral temporal alignment, and an evolution prediction is completed by constructing a multi-node collaborative relationship graph. Based on clear operational intent, an intent-guided set is constructed to achieve targeted control. This solves the problems of insufficient deep semantic understanding of the scene and inaccurate judgment of state and intent in traditional solutions. It upgrades the clothes rack from basic environmental detection to a personalized service that actively adapts to needs, effectively improving the convenience of use and promoting the intelligent and humanized upgrade of smart clothes racks.

[0147] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An adaptive control system based on clothing status recognition, characterized in that, include: The module consists of a perception module, an analysis module, and a decision control module. The sensing module is used to collect visual perception data in the drying area, and combine it with scene auxiliary data to perform feature association extraction and generate a scene feature set; The parsing module is used to separate scene feature sets into individual items, generate parsing subsets for individual garments, integrate visual features, thermal imaging features, and environmental temporal features to calculate multidimensional drying probability and determine drying status. Through audio and behavioral temporal alignment, it analyzes and identifies user intent, constructs a collaborative relationship graph containing garment status, intent, user status, and environmental nodes, and performs evolution prediction. Among these, user intent includes explicit operational intent, and an intent-guided set is constructed for explicit operational intent. The decision control module is used to read the intent guidance set in the parsing module and perform intent guidance control based on the target intent.

2. The adaptive control system based on clothing status recognition according to claim 1, characterized in that, The steps of feature association extraction include: Based on the configured camera, images of clothes in the drying area are acquired, including raw color images and infrared thermal images; Configure the camera with a dual-mode triggering mechanism, including timed sampling and event triggering. Event triggering includes clothing quantity triggering, offset triggering, and temperature change triggering. The Gaussian filtering algorithm is used to remove environmental noise from clothing images, and the scale-invariant feature transformation algorithm is used to align clothing images at different sampling times. Based on the denoised and aligned color image, the texture features and color saturation features of the clothing are extracted to form a visual feature map of the clothing. Based on denoised and aligned infrared thermal imaging, an adaptive threshold segmentation algorithm is used to extract the temperature profile of clothing, and a temperature distribution matrix is ​​generated by quantizing the temperature of each pixel. Integrate the visual feature map and the temperature distribution matrix to construct a clothing feature set.

3. The adaptive control system based on clothing status recognition according to claim 2, characterized in that, The steps of feature association extraction also include: Acquire environmental time-series data of the drying area, including temperature, humidity, wind speed, and light intensity curves; calculate the rate of environmental change through a sliding window; and adjust the sampling frequency based on the change threshold. Based on the audio sensor and the camera, the camera images are analyzed in real time to capture user behavior, including audio signals and human image frames; Obtain subsets of audio features and behavioral features to generate an interaction feature set; Get and quantify environmental warnings for a future preset time period, get the current day / night time and environmental attributes, convert them into label features, and generate scene auxiliary features in combination with the environmental warnings; A scene feature set is constructed based on time synchronization and regional identification.

4. The adaptive control system based on clothing status recognition according to claim 3, characterized in that: The parsing module includes a state determination unit and an association reasoning unit; The state determination unit is used to separate clothing from the scene feature set, calculate the probability of collecting a single piece of clothing by combining visual features, thermal imaging features, and environmental time series features, determine the dry state, and form a clothing state list. The association reasoning unit is used to construct a user behavior sequence, determine the user's intent by combining the clothing status list, generate intent tags, perform environmental trend analysis on the scene auxiliary features and the environmental time series data, and perform evolution prediction by combining clothing status and user intent to generate scene evolution trend tags.

5. The adaptive control system based on clothing status recognition according to claim 4, characterized in that, The steps for determining the drying state include: The visual outline of the clothing is extracted from the visual feature map using the Canny edge detection algorithm. Based on the spatial mask of the visual contour, the overlapping regions in the temperature distribution matrix are targeted and thresholded to extract the temperature contour. Calculate the coordinate intersection rate. Only when the coordinate intersection rate is not less than the confidence threshold is it determined that the visual contour and the temperature contour belong to the same garment, and pixel-level fusion is performed to generate a composite contour. The coordinate range of the region identifier is divided into mutually exclusive parts to obtain the outline center, and the reference region identifier is obtained by comparing them one by one. Obtain the region area threshold and outline area to classify clothing categories; For a single garment that has been categorized, corresponding exclusive features are extracted based on the scene feature set to generate a parsed subset for that garment.

6. The adaptive control system based on clothing status recognition according to claim 5, characterized in that, The steps for determining the drying state also include: Based on the aforementioned parsed subset, texture features and color saturation features are weighted and fused to generate drying visual features, which are then corrected by combining scene auxiliary features. Using a visual judgment model, with the aforementioned drying visual features as input, a visual dryness probability is generated; Obtain the average surface temperature of a single garment and the average ambient temperature, and calculate the average temperature difference. Set the imaging temperature difference threshold and adjust it based on scene auxiliary features; Construct a temperature feature classification system and generate thermal imaging dryness probability based on the currently calculated average temperature difference; Using a time-series determination model, the probability of an environment being dry is generated by taking the hanging time of a single garment, environmental time-series data, and environmental early warning as inputs. Based on scene-assisted features, the weights of the drying probabilities of each dimension are configured, the collection probability is calculated, and the drying status is generated using a probability mapping table, thereby generating a clothing status list for each garment.

7. The adaptive control system based on clothing status recognition according to claim 6, characterized in that, The steps for determining user intent include: For the subsets of audio features and the subsets of behavioral features, a synchronization sequence is constructed through temporal alignment; By utilizing a long-short dependency capture mechanism, sequence features are extracted, and user behavior sequences are generated based on time. Pre-defined scene intents are used to generate an initial candidate intent set. Based on prior probability estimation, and combined with historical interaction datasets, base probabilities are configured for various candidate intents. Set the target region and match it with the region identifiers that exist in the parsed subset; Configure scene correction factors and correct the base probabilities of various candidate intentions; Extract key signals from a subset of audio features, and perform intent enhancement on the audio features based on the key signals; Using an intent classification model and user behavior sequences, probability distributions of three types of candidate intents are generated. The highest probability is taken as the intent confidence level, and combined with a secondary confidence threshold, the user intent label is determined.

8. The adaptive control system based on clothing state recognition according to claim 7, characterized in that, The steps involved in evolutionary prediction include: Set up evolution nodes, including clothing state nodes, intent nodes, user state nodes, and environment nodes, and construct a collaborative relationship graph by combining directed causal edges; Based on the operational intent, an evolution prediction is made to obtain the predicted collection time. For the remaining clothes after the predicted collection, the drying optimization requirements are generated through the collaborative relationship graph. Based on environmental changes, the system predicts evolution and targets non-dry clothing. Using scene-assisted features, it obtains environmental warnings for a preset time period in the future, calculates the probability of non-dry clothing getting wet in the rain, and makes corresponding adjustments. Evolutionary prediction is performed based on the intention to dry clothes. Scenarios with clear operational intentions and target intentions of preparing to dry clothes are selected. The drying curve of damp clothes in the current environment is simulated to generate the amount of water evaporation per unit time and generate decision-making schemes. Based on scenario evolution, key time nodes and corresponding associated parameters are obtained, and a predictive evolution table is generated by combining the collaborative relationship diagram.

9. The adaptive control system based on clothing state recognition according to claim 8, characterized in that, The steps of intent-guided control include: If the target's intention is to collect, obtain their height and the range of the drying pole's height. Set a safety redundancy height, calculate the target height of the drying rack, and adjust it according to the range of drying rack heights; The target height of the drying rack is converted into a pulse signal to generate an adjustment command; The actual height of the drying rack is collected in real time, and the absolute deviation is calculated by combining it with the target height of the drying rack. Once the absolute deviation exceeds the deviation threshold, a secondary fine-tuning is performed using a PID control algorithm. Set a lighting activation threshold, combine the environmental time series data with the scene auxiliary features, perform auxiliary lighting linkage, and generate supplementary lighting commands; Real-time monitoring of target clothing, combined with composite contours to calculate contour similarity; Once the contour similarity is less than the contour determination threshold within the disappearance determination interval, it is determined that the clothing has been removed, a state update instruction is generated, and the scene feature set is updated.

10. The adaptive control system based on clothing state recognition according to claim 9, characterized in that, The steps of intent-guided control also include: If the target's intention is to prepare for drying clothes, calculate the target's hanging distance based on the current environmental parameters and the influence of clothes drying. Based on the aforementioned subset of behavioral features, the user's movement path is analyzed to confirm that the user is approaching the target area. Simultaneously, the infrared sensing device in the target area is activated to collect sensing data in real time. Once the user's handheld object enters the preset sensing range, the command to lower the drying rack is triggered. When the clothesline descends to the target height, a descent stop signal is triggered, and the images of the clothes in the target area are monitored in real time, comparing the images before and during the hanging process. If a new composite outline is detected, the garment hanging is considered complete; otherwise, monitoring continues. Once the hanging is confirmed, a collection association instruction is generated, the scene feature set is updated, and a new parsing subset for each piece of clothing is generated.

Images