A self-learning robot system and method based on real-time thermal comfort monitoring

Abstract

The application relates to a self-learning robot system based on real-time thermal comfort monitoring, comprising a mobile robot platform, an anti-interference human body perception module, a multi-target feature construction module, a public interaction and individualized parameter module, a semantic reasoning module, an algorithm adaptive individualized thermal comfort prediction module and an online learning and model updating module. The anti-interference human body perception module identifies real human body targets; the multi-target feature construction module performs target consistency correlation and establishes a multi-modal human body feature set; the public interaction and individualized parameter module obtains current thermal sensation feedback, and calls an individualized thermal comfort parameter set in response to an individualized thermal comfort service request; the semantic reasoning module outputs structured thermal comfort semantic features; the algorithm adaptive individualized thermal comfort prediction module outputs real-time thermal sensation prediction; and the online learning and model updating module performs incremental updating. The application can accurately reflect the real thermal sensation of the current interactive personnel.

Description

Technical Field

[0001] This invention relates to the fields of indoor environmental monitoring and robot adaptive learning technology, and in particular to a self-learning robot system and method based on real-time thermal comfort monitoring. Background Technology

[0002] Thermal comfort is a crucial factor influencing the indoor environmental experience of building users. Existing building thermal environment assessments typically employ models such as Predicted Mean Vote (PMV), calculating the average thermal sensation of a group based on parameters like air temperature, mean radiant temperature, relative humidity, air velocity, human metabolic rate, and clothing thermal resistance. While these methods provide a foundation for traditional indoor thermal environment assessments, they primarily rely on assumptions of a relatively stable environment and group average response. However, in large public buildings like shopping malls, the indoor thermal environment and occupant status exhibit significant spatial and temporal dynamics. On one hand, local heat sources, air conditioning airflow organization, different spatial locations, and occupant densities lead to continuous changes in the actual thermal environment; on the other hand, there are significant differences in clothing, activity levels, behavioral states, spatial locations, and long-term thermal preferences among different individuals. Therefore, relying solely on fixed-location environmental sensors or group average models is insufficient to accurately reflect the true thermal sensation of specific individuals at specific times and locations. The main limitation of group steady-state methods like PMV in dynamic public spaces lies in their inability to reflect individual feedback, short-term occupant changes, and behavioral differences.

[0003] While existing computer vision and multimodal perception technologies can acquire information such as clothing, posture, behavior, and body surface temperature through RGB, depth information, and thermal imaging, they still face the following challenges in real public buildings: First, complex visual interference. Shopping malls are plagued by human models, advertisements, glass reflections, local overexposure, and non-uniform lighting. During peak hours, the large number of overlapping, occluding, and rapidly crossing people can easily lead to false detections, missed detections, and cross-frame target association switching. Second, maintaining target consistency is difficult. In dynamic environments with multiple people, if the information on clothing, behavior, posture, and position obtained from a single frame cannot be continuously associated with the corresponding human target in consecutive frames, it will cause mixing of different target features, affecting subsequent thermal comfort predictions. Third, there is a lack of effective fusion between subjective feedback and objective perception information. A person's actual thermal sensation is not only related to the environment but also influenced by factors such as clothing, activity, historical thermal preferences, and current subjective feelings. Relying solely on environmental parameters or visual features is insufficient to establish a stable thermal comfort representation. Fourth, most existing thermal comfort models are offline static models. Once the model is trained, its parameters are basically fixed. When personnel composition, environmental conditions, personnel behavior, or individual preferences change over time, the model is difficult to adjust in a timely manner, and long-term use can easily lead to prediction bias. Fifth, fixed sensing systems have limited spatial coverage. A single or a small number of fixed sensors can only reflect the environment of a local measurement point, making it difficult to continuously reflect the status of personnel and changes in the thermal environment in different spaces; increasing the number of fixed devices will increase the complexity of system deployment and maintenance.

[0004] Therefore, there is a need for a real-time thermal comfort monitoring technology that can autonomously move and sense in real and complex public spaces, resist visual environmental interference and human occlusion, maintain the consistency of continuous frame target and multimodal human body information, obtain subjective thermal sensations through public active interaction, and update the prediction model based on continuous feedback.

[0005] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The technical problem addressed by this application is: "In real dynamic public indoor spaces, traditional thermal comfort monitoring methods struggle to maintain a correct correspondence between human body state characteristics and current thermal sensation feedback under complex visual interference and dynamic changes in multiple people, making it difficult to provide a reliable basis for individualized environmental control."

[0007] The technical solution adopted in this application to solve the above-mentioned technical problems is as follows.

[0008] This invention provides a self-learning robot system based on real-time thermal comfort monitoring, comprising: The mobile robot platform is configured to move autonomously in the public indoor space to be monitored. The mobile robot platform is equipped with environmental sensing devices, visual sensing devices, infrared thermal imaging devices, and human-computer interaction terminals. An anti-interference human body perception module is configured to identify real human targets from video frames acquired by a visual sensing device. The multi-target feature construction module is configured to perform multi-person pose estimation and cross-frame target association on real human targets. It forms a continuous frame target sequence based on the confidence of human keypoints in continuous video frames, current spatial position, pose similarity, target bounding box features, and temporal continuity, and establishes a multimodal human feature set with target consistency for each real human target. The multimodal human feature set includes human pose features, clothing status, behavioral status, current spatial position and its corresponding environmental parameters, and human surface temperature-related features extracted based on infrared thermal images. Among them, the environmental parameters are obtained based on environmental sensing devices, and the infrared thermal images are obtained based on infrared thermal imaging devices. The public interaction and personalized parameter module is configured to obtain the current thermal sensation feedback actively submitted by the user through the human-computer interaction terminal; in response to the personalized thermal comfort service request initiated by the user through the human-computer interaction terminal, it obtains the interaction identifier provided or confirmed by the user, and calls the personalized thermal comfort parameter set corresponding to the interaction identifier. The personalized thermal comfort parameter set includes historical thermal sensation feedback, individual thermal preferences, historical interaction records and individual deviation parameters. The semantic reasoning module is configured to encode the multimodal human feature set and the current thermal sensation feedback into structured input, and add the personalized thermal comfort parameter set into the structured input when the user initiates a personalized thermal comfort service request; it uses a large language model to perform semantic consistency analysis and cross-modal information fusion on the structured input, and outputs structured thermal comfort semantic features; The algorithm-adaptive personalized thermal comfort prediction module is configured as follows: it takes environmental parameters and structured thermal comfort semantic features as basic inputs, and uses them as additional inputs when a personalized thermal comfort parameter set exists, outputting a real-time thermal sensation prediction for the current user. The real-time thermal sensation prediction is obtained by superimposing a basic prediction with an individual deviation parameter. The basic prediction is the predicted value output by this module without the individual deviation parameter. When the personalized thermal comfort parameter set corresponding to the interaction identifier is not invoked, the individual deviation parameter is zero. When the personalized thermal comfort parameter set corresponding to the interaction identifier is invoked, the individual deviation parameter is determined based on the deviation between historical thermal sensation feedback and the corresponding basic prediction. Its initial value is zero, and it is continuously updated based on the thermal sensation feedback subsequently submitted by the user. The online learning and model update module is configured to use newly acquired environmental parameters, multimodal human feature sets, and thermal sensation feedback confirmed by the interacting personnel to form training samples, and incrementally update the model parameters used to generate basic predictions in the algorithm adaptive personalized thermal comfort prediction module, so as to reduce the basic prediction error in subsequent predictions.

[0009] In some embodiments, the anti-interference human body sensing module includes: An anti-interference human detection unit is configured to perform initial human detection on video frames and obtain initial detection results including real human candidate regions and interference candidate regions; the interference candidate regions include at least one of human models, human advertisements, and human images formed by glass reflection. The difficult background suppression unit is configured to define the interference candidate region as a set of difficult negative samples and construct a difficult background weighted loss based on the set of difficult negative samples, so that the interference candidate region receives a higher misclassification penalty than the background region other than the set of difficult negative samples, thereby filtering out the interference candidate region from the initial detection results. The spatial consistency constraint unit for crowded scenes is configured as follows: when the overlap between adjacent detection boxes in the current frame image exceeds a preset overlap threshold, the current scene is determined to be a crowded environment; when the current scene is not a crowded environment, a first IoU threshold is used as an admission condition for spatial filtering of real human candidate regions in the current frame; when the current scene is a crowded environment, a second IoU threshold is used as an admission condition for spatial filtering of real human candidate regions in the current frame, wherein the second IoU threshold is greater than the first IoU threshold, and real human candidate regions that pass the spatial filtering are retained as human regions to be segmented. The human body instance segmentation unit is configured to perform pixel-level instance segmentation on the human body region to be segmented, so as to correct the human body boundary and remove non-human body pixels inside the human body detection box, thereby obtaining the real human body target.

[0010] In some embodiments, the multi-target feature construction module performs cross-frame target association through a multi-stage target consistency matching mechanism, which includes: The first stage is cross-frame candidate association: based on the current spatial location and temporal continuity, candidate real human targets corresponding to the target sequence of the previous continuous frame are selected from the real human targets in the current frame. The second stage is human pose consistency matching: using the confidence of human key points, the spatial position of key points and the overall pose similarity, the degree of pose consistency between the candidate real human target and the target sequence of consecutive frames in the previous moment is judged. The third stage is target appearance consistency matching: using the clothing status, human body contour features and target box features of the candidate real human target, the appearance consistency between the candidate real human target and the target sequence of consecutive frames in the previous time step is determined. The fourth stage, multimodal feature binding, assigns or extends temporary target identifiers to real human targets matched in the first three stages, uses the temporary target identifiers as the association index in the current monitoring process, and binds the multimodal human feature set to the corresponding real human target.

[0011] In some embodiments, the mobile robot platform is configured to move autonomously within the public indoor space to be monitored according to a preset inspection route and stay at preset key monitoring locations for a predetermined time.

[0012] In some embodiments, the human-computer interaction terminal includes at least one of a voice interaction terminal and a touch interaction terminal; the current thermal sensation feedback includes a standardized thermal sensation vote (TSV), represented by a preset graded thermal sensation scale; the personalized thermal comfort parameter set includes thermal preference information, interaction record information, and individual bias parameters; wherein, the thermal preference information includes historical thermal sensation feedback and individual thermal preference, the historical thermal sensation feedback being the standardized thermal sensation vote (TSV) submitted by the user in previous active interactions, the interaction record information including the time and interaction method of each active interaction, and the individual thermal preference being the user's preference tendency relative to the group average thermal sensation; the interaction identifier is used to invoke the corresponding personalized thermal comfort parameter set when the user initiates a personalized thermal comfort service request.

[0013] In some embodiments, the semantic reasoning module is further configured as follows: Structured input is encoded in JSON, key-value pairs, table structures, or data formats with equivalent field constraints; Event-triggered inference is used instead of continuous execution for each video frame; for the same consecutive frame target sequence, a minimum time interval is set between two adjacent large language model calls. When the preset triggering conditions are met and the minimum time interval has expired, the large language model is invoked. The preset triggering conditions include at least one of the following: receiving new current thermal sensation feedback, changes in human posture features in the multimodal human feature set, changes in clothing status in the multimodal human feature set, changes in behavioral status in the multimodal human feature set, changes in the current spatial position in the multimodal human feature set exceeding a preset spatial displacement threshold, or the previously generated structured thermal comfort semantic features are continuously reused for a preset reuse duration or a preset number of reuses. When the preset triggering conditions are not met, the structured thermal comfort semantic features generated by the semantic reasoning module in the previous call are reused.

[0014] In some embodiments, the algorithm-adaptive personalized thermal comfort prediction module is further configured as follows: The basic prediction is generated by a regression tree ensemble model, which takes environmental parameters and structured thermal comfort semantic features as basic inputs, and uses them as additional inputs when there is a set of personalized thermal comfort parameters. The basic prediction is output by weighted summation of multiple regression tree functions. Individual deviation parameters Updated using an exponential moving average method:

[0015] in, This is the current thermal feedback actively submitted by the user at time t. This is the basic prediction for the corresponding time. For smoothing coefficients; If the set of personalized thermal comfort parameters corresponding to the interactive identifier has not been updated within a preset time, time decay is applied to the individual deviation parameters in it, so that the individual deviation gradually approaches zero as the time without updating increases; Based on the statistical correlation between changes in environmental parameters within a preset sliding window and the real-time thermal sensing prediction error, the influence intensity of the current environmental conditions on the real-time thermal sensing prediction error is dynamically determined, and the contribution of the training samples within the preset sliding window to the incremental update process of the model parameters used to generate the basic prediction is adjusted accordingly. The real-time thermal sensing prediction error is the difference between the current thermal sensing feedback and the real-time thermal sensing prediction. Calculate prediction confidence

[0016]

[0017] in The variance is calculated based on the real-time thermal sensing prediction error statistics within a preset sliding window. ε is a positive number set to prevent the denominator from being zero. The prediction confidence level is used to characterize the stability and reliability of the real-time thermal sensing prediction.

[0018] In some embodiments, the public interaction and personalized parameter module is further configured to: provide a temporary interaction path for interactive personnel who have not initiated a personalized thermal comfort service request; in the temporary interaction path, assign a temporary interaction identifier to the current interaction, and associate the current multimodal human feature set, environmental parameters and current thermal sensation feedback based on the temporary interaction identifier.

[0019] In some embodiments, an output interface is also included, configured to send real-time thermal sensation predictions to the environmental control system of the public indoor space to be monitored, so as to provide data and decision-making basis for the environmental control of the public indoor space to be monitored.

[0020] In some embodiments, a self-learning robot monitoring method based on real-time thermal comfort monitoring is also provided, applied to a self-learning robot system based on real-time thermal comfort monitoring. The real-time thermal comfort monitoring self-learning robot system includes a mobile robot platform, an anti-interference human perception module, a multi-target feature construction module, a public interaction and personalized parameter module, a semantic reasoning module, an algorithm-adaptive personalized thermal comfort prediction module, and an online learning and model update module. The real-time thermal comfort monitoring self-learning robot monitoring method includes: Control the mobile robot platform to move autonomously within the public indoor space to be monitored; The anti-interference human perception module identifies real human targets from video frames acquired by the visual sensing device. A multi-target feature construction module performs multi-person pose estimation and cross-frame target association on real human targets. Based on the confidence of human keypoints in continuous video frames, current spatial position, pose similarity, target bounding box features, and temporal continuity, a continuous frame target sequence is formed, and a multimodal human feature set with target consistency is established for each real human target. The multimodal human feature set includes human pose features, clothing status, behavioral status, current spatial position and its corresponding environmental parameters, as well as human surface temperature-related features extracted based on infrared thermal images. Among them, the environmental parameters are obtained based on the environmental sensing device on the mobile robot platform, and the infrared thermal images are obtained based on the infrared thermal imaging device on the mobile robot platform. The system obtains current thermal sensation feedback actively submitted by the user through the human-computer interaction terminal on the mobile robot platform via the public interaction and personalized parameter module; in response to the personalized thermal comfort service request initiated by the user through the human-computer interaction terminal, it obtains the interaction identifier provided or confirmed by the user, and calls the personalized thermal comfort parameter set corresponding to the interaction identifier. The personalized thermal comfort parameter set includes historical thermal sensation feedback, individual thermal preferences, historical interaction records and individual deviation parameters. The semantic reasoning module encodes the multimodal human feature set and the current thermal sensation feedback into structured input, and adds the personalized thermal comfort parameter set into the structured input when the user initiates a personalized thermal comfort service request; the structured input is then analyzed for semantic consistency and fused with cross-modal information using a large language model to output structured thermal comfort semantic features; The algorithm-adaptive personalized thermal comfort prediction module takes environmental parameters and structured thermal comfort semantic features as basic inputs, and uses them as additional inputs when a personalized thermal comfort parameter set exists, outputting a real-time thermal sensation prediction for the current user. The real-time thermal sensation prediction is obtained by superimposing a basic prediction with an individual deviation parameter. The basic prediction is the predicted value output by the module without the individual deviation parameter. When the personalized thermal comfort parameter set corresponding to the interaction identifier is not invoked, the individual deviation parameter is zero. When the personalized thermal comfort parameter set corresponding to the interaction identifier is invoked, the individual deviation parameter is determined based on the deviation between historical thermal sensation feedback and the corresponding basic prediction. Its initial value is zero, and it is continuously updated based on subsequent active thermal sensation feedback submitted by the user. The online learning and model update module uses newly acquired environmental parameters, multimodal human feature sets, and thermal sensation feedback confirmed by the interacting personnel to form training samples. This incrementally updates the model parameters used to generate basic predictions in the algorithm's adaptive personalized thermal comfort prediction module, thereby reducing the basic prediction error in subsequent predictions.

[0021] The present invention has the following beneficial effects: This invention provides a self-learning robot system based on real-time thermal comfort monitoring. The system autonomously moves within the monitored public indoor space via a mobile robot platform, carrying environmental sensors, visual sensors, infrared thermal imaging devices, and a human-computer interaction terminal, overcoming the spatial coverage limitations of fixed sensors. An anti-interference human perception module identifies real human targets from video frames acquired by the visual sensors, reducing misidentification caused by visual interference such as human models, advertisements, reflections, and dense crowds in public indoor spaces. A multi-target feature construction module performs multi-person pose estimation and cross-frame target association on real human targets. Based on the confidence level, spatial location, pose similarity, bounding box features, and temporal continuity of human key points in continuous video frames, a continuous frame target sequence is formed, establishing a multimodal human feature set with target consistency, ensuring that human state features in dynamic multi-person scenarios are correctly associated with corresponding targets. A public interaction and personalized parameter module acquires current thermal sensation feedback actively submitted by interacting personnel and integrates this feedback with the interacting personnel's input. When a personalized thermal comfort service request is initiated, the system calls the personalized thermal comfort parameter set corresponding to the interaction identifier to provide continuous feedback information for personalized prediction. The semantic reasoning module encodes the multimodal human feature set, the personalized thermal comfort parameter set, and the current thermal sensation feedback into structured input, and uses a large language model for semantic consistency analysis and cross-modal information fusion to output structured thermal comfort semantic features, improving the reliability and completeness of the input features. The algorithm-adaptive personalized thermal comfort prediction module superimposes the basic prediction with individual deviation parameters to obtain a real-time thermal sensation prediction, and updates the individual deviation parameters based on subsequent active thermal sensation feedback submitted by the user. The online learning and model update module uses newly acquired environmental parameters, the multimodal human feature set, and valid thermal sensation feedback confirmed by the user to form training samples, incrementally updating the model parameters used to generate the basic prediction. This allows the basic prediction model to adapt to long-term dynamic factors such as seasonal changes, changes in population composition, and changes in the spatial thermal environment, continuously reducing the basic prediction error.

[0022] Based on the above technical features, this invention can accurately reflect the real thermal sensation of the interacting personnel in a real dynamic public indoor space with high real-time performance, anti-interference ability, target feature consistency and personalization, providing a reliable basis for individualized environmental control.

[0023] Other beneficial effects of the present invention will be further described below. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the technical process of the system of the present invention; Figure 2 This is a schematic diagram of the anti-interference identification system of the present invention; Figure 3 This is a schematic diagram of thermal comfort prediction in the system of the present invention. Detailed Implementation

[0025] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0026] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0027] This invention aims to solve existing technical problems such as limited coverage of fixed monitoring in dynamic public indoor environments, unstable human body recognition due to visual interference, unstable cross-frame correlation of multi-target human body features, insufficient personalization of traditional thermal comfort models, and the inability of models to be adjusted in a timely manner based on continuous public feedback.

[0028] To this end, the present invention achieves accurate perception of the actual thermal sensation of the current interacting person in a real dynamic public indoor space through the coordinated operation of a mobile robot platform, an anti-interference human perception module, a multi-target feature construction module, a public interaction and personalized parameter module, a semantic reasoning module, an algorithm-adaptive personalized thermal comfort prediction module, and an online learning and model update module.

[0029] Specifically, mobile robots are used to establish a dynamic environmental perception method for people, so as to establish a connection between environmental parameter collection and the spatial location of people, thereby improving the spatial coverage capability of thermal environment monitoring in public indoor spaces. Establish an anti-interference human perception mechanism for human models, human advertisements, reflections, complex lighting, and dense crowd occlusion to improve the reliability of human detection, cross-frame target association, and target feature preservation in complex architectural scenes. The relevant features of human posture, clothing, behavior, current spatial location and human surface temperature in continuous video frames are uniformly constructed to form dynamic human thermal comfort features that are consistent with the target sequence of continuous frames. Establish a public-participatory thermal feedback mechanism, obtain real thermal feeling votes through voice and / or touch interaction, so that the system can obtain the subjective thermal feelings of actual users as model supervision information; By utilizing a large language model, semantic understanding, consistency verification, and cross-modal information fusion are performed on multimodal human features, personalized thermal comfort parameters, current interaction information, and thermal sensation feedback, thereby reducing the impact of errors or omissions in single visual information on subsequent predictions. An algorithm-adaptive personalized thermal comfort prediction model is established that can be continuously updated based on individual differences, environmental changes, and time variations. Through individual bias learning, environmental adaptation, time weighting, sliding window, prediction confidence, and online incremental updates, the accuracy and stability of thermal sensation prediction under long-term dynamic operating conditions are improved.

[0030] Through the above approach, the present invention ultimately constitutes a closed-loop system of "robot perception - personnel feature construction - public interaction feedback - semantic fusion - thermal sensation prediction - online update - re-prediction", which transforms thermal comfort prediction from a traditional one-time offline model into a long-term adaptive model driven by public participation.

[0031] The technical solution of the present invention will be described by way of example below.

[0032] Those skilled in the art will understand that the descriptions in this invention are merely illustrative and are not intended to limit the scope of protection of this invention. The specific naming conventions used in this invention should not be considered as limitations on the technical solutions of this invention. In this invention, the "environmental sensing device" can be, for example, an environmental sensing unit or an environmental sensing device; the "visual sensing device" can be, for example, a visual sensing device, and more specifically, an RGB or RGB-D camera device; the "anti-interference human body perception module" can be, for example, an anti-interference perception model AIT-A; the "multimodal human body feature set" can be, for example, a multimodal feature set; the "semantic reasoning module" can be, for example, a large language model-assisted semantic reasoning module or an LLM semantic processing module; the "algorithm-adaptive personalized thermal comfort prediction module" can be, for example, an algorithm-adaptive personalized thermal comfort model AAPCM, wherein the "basic prediction" can be, for example, a group-based basic prediction; and the "human-computer interaction terminal" can be, for example, a touch terminal or a voice interaction module. Furthermore, the "difficult background suppression unit" described in this invention can be implemented, for example, by employing difficult background suppression or difficult background weighted suppression; the "crowded scene spatial consistency constraint unit" can be implemented, for example, by employing crowded scene adaptive IoU constraint; the "human instance segmentation unit" can be implemented, for example, by employing a lightweight instance segmentation model (such as FastSAM) to perform instance segmentation; for the "multi-stage target consistency matching mechanism," it can adopt a step-by-step processing method of spatial / temporal candidate screening, human posture consistency matching, target appearance consistency matching, and multimodal feature binding; the "voice interaction terminal" and "touch interaction terminal" can be, for example, voice interaction modules and touch interaction modules; the "preset graded thermal sensation scale" can be, for example, a seven-level thermal sensation scale from -3 to +3; the "regression tree ensemble model" can be, for example, an XGBoost regression model; the "preset sliding window" can be, for example, a time-weighted sliding window; as for the "statistical correlation degree," it can be characterized, for example, by calculating the statistical correlation degree (e.g., correlation coefficient) between changes in environmental variables and thermal comfort prediction errors; and the "temporary target identifier" and "temporary interaction identifier" can be used for target association in the current monitoring process and data association in the current interaction, respectively. Those skilled in the art can make adaptive adjustments according to specific application scenarios, and this should not be regarded as a limitation on the technical solution of the present invention.

[0033] In some embodiments, the present invention provides a self-learning robot system based on real-time thermal comfort monitoring, comprising at least: a mobile robot and environment perception module, an anti-interference human perception module, a multi-target feature construction module, a public interaction and personalized parameter module, a semantic reasoning module, an algorithm-adaptive personalized thermal comfort prediction module, and an online learning and model update module, wherein each module sequentially transmits data and forms a closed-loop feedback.

[0034] (a) Mobile robot and environmental perception module

[0035] The mobile robot is equipped with visual sensing devices, environmental sensing devices, infrared thermal imaging devices, and human-computer interaction terminals. It moves in the public indoor space to be monitored according to a preset inspection route and can stay briefly at preset key monitoring locations.

[0036] The environmental sensing unit acquires environmental parameters, including at least air temperature, relative humidity, and air velocity, in real time or periodically, and obtains infrared thermal information to provide environmental exposure data near the location of personnel for thermal comfort prediction.

[0037] (II) Anti-interference human body perception module

[0038] To address the issue of false detection caused by human models, advertising figures, glass reflections, complex backgrounds, and dense crowds in public buildings, an anti-interference perception model, AIT-A, is established.

[0039] This approach prioritizes spatial consistency in human target matching within crowded scenarios, reducing target association drift caused by overlapping, occlusion, and cross-movement of people. Specifically, the adaptive IoU constraint in crowded scenarios employs spatial gating rather than simply adjusting the weight of the IoU metric in the overall matching function. For a continuous frame target sequence established in the previous time step and a candidate human target detected in the current frame, the Intersection over Union (IoU) between their bounding boxes is first calculated, and a corresponding IoU admission threshold is selected based on the current scene state. A first IoU threshold is used when the current scene is a normal human environment; a second IoU threshold is used when the current scene is a crowded environment, and this threshold must be satisfied. Only when the IoU between a candidate target and the target sequence from the previous time step reaches the corresponding threshold for the current scene can the candidate target enter the subsequent pose similarity and target consistency matching stage; candidate matching relationships that do not reach the threshold are discarded. This improves the spatial continuity condition required for candidate targets to enter the subsequent association stage in crowded environments, reducing erroneous target associations caused by overlapping of adjacent people, cross-movement, and detection box drift.

[0040]

[0041] Where, =1 indicates that the i-th consecutive frame target sequence and the j-th candidate body in the current frame can enter the subsequent target consistency matching through spatial consistency gating.

[0042] Furthermore, the standard human detection loss, difficult background suppression loss, and crowded scene constraints are jointly used to construct the training objective. In some embodiments, this invention also employs lightweight FastSAM to segment human candidate regions, thereby refining human body boundaries and reducing false positives caused by decorations, advertisements, and reflective areas.

[0043] (III) Multi-target feature construction module

[0044] After human detection is completed, multi-person pose estimation and cross-frame target association are performed for each human target. For human targets in consecutive frames, a continuous frame target sequence is formed based on human keypoint confidence, spatial distance, pose similarity, target box features and temporal continuity. Then, a multi-stage target consistency matching mechanism is used to perform target-level association of human state information from different sources.

[0045] For each real human target, establish a corresponding multimodal feature set. The features include at least one or more of the following: human posture key point features, clothing status, behavior status, activity status, current spatial location, human surface temperature-related features extracted based on infrared thermal images, and environmental parameters corresponding to the current spatial location.

[0046] By employing cross-frame target association and multi-stage target consistency matching, the same human target in consecutive video frames maintains consistent target features, avoiding feature mismatch between different targets in multi-person scenes. In some embodiments, this invention uses AlphaPose for multi-person pose estimation and the aforementioned multi-stage target consistency matching mechanism for target-level multimodal feature fusion.

[0047] (iv) Public interaction and personalized parameter module

[0048] The system is configured to allow public participation through at least one of voice or touch interaction methods.

[0049] When a person enters the preset interaction area and initiates an interaction, the system receives the interaction request through the human-machine interface terminal on the robot. The RGB-D camera on the robot can be used to detect human targets within the interaction area, and a lightweight target detection network based on Tiny-YOLO is used as the interaction trigger. When a human target is detected entering the preset interaction area and a touch or voice interaction request is received, the current interaction processing flow is initiated. Tiny-YOLO is used for low-latency human presence detection and interaction module triggering, while human perception, segmentation, and target consistency feature construction in complex environments are still completed by the aforementioned AIT-A and multi-stage target consistency matching mechanism.

[0050] For users who proactively initiate personalized thermal comfort service requests, the system obtains the interaction identifier provided or confirmed by the user through the human-computer interaction terminal, and uses the interaction identifier as a call index to read the corresponding personalized thermal comfort parameter set. The personalized thermal comfort parameter set is stored in a structured manner and dynamically updated with new proactive interaction results, and may include historical thermal sensation feedback, individual thermal preferences, historical interaction records, and individual deviation parameters.

[0051] Users can actively input their current subjective thermal sensation via voice or touchscreen. In voice interaction, the system uses an LST-RPESR automatic speech recognition model to process the user's input. The language model employs a multi-head attention mechanism based on a gated linear unit (GLU) to enhance the extraction of key semantic information and suppress irrelevant speech content. The speech recognition results are further converted into structured interactive information and thermal sensation feedback, which are used to update the personalized thermal comfort parameter set corresponding to the current personalized service request. Thermal sensation is represented using the standard thermal sensation voting TSV; in this embodiment, a seven-level thermal sensation scale from -3 to +3 is used.

[0052] For users who do not request personalized thermal comfort services, the system employs a temporary interaction path. This path involves the human-computer interaction terminal receiving current thermal sensations and necessary thermal comfort-related status information, assigning a temporary interaction identifier to this interaction, and using this identifier to correlate current multimodal human characteristics, environmental parameters, and thermal sensation feedback to form a data sample for current thermal comfort prediction and online model updates.

[0053] Thus, this system forms a personalized interaction process of "personnel presence detection - user active interaction - interaction identifier confirmation - personalized thermal comfort parameter call - current feedback - parameter dynamic update - personalized prediction", and a temporary interaction process of "personnel presence detection - user active interaction - temporary interaction identifier - current feedback - current human body and environmental feature association - group basic prediction / online sample".

[0054] (v) Semantic Reasoning Module

[0055] The data output by the aforementioned modules is encoded according to a predefined field-value structure, and a structured input record is formed using the temporary target identifier corresponding to the current continuous frame target sequence or the interaction identifier corresponding to the current interaction as a data association index. The structured input includes at least: acquisition time, human posture and behavioral characteristics, clothing status, activity status, current spatial location, current environmental parameters, current thermal sensation feedback, and personalized thermal comfort parameters invoked under a personalized thermal comfort service request. Continuous features are represented numerically, categorical features are represented using predefined enumeration labels, and temporarily missing or insufficiently reliable features are represented using null values ​​or uncertainty markers. The structured input can use JSON, key-value pairs, table structures, or data formats with equivalent field constraints. For example: Target identifier: T_0021; Human features: {Posture: Standing, Activity state: Walking}; Clothing status: {Upper garment: Long sleeve, Lower garment: Long pants}; Current spatial location: {Area: Atrium, Location coordinates: Corresponding robot coordinate system position}; Personalized thermal comfort parameters: {Thermal preference: Cool, Historical effective TSV: ...}; Current interaction: {Interaction time: ..., Current TSV: ...}; Current feedback: {TSV: +1, feedback method: touch}. The large language model performs semantic reasoning according to a preset prompt template containing role instructions, structured multimodal input blocks, and constrained output formats. Role instructions limit the model to a semantic fusion module oriented towards thermal comfort; the structured input blocks provide multimodal information about the current human target and the interaction process according to the aforementioned fields; the constrained output format requires the model to output only predefined fields such as activity state, the impact of clothing on thermal state, thermal preference tendency, information confidence level, and uncertainty markers, and encodes the output results as structured variables before inputting them into the personalized thermal comfort prediction module.

[0056] To reduce the computational load on the mobile robot during real-time operation, the large language model employs event-triggered reasoning instead of continuous execution for each video frame. For the same consecutive frame target sequence, the minimum time interval between two adjacent large language model calls is set to 30 seconds. The system detects "new interactive feedback events" and "human state change events." A new interactive feedback event is determined as soon as new voice or touch feedback is received. Human state changes are determined based on the activity state, posture state, clothing state, and current spatial position at adjacent update times of the consecutive frame target sequence. A significant change in human state is determined when at least one of the following occurs: a change in activity state category, a posture similarity below a preset posture threshold, a change in clothing state category, or a human target spatial displacement exceeding a preset distance threshold. The large language model is then re-called for semantic fusion if the minimum call interval is met. If no new interactive feedback is received, and none of the above human state change conditions are met, the previously generated structured semantic description is reused. The maximum continuous reuse time for the same semantic description is 60 seconds or three consecutive target sequence update cycles, whichever comes first. Once the reuse limit is reached, semantic reasoning is re-executed.

[0057] Let the human state of the same consecutive frame target sequence u at time t be represented as:

[0058] in, In active state, Human posture characteristics. For clothing status, For spatial location. Define personnel status change indicators:

[0059] The LLM trigger condition is:

[0060] in, =1 indicates that a new voice or touch response has been received since the last semantic reasoning.

[0061] (vi) Online learning and model update module

[0062] The algorithm adaptively personalizes thermal comfort prediction.

[0063] The following three types of information constitute the thermal comfort prediction input vector: environmental parameters such as air temperature, relative humidity, and air velocity; multimodal human body and behavioral semantic features processed by a large language model; and personalized thermal comfort parameters obtained from public interaction and personalized parameter modules.

[0064] An algorithmically adaptive personalized thermal comfort model (AAPCM) is established with real-time thermal sensation of personnel as the prediction target. In some embodiments, this invention uses an XGBoost regression model as the basic predictor, and adds individual bias learning, environmental adaptation, time weighting, sliding window management, online learning, and prediction confidence estimation mechanisms. When participating for the first time or using a temporary interaction path, the individual bias term is set to 0. When an individual actively initiates a personalized thermal comfort service and provides or confirms an interaction identifier, the individual bias is continuously updated based on the valid thermal sensation feedback corresponding to that interaction identifier. When the corresponding personalized thermal comfort parameter set has not been updated for a long time, a time decay is applied to historical individual biases, allowing long-term unupdated individual biases to gradually return to the group baseline, reducing the impact of outdated preferences on the current prediction.

[0065] Environmental adaptation, time weighting, and prediction confidence.

[0066] To address the issue of indoor environmental conditions changing with population density, time, and spatial location, this implementation method establishes an environmental adaptive mechanism based on the correlation between recent prediction residuals and environmental variables. Instead of pre-setting fixed weights for air temperature, relative humidity, and air velocity, this mechanism dynamically determines the impact of current environmental conditions on model error during online operation based on the statistical correlation between recent changes in environmental variables within a sliding window and the thermal comfort prediction error. Based on this, it adjusts the contribution of recent training samples in the online model update process.

[0067] Let the actual thermal feedback value at time t be The model predicts the value. The prediction residual is then defined as:

[0068] The correlation coefficient between this environmental variable and the predicted residual:

[0069] Calculate the normalized environmental sensitivity of each environmental variable based on the absolute value of the correlation coefficient:

[0070] Where J represents the number of environmental variables involved in environmental adaptation. A positive number is set to prevent the denominator from being zero. Therefore, the following condition is met:

[0071] When the association between recent changes in an environmental variable and the predicted residual strengthens, the corresponding environmental sensitivity increases. Automatically increases sensitivity; as the correlation weakens, its sensitivity decreases accordingly. Therefore, there is no need to manually predefine fixed relationships such as "air temperature is always more important than humidity." The environmental response is calculated based on the degree of deviation of the environmental state from the current window reference state.

[0072] Then, construct adaptive sample weights based on the environmental response:

[0073] Online learning driven by public feedback and closed-loop self-evolution.

[0074] The robot continuously collects new environmental data, human body condition characteristics, and thermal sensation feedback actively submitted by the public. Valid thermal sensation votes obtained and confirmed through voice or touch interaction are used as supervisory information, forming new training samples together with corresponding environmental parameters, multimodal human body characteristics, and currently available personalized thermal comfort parameters. The online learning module uses recent samples in a time-weighted sliding window to perform incremental model updates, eliminating the need for complete offline retraining on all historical data each time.

[0075] The following provides an exemplary description of specific embodiments of the present invention.

[0076] 1.1 System Architecture

[0077] Reference Figure 1 The present invention adopts a layered architecture of "mobile perception layer - human perception and feature construction layer - interaction and semantic processing layer - thermal comfort prediction layer - online learning and update layer". Each layer transmits information through a unified data interface, and together they form a closed-loop thermal comfort monitoring system.

[0078] 1.1.1 Motion Perception Layer

[0079] This invention utilizes an indoor mobile robot as a carrier for a mobile sensing layer, comprising a mobile chassis, a visual sensing device, an environmental sensing device, an infrared thermal imaging device, a human-machine interaction terminal, and a data processing unit. In one specific embodiment, the mobile robot autonomously moves within the public indoor space to be monitored according to a pre-set inspection route, and can perform short-term stationary monitoring in key areas such as entrances, traffic nodes, rest areas, queuing areas, and intersections of commercial spaces. The environmental sensing device includes at least an air temperature sensor, a relative humidity sensor, and an air velocity sensor; it may also be equipped with an infrared thermal imaging device to acquire infrared thermal images of the human body within the robot's field of view; combining human detection, instance segmentation, and cross-frame target association results to determine the human body region corresponding to each real human target, and providing raw data for subsequent extraction of human surface temperature-related features. Infrared thermal images and environmental parameters such as air temperature, relative humidity, and air velocity are collected separately, with the infrared thermal images mainly reflecting the thermal state of personnel, and air temperature and other parameters mainly reflecting the thermal exposure state of personnel in their environment. In a corresponding embodiment, environmental parameters are collected at 1-minute intervals, and the sensor is installed at a height of approximately 0.8m above the ground to obtain thermal environment parameters close to the actual exposure height of personnel. The visual perception device continuously acquires RGB or RGB-D images of the robot's surroundings and inputs the video frames into a subsequent anti-interference human recognition module. The mobile robot simultaneously records its own position and timestamp to spatiotemporally correlate the current human state, current spatial location, and environmental data.

[0080] 1.1.2 Human Perception and Feature Construction Layer

[0081] The human perception and feature construction layer mainly includes: 1. Anti-interference human body detection unit; 2. Human body instance segmentation unit; 3. Multi-person pose estimation unit; 4. Cross-frame human body association unit; 5. Multi-stage target consistency matching unit; 6. Multimodal human feature construction unit Its function is to convert continuous video data collected by robots into structured human information with target consistency that can be used for thermal comfort prediction. Addressing challenges in commercial buildings such as posters of people, human models, glass reflections, strong light, backlighting, and occlusion by high-density crowds, the system first uses an anti-interference human detection model to identify real human targets. Then, it segments the candidate human body regions into human instances and extracts features such as human pose, behavioral state, clothing status, and current spatial location. Cross-frame target association maintains feature consistency for the same human target across consecutive video frames. In the specific implementation, FastSAM can be used as a lightweight human instance segmentation model, AlphaPose can be used for multi-person pose estimation, and a multi-stage target consistency matching mechanism can be used for cross-frame target association.

[0082] 1.1.3 Interaction and Semantic Processing Layer

[0083] The interaction and semantic processing layer includes a personalized parameter module, a voice interaction module, a touch interaction module, and a large language model-assisted semantic fusion module. For users who proactively initiate personalized thermal comfort service requests, the system obtains the interaction identifier provided or confirmed by the user through the human-computer interaction terminal and calls the personalized thermal comfort parameter set corresponding to that interaction identifier. When the corresponding parameter set has not yet formed historical interaction data, fields such as historical thermal sensation feedback, historical interaction records, and individual thermal preferences are initialized to empty, and the number of interactions is initialized to 0.

[0084] The individual bias term used for personalized thermal comfort prediction is initialized to 0, meaning the initial personalized prediction uses the output of the population-based model without individual bias correction. Currently observable fields such as clothing status, behavioral status, and activity status are directly written into the current interaction record based on the perception results, along with the corresponding acquisition time. After a valid voice or touch interaction, the system uses the interaction identifier as an index to append a new interaction record to the corresponding personalized thermal comfort parameter set, including the thermal sensation vote TSV, interaction time, interaction method, corresponding environmental parameters, and current human characteristics. For consecutive valid thermal sensation feedback obtained under the same interaction identifier, the system continuously updates its individual bias term based on the deviation between the actual thermal sensation and the model prediction results.

[0085] Here, represents the actual thermal sensation vote actively submitted by the interacting personnel at time t, and represents the base prediction value at that time. As the number of valid interaction samples gradually increases, the individual bias term gradually reflects the stable deviation of the thermal sensation corresponding to the interaction identifier relative to the group average thermal sensation from the initial value of 0, and serves as an important parameter for subsequent personalized thermal comfort prediction.

[0086] Users submit their current thermal sensations via a touchscreen terminal on the robot or through voice interaction. Thermal sensations can be represented using a standardized thermal sensation vote (TSV), and in this embodiment, a seven-level thermal sensation scale ranging from -3 to +3 is used.

[0087] Personalized thermal comfort parameter sets may include: 1. Historical thermal sensation feedback; 2. Individual heat preference; 3. Historical interaction records; 4. Individual deviation parameters; 5. Clothing status corresponding to each active interaction; 6. The behaviors and activity states corresponding to each active interaction.

[0088] Therefore, the personalized thermal comfort parameter set is dynamically formed incrementally. When a personalized thermal comfort service is initiated for the first time, the system calls or establishes the corresponding parameter set based on the interaction identifier provided or confirmed by the user; after the first effective interaction, the first thermal sensation feedback and interaction record are written; as the number of subsequent active interactions increases, historical TSVs and interaction records are continuously added, and individual bias and thermal preference-related parameters are updated. When the user initiates a personalized thermal comfort service again and provides or confirms the same interaction identifier, the system reads the corresponding parameter set and uses the information obtained this time together with the existing parameters for the current personalized thermal comfort prediction. For information obtained from different data sources, the large language model is responsible for performing semantic alignment, consistency detection, and feedback interpretation, transforming visual, human-computer interaction, and personalized parameters into structured thermal comfort semantic features.

[0089] 1.1.4 Thermal Comfort Prediction and Online Learning Layer

[0090] The thermal comfort prediction layer receives environmental parameters, target-aligned human characteristics, personalized thermal comfort parameters, and semantic fusion results, and outputs the predicted thermal sensation value for the current interacting person. The online learning layer continuously refines the model based on subsequent valid public thermal sensation feedback, allowing the prediction model to gradually adapt. 1. Long-term heat preferences of different individuals; 2. Environmental changes over different time periods; 3. Changes in population density; 4. Changes in the space thermal environment; 5. Changes in behavior and clothing This leads to: A closed-loop operating architecture consisting of: mobile sensing → human body recognition → multimodal feature construction → public feedback → semantic fusion → thermal sensing prediction → online update → re-prediction.

[0091] 1.2 Software and Algorithm Design

[0092] 1.2.1 Anti-interference human recognition algorithm

[0093] To address complex visual interference in public buildings, this implementation method employs the anti-interference human perception model AIT-A.

[0094] First, non-human regions that are easily confused with the human body, such as human models, advertisements featuring people, and human-shaped images formed by glass reflections, are set as a difficult negative sample set (N_h). This causes the model to incur a greater loss penalty for misidentifying difficult backgrounds during training, thereby improving its ability to distinguish real human bodies under complex background conditions.

[0095] For densely populated areas, an adaptive IoU constraint for crowded scenarios is set. That is, a stricter spatial overlap matching constraint is used in crowded areas to reduce target drift and target association switching when different human targets cross paths.

[0096] The aforementioned difficult background suppression mechanism and crowded scene constraints, together with the standard human detection loss, form the training objective, thus balancing human recognition accuracy and robustness in complex scenes. In the corresponding experiments, after adding human instance segmentation, difficult background suppression, crowded scene IoU constraints, and multi-stage target consistency matching, the mAP of AIT-A in the decorative interference scene improved from 0.81 to 0.87, and in the composite interference scene improved from 0.75 to 0.82.

[0097] The anti-interference perception model AIT-A in this embodiment does not only use a general human detection model for target recognition, but also sets up a four-level processing mechanism on the basic human detection network, including difficult background adaptive suppression, crowded scene spatial consistency constraint, human instance boundary correction, and cross-frame target consistency maintenance, in order to reduce the cascading errors caused by non-human target misidentification, human missed detection, boundary drift, and target association switching in complex public indoor environments on subsequent thermal comfort prediction.

[0098] First, the current image captured by the robot's vision sensor is denoted as... The basic detection network targets images Generate a set of human candidate regions. For candidate regions consisting of people in advertisements, human models, human-shaped regions formed by glass reflections, and other non-human candidate regions with strong visual similarity to humans, define them as the set of difficult negative samples:

[0099] For the Each background candidate region is assigned an adaptive weight based on whether it belongs to the set of difficult negative samples:

[0100] in, The hard background enhancement coefficient is used. A hard background weighted loss is constructed based on this weight:

[0101] In the formula, Indicates the first The probability that a candidate region is predicted as background. This indicates the number of candidate regions in the current training batch. Through the above settings, human models, human-shaped advertisements, and human-shaped pseudo-targets formed by reflections receive a higher misclassification penalty than ordinary backgrounds during model optimization, thereby improving the ability to distinguish between real human bodies and human-shaped interference targets in complex backgrounds. This processing does not simply delete complex backgrounds, but rather achieves targeted anti-interference by changing the contribution of different background samples to the optimization objective.

[0102] Secondly, a spatial consistency assessment of crowded scenes is performed for scenes with high population density. The set of images identified as densely populated environments is denoted as... Dynamically select IoU constraints based on whether the current image belongs to a crowded scene:

[0103] And satisfy:

[0104] That is, when there is obvious occlusion between people, large-scale overlap of detection boxes, or cross movement of people, the target space matching constraint is improved, so that subsequent association depends more on the consistency of human space between consecutive frames, rather than just matching based on instantaneous classification confidence, thereby suppressing detection box drift and target association exchange in high-density environments.

[0105] The overall training objective of AIT-A is:

[0106] in, Indicates model parameters; This indicates the loss in basic human body detection. Indicates that a difficult background inhibits loss; and The contributions of the two types of anti-interference constraints to the overall training objective are controlled separately.

[0107] After obtaining the human candidate region, instead of directly using rectangular detection boxes as the subsequent human feature extraction region, a lightweight instance segmentation model is further employed to perform pixel-level boundary correction on the candidate human region. In this embodiment, FastSAM is used, performing instance segmentation only on the detected human candidate regions, rather than performing large-scale segmentation on the entire image, to balance human contour accuracy and the real-time computing power of the mobile robot. The segmentation mask is used to remove non-human pixels such as advertising backgrounds and glass reflections within the human detection box, and to provide human boundary priors for subsequent pose estimation and human attribute extraction. Finally, pose estimation and cross-frame target association are performed on the segmented human. In this embodiment, human keypoints are obtained through AlphaPose, and the human poses in two consecutive frames are denoted as follows:

[0108]

[0109] Here, and represent the positions of the j-th human body keypoints. Cross-frame matching comprehensively considers keypoint confidence, keypoint spatial proximity, human pose similarity, human bounding box feature similarity, and temporal continuity to determine whether two human instances in consecutive video frames belong to the same consecutive frame target sequence. This invention explicitly uses the above-mentioned joint information to establish cross-frame pose association, rather than matching solely based on detection boxes. Therefore, AIT-A is actually formed as follows: Candidate body detection → Weighted suppression of difficult backgrounds → Adaptive IoU constraint for crowded scenes → FastSAM human instance boundary correction → Pose estimation → Cross-frame target association.

[0110] The hierarchical anti-interference processing path outputs not a single human detection box, but a human target with high spatial reliability and the ability to further establish target consistency relationships.

[0111] 1.2.2 Multi-target pose and consistency matching algorithm (refer to...) Figure 2 )

[0112] A multi-stage target consistency matching mechanism is used to address the issue that human body status, environmental information, and current subjective thermal sensation feedback may be bound to incorrect targets during frequent cross-traffic, occlusion, and re-emergence of multiple people in public spaces. In this embodiment, the multi-stage target consistency matching mechanism operates in conjunction with the aforementioned adaptive IoU spatial gating for crowded scenarios. The IoU spatial gating is used to form a candidate association set that meets the spatial continuity requirement; for candidate targets that pass the spatial gating, multi-stage consistency matching is then performed sequentially using posture information, temporal continuity, and target appearance information.

[0113] Its core is to apply phased consistency constraints to human targets in continuous video frames based on the current spatial location, key points of human posture, target box features, clothing status and temporal continuity.

[0114] The first stage is cross-frame candidate association. For the current frame human instance and the continuous frame target sequence formed in the previous moment, candidate matching objects that may belong to the same target sequence are first screened based on the current spatial position of the human and the temporal continuity, so as to exclude target combinations with obviously unreasonable spatial distances.

[0115] The second stage is human pose consistency matching. For candidate objects, the consistency between the current human instance and the target sequence at the previous time step is determined by using human keypoint confidence, keypoint spatial location, and overall pose similarity. Further corrections are made by combining human detection box features and temporal continuity. Cross-frame pose association is accomplished through keypoint confidence, spatial proximity, pose similarity, and temporal continuity, thereby improving the stability of target association under occlusion and motion blur conditions.

[0116] The third stage is target appearance consistency matching. For candidate human targets that have passed the screening in the first two stages, the appearance consistency between the current human instance and the target sequence at the previous time step is further determined by using clothing status, human body contour features and target bounding box features, and the target association result is determined by combining the aforementioned spatial and pose information.

[0117] The fourth stage is multimodal feature binding. After completing target consistency matching, temporary target identifiers are assigned or extended to the corresponding real human targets, and the temporary target identifiers are used as the association index in the current monitoring process to bind human posture, clothing status, behavioral status, current spatial location, human surface temperature and related features and corresponding environmental parameters to the same target data entity.

[0118] Through the above-described hierarchical constraints, a multi-stage goal consistency matching mechanism is formed: Spatial / temporal candidate selection → pose consistency matching → target appearance consistency matching → target consistency construction process of multimodal feature binding.

[0119] The design maintains a continuous frame sequence of targets in the current monitoring process by using human posture, current spatial location, target appearance features, and temporal continuity. When the overall consistency of candidate associations is insufficient, temporary target identifiers can be reassigned to reduce the impact of erroneous associations on the subsequent construction of thermal comfort features.

[0120] In the field composite interference experiment, there were 112 target association switching (ID Switch) events without target-level fusion; after adding visual-level fusion, the number dropped to 74; after adding multi-stage target consistency matching, the number dropped further to 51; and after combining with LLM semantic fusion, the number dropped to 32.

[0121] 1.2.3 Interactive Triggering and Personalized Parameter Management Algorithm

[0122] The public interaction and personalized parameter module in this embodiment is used to predict thermal comfort by incorporating current thermal sensation feedback, current human body state, and optional historical thermal preference parameters when a user actively initiates a service request. The module executes in the following order: "personnel presence detection—interaction request triggering—interaction identifier acquisition—personalized thermal comfort parameter invocation or creation—current feedback acquisition—parameter dynamic update".

[0123] (1) User presence detection and interaction module triggered

[0124] To prevent the robot from continuously performing computationally intensive interactive processes without human interaction, this implementation first sets up a lightweight human presence detection module.

[0125] The robot's RGB-D camera continuously captures images of the current interaction area. When a human enters the robot's preset interaction area, a lightweight object detection network is used to detect the user's presence. In a specific embodiment, a real-time detection network based on the Tiny-YOLO architecture is used as the interaction trigger.

[0126] The input RGB image tensor has a size of 1×3×416×416, and the detection network output tensor has a size of 1×255×26×26. The output channels contain information such as candidate target category probabilities, target presence probabilities, and target bounding box coordinates. Therefore, this module forms: Continuous image input → Lightweight human presence detection → Determine if a human target has entered the interactive area → Yes: Wait for and respond to user-initiated interaction requests; No: Maintain low computational load monitoring state.

[0127] (2) Personalized service request and interaction identifier acquisition

[0128] When an interactive user initiates a personalized thermal comfort service request via a touch terminal or voice terminal, the system enters the personalized interaction process.

[0129] The human-computer interaction terminal obtains the interaction identifier provided or confirmed by the interaction personnel and uses the interaction identifier as the index for calling the personalized thermal comfort parameter set; for the current continuous frame target sequence, the system also retains a temporary target identifier, which is used to match the human body state characteristics during this interaction process with the current environmental parameters and the current thermal sensation feedback.

[0130] (3) Personalized thermal comfort parameter call

[0131] The system retrieves the corresponding personalized thermal comfort parameter set based on the interaction identifier. The parameter set may include historical thermal sensation feedback, individual thermal preferences, historical interaction records, and individual deviation parameters. When the interaction identifier is used for the first time in the personalized thermal comfort service, a corresponding parameter set can be established and the individual deviation parameters can be initialized to 0.

[0132] Personalized thermal comfort parameters are accessed using interaction identifiers as the index and are continuously updated based on new and valid thermal sensation feedback during subsequent active interactions initiated by the same interaction identifier.

[0133] (4) Interaction state determination and parameter update

[0134] For the current personalized service request, the system reads the personalized thermal comfort parameters corresponding to the interaction identifier, and combines them with the current environmental parameters, current multimodal human characteristics, and current thermal sensation feedback to form a structured input; after completing this interaction, the new effective feedback and the corresponding interaction time and interaction method are written into the corresponding parameter set.

[0135] For interaction identifiers used for the first time, the system establishes a corresponding set of personalized thermal comfort parameters; for interaction identifiers with existing parameter sets, the system calls the existing parameters and performs incremental updates based on the results of this active interaction.

[0136] (5) Personalized thermal comfort parameter data structure

[0137] Use the interaction identifier as the call index to create or call a personalized thermal comfort parameter set.

[0138] Personalized thermal comfort parameter sets include at least three types of information: The first category is interactive record information.

[0139] This includes the timing of proactive interactions, the methods of interaction, and the corresponding standardized thermal sensory voting (TSV).

[0140] The second category is thermal comfort-related status information.

[0141] This includes thermal comfort-related status information such as clothing status, behavior status, activity status, and current spatial location corresponding to each active interaction.

[0142] The third category is hot preference information.

[0143] This includes current and historical thermal perception voting, individual thermal preference trends, and individual bias parameters. In this embodiment, a seven-point TSV thermal perception scale (-3 to +3) is used to obtain the actual thermal perceptions actively submitted by the participants.

[0144] Therefore, the personalized thermal comfort parameter set corresponding to the u-th interaction identifier can be represented as: (6) Acquisition and structuring of speech information For users who actively engage in voice interaction, the robot acquires their voice recordings via microphone. The speech recognition and language understanding model is trained using the THCHS-30 and AISHELL-1 Chinese speech datasets. THCHS-30 contains approximately 30 hours of Chinese speech, while AISHELL-1 contains speech data from 400 speakers with different accents. The system further converts the speech recognition results into structured interactive information and thermal feedback before inputting them into the subsequent LLM semantic fusion module.

[0145] (7) Temporary interaction mechanism

[0146] For users who do not initiate a request for personalized thermal comfort services, the system uses a temporary interaction path.

[0147] Interacting personnel can submit their current thermal sensations and necessary thermal comfort-related status information through the robot's touch terminal or voice terminal. The system assigns a temporary interaction identifier to this interaction and associates the current human characteristics, environmental parameters, and TSV based on the temporary interaction identifier to form the data sample required for current thermal comfort prediction and online model updates.

[0148] Therefore, this implementation method forms two parallel running paths: Personalized interaction path: Personnel presence detection → User initiates personalized service → Interaction identifier confirmation → Invoke / create personalized thermal comfort parameter set → Current feedback → Personalized prediction.

[0149] Temporary interaction path: Personnel presence detection → User-initiated interaction → Temporary interaction identifier → Current feedback → Current human and environmental characteristic association → Population basic prediction / online sample.

[0150] Both of these paths are initiated by the interaction personnel and are used for personalized predictions that require historical parameter calls and real-time thermal comfort predictions that are based solely on current interaction information, respectively.

[0151] 1.2.4 Large Language Model-Assisted Semantic Reasoning Algorithm

[0152] To reduce the impact of visual misidentification, missing human features, and semantic inconsistencies between different modal data on thermal comfort prediction, the data that has completed the consistency association of the target and corresponds to the current interaction information is input into the large language model to assist the semantic processing module.

[0153] The large language model does not directly participate in the original video human detection, nor does it modify the human detection bounding boxes, target categories, or detection model parameters output by AIT-A, nor does it re-execute human detection through backfeedback. Its role is limited to semantic layer processing of multimodal human features that have already achieved target consistency association, current interaction feedback, and available personalized thermal comfort parameters. Therefore, the output of the large language model affects the subsequent semantic representation of human state and thermal comfort prediction input, rather than the front-end target detection results.

[0154] In a specific implementation, the large language model adopts the Qwen3-8B model with lightweight parameter adaptation.

[0155] The input content uses a structured template and must include at least: Part 1: Role Description. The designated large language model will perform the thermal comfort information fusion task.

[0156] Part Two: Structured Input. This includes human posture, clothing status, behavioral status, current spatial location, environmental parameters, current thermal sensation feedback, as well as historical thermal sensation parameters and individual thermal preference parameters invoked under personalized thermal comfort service requests.

[0157] Part Three: Constraint Output. The model is required to output only predefined structured features, such as: activity status; meaning of clothing-related thermal states; thermal preference trends; feature confidence levels; and uncertainty labels.

[0158] The Large Language Model (LLM) does not directly output the final TSV value. Instead, it further encodes the output into structured variables before inputting them into the thermal comfort prediction model. To balance real-time performance with mobile computing load, the LLM employs an event-triggered mechanism rather than calling each frame of the video. In this embodiment, the time interval between two adjacent LLM calls to the same consecutive target sequence is no less than 30 seconds. When there are no significant changes in human state, current feedback, or target features, existing semantic results can be reused for a maximum of 60 seconds or three target sequence update cycles. When there is a contradiction between visual recognition information and feedback actively submitted by the user, the LLM simultaneously retains information from each data source and outputs conflict markers, confidence levels, and uncertainty markers. The AAPCM then combines environmental information, available historical thermal sensation parameters, and individual biases to perform subsequent predictions.

[0159] 1.2.5 Specific Implementation of the Algorithm-Adaptive Personalized Thermal Comfort Model (AAPCM)

[0160] The fused feature input algorithm adaptively personalizes the thermal comfort model (AAPCM).

[0161] AAPCM does not simply use XGBoost to output thermal perception values, but rather establishes a joint adaptive mechanism on the basis of the basic regression model, which includes population baseline prediction, individual bias learning, environmental adaptation, time weighting, sliding window management, online incremental learning, and prediction confidence assessment.

[0162] First, construct the multi-source input vector:

[0163] Among them, environmental parameters such as air temperature, relative humidity, and air velocity are represented; clothing, posture, activity, and other human body characteristics are represented; structured variables such as activity status, clothing thermal meaning, thermal preference trend, confidence level, and uncertainty are represented by LLM semantic fusion; and currently available personalized thermal comfort parameters and historical thermal preference information are represented. This invention explicitly uses environmental information, LLM-enhanced human body characteristics, and personalized thermal comfort parameters as the three main inputs of AAPCM. Basic thermal sensation prediction adopts a regression tree ensemble approach.

[0164] Its basic optimization objective is:

[0165] Where is the actual TSV, and is the model-predicted thermal sensation. This embodiment of the invention uses XGBoost as the regression skeleton, but the innovative focus of AAPCM is not on XGBoost itself, but on its surrounding continuous adaptive mechanism. For the interactive process of calling a personalized thermal comfort parameter set through interactive identifiers and obtaining effective thermal sensation feedback, the system maintains the corresponding individual deviation parameters and updates them using an exponential moving average method:

[0166] Among them, represents the actual TSV actively submitted by the interacting personnel, and represents the output of the group's basic model.

[0167] The personalized prediction output is:

[0168] Set the interaction identifier when using it for the first time or when using a temporary interaction path:

[0169] That is, the group model is used to output first; individual corrections are gradually formed after valid TSVs are actively submitted through the same interaction identifier.

[0170] For personalized thermal comfort parameter sets that have not been updated for a long period of time, time decay is applied to the individual bias parameters to prevent outdated preferences from continuing to affect the prediction:

[0171]

[0172] With non-interaction time As historical individual biases gradually decrease towards zero, the model gradually reverts to the population baseline. In the environmental adaptation layer, the model dynamically adjusts the contributions of environmental features such as air temperature, relative humidity, and air velocity to the prediction results based on the correlation between recent environmental variables and the prediction residuals. This allows the model to respond to changes in population density, spatial location, and short-term environmental conditions, rather than continuously using fixed environmental variable contribution relationships. The online optimization objective of AAPCM can be further expressed as:

[0173] Prediction confidence is defined as:

[0174] in, It is a positive numerically stable term. When the recent residual fluctuation is small, the confidence level is high; when environmental changes, significant changes in personnel composition, or abnormal perception inputs lead to increased fluctuations in the prediction residual, the confidence level decreases, thus providing a basis for subsequent judgment on the reliability of the current prediction.

[0175] Therefore, the complete data flow of AAPCM is as follows: multi-source feature input → population-based prediction → individual bias correction → environmental contribution adaptation → time weighting → sliding window sample management → public TSV feedback → online incremental update → prediction confidence evaluation → next cycle prediction.

[0176] 1.3 Data Processing and Fusion (Refer to) Figure 3 )

[0177] 1.3.1 Data Time Synchronization

[0178] The robot's vision system continuously acquires video frames, environmental sensors collect environmental parameters according to a preset sampling period, and public feedback is non-fixed-time event data actively triggered by interacting personnel. Therefore, a unified timestamp is added to each type of data. For a continuous frame sequence of human targets formed at time (t), the system selects the most recent valid environmental data at that time as the corresponding environmental state and establishes a temporal correlation between the current human features, robot position, and environmental parameters.

[0179] 1.3.2 Target-level data fusion

[0180] Multimodal data is not simply stitched together from video frames, but is first organized as a sequence of consecutive target frames. Through target-level fusion, it can be ensured that the visual, environmental, and current subjective feedback data corresponding to the same consecutive target frame sequence enter the same thermal comfort prediction sample, avoiding the mixing of information from different human targets in dynamic multi-person scenes.

[0181] 1.3.3 Semantic-level data fusion

[0182] In situations where visual detection results may contradict actual feedback submitted by users, such as when the system visually determines that the target is wearing thick clothing, but current feedback consistently reports that the temperature is too cold, the visual classification results cannot be used directly for prediction. The system feeds the current human state, available historical thermal feedback, clothing status, and environmental parameters into the LLM semantic module for consistency analysis.

[0183] After LLM outputs a standardized semantic description, it is then converted into a numerical or categorical variable and input into AAPCM along with the original environmental parameters.

[0184] This results in a multi-layered data processing mechanism: raw sensor data → target-level fusion → semantic-level fusion → thermal comfort prediction features.

[0185] 1.3.4 Personalized Deviation Data Update

[0186] The system maintains individual thermal perception biases for personalized thermal comfort parameter sets invoked through interactive identifiers, initially using group prediction results; then, personalized corrections are gradually established after obtaining valid feedback actively submitted by the participants. When the corresponding parameter set has not been updated for a long period, its historical individual biases are decayed over time, thereby reducing the adverse impact of outdated historical preferences on future prediction results.

[0187] 1.4 Online Learning and Adaptive Update Mechanism

[0188] 1.4.1 Time-weighted

[0189] The closer the sample time is to the current moment, the higher its weight in model updates; the influence of earlier samples gradually decreases over time. This method can prevent earlier historical data from continuously dominating model parameters after changes in seasons, population density, and spatial conditions.

[0190] 1.4.2 Sliding Window Update

[0191] Let the sliding window length be (W), then the dataset used for the current model update is:

[0192] Where t represents the sequence number of the latest valid online learning sample formed up to the present, i represents the sequence number of each valid sample within the window, and W represents the number of valid samples retained by the sliding window. It should be noted that the above expression describes a sample sequence management mechanism that retains the most recent W valid samples, and does not assume that only one sample can be generated at each physical time point. In actual operation, when no new valid thermal sensory feedback is obtained at a certain moment, no new supervised learning sample is generated, and the sample sequence number does not increase; when multiple users generate valid feedback at the same time, multiple independent valid samples are formed, and different sample sequences are assigned to them sequentially. Therefore, t does not represent the actual physical time or number of minutes, but rather the sample sequence number; there is also no strict one-to-one correspondence between W and the actual physical time.

[0193] For example, in some embodiments, W=300 indicates that a maximum of the most recent 300 valid samples will be retained. Under the condition that experimental data is generated continuously at approximately 1-minute intervals, 300 samples correspond to approximately 300 minutes of runtime. However, this correspondence is only an approximation based on the frequency of on-site data generation, not a mathematically mandated equivalence. For situations where public feedback involves non-uniform intervals or multiple interactions at the same time, the system still manages the sliding window according to the valid sample sequence number and calculates the time weight based on the actual generation time of each sample.

[0194] The window always retains the W most recent valid samples and uses a first-in, first-out (FIFO) approach to discard older data. This ensures that the most recent state plays a major role in online model updates.

[0195] 1.4.3 Prediction Confidence Assessment

[0196] To identify low-reliability predictions that may occur under conditions such as rapid environmental changes, changes in personnel status, or sensory anomalies, this implementation calculates the prediction confidence level based on the degree of fluctuation in the prediction residual between recent actual thermal sensation feedback and model prediction results.

[0197] Let the user's actual thermal sensation value at time t be... The model predicts the thermal sensation value to be... The current prediction residual is defined as:

[0198] Select the L most recent prediction residuals with effective thermal feedback before the current time to form the recent prediction residual set:

[0199] Calculate the variance of the recent forecast residual set, and define the reciprocal of its standard deviation as the current forecast confidence level:

[0200] in, This represents the variance of the most recent L predicted residuals. ε is a positive number set to prevent the denominator from being zero. Optionally, the possible values ​​of ε can be: 10... -6 ~10 -4 Preferably 10 -5 .

[0201] The basic principle is that when the error between recent prediction results and actual user feedback is relatively stable, the dispersion between prediction residuals is small, i.e. Reduce, thus An increase in the recent prediction residual indicates that the current model has high stability; conversely, when environmental conditions change abruptly, human activity levels change significantly, human perception becomes abnormal, or the model has not yet adapted to the new data distribution, the recent prediction residual fluctuation increases, i.e. Increase, thus A decrease indicates that the uncertainty of the current forecast has increased, and further adjustments are needed based on new public sentiment feedback and subsequent online learning results.

[0202] This embodiment This is a confidence index used to characterize the stability and relative reliability of predictions, not a probability value. A higher value indicates smaller fluctuations in recent prediction residuals and a more stable current prediction; a lower value indicates larger fluctuations in recent prediction residuals and a relatively lower reliability of the current prediction.

[0203] 1.4.4 Public Interaction Model Revision

[0204] Valid TSVs (Transmissions Seeking Information) submitted proactively by users via voice or touch terminals are considered authentic supervisory data. The system correlates this feedback with corresponding multimodal human features and environmental parameters to generate new model samples. These new samples enter a sliding window and participate in subsequent incremental model updates.

[0205] Therefore, each user participation not only yields thermal comfort evaluation data, but also becomes a data source for subsequent model optimization.

[0206] During continuous operation, the system follows the following: The process of continuously updating the model involves a new environmental state, a new human body state, a new TSV, calculating the predicted residual, updating the available individual biases, updating the sample window, and incrementally updating the model.

[0207] This method eliminates the need for frequent offline retraining using all historical data, which helps maintain model update efficiency under long-term deployment conditions. This invention explicitly defines this mechanism as an online incremental learning and long-term self-evolutionary process involving public feedback.

[0208] 1.5 Implementation Example

[0209] 1.5.1 Experimental Period and Data Acquisition

[0210] The system will operate continuously in the field from November 3 to December 29, 2025, for a total of 57 days.

[0211] A total of the following data were collected during the operation: 1557 valid thermal sensation feedback responses were received; 39,181 sets of environmental parameter samples; Processing more than (1.05 x 10) video frames 7 ) frame.

[0212] The field experiments covered weekdays and weekends, different population densities, and different environmental interference conditions to evaluate the robot's long-term operation, complex visual recognition, multi-target interaction, and online learning capabilities.

[0213] 1.5.2 Experimental Site and Operating Conditions

[0214] The experimental site was set up in the public area on the first floor of a large commercial complex in Shenzhen. The experimental area covers an area of ​​approximately 2,100 m² and features semi-open commercial space characteristics, including various typical scenarios such as the main passageway, the intersection of the atrium, the entrance, and the areas where people stay.

[0215] The area experiences high daily pedestrian traffic, with an average daily visitor flow of approximately 44,000 during the experiment. Weekday visitor flow accounted for about 34.7%, while weekend and holiday visitor flow accounted for about 65.3%. Therefore, the experimental site was able to simultaneously cover low, medium, and high population densities, as well as various usage scenarios such as rapid movement, lingering, queuing, and gathering of people.

[0216] Meanwhile, the experimental environment contained several typical interference factors that affect machine vision perception, including large-sized advertisements featuring human figures, mannequins, glass curtain walls and reflective surfaces, non-uniform lighting caused by the mixture of natural and artificial light, localized strong reflections, and occlusion and cross-movement among high-density crowds. These conditions can simulate the complex dynamic environment that robots face during long-term operation in real large public buildings.

[0217] The field experiment lasted from November 3, 2025 to December 29, 2025, a total of 57 consecutive days. The mobile robot moved autonomously along a pre-set inspection route and stayed at selected key observation locations for a short period of about 5 minutes to obtain environmental and human data under different spatial locations and different personnel densities.

[0218] Three independent fixed thermal environment reference monitoring devices were set up in the experimental area to provide fixed-point environmental benchmarks and verify environmental measurement results. In this embodiment, a JT2180 indoor thermal comfort measuring instrument was used to continuously collect indoor thermal environment parameters such as air temperature, relative humidity, air velocity, and black sphere temperature. The fixed reference monitoring devices and the environmental sensors mounted on the mobile robot operated independently: the fixed devices continuously recorded local thermal environment changes at predetermined locations, which was used to compare with the mobile environmental data acquired during the robot's inspection process in time and space, helping to determine whether there were obvious anomalies or drifts in the robot's environmental data collection results, and providing a reference for the overall thermal environment status during the experiment. It should be noted that the fixed devices are not used as a direct, true label for personalized thermal comfort prediction results; the thermal comfort prediction performance is still primarily verified by the user's actual thermal sensation vote (TSV) provided through voice or touch interaction. The environmental parameters collected by the fixed devices serve as environmental input for the AAPCM and as reference data for the experimental environment. Throughout the experiment, a total of 1,557 valid thermal sensation votes (TSVs) and 39,181 valid environmental parameter samples were obtained. More than 1.05 × 10^7 frames of video data were processed, providing a data foundation for subsequent anti-interference human perception, semantic fusion, and long-term thermal comfort prediction.

[0219] 1.5.3 Experimental Procedure

[0220] After the robot begins operation, it first moves along a preset route within the experimental area, simultaneously acquiring RGB or RGB-D visual information, air temperature, relative humidity, airflow velocity, infrared thermal information, and the robot's current position. Subsequently, the anti-interference human perception module identifies real human targets from the video data and performs instance segmentation of the human body region. The system further performs multi-person pose estimation, cross-frame target association, and multi-stage target consistency matching to form a continuous frame target sequence for different human targets, and associates clothing, posture, behavior, current spatial location, and human surface temperature-related features with the corresponding target sequence. When a person enters the robot's interaction area and actively initiates interaction, the robot acquires their current thermal sensation through voice or touch. Thermal sensation is represented using a seven-level TSV scale from -3 to +3. For users who actively initiate personalized thermal comfort services, the system calls the corresponding personalized thermal comfort parameter set based on the interaction identifier they provide or confirm; for users using temporary interaction paths, the system associates the current feedback with the current human state and environmental parameters based on the temporary interaction identifier of this interaction.

[0221] After obtaining environmental data, human characteristics, and public feedback, the system inputs the consistent multimodal features into the LLM semantic processing module. This module performs semantic alignment and consistency analysis on human state, clothing information, available historical thermal comfort parameters, and current feedback. The output structured variables, including activity state, clothing thermal meaning, thermal preference trend, confidence level, and uncertainty markers, are then input into the AAPCM (Activity-Aspect-Current Model). The AAPCM outputs the predicted thermal sensation value for the current interacting person. Upon obtaining a new valid TSV (Thermal Sensitivity Value), the system calculates the residual between the actual and predicted values ​​and updates the individual bias parameters of the corresponding interaction identifier under the personalized interaction path. Simultaneously, the new sample is added to a time-weighted sliding window and participates in subsequent online model updates.

[0222] 1.5.4 Results and Analysis of Anti-interference Human Perception Experiment

[0223] To evaluate the robot's human perception performance in complex public spaces, the on-site video data was divided into four typical scenarios based on the main types of visual interference: S1 is the decorative interference scenario, mainly including human models, posters, and shop displays; S2 is the lighting and reflection interference scenario, mainly including backlighting, glass curtain wall reflection, and local overexposure; S3 is the dense crowd occlusion scenario, including frequent overlap and rapid crossing of people; and S4 is the composite interference scenario where multiple factors from the above coexist.

[0224] A stepwise ablation experiment was conducted on each component module of AIT-A. The results show that the anti-interference modules do not repeatedly perform the same function, but rather improve the recognition results step by step for different error sources. FastSAM provides more accurate human instance boundaries, thereby reducing false detections caused by glass reflections, advertising backgrounds, etc.; difficult background suppression further improves the ability to distinguish between human bodies and non-human targets; crowded scene IoU constraint mainly improves the stability of position matching under conditions of overlapping and intersection of people; and multi-stage target consistency matching further ensures the consistency of human targets in consecutive frames.

[0225] In the decorative interference scenario S1, the complete AIT-A achieved an mAP of 0.87 and a recall of 0.89; in the more complex interference scenario S4, it still maintained an mAP of 0.82 and a recall of 0.85, indicating that the present invention can reduce the impact of complex backgrounds, occlusions and rapid movement of people on the subsequent construction of thermal comfort features.

[0226] 1.5.5 Experimental Results and Analysis of Multimodal and LLM Semantic Fusion

[0227] To further evaluate the effects of target-level fusion and semantic-level fusion on the stability of multi-person dynamic features, we compared non-fusion, visual-level fusion, multi-stage target consistency matching, and complete LLM-assisted semantic fusion schemes under the composite interference scenario S4.

[0228] Experimental results show that without fusion, mAP is 0.75, Recall is 0.77, and Precision is 0.79, with 112 target association switching (ID switches). After visual-level fusion, the number of target association switching decreases to 74; after adding multi-stage target consistency matching, it decreases to 51; after adding LLM-assisted semantic fusion, mAP increases to 0.84, Recall increases to 0.87, Precision reaches 0.88, and the number of target association switching further decreases to 32. Compared with the non-fusion scheme, the number of target association switching is reduced by approximately 71.4%. The above results indicate that for the thermal comfort monitoring task of this invention, simply improving the accuracy of single-frame human detection is insufficient to guarantee the reliability of subsequent predictions. In multi-person public spaces, it is also necessary to address two issues: whether the current features belong to the same continuous frame target sequence and whether the information from different modalities is semantically consistent.

[0229] Multi-stage target consistency matching primarily addresses the consistency issues of cross-frame targets and multimodal state features, while LLM further performs semantic layer verification based on human body state, clothing, available historical thermal comfort parameters, and current feedback. When a single visual feature is temporarily missing or uncertain, the system can utilize other modal and interaction information to reduce the impact of anomalous features on the final prediction.

[0230] Further feature distribution analysis revealed that the intra-class variance of the target state features corresponding to the complete model decreased to 0.64, while the inter-class distance increased to 3.71. This indicates that the features of the same consecutive frame target sequence are more concentrated, while the feature separation between different target sequences is further improved. Therefore, the experiment supports the use of a hierarchical processing approach of "visual perception—target-level fusion—semantic-level fusion" in this invention, rather than directly concatenating data from different sources and inputting it into the prediction model.

[0231] 1.5.6 Results and Analysis of Personalized Thermal Comfort Prediction Experiment

[0232] To verify the predictive ability of AAPCM in a long-term dynamic environment, two representative full operating days were selected from the long-term field experiment for analysis.

[0233] The first evaluation date was November 25, 2025. This date followed approximately 20 days of continuous online operation, during which time the model had received feedback from various environmental conditions, crowd densities, and user feedback, thus establishing a preliminary stable online learning state. This was a weekday with relatively moderate overall traffic, but significant crowd gathering, increased activity, and environmental non-steady-state changes occurred during the evening peak hours of 5:30 PM to 7:30 PM. Using real user TSVs over 612 consecutive effective operating minutes as a reference, the robot's predictions generally tracked the actual trends in thermal sensation. As a criterion for correct prediction, AAPCM achieved a prediction accuracy of 88.4% on this day.

[0234] The second evaluation date was December 28, 2025, approximately one month after the first evaluation. During this period, the system continuously utilized public feedback for online updates. This day was a weekend, with significantly higher crowd density than weekdays. The composition and activities of the population were more complex, with frequent movement between different areas and various behaviors including family groups, individual shoppers, queuing, and prolonged stays. Therefore, the actual TSV exhibited greater short-term fluctuations. Despite the stronger non-stationarity of the input data, the model was still able to track the main trends in thermal sensation. With a TSV tolerance of ±0.5, the prediction accuracy for this day further improved to 89.8%.

[0235] Traditional PMV (Personalized Motion Measure) primarily reflects steady-state thermal sensations within a group, resulting in significant prediction errors in dynamic crowd environments. While static machine learning methods like XGBoost and Random Forest offer substantial improvements over PMV, their parameters remain fixed after training, making it difficult to continuously reflect environmental and crowd changes. Offline PCM-style models further incorporate individual descriptive information, but still lack continuous updates driven by public feedback, thus their overall performance remains lower than AAPCM (Average Personalized Motion Measure).

[0236] Meanwhile, ablation results showed that the accuracy dropped to 84.6% after removing individual bias correction and to 86.1% after removing time weighting. This indicates that individual thermal preference learning and time adaptation based on active interactive feedback both contribute to long-term thermal comfort prediction, and the performance improvement obtained by the complete AAPCM does not come solely from the basic XGBoost regression model.

[0237] Further analysis of the spatial distribution of people within the experimental area was conducted using robot trajectory and visual recognition results.

[0238] During weekday experiments, high-density crowds were mainly concentrated at the intersections of main passageways, atriums, and main entrance areas. During the evening rush hour, the intensity and coverage of these hotspots increased significantly.

[0239] On weekends, areas of high population density are more numerous, larger in area, and longer in duration. Particularly between 5:30 PM and 7:30 PM, continuous high-density areas appear at some major traffic nodes, reflecting significant spatial non-steadiness caused by queuing, waiting, and crowd gathering. From a thermal comfort perspective, crowd gathering not only alters human activity and metabolic heat release but may also change local airflow and heat load; therefore, predicting thermal sensation is essentially a dynamic problem that varies with both time and space.

[0240] It is noteworthy that while environmental complexity and population density were higher on weekends than on weekdays, the accuracy of the full AAPCM prediction did not decrease; instead, it increased from 88.4% to 89.8%. Considering the approximately one month of continuous online operation between the two evaluations, this result suggests that the online learning mechanism can continuously utilize new environmental data and real TSVs to correct individual bias parameters and recent environmental response relationships, thereby reducing performance degradation caused by changes in population distribution and environmental data drift.

[0241] 1.5.7 Comprehensive Experimental Analysis

[0242] Based on the above experimental results, this embodiment verifies different links in the technical chain of the present invention.

[0243] Anti-interference experiments show that difficult background suppression, human instance segmentation, crowded scene constraints, and multi-stage target consistency matching can improve the reliability of human perception in complex public spaces. Target-level and semantic-level fusion experiments further show that improving single-frame detection performance alone is insufficient to support long-term multi-person thermal comfort monitoring, while the consistency of targets in consecutive frames and the semantic consistency between different information play an important role in the stable operation of the system.

[0244] Long-term thermal comfort prediction experiments show that, when environmental parameters, human behavior, clothing, public subjective feedback, and online adaptation information are all involved in the prediction, AAPCM has lower prediction error and higher ±0.5 TSV tolerance accuracy compared to PMV, static machine learning models, and PCM models that do not have continuous feedback and update capabilities.

[0245] The 57-day on-site deployment generated 1,557 valid thermal sensation feedbacks and 39,181 environmental samples, while also completing the processing of over ten million video frames. This demonstrates that the proposed "anti-interference perception—target-consistent feature construction—public proactive feedback—LLM semantic fusion—personalized thermal comfort prediction—online incremental update" system is not only applicable to short-term offline experiments, but can also function as a complete system in real high-traffic public building environments.

[0246] Therefore, the above experiments have verified the effectiveness and engineering feasibility of the technical solution of the present invention from multiple aspects, such as perception accuracy, target association continuity, cross-modal information reliability, thermal comfort prediction accuracy, and long-term online adaptability. They also provide a real-time human thermal demand data foundation for subsequently integrating the prediction results into building automation systems, HVAC control systems, or other intelligent environmental control systems.

[0247] Compared with existing thermal comfort monitoring methods based on single-point data acquisition and static model prediction, this invention has the following advantages and effects: 1. Improve the human body's ability to resist interference in complex public environments.

[0248] By employing a weighted approach for challenging background samples, IoU constraints for crowded scenes, human instance segmentation, and a multi-stage target consistency matching mechanism, AIT-A effectively reduces false detections and target association drift caused by human models, advertisements featuring people, glass reflections, complex lighting, and occlusion by dense crowds. In actual testing, AIT-A achieved an mAP of 0.87 in decorative interference scenarios and still reached 0.82 in complex scenarios with multiple superimposed interferences.

[0249] 2. Improve the consistency of targets and characteristics during continuous monitoring of multiple people.

[0250] By continuously mapping human posture, current spatial location, clothing, behavior, and human surface temperature-related features to a continuous sequence of target frames, the interruption of the target sequence and feature mismatch caused by human occlusion, rapid crossover, and re-emergence can be reduced, providing a stable data foundation for thermal comfort prediction.

[0251] The anti-interference human perception model AIT-A achieved mAP of 0.87 and 0.82 in decorative interference scenarios and complex interference scenarios, respectively. Building upon this, multi-stage target consistency matching and large language model-assisted semantic fusion improved cross-frame target consistency and the reliability of multimodal human feature representation. Through target consistency matching and large language model-assisted semantic alignment, conflict identification, and uncertainty handling, target mismatch and feature concatenation caused by occlusion, personnel overlap, and local perception anomalies were reduced. In complex interference scenarios, the number of target association switching times decreased from 112 times without fusion to 32 times with the complete fusion scheme, a reduction of approximately 71.4%.

[0252] 3. To achieve the integration of objective human perception and subjective thermal sensation feedback.

[0253] Unlike systems that rely solely on environmental parameters or human visual characteristics for prediction, this invention uses real-time thermal sensations from the public as model input and subsequent monitoring information. This allows the model to directly utilize people's real subjective feelings to correct prediction biases, thus transforming the model from simply "monitoring the environment" to "perceiving people's real thermal needs."

[0254] 4. Improve personalized prediction capabilities under proactive personalized interaction conditions.

[0255] By calling the individual deviation parameters corresponding to the interaction identifier when the user actively initiates a personalized thermal comfort service request, and updating them using the user's subsequent active feedback, the stable thermal preferences under the corresponding interaction identifier can be gradually learned, thereby reducing the systematic error caused by the "average user assumption" in the traditional group average model.

[0256] 5. It has the ability to adapt to long-term online operation.

[0257] By employing time weighting, sliding windowing, individual bias decay, online incremental updates, and prediction confidence assessment, recent effective data can play a greater role in the model, while gradually reducing the impact of failed historical data on prediction results. This allows the system to adapt to changes in the environment, personnel behavior, and personnel composition without the need for frequent retraining of the complete offline model.

[0258] 6. Improve the accuracy of thermal perception prediction in dynamic public environments.

[0259] Actual long-term deployment results show that, with a correct prediction tolerance of ±0.5 TSV, the prediction accuracy of this invention's AAPCM reaches 88.4% on representative weekdays, corresponding to a MAE of 0.36 and an RMSE of 0.49. In weekend scenarios with higher population density and more significant environmental fluctuations, the prediction accuracy further reaches 89.8%, with an MAE of 0.34 and an RMSE of 0.47. In comparison, the accuracy of the traditional PMV model in the control experiment was 56.7%, the static XGBoost model was 77.6%, and the offline PCM model was 80.3%.

[0260] 7. Possesses engineering feasibility for long-term deployment in real public buildings.

[0261] This invention was not validated only in laboratory or short-term dataset conditions, but was continuously deployed in a real large commercial building for 57 days, obtaining 1,557 valid thermal sensory feedbacks, 39,181 sets of environmental data, and more than (1.05×10^7) frames of processed video data. This verified that the complete system consisting of robot perception, multi-object association, public active interaction, model prediction, and online learning can work continuously in a real dynamic environment.

[0262] 8. It is conducive to the formation of a user-oriented intelligent monitoring foundation for the building environment.

[0263] This invention expands the traditional monitoring model, which is based on fixed environmental parameters, into a thermal comfort perception model that involves environmental information, human dynamic information, and subjective feelings actively submitted by the public. It can generate dynamic thermal comfort information under different spaces, times, and interactive conditions, providing more refined data and prediction results for building operation management systems and subsequent human-centered environmental regulation.

[0264] In summary, this invention significantly outperforms existing technologies in terms of real-time performance, accuracy, personalization, and spatial visualization. It can provide reliable support for intelligent building environmental monitoring, energy-saving control of HVAC systems, and the development of green buildings, demonstrating outstanding technological advancement and broad application prospects.

[0265] The background section of this invention may include background information about the problems or environment in which the invention is being developed, and is not necessarily a description of prior art. Therefore, the content included in the background section does not constitute an admission of prior art by the applicant.

[0266] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A self-learning robot system based on real-time thermal comfort monitoring, characterized in that, include: A mobile robot platform is configured to move autonomously in the public indoor space to be monitored. The mobile robot platform is equipped with an environmental sensing device, a visual sensing device, an infrared thermal imaging device, and a human-computer interaction terminal. An anti-interference human body perception module is configured to identify real human targets from video frames acquired by the visual sensing device. A multi-target feature construction module is configured to perform multi-person pose estimation and cross-frame target association on the real human targets. It forms a continuous frame target sequence based on the confidence of human keypoints in consecutive video frames, current spatial position, pose similarity, target bounding box features, and temporal continuity. It also establishes a multimodal human feature set with target consistency for each real human target. The multimodal human feature set includes human pose features, clothing status, behavioral status, current spatial position and its corresponding environmental parameters, and human surface temperature-related features extracted based on infrared thermal images. The environmental parameters are obtained based on the environmental sensing device, and the infrared thermal images are obtained based on the infrared thermal imaging device. The public interaction and personalized parameter module is configured to obtain the current thermal sensation feedback actively submitted by the user through the human-computer interaction terminal; in response to the personalized thermal comfort service request initiated by the user through the human-computer interaction terminal, obtain the interaction identifier provided or confirmed by the user, and call the personalized thermal comfort parameter set corresponding to the interaction identifier, wherein the personalized thermal comfort parameter set includes historical thermal sensation feedback, individual thermal preferences, historical interaction records and individual deviation parameters; The semantic reasoning module is configured to encode the multimodal human feature set and the current thermal sensation feedback into a structured input, and add the personalized thermal comfort parameter set to the structured input when the interacting person initiates a personalized thermal comfort service request; and perform semantic consistency analysis and cross-modal information fusion on the structured input using a large language model to output structured thermal comfort semantic features. The algorithm-adaptive personalized thermal comfort prediction module is configured to: take the environmental parameters and the structured thermal comfort semantic features as basic inputs, and use the personalized thermal comfort parameter set as additional input when it exists, outputting a real-time thermal sensation prediction for the current interacting user; wherein, the real-time thermal sensation prediction is obtained by superimposing the basic prediction and the individual deviation parameter, and the basic prediction is the prediction value output by the algorithm-adaptive personalized thermal comfort prediction module when the individual deviation parameter is not superimposed; when the personalized thermal comfort parameter set corresponding to the interaction identifier is not invoked, the individual deviation parameter takes a value of zero; when the personalized thermal comfort parameter set corresponding to the interaction identifier is invoked, the individual deviation parameter is determined according to the deviation between the historical thermal sensation feedback and the corresponding basic prediction, its initial value is zero, and it is continuously updated according to the thermal sensation feedback subsequently actively submitted by the interacting user; The online learning and model update module is configured to: use newly acquired environmental parameters, the multimodal human feature set, and thermal sensation feedback confirmed by the interacting personnel to form training samples, and incrementally update the model parameters used to generate the basic prediction in the algorithm adaptive personalized thermal comfort prediction module, so as to reduce the basic prediction error in subsequent predictions.

2. The self-learning robot system based on real-time thermal comfort monitoring according to claim 1, characterized in that, The anti-interference human body sensing module includes: An anti-interference human body detection unit is configured to perform initial human body detection on the video frame and obtain initial detection results including real human body candidate regions and interference candidate regions; the interference candidate regions include at least one of human body models, human advertisements, and human-shaped images formed by glass reflection. The difficult background suppression unit is configured to define the interference candidate region as a set of difficult negative samples and construct a difficult background weighted loss based on the set of difficult negative samples, so that the interference candidate region receives a higher misclassification penalty than the background region other than the set of difficult negative samples, thereby filtering out the interference candidate region from the initial detection result; The crowded scene spatial consistency constraint unit is configured as follows: when the overlap between adjacent detection boxes in the current frame image exceeds a preset overlap threshold, the current scene is determined to be a crowded environment; when the current scene is not a crowded environment, the real human candidate region in the current frame is spatially filtered using a first IoU threshold as an admission condition; when the current scene is a crowded environment, the real human candidate region in the current frame is spatially filtered using a second IoU threshold as an admission condition, wherein the second IoU threshold is greater than the first IoU threshold, and the real human candidate region that passes the spatial filtering is retained as the human region to be segmented. The human body instance segmentation unit is configured to perform pixel-level instance segmentation on the human body region to be segmented, so as to correct the human body boundary and remove non-human body pixels inside the human body detection box, thereby obtaining the real human body target.

3. The self-learning robot system based on real-time thermal comfort monitoring according to claim 1, characterized in that, The multi-target feature construction module performs cross-frame target association through a multi-stage target consistency matching mechanism, which includes: The first stage is cross-frame candidate association: based on the current spatial location and temporal continuity, candidate real human targets corresponding to the target sequence of the previous continuous frame are selected from the real human targets in the current frame. The second stage is human posture consistency matching: using the confidence of human key points, the spatial position of key points and the overall posture similarity, the degree of posture consistency between the candidate real human target and the target sequence of consecutive frames in the previous moment is determined. The third stage is target appearance consistency matching: using the clothing status, human body contour features and target box features of the candidate real human target, the appearance consistency between the candidate real human target and the target sequence of consecutive frames in the previous time step is determined. The fourth stage, multimodal feature binding: assigning or extending temporary target identifiers to the real human targets matched through the first three stages, using the temporary target identifiers as the association index in the current monitoring process, and binding the multimodal human feature set to the corresponding real human targets.

4. The self-learning robot system based on real-time thermal comfort monitoring according to claim 1, characterized in that, The mobile robot platform is configured to move autonomously within the public indoor space to be monitored according to a preset inspection route, and to stay at preset key monitoring locations for a predetermined time.

5. The self-learning robot system based on real-time thermal comfort monitoring according to claim 1, characterized in that, The human-computer interaction terminal includes at least one of a voice interaction terminal and a touch interaction terminal; the current thermal sensation feedback includes a standardized thermal sensation voting (TSV) and is represented by a preset graded thermal sensation scale. The personalized thermal comfort parameter set includes thermal preference information, interaction record information, and individual bias parameters; wherein, the thermal preference information includes the historical thermal sensation feedback and the individual thermal preference, the historical thermal sensation feedback is the standardized thermal sensation vote (TSV) submitted by the interacting person in previous active interactions, the interaction record information includes the time and interaction method of each active interaction, and the individual thermal preference is the interacting person's preference tendency relative to the group average thermal sensation. The interaction identifier is used to invoke the corresponding personalized thermal comfort parameter set when the interacting person initiates a personalized thermal comfort service request.

6. The self-learning robot system based on real-time thermal comfort monitoring according to claim 1, characterized in that, The semantic reasoning module is further configured as follows: The structured input is encoded in JSON, key-value pairs, table structures, or data formats with equivalent field constraints; Event-triggered inference is used instead of continuous execution for each video frame; for the same consecutive frame target sequence, a minimum time interval is set between two adjacent large language model calls. When the preset triggering conditions are met and the minimum time interval has expired, the large language model is invoked; the preset triggering conditions include at least one of the following: receiving new current thermal sensation feedback, changes in human posture features in the multimodal human feature set, changes in clothing status in the multimodal human feature set, changes in behavioral status in the multimodal human feature set, changes in the current spatial position change in the multimodal human feature set exceeding a preset spatial displacement threshold, or the previously generated structured thermal comfort semantic features are continuously reused for a preset reuse duration or a preset number of reuses; When the preset triggering condition is not met, the structured thermal comfort semantic features generated by the semantic reasoning module in the previous call are reused.

7. The self-learning robot system based on real-time thermal comfort monitoring according to claim 1, characterized in that, The algorithm-adaptive personalized thermal comfort prediction module is further configured as follows: The basic prediction is generated by a regression tree ensemble model, with the environmental parameters and the structured thermal comfort semantic features as the basic inputs, and the personalized thermal comfort parameter set as an additional input when it exists. The basic prediction is output by weighted summation of multiple regression tree functions. The individual deviation parameter Updated using an exponential moving average method: in, This refers to the current thermal sensation feedback actively submitted by the interacting person at time t. This is the basic prediction for the corresponding time. For smoothing coefficients; When the set of personalized thermal comfort parameters corresponding to the interaction identifier has not been updated within a preset time period, a time decay is applied to the individual deviation parameters therein, so that the individual deviation gradually approaches zero as the time without updating increases; Based on the statistical correlation between changes in environmental parameters within a preset sliding window and the real-time thermal sensing prediction error, the influence intensity of the current environmental conditions on the real-time thermal sensing prediction error is dynamically determined, and the contribution of the training samples within the preset sliding window to the incremental update process of the model parameters used to generate the basic prediction is adjusted accordingly. The real-time thermal sensing prediction error is the difference between the current thermal sensing feedback and the real-time thermal sensing prediction. Calculate prediction confidence in The variance is calculated based on the real-time thermal sensing prediction error statistics within the preset sliding window. ε is a positive number set to prevent the denominator from being zero. The prediction confidence level is used to characterize the stability and reliability of the real-time thermal sensing prediction.

8. The self-learning robot system based on real-time thermal comfort monitoring according to claim 1, characterized in that, The public interaction and personalized parameter module is also configured to provide a temporary interaction path for users who have not initiated a personalized thermal comfort service request. In the temporary interaction path, a temporary interaction identifier is assigned to this interaction, and the current multimodal human feature set, environmental parameters and current thermal sensation feedback are associated based on the temporary interaction identifier.

9. The self-learning robot system based on real-time thermal comfort monitoring according to claim 1, characterized in that, It also includes an output interface configured to send the real-time thermal sensing prediction to the environmental control system of the public indoor space to be monitored, so as to provide data and decision-making basis for the environmental control of the public indoor space to be monitored.

10. A self-learning robot monitoring method based on real-time thermal comfort monitoring, characterized in that, A self-learning robot system based on real-time thermal comfort monitoring is applied. This system includes a mobile robot platform, an anti-interference human perception module, a multi-target feature construction module, a public interaction and personalized parameter module, a semantic reasoning module, an algorithm-adaptive personalized thermal comfort prediction module, and an online learning and model update module. The monitoring method of the real-time thermal comfort monitoring self-learning robot includes: The mobile robot platform is controlled to move autonomously within the public indoor space to be monitored. The anti-interference human body perception module identifies real human targets from video frames acquired by the visual sensing device. The multi-target feature construction module performs multi-person pose estimation and cross-frame target association on the real human targets. Based on the confidence of human key points in continuous video frames, current spatial position, pose similarity, target box features, and temporal continuity, a continuous frame target sequence is formed, and a multimodal human feature set with target consistency is established for each real human target. The multimodal human feature set includes human pose features, clothing status, behavioral status, current spatial position and its corresponding environmental parameters, and human surface temperature-related features extracted based on infrared thermal images. The environmental parameters are obtained based on the environmental sensing device on the mobile robot platform, and the infrared thermal images are obtained based on the infrared thermal imaging device on the mobile robot platform. The public interaction and personalized parameter module obtains the current thermal sensation feedback actively submitted by the user through the human-computer interaction terminal mounted on the mobile robot platform; in response to the personalized thermal comfort service request initiated by the user through the human-computer interaction terminal, it obtains the interaction identifier provided or confirmed by the user, and calls the personalized thermal comfort parameter set corresponding to the interaction identifier. The personalized thermal comfort parameter set includes historical thermal sensation feedback, individual thermal preferences, historical interaction records, and individual deviation parameters. The semantic reasoning module encodes the multimodal human feature set and the current thermal sensation feedback into a structured input, and adds the personalized thermal comfort parameter set to the structured input when the user initiates a personalized thermal comfort service request; the structured input is then subjected to semantic consistency analysis and cross-modal information fusion using a large language model to output structured thermal comfort semantic features; The algorithm-adaptive personalized thermal comfort prediction module takes the environmental parameters and the structured thermal comfort semantic features as basic inputs, and uses the personalized thermal comfort parameter set as additional input when it exists, to output a real-time thermal sensation prediction for the current user. The real-time thermal sensation prediction is obtained by superimposing a basic prediction with an individual deviation parameter. The basic prediction is the prediction value output by the module without the individual deviation parameter. When the personalized thermal comfort parameter set corresponding to the interaction identifier is not invoked, the individual deviation parameter is zero. When the personalized thermal comfort parameter set corresponding to the interaction identifier is invoked, the individual deviation parameter is determined based on the deviation between the historical thermal sensation feedback and the corresponding basic prediction. Its initial value is zero, and it is continuously updated based on the thermal sensation feedback subsequently submitted by the user. The online learning and model update module uses newly acquired environmental parameters, the multimodal human feature set, and thermal sensation feedback confirmed by the interacting personnel to form training samples. This data is then used to incrementally update the model parameters used to generate the basic prediction in the algorithm's adaptive personalized thermal comfort prediction module, thereby reducing the basic prediction error in subsequent predictions.

Images