LLM-based ambient temperature control method, apparatus and device, and storage medium

Abstract

The invention relates to the cross field of block chain technology and artificial intelligence, and provides an LLM-based environment temperature control method, device and equipment and a storage medium. The method comprises the following steps: triggering an image acquisition device through a multi-modal fusion triggering mechanism to obtain environment image data of a target area; inputting the environment image data into an LLM system for multi-modal joint data analysis, and identifying personnel category attributes, the number of people in each age group and behavior state characteristics in the target area; generating a temperature regulation and control instruction through a dynamic role prompt technology based on the personnel category attributes in the target area, the number of people in each age group, the behavior state characteristics and a preset temperature preference rule; according to the digital twin sandbox temperature control system, the temperature control instruction is converted into the temperature control signal, and the temperature is dynamically adjusted after the digital twin sandbox preview verification, so that the bottleneck of high cost and low intelligence of the traditional temperature control system is broken through, and an environment control mode with economical efficiency, comfort and intelligence is constructed.

Description

Technical Field

[0001] This disclosure relates to the intersection of blockchain technology and artificial intelligence, and in particular to an environmental temperature control method, apparatus, device and storage medium based on LLM. Background Technology

[0002] Currently, in the field of environmental control, traditional AI temperature control methods often require the deployment of multiple sensors to collect a large amount of data. For example, a common park temperature control system needs to deploy at least 3-5 temperature sensors, and also needs to be combined with personnel counting sensors to collect environmental parameters. Then, control commands are generated through preset rules or machine learning models. However, this approach has the following drawbacks:

[0003] (1) The deployment cost of multiple sensors is high, and in special scenarios such as open kitchens and restaurants with high ceilings, traditional sensors cannot accurately capture the temperature of the area where people are active. For example, the temperature sensor error in hot pot restaurants is greater than 2 degrees Celsius due to oil fume pollution.

[0004] (2) Temperature preferences are determined by on-site voting, but this involves personal privacy and has a high margin of error. It cannot be effectively carried out when there are not enough voters.

[0005] (3) It is impossible to predict the flow of people during the target time period, to adjust the indoor / park temperature in advance, and to restore the basic temperature in time when the flow of people decreases, so the real-time performance is not strong. Summary of the Invention

[0006] This disclosure provides an LLM-based environmental temperature control method, device, equipment, and storage medium, aiming to construct a new environmental control mode that combines economy, comfort, and intelligence, in order to reduce development costs and energy costs, and improve the accuracy, comfort, and applicability of temperature control range.

[0007] According to a first aspect of this disclosure, an LLM-based environmental temperature control method is provided, comprising:

[0008] The image acquisition device is triggered by a multimodal fusion triggering mechanism to acquire environmental image data of the target area;

[0009] The environmental image data is input into the LLM system for multimodal joint data analysis to identify the category attributes of people in the target area and the number of people in each age group, while analyzing the behavioral state characteristics of people in the target area.

[0010] Based on the personnel category attributes, the number of people in each age group, behavioral characteristics, and preset temperature preference rules within the target area, temperature control instructions are generated through dynamic role prompting technology.

[0011] The temperature control command is converted into a temperature control signal for the temperature regulation device in the target area, and the temperature in the target area is dynamically adjusted after pre-verification through a digital twin sandbox.

[0012] As a preferred embodiment, the preset triggering mechanism includes at least one of the following triggering methods:

[0013] Time-triggered: Environmental image data acquisition is triggered periodically at preset time intervals, wherein the time interval is dynamically adjusted according to the time entropy value of historical personnel flow.

[0014] Personnel number change threshold trigger: When the change rate of the total number of personnel detected twice consecutively exceeds the preset change rate threshold, environmental image data acquisition is triggered;

[0015] Ambient temperature deviation threshold trigger: When the difference between the real-time monitored temperature and the current set temperature exceeds the preset temperature fluctuation range, environmental image data acquisition is triggered. The preset temperature fluctuation range is corrected in real time according to the intensity of human activity in the target area.

[0016] In a preferred embodiment, the environmental image data is input into an LLM system for multimodal joint data analysis to identify the category attributes of people within the target area and the number of people in each age group, while simultaneously analyzing the behavioral state characteristics of people within the target area, including:

[0017] Visual semantic features are extracted using the CLIP model, and a multimodal feature space is constructed by fusing thermal imaging and audio data.

[0018] Based on the Few-Shot learning capability of the LLM system, personnel category attributes are identified, and the number of people in each age group is estimated through Monte Carlo sampling.

[0019] Combining DINOv2 attitude analysis and optical flow field calculation, the static / dynamic behavioral characteristics and thermal comfort index of personnel are analyzed based on the LLM system.

[0020] As a preferred implementation, based on the personnel category attributes, the number of people in each age group, behavioral characteristics, and preset temperature preference rules within the target area, temperature control instructions are generated using dynamic role-based prompting technology, including:

[0021] Based on a pre-built adaptive weighting model, the weight ratio of the number of people in each age group in temperature control is calculated.

[0022] Natural language prompts are constructed based on the personnel category attributes, the number of people in each age group, the current ambient temperature, behavioral characteristics, preset temperature preference rules, and the weight ratio of people in each age group in temperature control.

[0023] The natural language prompts are semantically parsed to generate temperature control instructions.

[0024] As a preferred embodiment, the step of dynamically adjusting the temperature within the target area after pre-verification via a digital twin sandbox includes:

[0025] Set a safety threshold for temperature adjustment, wherein the safety threshold includes an upper temperature threshold and a lower temperature threshold;

[0026] The temperature control signal is input into a digital twin sandbox to predict the temperature change trend within a preset time.

[0027] If the predicted temperature change trend is within the safe threshold range, the temperature in the target area will be dynamically adjusted according to the temperature control signal.

[0028] If the predicted temperature change trend exceeds the safety threshold within the safe threshold range, a multi-level alarm is triggered, and the temperature of the target area is maintained within the safety threshold.

[0029] In a preferred embodiment, the method further includes dynamically adjusting the temperature control command based on changes in pedestrian traffic over different time periods, specifically including:

[0030] Historical data was statistically analyzed, and an LSTM neural network was used to establish a traffic flow prediction model for different time periods. The input parameters of the traffic flow prediction model included historical passenger flow, holiday labels, and external environmental data.

[0031] Based on the personnel flow prediction model for different time periods, the personnel flow within the target time period is predicted;

[0032] A temperature pre-adjustment strategy is executed based on the predicted flow of people within the target time period, wherein the temperature pre-adjustment strategy includes the advance temperature adjustment duration and the temperature adjustment rate.

[0033] In a preferred embodiment, the temperature pre-adjustment strategy is executed based on the predicted passenger flow within the target time period. The temperature pre-adjustment strategy includes the duration of advance temperature adjustment and the temperature adjustment rate, comprising:

[0034] When the predicted flow of people during the target time period exceeds the preset multiple of the existing flow of people, the temperature adjustment duration is determined in advance based on the predicted growth rate of the flow of people.

[0035] When the predicted rate of change of personnel flow in the target time period exceeds the preset rate of change of personnel flow, the temperature adjustment rate is determined according to the pre-established linear relationship between the rate of change of personnel flow and the temperature adjustment rate, and the temperature of the target area after adjustment still does not exceed the safety threshold.

[0036] When the predicted flow of people during the target time period is lower than the preset multiple of the current flow of people, the temperature is gradually restored to the baseline temperature.

[0037] In a preferred embodiment, the method further includes:

[0038] The preset temperature preference rules are modified based on the external environmental factors of the target area.

[0039] If the external environment is during the high-temperature period of summer, the high-temperature correction factor will be activated, and the preset temperature preference will be adjusted to the product of the original temperature preference and the high-temperature correction factor.

[0040] If the external environment is in a period of low winter temperatures, the low temperature correction factor will be activated, and the preset temperature preference will be adjusted to the product of the original temperature preference and the low temperature correction factor.

[0041] If the external environment is during peak dining hours, the personnel density correction factor will be activated, and the preset temperature preference will be adjusted to the product of the original temperature preference and the personnel density correction factor.

[0042] According to a second aspect of this disclosure, an LLM-based ambient temperature control device is provided, comprising:

[0043] The environmental image data acquisition module is used to trigger the image acquisition device through a multimodal fusion triggering mechanism to acquire environmental image data of the target area;

[0044] The data analysis and recognition module is used to input the environmental image data into the LLM system for multimodal joint data analysis, identify the personnel category attributes and the number of people in each age group in the target area, and analyze the behavioral state characteristics of the people in the target area.

[0045] The temperature control instruction generation module is used to generate temperature control instructions based on the personnel category attributes, the number of people in each age group, behavioral status characteristics, and preset temperature preference rules within the target area, using dynamic role prompting technology.

[0046] The temperature dynamic adjustment module is used to convert the temperature control command into a temperature control signal for the temperature regulation device in the target area, and to dynamically adjust the temperature in the target area after verification through a digital twin sandbox.

[0047] According to a third aspect of this disclosure, an electronic device is provided, comprising at least one processor and a memory communicatively connected to said at least one processor; wherein the memory stores a computer program executable by said at least one processor, said computer program being executed by said at least one processor to enable said at least one processor to perform the method as described in any of the preceding claims.

[0048] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided, wherein computer instructions are provided to cause the computer to perform the method described in any of the preceding claims.

[0049] Compared with the prior art, this disclosure achieves the following beneficial effects:

[0050] (1) This disclosure utilizes a multimodal fusion triggering mechanism, combining various triggering methods such as time, changes in the number of people, and deviations in ambient temperature, and dynamically adjusts the triggering conditions. For example, it adjusts the time interval period based on the temporal entropy value of historical personnel flow and corrects the preset temperature fluctuation range based on the intensity of personnel activity, enabling more timely and accurate acquisition of environmental image data. Furthermore, it employs an LLM system for multimodal joint data analysis to identify personnel category attributes, age group numbers, and behavioral characteristics. Combined with dynamic role prompting technology, it generates temperature control instructions, ensuring that temperature regulation accurately matches personnel needs and environmental changes. For instance, it achieves refined temperature control by addressing the differences in temperature preferences among different age groups and the impact of static and dynamic behaviors on thermal comfort.

[0051] (2) This disclosure utilizes the Few-Shot learning capability and dynamic role prompting technology of LLM to reduce the dependence on a large amount of labeled data and can quickly adapt to new scenarios based on a small number of samples; it establishes a traffic flow prediction model through LSTM neural network, and combines historical traffic flow, holiday labels and external environment data to achieve accurate prediction of traffic flow, and executes temperature pre-adjustment strategy accordingly; the digital twin sandbox pre-performance verification mechanism predicts temperature change trends by simulating different temperature control schemes, discovers potential problems in advance and optimizes control strategies, so that the system has stronger intelligent decision-making and adaptive capabilities.

[0052] (3) This disclosure takes into account external environmental factors (summer, winter, peak dining season, etc.) to modify the preset temperature preference rules, and combines real-time environmental data and personnel status in the target area to enable the system to maintain good temperature control effect in a variety of complex scenarios, thereby improving user thermal comfort and user experience.

[0053] (4) The technical solution disclosed herein does not require the deployment of a large number of temperature sensors as in traditional temperature control solutions. Environmental data collection can be completed solely through existing video monitoring systems, reducing hardware costs. Furthermore, it does not require extensive data annotation and model training for specific scenarios. Temperature strategy optimization can be achieved through prompts, greatly reducing development costs.

[0054] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0055] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0056] Figure 1 A flowchart of an LLM-based ambient temperature control method according to an embodiment of this disclosure is shown;

[0057] Figure 2 A block diagram of an LLM-based ambient temperature control device according to an embodiment of the present disclosure is shown;

[0058] Figure 3 A schematic diagram of an exemplary electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0060] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0061] like Figure 1 The diagram shows a flowchart of an LLM-based environmental temperature control method disclosed herein. The method 100 includes:

[0062] S110: The image acquisition device is triggered by a multimodal fusion triggering mechanism to acquire environmental image data of the target area.

[0063] In some embodiments, the above-mentioned preset triggering mechanism includes at least one of the following methods:

[0064] (1) Time-triggered: Environmental image data acquisition is triggered periodically at preset time intervals (e.g., every 10 minutes). The time interval is dynamically adjusted based on the temporal entropy value of historical personnel flow. For example, the system calculates the temporal entropy value of the flow for each time period of the day based on three months of historical personnel flow data using the Transformer time series model. When the entropy value is higher than the threshold (e.g., entropy value > 0.8 at noon on weekdays), the acquisition interval is automatically shortened to 5 minutes; when the entropy value is lower (e.g., entropy value < 0.3 at night), the interval is extended to 30 minutes, achieving adaptive matching between the data acquisition frequency and the pattern of personnel activity.

[0065] (2) Threshold trigger for personnel number change: Analyze the collected environmental image data, and trigger the collection of environmental image data when the change rate of the total number of personnel detected twice in a row exceeds the preset change rate threshold (e.g., 15%). For example, if the change rate of the total number of personnel detected twice in a row (10 seconds apart) exceeds 15%, the collection will be triggered immediately.

[0066] (3) Triggering the deviation of ambient temperature from the threshold: When the difference between the real-time monitored temperature and the current temperature exceeds the preset temperature fluctuation range (e.g., 2 degrees Celsius), the acquisition of environmental image data is triggered. At the same time, the system can also dynamically adjust the threshold according to the intensity of human activity in the target area: when the proportion of dynamic activity exceeds 30%, the threshold is tightened to ±1.5℃; when static activity is the main activity, it is relaxed to ±2.5℃.

[0067] (4) Triggering of personnel activity intensity: DINOv2 self-supervised visual model is used to analyze personnel posture and optical flow field algorithm is used to calculate personnel movement speed. When the proportion of dynamic activity personnel exceeds 30%, high frequency acquisition mode is started to trigger high frequency acquisition (e.g., 5 minutes / time).

[0068] Among them, the aforementioned image acquisition devices include, but are not limited to, intelligent image data acquisition devices such as video surveillance systems or drone aerial photography systems that can respond to trigger mechanisms and output environmental image data, and can transmit image data to the LLM system in real time via Modbus protocol or HTTP interface.

[0069] S120: Input the environmental image data into the LLM system for multimodal joint data analysis, identify the personnel category attributes and the number of people in each age group in the target area, and analyze the behavioral state characteristics of the people in the target area.

[0070] In some implementations, environmental image data is input into an LLM system for deep multimodal joint data analysis. The specific process is as follows:

[0071] First, visual semantic features are extracted using the CLIP model, and then thermal imaging and audio data are fused to construct a multimodal feature space:

[0072] Secondly, personnel category identification: Based on the Few-Shot learning capability of LLM, combined with 5-10 labeled samples, the Tongyi Qianwen model is guided by Prompt Engineering technology to quickly identify categories such as children, young people, middle-aged people, and the elderly.

[0073] Age group quantity estimation: Construct an age distribution prediction head based on Transformer, use Monte Carlo sampling method to output the probability distribution of each age group (0-12 years, 13-25 years, 26-45 years, 46-65 years, 66 years and above), and generate 95% confidence intervals.

[0074] Behavioral state analysis: The DINOv2 model is used to analyze human posture (sitting, standing, and moving postures), combined with OpenPose keypoint detection technology to calculate human movement speed and direction. Simultaneously, facial recognition technology is used to detect human micro-expressions (such as wiping sweat or frowning) to construct thermal comfort behavior indicators and quantify human thermal perception status.

[0075] The LLM system can be deployed locally or on a cloud server.

[0076] S130: Based on the personnel category attributes, the number of people in each age group, behavioral status characteristics, and preset temperature preference rules within the target area, a temperature control command is generated through dynamic role prompting technology.

[0077] In some embodiments, the above-mentioned preset temperature preference rules are: age: 20-30: young people (20-23 degrees Celsius), age: 30-50: middle-aged people (22-25 degrees Celsius), age: 50-80: elderly people (25-27 degrees Celsius).

[0078] Furthermore, to improve the accuracy of temperature control, the preset temperature preference rules can be modified by taking into account external environmental factors of the target area:

[0079] For example, if the external environment is during the high-temperature period of summer, the high-temperature correction factor (e.g., 0.9) will be activated, and the preset temperature preference (young people (20-23 degrees Celsius)) will be adjusted to the product of the original temperature preference and the high-temperature correction factor (young people (20-23 degrees Celsius) * 0.9).

[0080] If the external environment is during a period of low temperatures in winter, the low temperature correction factor (e.g., 1.2) will be activated, and the preset temperature preference (young people (20-23 degrees Celsius)) will be adjusted to the product of the original temperature preference and the low temperature correction factor (young people (20-23 degrees Celsius) * 1.2).

[0081] If the external environment is during peak dining hours, the personnel density correction factor (e.g., 0.8) will be activated, and the preset temperature preference (young people (20-23 degrees Celsius)) will be adjusted to the product of the original temperature preference and the personnel density correction factor (young people (20-23 degrees Celsius) * 0.8).

[0082] In some embodiments, based on the personnel category attributes, the number of people in each age group, behavioral characteristics, and preset temperature preference rules within the target area, temperature control instructions are generated through dynamic role-based prompting technology, specifically including the following steps:

[0083] First, based on the pre-built adaptive weighting model, calculate the weight ratio of people in each age group in temperature control (e.g., weight ratio = based on people in each age group / total number of people in the target area);

[0084] The system constructs natural language prompts based on the personnel category attributes, number of people in each age group, behavioral characteristics, preset temperature preference rules, and the weighting of each age group in temperature control within the target area. For example, the prompt could be: "You are a restaurant air conditioning control system. Currently, there are 30 young people, 10 elderly people, and 3 children. The current temperature is 32 degrees Celsius. It is known that the elderly prefer an environment of 25-28 degrees Celsius, the young people prefer an environment of 21-25 degrees Celsius, and the children prefer an environment of 23-25 ​​degrees Celsius. How should the air conditioning temperature be adjusted to meet the needs of everyone as much as possible, and output the SQL statement?"

[0085] Finally, the semantic understanding capabilities of LLM are used to perform semantic parsing on the natural language prompts to generate temperature control instructions.

[0086] S140: Convert the temperature control command into a temperature control signal for the temperature regulation device in the target area, and dynamically adjust the temperature in the target area after verification through a digital twin sandbox.

[0087] In some embodiments, the temperature within the target area is dynamically adjusted after pre-verification via a digital twin sandbox, including:

[0088] Set a safety threshold for temperature adjustment, where the safety threshold includes an upper temperature threshold (e.g., 28 degrees Celsius) and a lower temperature threshold (e.g., 20 degrees Celsius);

[0089] Input the temperature control signal into the digital twin sandbox to predict the temperature change trend within a preset time (e.g., within the next 15 minutes);

[0090] If the predicted temperature change trend is within the safe threshold range, the temperature in the target area will be dynamically adjusted according to the temperature control signal.

[0091] If the predicted temperature change trend exceeds the safety threshold, a multi-level alarm (yellow warning, orange warning, red warning) will be triggered, and the system will automatically switch to the backup temperature control scheme (such as prioritizing the activation of the fresh air system to reduce the perceived temperature) and maintain the temperature of the target area within the safety threshold.

[0092] In some embodiments, in order to adjust the air conditioning temperature in a timely manner to cope with changes in personnel flow and provide a comfortable indoor environment, dynamic adjustment of the temperature in the target area may further include dynamically adjusting the temperature control command according to the pattern of personnel flow changes over different time periods, specifically including the following steps:

[0093] By statistically analyzing historical data, an LSTM neural network was used to build a traffic flow prediction model for different time periods. The input parameters of the model include historical traffic flow, holiday labels (weekdays / weekends / holidays), and external environmental data (temperature, humidity, wind speed).

[0094] Based on the pedestrian flow prediction model for different time periods, predict the pedestrian flow within the target time period (e.g., within the next hour).

[0095] Implement temperature pre-adjustment strategies based on the predicted passenger flow during the target time period, for example:

[0096] When it is predicted that the flow of people in the target time period will exceed a preset multiple of the current flow of people (e.g., 1.2 times the current flow of people), the temperature will be adjusted in advance, and the duration of the advance temperature adjustment will be determined according to the predicted growth rate of the flow of people.

[0097] When the predicted rate of change of personnel flow within the target time period exceeds the preset rate of change of personnel flow, the temperature adjustment rate is increased, and the temperature of the target area after adjustment still does not exceed the safety threshold.

[0098] When the predicted flow of people during the target time period is lower than a preset multiple of the current flow of people (e.g., lower than 0.8 times the current flow of people), the temperature is gradually restored to the baseline temperature.

[0099] In some embodiments, the duration of temperature adjustment in advance can be determined based on the predicted rate of increase in passenger flow using the following formula:

[0100]

[0101] Among them, T 提前 In order to adjust the temperature and duration in advance, T 基础 The base time for the temperature control equipment to reach the target temperature is given; the personnel growth rate is the increase in personnel per unit time (persons / minute); and K is the scene correction coefficient (1.2 in summer, 0.8 in winter, 1.0 by default).

[0102] According to the above embodiments of this disclosure, the following technical effects are achieved:

[0103] (1) This disclosure utilizes a multimodal fusion triggering mechanism, combining various triggering methods such as time, changes in the number of people, and deviations in ambient temperature, and dynamically adjusts the triggering conditions. For example, it adjusts the time interval period based on the temporal entropy value of historical personnel flow and corrects the preset temperature fluctuation range based on the intensity of personnel activity, enabling more timely and accurate acquisition of environmental image data. Furthermore, it employs an LLM system for multimodal joint data analysis to identify personnel category attributes, age group numbers, and behavioral characteristics. Combined with dynamic role prompting technology, it generates temperature control instructions, ensuring that temperature regulation accurately matches personnel needs and environmental changes. For instance, it achieves refined temperature control by addressing the differences in temperature preferences among different age groups and the impact of static and dynamic behaviors on thermal comfort.

[0104] (2) This disclosure utilizes the Few-Shot learning capability and dynamic role prompting technology of LLM to reduce the dependence on a large amount of labeled data and can quickly adapt to new scenarios based on a small number of samples; it establishes a traffic flow prediction model through LSTM neural network, and combines historical traffic flow, holiday labels and external environment data to achieve accurate prediction of traffic flow, and executes temperature pre-adjustment strategy accordingly; the digital twin sandbox pre-performance verification mechanism predicts temperature change trends by simulating different temperature control schemes, discovers potential problems in advance and optimizes control strategies, so that the system has stronger intelligent decision-making and adaptive capabilities.

[0105] (3) This disclosure takes into account external environmental factors (summer, winter, peak dining season, etc.) to modify the preset temperature preference rules, and combines real-time environmental data and personnel status in the target area to enable the system to maintain good temperature control effect in a variety of complex scenarios, thereby improving user thermal comfort and user experience.

[0106] (4) The technical solution disclosed herein does not require the deployment of a large number of temperature sensors as in traditional temperature control solutions. Environmental data collection can be completed solely through existing video monitoring systems, reducing hardware costs. Furthermore, it does not require extensive data annotation and model training for specific scenarios. Temperature strategy optimization can be achieved through prompts, greatly reducing development costs.

[0107] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0108] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0109] Figure 2 A block diagram of a knowledge base construction apparatus based on a multimodal large language model according to an embodiment of the present disclosure is shown. Figure 2 As shown, the device 200 includes:

[0110] The environmental image data acquisition module 210 is used to trigger the image acquisition device through a multimodal fusion triggering mechanism to acquire environmental image data of the target area;

[0111] The data analysis and recognition module 220 is used to input the environmental image data into the LLM system for multimodal joint data analysis, identify the category attributes of people in the target area and the number of people in each age group, and at the same time analyze the behavioral state characteristics of people in the target area.

[0112] Temperature control command generation module 230 is used to generate temperature control commands based on the personnel category attributes, the number of people in each age group, behavioral status characteristics, and preset temperature preference rules in the target area, through dynamic role prompting technology.

[0113] The temperature dynamic adjustment module 240 is used to convert the temperature control command into a temperature control signal for the temperature regulation device in the target area, and to dynamically adjust the temperature in the target area after verification through a digital twin sandbox.

[0114] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0115] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0116] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0118] Figure 3A schematic block diagram of an electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0119] Electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in ROM 302 or a computer program loaded into RAM 303 from storage unit 308. RAM 303 can also store various programs and data required for the operation of electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via bus 304. I / O interface 305 is also connected to bus 304.

[0120] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0121] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as method 100. For example, in some embodiments, method 300 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of method 100 described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform method 100 by any other suitable means (e.g., by means of firmware).

[0122] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0124] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0126] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0127] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0128] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0129] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0130] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An environmental temperature control method based on LLM, characterized in that, include: The image acquisition device is triggered by a multimodal fusion triggering mechanism to acquire environmental image data of the target area; The environmental image data is input into the LLM system for multimodal joint data analysis to identify the category attributes of people in the target area and the number of people in each age group, while analyzing the behavioral state characteristics of people in the target area. Based on the personnel category attributes, the number of people in each age group, behavioral characteristics, and preset temperature preference rules within the target area, temperature control instructions are generated through dynamic role prompting technology. The temperature control command is converted into a temperature control signal for the temperature regulation device in the target area, and the temperature in the target area is dynamically adjusted after pre-verification through a digital twin sandbox.

2. The method according to claim 1, characterized in that, The multimodal fusion triggering mechanism includes at least one of the following triggering methods: Time-triggered: Environmental image data acquisition is triggered periodically at preset time intervals, wherein the time interval is dynamically adjusted according to the time entropy value of historical personnel flow. Personnel number change threshold trigger: When the change rate of the total number of personnel detected twice consecutively exceeds the preset change rate threshold, environmental image data acquisition is triggered; Ambient temperature deviation threshold trigger: When the difference between the real-time monitored temperature and the current set temperature exceeds the preset temperature fluctuation range, environmental image data acquisition is triggered. The preset temperature fluctuation range is corrected in real time according to the intensity of human activity in the target area.

3. The method according to claim 1, characterized in that, The environmental image data is input into an LLM system for multimodal joint data analysis to identify the category attributes of people and the number of people in each age group within the target area. Simultaneously, the behavioral characteristics of people within the target area are analyzed, including: Visual semantic features are extracted using the CLIP model, and a multimodal feature space is constructed by fusing thermal imaging and audio data. Based on the Few-Shot learning capability of the LLM system, personnel category attributes are identified, and the number of people in each age group is estimated through Monte Carlo sampling. Combining DINOv2 attitude analysis and optical flow field calculation, the static / dynamic behavioral characteristics and thermal comfort index of personnel are analyzed based on the LLM system.

4. The method according to claim 1, characterized in that, Based on the personnel category attributes, the number of people in each age group, behavioral characteristics, and preset temperature preference rules within the target area, temperature control instructions are generated through dynamic role-based prompting technology, including: Based on a pre-built adaptive weighting model, the weight ratio of the number of people in each age group in temperature control is calculated. Natural language prompts are constructed based on the personnel category attributes, the number of people in each age group, the current ambient temperature, behavioral characteristics, preset temperature preference rules, and the weight ratio of people in each age group in temperature control. The natural language prompts are semantically parsed to generate temperature control instructions.

5. The method according to claim 1, characterized in that, The dynamic adjustment of temperature within the target area after pre-verification via a digital twin sandbox includes: Set a safety threshold for temperature adjustment, wherein the safety threshold includes an upper temperature threshold and a lower temperature threshold; The temperature control signal is input into a digital twin sandbox to predict the temperature change trend within a preset time. If the predicted temperature change trend is within the safe threshold range, the temperature in the target area will be dynamically adjusted according to the temperature control signal. If the predicted temperature change trend exceeds the safety threshold within the safe threshold range, a multi-level alarm is triggered, and the temperature of the target area is maintained within the safety threshold.

6. The method according to claim 5, characterized in that, The method further includes dynamically adjusting the temperature control command based on changes in pedestrian traffic over different time periods, specifically including: Historical data was statistically analyzed, and an LSTM neural network was used to establish a traffic flow prediction model for different time periods. The input parameters of the traffic flow prediction model included historical passenger flow, holiday labels, and external environmental data. Based on the personnel flow prediction model for different time periods, the personnel flow within the target time period is predicted; A temperature pre-adjustment strategy is executed based on the predicted flow of people within the target time period, wherein the temperature pre-adjustment strategy includes the advance temperature adjustment duration and the temperature adjustment rate.

7. The method according to claim 6, characterized in that, The temperature pre-adjustment strategy is executed based on the predicted passenger flow within the target time period. This pre-adjustment strategy includes the duration of advance temperature adjustment and the temperature adjustment rate, comprising: When the predicted flow of people during the target time period exceeds the preset multiple of the existing flow of people, the temperature adjustment duration is determined in advance based on the predicted growth rate of the flow of people. When the predicted rate of change of personnel flow in the target time period exceeds the preset rate of change of personnel flow, the temperature adjustment rate is determined according to the pre-established linear relationship between the rate of change of personnel flow and the temperature adjustment rate, and the temperature of the target area after adjustment still does not exceed the safety threshold. When the predicted flow of people during the target time period is lower than the preset multiple of the current flow of people, the temperature is gradually restored to the baseline temperature.

8. The method according to claim 1, characterized in that, The method further includes: The preset temperature preference rules are modified based on the external environmental factors of the target area. If the external environment is during the high-temperature period of summer, the high-temperature correction factor will be activated, and the preset temperature preference will be adjusted to the product of the original temperature preference and the high-temperature correction factor. If the external environment is in a period of low winter temperatures, the low temperature correction factor will be activated, and the preset temperature preference will be adjusted to the product of the original temperature preference and the low temperature correction factor. If the external environment is during peak dining hours, the personnel density correction factor will be activated, and the preset temperature preference will be adjusted to the product of the original temperature preference and the personnel density correction factor.

9. An environmental temperature control device based on LLM, characterized in that, include: The environmental image data acquisition module is used to trigger the image acquisition device through a multimodal fusion triggering mechanism to acquire environmental image data of the target area; The data analysis and recognition module is used to input the environmental image data into the LLM system for multimodal joint data analysis, identify the personnel category attributes and the number of people in each age group in the target area, and analyze the behavioral state characteristics of the people in the target area. The temperature control instruction generation module is used to generate temperature control instructions based on the personnel category attributes, the number of people in each age group, behavioral status characteristics, and preset temperature preference rules within the target area, using dynamic role prompting technology. The temperature dynamic adjustment module is used to convert the temperature control command into a temperature control signal for the temperature regulation device in the target area, and to dynamically adjust the temperature in the target area after verification through a digital twin sandbox.

10. An electronic device, characterized in that, The electronic device includes: At least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

Images