Intelligent housekeeper system based on multi-modal perception and regulation and control method

Abstract

The invention discloses an intelligent housekeeper system based on multi-modal perception and a regulation and control method, and belongs to the technical field of artificial intelligence, and the system comprises a multi-modal perception layer which is used for obtaining environment parameters of a scene and state data of a scene object; the AI decision center layer is used for generating a regulation and control scheme meeting multi-target balance according to the acquired real-time data; the terminal control layer comprises a general control interface module, an execution terminal and a state feedback unit; the interactive feedback layer is used for realizing bidirectional information interaction between the system and a user and comprises a universal interactive interface and a multi-modal feedback module, the universal interactive interface supports multiple input modes and receives user instructions and settings, and the multi-modal feedback module displays system decision basis, execution progress and abnormal early warning information to the user; according to the intelligent housekeeper system based on multi-modal perception and the regulation and control method provided by the invention, the practicability of the intelligent management system is remarkably improved, and the intelligent housekeeper system has a wide application prospect in multiple scenes such as life, industry, business and the like.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an intelligent home management system and control method based on multimodal perception. Background Technology

[0002] With the rapid development of artificial intelligence and the Internet of Things (IoT) technologies, intelligent management systems have gradually penetrated into various fields such as daily life, industry, and commerce. Their core requirement is to achieve automated and personalized control and management through environmental and object state perception. In existing technologies, intelligent systems for daily life scenarios mostly focus on home appliance control and health monitoring, systems for commercial scenarios emphasize customer flow statistics and marketing conversion, and systems for industrial scenarios focus on equipment operation and maintenance and production efficiency improvement. Related technologies have made some progress: for example, some smart home systems use temperature and humidity sensors to control air conditioners, humidifiers, and other equipment to improve living comfort; equipment monitoring systems in the industrial field use vibration and temperature sensors to monitor equipment operating status and provide fault warnings; and intelligent navigation systems in commercial scenarios guide crowd flow through customer flow statistics to optimize space utilization efficiency. However, existing technologies still have the following shortcomings: most existing systems are customized for specific scenarios, lacking versatility across multiple scenarios; and they rely on single-dimensional data for decision-making, resulting in an inability to match the core needs of different scenarios. Summary of the Invention

[0003] The purpose of this invention is to provide an intelligent home management system and control method based on multimodal perception, so as to solve the problems existing in the above-mentioned background technology.

[0004] To achieve the above objectives, the present invention provides an intelligent butler system based on multimodal perception, comprising: The multimodal perception layer is used to acquire environmental parameters and state data of scene objects. Through heterogeneous sensors and adapter interfaces, it can achieve real-time acquisition, preliminary filtering and standardization of data of different dimensions and types. The AI ​​decision-making center layer communicates and connects with the multimodal perception layer, and generates a control scheme that satisfies the balance of multiple objectives based on the real-time data obtained by the multimodal perception layer. The terminal control layer communicates with the AI ​​decision-making center layer and includes a general control interface module, execution terminals and a status feedback unit. The general control interface module supports standardized docking of execution terminals with different protocols and types. The execution terminals perform corresponding operations according to the control scheme generated by the AI ​​decision-making center layer. The status feedback unit collects the operating parameters and execution results of the execution terminals in real time to form a closed-loop control data chain. The interactive feedback layer is used to realize two-way information interaction between the system and the user. It includes a general interactive interface and a multimodal feedback module. The general interactive interface supports multiple input methods and receives user commands and settings. The multimodal feedback module displays the system's decision-making basis, execution progress, and abnormal warning information to the user.

[0005] Preferably, the AI ​​decision-making central layer includes a scene recognition module, a dynamic target weight engine, a multi-dimensional optimization solution module, and a self-learning update module, wherein: The scene recognition module is used to fuse and analyze the environmental parameters and object state data acquired by the multimodal perception layer, extract scene feature vectors, and identify the corresponding scene. The dynamic target weight engine is used to dynamically allocate priority weights for each target based on real-time feature vectors and a multi-dimensional target system. The multi-dimensional optimization solution module generates a control scheme that satisfies multiple objectives based on weight constraints and the range of decision variables. The self-learning update module continuously optimizes the logic of the scene feature extraction module and the dynamic target weight engine by receiving feedback data.

[0006] This invention also provides a smart home control method based on multimodal perception, comprising the following steps: S1. Real-time acquisition of environmental data and state data of scene objects, which are then preprocessed and transmitted to the AI ​​decision-making center layer; S2, the AI ​​decision-making center layer extracts the static and dynamic features of the acquired data, outputs the probability of scene type, and determines the current scene type; S3. Assign real-time weights to control targets based on scenario type; S4. Generate a terminal control scheme that satisfies the balance of multiple objectives under weight constraints; S5. The terminal control layer executes the corresponding control commands, the interactive feedback layer pushes the execution results and receives user adjustments, and the system records and updates user preferences.

[0007] Preferably, in step S1, the obtained environmental parameter vector and object state vector Standardization is performed using Min-Max normalization: ; in, The original data, , These are the maximum and minimum values ​​of the data.

[0008] Preferably, step S2 specifically includes: S21. Based on the data from step S1, construct a multi-dimensional feature vector set, including static feature vectors and dynamic feature vectors. The feature vector dimensions are combined into 2N+5N according to the data type, where N is the number of core parameters. S22. Construct a parallel LSTM network architecture, inputting different types of feature vectors into dedicated LSTM channels, with each channel extracting features through a gating mechanism; then concatenate and fuse the feature vectors output from each channel to obtain a high-dimensional general feature vector. It retains both temporal dependency features and static association features; S23. Input the fused feature vector into the fully connected layer and output the scene type probability through the softmax function. ,in: ; in, The logits value for the scene. For the total scene type; when the probability of a certain scene is... When the current scene is identified as the main scene, the sub-scene is located by combining the sub-features in the feature vector.

[0009] Preferably, the expression for the parallel LSTM network architecture in step S22 is: ; in, , , These are the input gate, forget gate, and output gate, respectively. For memory cell state, Output in hidden state , , , Both are weight matrices. , , , For bias terms, For activation function, for The output of the LSTM hidden layer at time 1. The input feature vector at the current time step. for The state of memory cells at any given moment.

[0010] Preferably, step S3 assigns real-time weights to multi-dimensional control targets based on the identified scene type, ensuring that the target priority matches the scene requirements, including: S31. Define the target system Based on the feature vector output in step S22 Calculate the real-time value of each objective. : ; in, These are the feature values ​​corresponding to the target in the feature vector. This represents the ideal eigenvalue of the target. S32. Introduce Generalized Advantage Estimation (GAE) to optimize objective value assessment: ; ; in, for The advantage estimate at time, The total duration of the time series data. This represents the offset at a future time. As a discount factor, To balance the parameters, for The timing difference error at time t. for The timing difference error at time 10:00. For the state value function, For instant rewards, for The state-value function at time t; S33. The PPO pruning mechanism is used to constrain the weight update magnitude. The weight allocation formula is as follows: ; in, For the first Scenario-specific learning coefficients for each target. The overall control target.

[0011] Preferably, step S4 generates an optimal control scheme that satisfies multi-objective balance, using the control parameters as decision variables, based on the weight allocation result of step S33. Specifically: S41, Decision Variables It includes control parameters and action commands, the range of which is determined by the physical limitations of the equipment and safety specifications. The constraints include equipment operation constraints, power balance constraints and safety threshold constraints. S42, Combining weights Construct the overall objective function: ; in, Decision variables The corresponding actual target value, The target ideal value; S43. Use PPO-NSGA-Ⅲ to solve for the optimal solution and select the largest solution as the terminal control scheme.

[0012] Preferably, step S43 specifically includes: Initialize the population: 30% generated by the deterministic PPO strategy, 30% by the stochastic PPO strategy, and 40% by random generation. The population size is dynamically adjusted according to the variable dimensions. Adaptive crossover mutation: Adjusting the crossover rate using a cosine decreasing strategy. and variability : ; ; in, This represents the minimum crossover probability. This represents the maximum crossover probability. This represents the current iteration number. The maximum number of iterations, This represents the minimum probability of mutation. This represents the maximum mutation probability. Non-dominated sorting and optimal solution selection: Pareto optimal solutions are selected using the reference point mechanism of NSGA-III, and then the relative proximity is calculated using the TOPSIS method. ; ; ; in, For the first The Euclidean distance from each candidate solution to the ideal solution. For the first The Euclidean distance from each candidate solution to the negative ideal solution. For the first The relative similarity of the candidate solutions For the first The candidate solution is in the... Deviation value on each target For the first The optimal deviation value for each target. For the first The worst deviation value of each target; choose The largest solution is taken as the final control scheme.

[0013] Preferably, in step S5, the general control interface module of the terminal control layer parses the generated control scheme into standardized control commands and sends them to the execution terminal. The execution terminal performs the operation according to the control commands, and the status feedback unit collects the terminal operation data and result data in real time and sends them back to the AI ​​decision center layer.

[0014] Therefore, the present invention employs the above-mentioned intelligent butler system and control method based on multimodal perception, which has the following beneficial effects: (1) It adopts a general hardware architecture and algorithm framework, which eliminates the need for customized development for specific scenarios and adapts to various scenarios such as life, industry, and commerce, thereby reducing hardware investment; (2) The control scheme is based on multiple parameters. Compared with the use of a single parameter, this control scheme is more in line with the core needs of the scenario. (3) Based on the improved PPO algorithm, the priority of core objectives in different scenarios and under different states is guaranteed by dynamically allocating and adjusting the target weights, so as to achieve dynamic balance of different objectives; (4) By receiving user instructions and preference feedback through the interactive feedback layer, the system gradually adapts to the user's personalized needs, thereby improving the practicality of human-computer collaboration.

[0015] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of an intelligent butler system based on multimodal perception according to the present invention; Figure 2 This is a flowchart of a smart home control method based on multimodal perception according to the present invention. Detailed Implementation

[0017] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0018] Please see Figure 1 A smart home management system based on multimodal perception includes: The multimodal perception layer is used to acquire environmental parameters (temperature, humidity, energy consumption, etc.) and state data of scene objects (human biological indicators, passenger flow density, equipment vibration, etc.). Through heterogeneous sensors and adapter interfaces, it can achieve real-time acquisition, preliminary filtering and standardization of data of different dimensions and types. The AI ​​decision-making central layer communicates with the multimodal perception layer and generates a control scheme that satisfies the balance of multiple objectives based on real-time data acquired by the multimodal perception layer; including: The scene recognition module is used to fuse and analyze the environmental parameters and object state data acquired by the multimodal perception layer, extract scene feature vectors, and identify the corresponding scene. The dynamic target weight engine is used to dynamically allocate priority weights for each target based on real-time feature vectors and a multi-dimensional target system. The multi-dimensional optimization solution module generates a control scheme that satisfies multiple objectives based on weight constraints and the range of decision variables. The self-learning update module continuously optimizes the logic of the scene feature extraction module and the dynamic target weight engine by receiving feedback data.

[0019] The terminal control layer communicates and connects with the AI ​​decision-making center layer. It includes a general control interface module, an execution terminal and a status feedback unit. It connects to terminal devices in living, industrial and commercial scenarios, executes control commands and provides feedback on operating status, and integrates a cross-scenario safety monitoring module to achieve risk-level response. The interactive feedback layer is used to realize two-way information interaction between the system and the user. It displays the decision results to the user through the terminal device and receives adjustment instructions to achieve human-machine collaborative optimization.

[0020] like Figure 2 As shown, the control method based on the above-mentioned smart home system includes the following steps: S1. Real-time acquisition of environmental data and object status data for the scene. Environmental parameters include temperature and humidity, air quality, energy consumption load, acoustic parameters (noise), optical parameters (illuminance), and spatial field strength. Object status data includes physiological indicators of biological objects, behavioral characteristics of group objects, operating parameters of equipment objects, and attributes and status of material objects. The acquired environmental parameter vectors... and object state vector Standardization is performed using Min-Max normalization: ; in, The original data, , These are the maximum and minimum values ​​of the data.

[0021] S2, the AI ​​decision-making center layer extracts static and dynamic features from the acquired data, outputs scene type probability, and determines the current scene type; specifically including: S21. Based on the data from step S1, construct a multi-dimensional feature vector set, including static feature vectors and dynamic feature vectors. The feature vector dimensions are combined into 2N+5N according to the data type, where N is the number of core parameters. S22. Construct a parallel LSTM network architecture, inputting different types of feature vectors into dedicated LSTM channels, and extracting features from each channel through a gating mechanism: ; in, , , These are the input gate, forget gate, and output gate, respectively. For memory cell state, Output in hidden state , , , Both are weight matrices. , , , For bias terms, For activation function, for The output of the LSTM hidden layer at time 1. The input feature vector at the current time step. for The memory cell state at any given time; the feature vectors output from each channel are concatenated and fused to obtain a high-dimensional universal feature vector. It retains both temporal dependency features and static association features; S23. Input the fused feature vector into the fully connected layer and output the scene type probability through the softmax function. ,in: ; in, The logits value for a scene, when the probability of a certain scene is... When the current scene is identified as the main scene, the sub-scene is located by combining the sub-features in the feature vector.

[0022] S3. Based on the identified scene type, assign real-time weights to multi-dimensional control targets to ensure that target priorities match scene requirements, including: S31. Define the target system Adjust the target connotation based on the scene label in step S2 (e.g., in everyday life scenes). (For health compatibility, in industrial scenarios it's device compatibility); based on the feature vector output in step S22. Calculate the real-time value of each objective. : ; in, These are the feature values ​​corresponding to the target in the feature vector. This represents the ideal eigenvalue of the target. S32. Introduce Generalized Advantage Estimation (GAE) to optimize objective value assessment: ; ; in, for The advantage estimate at time, The total duration of the time series data. This represents the offset at a future time. As a discount factor, To balance the parameters, for The timing difference error at time 10:00. for The timing difference error at time 10:00. For the state value function, For instant rewards, for The state-value function at time t; S33. The PPO pruning mechanism is used to constrain the weight update magnitude. The weight allocation formula is as follows: ; in, For the first Scenario-specific learning coefficients for each objective. Adding weighted constraints for specific scenarios: everyday life scenarios. Industrial scenarios (Raise to when equipment malfunctions) ), peak hours in commercial scenarios .

[0023] S4. Based on the weight allocation results of step S33, using the control parameters as decision variables, generate the optimal control scheme that satisfies the balance of multiple objectives, specifically as follows: S41, Decision Variables It includes control parameters and action commands, the range of which is determined by the physical limitations of the equipment and safety specifications. The constraints include equipment operation constraints, power balance constraints and safety threshold constraints. S42, Combining weights Construct the overall objective function: ; in, Decision variables The corresponding actual target value, The target ideal value; S43. Employ the PPO-NSGA-Ⅲ collaborative solution to obtain the optimal solution, and select the solution with the largest value as the terminal control scheme. Specifically, this includes: Initialize the population: 30% generated by the deterministic PPO strategy, 30% by the stochastic PPO strategy, and 40% by random generation. The population size is dynamically adjusted according to the variable dimensions. Adaptive crossover mutation: Adjusting the crossover rate using a cosine decreasing strategy. and variability : ; ; in, This represents the minimum crossover probability. This represents the maximum crossover probability. This represents the current iteration number. The maximum number of iterations, This represents the minimum probability of mutation. This represents the maximum mutation probability. Non-dominated sorting and optimal solution selection: Pareto optimal solutions are selected using the reference point mechanism of NSGA-III, and then the relative proximity is calculated using the TOPSIS method. ; ; ; in, For the first The Euclidean distance from each candidate solution to the ideal solution. For the first The Euclidean distance from each candidate solution to the negative ideal solution. For the first The relative similarity of the candidate solutions For the first The candidate solution is in the... Deviation value on each target For the first The optimal deviation value for each target. For the first The worst deviation value of each target; choose The largest solution is taken as the final control scheme.

[0024] The S5 terminal control layer's general control interface module parses the generated control scheme into standardized control commands and sends them to the execution terminal. The execution terminal performs operations according to the control commands, and the status feedback unit collects terminal operation data and result data in real time and sends them back to the AI ​​decision-making center layer.

[0025] To verify the practicality of the aforementioned smart home system, this embodiment uses a three-person urban residential living scenario (two bedrooms, one living room, one kitchen, one bathroom, with a building area of ​​90m²) as an example. 2Taking this example, the core requirements include: ① health adaptation for family members (elderly people with high blood sugar, children with allergies, adults who sit for long periods at work); ② optimization of living comfort (temperature and humidity, air quality, noise control); ③ energy cost savings; ④ home safety protection (gas leak, fire, fall warning). The system deployment covers the living room, master bedroom (elderly residence), secondary bedroom (children's residence), study (adult office), kitchen, and bathroom, realizing 24-hour intelligent control and health management across all scenarios. The deployment of the multimodal perception layer and terminal control layer is shown in Tables 1 and 2.

[0026] Table 1 Deployment of Multimodal Sensing Layer

[0027] Table 2 Terminal Control Layer Deployment

[0028] Experimental results: Health fit: Blood glucose levels in the elderly dropped to 6.5 mmol / L (normal range) after 2 hours; dust mite concentrations in children met the standards; and sedentary time in adults decreased by 40%. Comfort level: The temperature and humidity throughout the house are stable at 24-26℃ and 45%-55%, with excellent air quality (PM2.5 < 20 μg / m³). 3 (CO2 < 800 ppm) Energy consumption control: Total energy consumption is reduced by 8% compared to traditional manual control. Energy consumption of core equipment (air conditioner and air purifier) ​​is increased but does not exceed the preset threshold. Safety protection: The system continuously monitors gas and fall risks, and no abnormal warnings are triggered, effectively verifying the proactive protection capabilities.

[0029] Therefore, the present invention adopts the above-mentioned intelligent butler system and control method based on multimodal perception, which significantly improves the practicality of the intelligent management system and has broad application prospects in various scenarios such as life, industry, and commerce.

[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A smart butler system based on multi-modal perception, characterized in that, Comprise: A multi-modal perception layer for acquiring environmental parameters of a scene and state data of scene objects, realizing real-time acquisition, preliminary filtering and standardization processing of different dimensions and different types of data through a heterogeneous sensor and an adaptive interface; An AI decision hub layer in communication connection with the multi-modal perception layer, generating a regulation scheme meeting multi-objective balance according to real-time data acquired by the multi-modal perception layer; A terminal control layer in communication connection with the AI decision hub layer, comprising a general control interface module, an execution terminal and a state feedback unit, the general control interface module supporting standardized docking of different protocols and different types of execution terminals, the execution terminal executing corresponding operations according to the regulation scheme generated by the AI decision hub layer, and the state feedback unit acquiring running parameters and execution results of the execution terminal in real time to form a closed-loop control data chain; An interactive feedback layer for realizing bidirectional information interaction between the system and the user, comprising a general interactive interface and a multi-modal feedback module, the general interactive interface supporting multiple input modes to receive user instructions and settings, and the multi-modal feedback module showing the user system decision basis, execution progress and abnormal warning information.

2. The intelligent housekeeper system based on multi-modal perception according to claim 1, characterized in that, The AI decision hub layer comprises a scene recognition module, a dynamic target weight engine, a multi-dimensional optimization solving module and a self-learning updating module, wherein: The scene recognition module is used for fusion analysis of the environmental parameters and the object state data acquired by the multi-modal perception layer, extraction of a scene feature vector and recognition of a corresponding scene; The dynamic target weight engine is used for dynamically allocating priority weights of each target according to real-time feature vectors and a multi-dimensional target system; The multi-dimensional optimization solving module generates a regulation scheme meeting multi-objective balance based on weight constraints and decision variable ranges; The self-learning updating module continuously optimizes the logic of the scene feature extraction module and the dynamic target weight engine by receiving feedback data.

3. The method of claim 1 or 2, wherein the method is applied to the system of claim 1 or 2. Comprise the following steps: S1, real-time acquisition of environmental data of a scene and state data of scene objects, and transmission of the preprocessed data to an AI decision hub layer; S2, the AI decision hub layer extracts static and dynamic features of the acquired data, outputs a scene type probability, and determines a current scene type; S3, based on the scene type, real-time weights are allocated to regulation targets; S4, a terminal regulation scheme meeting multi-objective balance is generated under weight constraints; S5, a terminal control layer executes corresponding regulation instructions, an interactive feedback layer pushes execution results and receives user adjustments, and the system records user preferences and updates.

4. The multi-modal perception based smart butler regulation method of claim 3, wherein, The acquired environmental parameter vector in step S1 is normalized by using Min-Max normalization: and the object state vector is normalized by using Min-Max normalization: ； wherein is the original data, , is the maximum and minimum values of the data.

5. The multi-modal perception based smart butler regulation method of claim 3, wherein, Step S2 specifically comprises: S21, based on the data of step S1, a multi-dimensional feature vector set is constructed, comprising static feature vectors and dynamic feature vectors, and the feature vector dimension is combined as 2N+5N according to the data type, wherein N is the number of core parameters; S22, build a parallel LSTM network architecture, input different types of feature vectors into exclusive LSTM channels respectively, and extract features through a gating mechanism in each channel; and splice and fuse the feature vectors output by the channels to obtain a high-dimensional general feature vector while retaining the time-dependent features and static correlation features S23, input the fused feature vector into a full connection layer, and output a scene type probability through a softmax function wherein: ； wherein, is the logits value of the scene, is the total scene type; when a certain scene probability is determined as the current main scene, and the sub-scene is located in combination with the sub-feature in the feature vector.

6. The multi-modal perception based smart butler regulation method of claim 5, wherein, The parallel LSTM network architecture expression in step S22 is: ； in, , , These are the input gate, forget gate, and output gate, respectively. For memory cell state, Output in hidden state , , , Both are weight matrices. , , , For bias terms, For activation function, for The output of the LSTM hidden layer at time 1. The input feature vector at the current time step. for The state of memory cells at any given moment.

7. The multi-modal perception based smart butler regulation method of claim 3, wherein, Step S3 allocates real-time weights to multi-dimensional regulation targets based on the recognized scene type to ensure that the target priority matches the scene demand, comprising: S31, defining target system based on the feature vector outputted in step S22 calculating the instant value of each target : ； wherein, is a characteristic value of the target in the feature vector, is an ideal characteristic value of the target; S32, a generalized advantage estimation GAE is introduced to optimize target value evaluation: ； ； wherein, is the advantage estimate at time step t, is the total duration of the time series data, is the future time step offset, is the discount factor, is the trade-off parameter, is the time series difference error at time step t, is the time series difference error at time step t, is the state value function, is the immediate reward, is the state value function at time step t; S33, a PPO clipping mechanism is used to constrain the weight update amplitude, and the weight allocation formula is: ； wherein, is the scene-specific learning coefficient for the th target, is the total control target. 8.The multi-modal perception based smart butler regulation method of claim 7, wherein, Step S4 generates an optimal regulation scheme satisfying multi-objective balance according to the weight distribution result of step S33, with the regulation parameter as the decision variable, specifically: S41, decision variable , including control parameters and action instructions, the variable range is determined by the physical limitations and safety specifications of the device, and the constraint conditions include device operation constraints, power balance constraints, and safety threshold constraints; S42, combine weights , construct total objective function: ； wherein, is a decision variable corresponding target actual value, is a target ideal value; S43, adopt PPO-NSGA-Ⅲ to solve the optimal solution, and select the largest solution as the terminal regulation scheme. 9.The multi-modal perception based smart butler regulation method of claim 8, wherein, Step S43 specifically includes: Initialize the population: 30% is generated by PPO deterministic strategy, 30% is generated by PPO random strategy, and 40% is randomly generated. The population size is dynamically adjusted according to the variable dimension; Adaptive crossover mutation: cosine decreasing strategy is adopted to adjust the crossover rate and mutation rate : ； ； wherein, is a minimum value of the crossover probability, is a maximum value of the crossover probability, is a current iteration number, is a maximum iteration number, is a minimum value of the mutation probability, is a maximum value of the mutation probability; Non-dominated sorting and optimal solution selection: filter the Pareto optimal solution through the reference point mechanism of NSGA-Ⅲ, and then calculate the relative closeness by the TOPSIS method: ； ； ； wherein, is the Euclidean distance of the th candidate solution to the positive ideal solution, is the Euclidean distance of the th candidate solution to the negative ideal solution, is the relative closeness of the th candidate solution, is the deviation value of the th candidate solution on the th objective, is the optimal deviation value of the th objective, is the worst deviation value of the th objective; selecting The largest solution as the final control scheme. 10.The smart butler regulation method based on multi-modal perception of claim 3, wherein, In step S5, the general control interface module of the terminal control layer parses the generated regulation scheme into standardized control instructions, which are issued to the execution terminal. The execution terminal executes the operation according to the control instructions. The state feedback unit collects terminal operation data and result data in real time and returns them to the AI decision hub layer.

Images