A smart curtain control method based on control pyramid and multimodal interaction

Abstract

The present invention discloses an intelligent curtain control method based on control pyramid and multimodal interaction, including: collecting real-time ambient light intensity, real-time multi-point temperature and humidity to generate first-level curtain control features; capturing on-site environmental sound features, separating voice commands from environmental noise in real time, and generating second-level curtain control features; regularly requesting control instructions from user terminal devices, analyzing and converting the control instructions, and generating third-level curtain control features; correlating and analyzing the first, second, and third-level curtain control features based on a multimodal fusion algorithm to form a curtain control pyramid; learning, converting, filling, and bidirectionally extracting the curtain control pyramid to determine the control intent, and inputting it into a curtain control model to control the opening and closing state; real-time monitoring of the motion state of the curtain, and feeding back the monitoring information to the curtain control model for dynamic adjustment. The present invention improves the intelligent level, accuracy, and adaptability of curtain control.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal interactive control, and in particular to an intelligent curtain control method based on a control pyramid and multimodal interaction. Background Art

[0002] With the rapid development of smart home technology, curtains, as an important part of the home environment, are gradually changing from traditional manual operation to intelligent control. Existing smart curtain control systems usually rely on a single control method, such as timer-based control, remote control control, light sensor control, or voice command control. However, these control methods have certain limitations in practical applications.

[0003] For example, the timer-based control method cannot dynamically adapt to real-time changes in the environment, resulting in the curtain opening and closing status not meeting user needs; although the light-sensing-based control method can achieve environmental adaptability, it is prone to insufficient control accuracy in complex environments, such as strong light interference or multi-source light changes; the voice-command-based control method requires high-precision noise suppression and voice recognition capabilities, but may reduce the reliability of voice recognition in noisy environments. In addition, existing systems often find it difficult to achieve fusion processing of multimodal information, and are unable to comprehensively consider multiple factors such as light intensity, temperature and humidity, sound, and user commands for refined control.

[0004] In the research and application of intelligent curtain control technology, how to achieve efficient fusion and dynamic response of multimodal information to meet users' personalized and refined needs for the opening and closing status of curtains has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides an intelligent curtain control method based on a control pyramid and multimodal interaction, which is used to solve the technical problems of low intelligence level, low precision and poor adaptability of the existing curtain intelligent control method, thereby improving the intelligence level, precision and adaptability of curtain control by integrating multiple perception methods to interact with users.

[0006] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0007] An intelligent curtain control method based on control pyramid and multimodal interaction comprises the following steps:

[0008] Collect real-time ambient light intensity, real-time multi-point temperature and humidity to generate first-level curtain control features;

[0009] Capture the sound characteristics of the on-site environment, separate voice commands from ambient noise in real time, and generate second-level curtain control features based on voice commands;

[0010] Regularly request control instructions from user terminal equipment, analyze and convert the control instructions, and generate third-level curtain control features;

[0011] The first, second and third level curtain control features are correlated and analyzed based on the multimodal fusion algorithm to form a curtain control pyramid;

[0012] The curtain control pyramid is studied, converted, filled, and bidirectionally extracted to determine the control intent;

[0013] Input the control intention into the curtain control model to perform fine control of the curtain's opening and closing status;

[0014] Monitor the movement status of the curtains in real time, and feed the real-time monitoring information back to the curtain control model for dynamic adjustment to control the effect;

[0015] Among them, the multimodal fusion algorithm includes:

[0016] By mapping the first, second and third level curtain control features, shared features and independent features are obtained, and the shared features and independent features are spliced ​​to obtain combined features;

[0017] A regression coefficient matrix is ​​constructed for the combined features, and a curtain control pyramid is formed by combining the combined features.

[0018] As a preferred embodiment of the present invention, when collecting real-time ambient light intensity, it includes:

[0019] The image plane of the light intensity detector is divided into multiple detection element areas according to the x coordinate, the real-time light is dispersed by the echelle grating, and the light is split by the transmission prism so that the light of different wavelengths falls on the image plane of the light intensity detector;

[0020] Obtaining the central wavelengths of different grating diffraction orders of the echelle grating;

[0021] According to the central wavelength, the prism refractive index at different central wavelengths is obtained, and further the relationship between the exit angle and the prism refractive index after the light of different wavelengths is incident from the echelle grating to the transmission prism is obtained;

[0022] According to the relationship between the exit angle and the refractive index of the prism, the difference between the exit angles of different central wavelengths and the selected reference wavelength is determined, and the detection element area where the light falls on the image plane of the light intensity detector is determined by the focal length of the focusing lens;

[0023] A fitting relationship between the detection element area and the diffraction order is established to determine the grating diffraction order and real-time light intensity value of the real-time light.

[0024] As a preferred embodiment of the present invention, when collecting real-time temperature and humidity, it includes:

[0025] Use the underlying data registers to collect multi-point temperature and humidity data in real time through the serial port, and parse and process the multi-point temperature and humidity data through the interrupt processing function;

[0026] Among them, when parsing and processing, it includes:

[0027] The cross correction coefficients of the multi-point temperature and humidity sensors are obtained, and a cross correction coefficient matrix is ​​constructed based on the cross correction coefficients, as shown in Formula 7:

[0028] (7);

[0029] In the formula, For the The temperature sensor and Cross correction factor between humidity sensors, is the number of temperature and humidity sensors, is the cross correction coefficient matrix;

[0030] Based on the cross correction coefficient matrix, accurate multi-point temperature and humidity data are obtained, as shown in Formula 8:

[0031] (8);

[0032] In the formula, , They are the actual multi-point temperature and humidity data collected, for The temperature data collected by the temperature sensor for The humidity data collected by the humidity sensor is , Expressing the The temperature sensor, All cross correction coefficients associated with the humidity sensors are averaged. , They are the corrected multi-point temperature and humidity data respectively.

[0033] As a preferred embodiment of the present invention, when capturing the sound characteristics of the on-site environment, it includes:

[0034] The captured on-site environmental sound signal is divided into frames to obtain several frames of sound signals, and each frame of sound signal is extracted dimensional feature vector;

[0035] Based on the dimensional feature vector, obtain the first The mean and variance of the dimension, where ;

[0036] Perform linear transformation on each frame of sound signal to eliminate the scale differences between different feature vectors;

[0037] Use the Gaussian mixture model to approximate the probability density distribution of the feature parameters, and further obtain the cumulative distribution function;

[0038] Acquire the spectrum characteristics of the on-site environmental sound, and form a noise suppression feature vector with the cumulative distribution function;

[0039] Given standard speech data, iterative estimation is performed based on the noise suppression feature vector to suppress the on-site environmental noise;

[0040] The spectral features include Characteristic spectrum and Score.

[0041] As a preferred embodiment of the present invention, when generating the second level curtain control feature, it includes: performing lexical and grammatical analysis on the voice command separated in real time to generate the second level curtain control feature;

[0042] Wherein, the real-time separation includes:

[0043] The noise-suppressed sound signal is input into the separation model, and the features of the sound signal are extracted through the front module in the separation model. After being processed and converted by the encoding layer composed of a deep CNN network, it is decoded by the CTC model and attention mechanism in the decoding layer, while the bidirectional long short-term memory network is used to take into account the contextual information.

[0044] As a preferred embodiment of the present invention, when analyzing and converting the control instruction, it includes:

[0045] Determine whether the control command is a text command or a voice command. If it is a text command, it is directly converted into a third-level curtain control feature. If it is a voice command, it is converted into a third-level curtain control feature after lexical and grammatical analysis.

[0046] When performing lexical and grammatical analysis, this includes:

[0047] The voice command string is scanned one by one by the lexical analyzer to form a lexical unit sequence. The grammatical analyzer performs recursive descent grammatical analysis on the input lexical unit sequence based on the grammar tree nodes and according to the strategy grammar to obtain a grammatical analysis tree, and generates the second or third level curtain control features based on the grammatical analysis tree.

[0048] As a preferred embodiment of the present invention, when obtaining the combined features, it also includes:

[0049] The first, second and third level curtain control features are mapped through the transformation matrix to obtain the shared features among the three levels of curtain control features. , and the three levels of curtains have their own unique independent features , , , and concatenate them to get the combined features ;

[0050] When constructing the regression coefficient matrix and forming the curtain control pyramid, it also includes:

[0051] Get the corresponding shared features , independent features , , The first, second, third, and fourth weight coefficients , , , , and construct the regression coefficient matrix ;

[0052] Based on the combined features and the regression coefficient matrix, the curtain control pyramid is formed, as shown in Formula 15:

[0053] (15);

[0054] In the formula, Control pyramid for curtains.

[0055] As a preferred embodiment of the present invention, when determining the control intention, it includes:

[0056] Using self-learning The curtain control pyramid is learned and transformed in a layer to obtain an initial effective feature set;

[0057] Filling the initial valid feature set with blank vectors until the maximum length of the sentence is reached to obtain a final valid feature set;

[0058] Based on The established bidirectional semantic extraction network performs bidirectional extraction on the final effective feature set to obtain a vector group expression of the curtain control pyramid in the feature space and a high-dimensional semantic vector representation;

[0059] The control intention is determined according to the vector group expression and the high-dimensional semantic vector representation.

[0060] As a preferred embodiment of the present invention, when performing fine opening and closing state control of the curtain, it includes:

[0061] Get the armature voltage of the curtain DC motor , and place the DC motor in Input voltage at the moment Fuzzy for the armature voltage As a fuzzy controller Output:

[0062] The control intention is combined with historical data to determine the target light intensity value of the current control area, which is compared with the real-time light intensity value contained in the first-level curtain control feature to obtain the real-time light intensity difference, which is used as the fuzzy controller The target input voltage is obtained.

[0063] The real-time input voltage of the DC motor is collected and compared with the target input voltage to obtain the input voltage difference. The input voltage difference is used as the fuzzy Controller input, and then determine Control the proportional gain change, integral gain change, differential gain change, and output The pulse signal is adjusted instantly until the input voltage difference is less than the preset threshold.

[0064] As a preferred implementation mode of the present invention, when performing dynamic adjustment, it includes:

[0065] Combining the control intention with historical data to determine the target travel amount of the curtain;

[0066] The stroke amount obtained by real-time monitoring is compared with the target stroke amount to obtain the stroke deviation amount, and further obtain the stroke deviation change rate;

[0067] The stroke deviation and stroke deviation change rate are taken as input and passed through The established fuzzy control table is Control the proportional gain change, integral gain change, and differential gain change for dynamic adjustment.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] (1) Accurate environmental perception and intelligent response

[0070] The present invention forms a multi-modal interactive control system by integrating real-time ambient light intensity, temperature and humidity, sound characteristics and user control instructions, which can sense environmental changes in real time and automatically adjust the opening and closing state of the curtains. Compared with the traditional single control method, it can more accurately adapt to different environmental conditions, such as indoor lighting changes, temperature and humidity fluctuations, and background noise interference, realize dynamic optimization of curtain opening and closing, and improve user living comfort and energy efficiency.

[0071] (2) Self-learning and refined control

[0072] The present invention introduces a multimodal fusion algorithm and a self-learning mechanism to enable the curtain control system to self-optimize according to real-time data. By learning ambient light intensity, temperature and humidity changes, sound signals, and user feedback, the system can gradually improve control accuracy and response speed, avoiding the limitations of human intervention and timing control, and providing a more intelligent and personalized curtain control experience.

[0073] (3) Noise suppression and accurate recognition of voice commands

[0074] The present invention adopts advanced noise suppression algorithms and deep learning technology, which can effectively separate voice commands from background noise in the case of high noise on-site environment, and improve the accuracy and response speed of voice recognition. Even in an environment with high noise interference, the user's voice commands can still be accurately recognized, which enhances the convenience of user operation.

[0075] (4) Fusion and optimization of multi-level control features

[0076] The present invention generates different levels of curtain control features by combining light intensity, temperature and humidity, sound characteristics and user instructions, and performs correlation analysis through a multimodal fusion algorithm to form a curtain control pyramid. This innovative structure can integrate data from different sources, automatically learn and adjust control intentions, and achieve refined and dynamic curtain control.

[0077] (5) Flexible dynamic adjustment and feedback mechanism

[0078] The present invention monitors the movement state of the curtain in real time and feeds the monitoring information back to the curtain control model to achieve dynamic adjustment. This feedback mechanism can ensure that the curtain always maintains the optimal opening and closing state during actual use, reduce energy waste, and make precise adjustments according to user needs.

[0079] (6) Energy conservation and environmental adaptability

[0080] The present invention combines real-time light intensity and temperature and humidity information during the curtain control process, and can effectively optimize the opening and closing state of the curtains, thereby automatically adjusting indoor light and temperature under different seasons and environmental conditions, avoiding energy waste, and improving the comfort and energy efficiency of the living environment.

[0081] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 It is a step diagram of the intelligent curtain control method based on control pyramid and multimodal interaction provided by the present invention. DETAILED DESCRIPTION

[0083] The intelligent curtain control method based on control pyramid and multimodal interaction provided by the present invention is as follows: Figure 1 As shown, the following steps are included:

[0084] Step S1: Collecting real-time ambient light intensity, real-time multi-point temperature and humidity to generate first-level curtain control features;

[0085] Step S2: capturing the sound characteristics of the on-site environment, separating the voice command from the ambient noise in real time, and generating the second-level curtain control characteristics based on the voice command;

[0086] Step S3: regularly requesting control instructions from the user terminal device, analyzing and converting the control instructions, and generating third-level curtain control features;

[0087] Step S4: performing correlation analysis on the first-level curtain control features, the second-level curtain control features, and the third-level curtain control features based on a multimodal fusion algorithm to form a curtain control pyramid;

[0088] Step S5: learning, converting, filling, and bidirectionally extracting the curtain control pyramid to determine the control intention;

[0089] Step S6: input the control intention into the curtain control model to perform fine opening and closing state control of the curtain;

[0090] Step S7: monitor the motion state of the curtain in real time, and feed the real-time monitoring information back to the curtain control model for dynamic adjustment to achieve the control effect;

[0091] Among them, the multimodal fusion algorithm includes:

[0092] By mapping the first, second and third level curtain control features, shared features and independent features are obtained, and the shared features and independent features are spliced ​​to obtain combined features;

[0093] A regression coefficient matrix is ​​constructed for the combined features, and the combined features are combined to form a curtain control pyramid.

[0094] Furthermore, when separating the voice command from the ambient noise in real time, if no new voice command is retrieved, the voice command at this time is determined based on the historical data and used to generate the second level curtain control feature;

[0095] When requesting control instructions from the user terminal device, if no new control instructions are retrieved, the control instructions at this time are determined based on historical data and used to generate the third level curtain control feature.

[0096] In the above step S1, when collecting the real-time ambient light intensity, it includes:

[0097] The image plane of the light intensity detector is divided into multiple detection element areas according to the x coordinate, the real-time light is dispersed by the echelle grating, and the light is split by the transmission prism so that the light of different wavelengths falls on the image plane of the light intensity detector;

[0098] Get the central wavelength of different grating diffraction orders of the echelle grating, as shown in Formula 1:

[0099] (1);

[0100] In the formula, is the central wavelength, is the grating constant, is the grating offset angle, is the diffraction angle of the central wavelength of each grating diffraction order, is the grating diffraction order;

[0101] According to the central wavelength, the refractive index of the prism at different central wavelengths is obtained, and the incident angle during the second refraction and the exit angle during the second refraction are further obtained, as shown in Formula 2, Formula 3 and Formula 4:

[0102] (2);

[0103] In the formula, is the prism refractive index at different central wavelengths, , , represents different dispersion effect strengths coefficient, , , represents the square of the absorption resonance wavelength coefficient;

[0104] (3);

[0105] (4);

[0106] In the formula, is the incident angle of the light when it is refracted for the second time by the transmission prism, is the prism apex angle, is the refraction angle of the light after the first refraction through the transmission prism, is the angle of incidence of the light when it is refracted for the second time by the transmission prism, is the incident angle of the light into the transmissive prism;

[0107] According to the incident angle and the exit angle during the second refraction, the relationship between the exit angle and the refractive index of the prism after the light of different wavelengths is incident from the middle step grating to the transmission prism is obtained, as shown in Formula 5:

[0108] (5);

[0109] According to the relationship between the exit angle and the refractive index of the prism, the difference in the exit angle between different central wavelengths and the selected reference wavelength is determined, and the detection element area where the light falls on the image plane of the light intensity detector is determined by the focal length of the focusing lens, as shown in Formula 6:

[0110] (6);

[0111] In the formula, is the detection element area where the light falls on the image plane of the light intensity detector, is the focal length of the focusing lens, is the difference in the exit angle between the central wavelength and the selected reference wavelength, is the number of central wavelengths, is the size of the detection element region;

[0112] An order fitting relationship between the detection element area and the diffraction order is established, the grating diffraction order of the real-time light is determined through the order fitting relationship, and the real-time light intensity value is determined based on the grating diffraction order.

[0113] In the above step S1, when collecting real-time temperature and humidity, it includes:

[0114] Enable the corresponding serial port clock by calling the clock control function, provide a serial port structure variable, and configure the baud rate of USART communication, the parity bit of the data frame, the number of stop bits transmitted at the end of each data frame, and the length of each frame of data in the serial port structure variable, and use the configured serial port structure variable to initialize the serial port once;

[0115] Provide a GPIO structure variable, configure the response speed of the GPIO pin, the working mode of the GPIO pin, the output type of the GPIO pin, and specify the GPIO pin to be configured for the GPIO structure variable, and use the configured GPIO structure variable to initialize the IO port once;

[0116] Open the corresponding interrupt function, configure the interrupt preemption priority, interrupt enable, response priority, and interrupt channel for the interrupt structure variable, use the configured interrupt structure variable to initialize the nested vector interrupt controller, and enable the serial port through the enable function;

[0117] Use the underlying data registers to collect multi-point temperature and humidity data in real time through the serial port, and parse and process the multi-point temperature and humidity data through the interrupt processing function;

[0118] Among them, when parsing and processing, it includes:

[0119] Obtain the cross correction coefficients of the multi-point temperature and humidity sensors, and construct a cross correction coefficient matrix based on the cross correction coefficients, as shown in Formula 7:

[0120] (7);

[0121] In the formula, For the The temperature sensor and Cross correction factor between humidity sensors, is the number of temperature sensors and humidity sensors, is the cross correction coefficient matrix;

[0122] Based on the cross correction coefficient matrix, accurate multi-point temperature and humidity data are obtained, as shown in Formula 8:

[0123] (8);

[0124] In the formula, is the multi-point temperature data actually collected, For the actual multi-point humidity data collected, for The temperature data collected by the temperature sensor for The humidity data collected by the humidity sensor is Expressing the All cross correction coefficients associated with each temperature sensor are averaged. Expressing the All cross correction coefficients associated with the humidity sensors are averaged. is the corrected multi-point temperature data, is the corrected multi-point humidity data.

[0125] In the above step S2, when capturing the sound characteristics of the on-site environment, it includes:

[0126] The captured on-site environmental sound signal is divided into frames to obtain several frames of sound signals, and each frame of sound signal is extracted dimensional feature vector;

[0127] based on dimensional feature vector, obtain the first The mean and variance of the dimension, where ;

[0128] Perform a linear transformation on each frame of sound signal to make the mean value 0 and the variance 1, eliminating the scale differences between different feature vectors;

[0129] Use the Gaussian mixture model to approximate the probability density distribution of the feature parameters, and further obtain the cumulative distribution function;

[0130] Acquire the spectrum characteristics of the on-site environmental sound and form a noise suppression feature vector with the cumulative distribution function;

[0131] Given standard speech data, iterative estimation is performed based on the noise suppression feature vector to suppress the scene environment noise.

[0132] Furthermore, when obtaining the spectrum characteristics of the on-site environmental sound, it includes:

[0133] Get the sound of the scene Characteristic spectrum and Spectrum, will Characteristic spectrum and The spectra are combined to obtain the spectral features;

[0134] Among them, in obtaining When composing, it includes:

[0135] Get the lowest and highest frequencies of the ambient sound, and The octave index relationship is different frequencies, as shown in Formula 9:

[0136] (9);

[0137] In the formula, is the number of different frequencies, For the number of copies, The highest frequency of the ambient sound. It is the lowest frequency of the ambient sound on site;

[0138] based on Different frequencies are used to construct frequency points, generate resolution width, and further obtain quality factors, as shown in Formula 10, Formula 11, and Formula 12:

[0139] (10);

[0140] (11);

[0141] (12);

[0142] In the formula, For the frequency points, For the frequency points, To resolve the width, is the quality factor;

[0143] Frame division and windowing are performed, and the truncation length is obtained based on the sampling frequency, quality factor and frequency point, as shown in Formula 13:

[0144] (13);

[0145] In the formula, is the cutoff length, is the sampling frequency;

[0146] Based on the cutoff length and quality factor, the angular frequency corresponding to the frequency point is obtained, as shown in Formula 14:

[0147] (14);

[0148] In the formula, Frequency point The corresponding angular frequency;

[0149] Based on the cutoff length and angular frequency, a discrete Fourier transform is performed to obtain Score.

[0150] In the above step S2, when generating the second level curtain control feature, it includes: performing lexical and grammatical analysis on the voice command separated in real time to generate the second level curtain control feature;

[0151] Among them, real-time separation includes:

[0152] The noise-suppressed sound signal is input into the separation model, and the features of the sound signal are extracted through the front module in the separation model. After being processed and converted by the encoding layer composed of a deep CNN network, it is decoded by the CTC model and attention mechanism in the decoding layer, while taking into account the context information using a bidirectional long short-term memory network.

[0153] Among them, when decoding, it includes:

[0154] The CTC model introduces the Label, to align the sound signal with the corresponding text label;

[0155] The defined attention mechanism uses context information to assign different weights to different input sound signal sequences;

[0156] The definition includes: defining the encoder network, the decoder network, adding the content attention mechanism and the position-aware attention mechanism;

[0157] The encoder network uses a long short-term memory network, and the decoder network uses The function processes the hidden vector output by the bidirectional long short-term memory network; the content attention mechanism uses the hyperbolic tangent activation function and is used to normalize the attention weights function; the position-aware attention mechanism is used to process the convolution of the function.

[0158] In the above step S3, when analyzing and converting the control instruction, it includes:

[0159] Determine whether the control command is a text command or a voice command. If it is a text command, it is directly converted into a third-level curtain control feature. If it is a voice command, it is converted into a third-level curtain control feature after lexical and grammatical analysis.

[0160] When performing lexical and grammatical analysis, this includes:

[0161] The lexical analyzer is used to scan the voice command string one by one to form a lexical unit sequence, and the grammatical analyzer is used to perform recursive descent grammatical analysis on the input lexical unit sequence based on the grammatical tree nodes and according to the strategy grammar to obtain a grammatical analysis tree, and the second-level curtain control feature and the third-level curtain control feature are generated based on the grammatical analysis tree;

[0162] Among them, the lexical units output as terminal symbols in the lexical unit sequence are composed of only category codes, and the lexical units output as non-terminal symbols in the lexical unit sequence are composed of category codes and attribute values; for single-character words, use their Code value is used as category code, and custom category code is used for words with two characters or more;

[0163] Function As a lexical analyzer The core of the class is used to scan strings, classify morphemes, and perform state transitions based on the type of characters. When scanning, the judgment function Are you ready to return lexical units to the parser? If so, use Class by using function Pre-read the next character to determine whether the current character is the last character of the lexical unit morpheme;

[0164] Among them, each non-terminal symbol of the strategy grammar corresponds to a specific implementation of a function, and the control flow in the function processing process corresponding to each non-terminal symbol is uniquely determined by the attribute value of the current parsing lexical unit in the input lexical unit sequence.

[0165] In the above step S4, when the combined feature is obtained, it also includes:

[0166] The first-level curtain control features, the second-level curtain control features, and the third-level curtain control features are mapped through the transformation matrix to obtain the shared features among the three levels of curtain control features. , and the three levels of curtains have their own unique independent features , , ;

[0167] Shared Features and independent features , , Splice to get the combined features ;

[0168] When constructing the regression coefficient matrix and forming the curtain control pyramid, it also includes:

[0169] Get the corresponding shared features , independent features , , The first, second, third, and fourth weight coefficients , , , , and construct the regression coefficient matrix ;

[0170] Based on the combined features and regression coefficient matrix, a curtain control pyramid is formed, as shown in Formula 15:

[0171] (15);

[0172] In the formula, Control pyramid for curtains.

[0173] In the above step S5, when determining the control intention, it includes:

[0174] Using self-learning The curtain control pyramid is learned and transformed layer by layer to obtain the initial effective feature set;

[0175] Fill the initial valid feature set with blank vectors until the maximum length of the sentence is reached to obtain the final valid feature set;

[0176] Based on The established bidirectional semantic extraction network performs bidirectional extraction on the final effective feature set, and obtains the vector group expression of the curtain control pyramid in the feature space and the high-dimensional semantic vector representation;

[0177] The control intention is determined based on the vector group expression and high-dimensional semantic vector representation.

[0178] In the above step S6, when performing fine opening and closing state control of the curtain, it includes:

[0179] Get the armature voltage of the curtain DC motor , and place the DC motor in Input voltage at the moment Fuzzy to armature voltage As a fuzzy controller The output is shown in Formula 16:

[0180] (16);

[0181] In the formula, is the armature voltage of the curtain DC motor, is the motor constant, is the air gap flux of the motor, For DC motors The speed of the moment, is the armature current, is the armature winding resistance, is the brush voltage drop;

[0182] Combine the control intention with historical data to determine the target light intensity value of the current control area, and compare it with the real-time light intensity value contained in the first-level curtain control feature to obtain the real-time light intensity difference;

[0183] Using the real-time light intensity difference as a fuzzy controller The target input voltage is obtained.

[0184] The real-time input voltage of the DC motor is collected and compared with the target input voltage to obtain the input voltage difference. The input voltage difference is used as the fuzzy Controller input, and then determine Control the proportional gain change, integral gain change, differential gain change, and output The pulse signal is adjusted instantly until the input voltage difference is less than the preset threshold, thus achieving precise control of the opening and closing state of the curtains;

[0185] in, When the motor rotates forward, the curtain opens. , the motor reverses and the curtain closes.

[0186] In the above step S7, when dynamic adjustment is performed, it includes:

[0187] Combine the control intention with historical data to determine the target travel volume of the curtain;

[0188] The stroke amount obtained by real-time monitoring is compared with the target stroke amount to obtain the stroke deviation amount, and further obtain the stroke deviation change rate;

[0189] The stroke deviation and stroke deviation change rate are taken as input and passed through The established fuzzy control table is Control the proportional gain change, integral gain change, and differential gain change for dynamic adjustment.

[0190] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. An intelligent curtain control method based on control pyramid and multimodal interaction, characterized in that: The following steps are involved: Collect real-time ambient light intensity, real-time multi-point temperature and humidity to generate first-level curtain control features; Capture the sound characteristics of the on-site environment, separate voice commands from ambient noise in real time, and generate second-level curtain control features based on voice commands; Regularly request control instructions from user terminal equipment, analyze and convert the control instructions, and generate third-level curtain control features; The first, second and third level curtain control features are correlated and analyzed based on the multimodal fusion algorithm to form a curtain control pyramid; The curtain control pyramid is studied, converted, filled, and bidirectionally extracted to determine the control intent; Input the control intention into the curtain control model to perform fine control of the curtain's opening and closing status; Monitor the movement status of the curtains in real time, and feed the real-time monitoring information back to the curtain control model for dynamic adjustment to control the effect; Among them, the multimodal fusion algorithm includes: By mapping the first, second and third level curtain control features, shared features and independent features are obtained, and the shared features and independent features are spliced ​​to obtain combined features; A regression coefficient matrix is ​​constructed for the combined features, and a curtain control pyramid is formed by combining the combined features.

2. The intelligent curtain control method based on control pyramid and multimodal interaction according to claim 1 is characterized in that: When collecting real-time ambient light intensity, it includes: The image plane of the light intensity detector is divided into multiple detection element areas according to the x coordinate, the real-time light is dispersed by the echelle grating, and the light is split by the transmission prism so that the light of different wavelengths falls on the image plane of the light intensity detector; Obtaining the central wavelengths of different grating diffraction orders of the echelle grating; According to the central wavelength, the prism refractive index at different central wavelengths is obtained, and further the relationship between the exit angle and the prism refractive index after the light of different wavelengths is incident from the echelle grating to the transmission prism is obtained; According to the relationship between the exit angle and the refractive index of the prism, the difference between the exit angles of different central wavelengths and the selected reference wavelength is determined, and the detection element area where the light falls on the image plane of the light intensity detector is determined by the focal length of the focusing lens; A fitting relationship between the detection element area and the diffraction order is established to determine the grating diffraction order and real-time light intensity value of the real-time light.

3. The intelligent curtain control method based on control pyramid and multimodal interaction according to claim 1 is characterized in that: When collecting real-time temperature and humidity, it includes: Use the underlying data registers to collect multi-point temperature and humidity data in real time through the serial port, and parse and process the multi-point temperature and humidity data through the interrupt processing function; Among them, when parsing and processing, it includes: The cross correction coefficients of the multi-point temperature and humidity sensors are obtained, and a cross correction coefficient matrix is ​​constructed based on the cross correction coefficients, as shown in Formula 7: （7）； In the formula, For the The temperature sensor and Cross correction factor between humidity sensors, is the number of temperature and humidity sensors, is the cross correction coefficient matrix; Based on the cross correction coefficient matrix, accurate multi-point temperature and humidity data are obtained, as shown in Formula 8: （8）； In the formula, , They are the actual multi-point temperature and humidity data collected, for The temperature data collected by the temperature sensor for The humidity data collected by the humidity sensor is , Expressing the The temperature sensor, All cross correction coefficients associated with the humidity sensors are averaged. , They are the corrected multi-point temperature and humidity data respectively.

4. The intelligent curtain control method based on control pyramid and multimodal interaction according to claim 1 is characterized in that: When capturing the sound characteristics of the live environment, include: The captured on-site environmental sound signal is divided into frames to obtain several frames of sound signals, and each frame of sound signal is extracted dimensional feature vector; Based on the dimensional feature vector, obtain the first The mean and variance of the dimension, where ; Perform linear transformation on each frame of sound signal to eliminate the scale differences between different feature vectors; Use the Gaussian mixture model to approximate the probability density distribution of the feature parameters, and further obtain the cumulative distribution function; Acquire the spectrum characteristics of the on-site environmental sound, and form a noise suppression feature vector with the cumulative distribution function; Given standard speech data, iterative estimation is performed based on the noise suppression feature vector to suppress the on-site environmental noise; The spectral features include Characteristic spectrum and Score.

5. The intelligent curtain control method based on control pyramid and multimodal interaction according to claim 4 is characterized in that: When generating the second level curtain control feature, it includes: performing lexical and grammatical analysis on the voice command separated in real time to generate the second level curtain control feature; Wherein, the real-time separation includes: The noise-suppressed sound signal is input into the separation model, and the features of the sound signal are extracted through the front module in the separation model. After being processed and converted by the encoding layer composed of a deep CNN network, it is decoded by the CTC model and attention mechanism in the decoding layer, while the bidirectional long short-term memory network is used to take into account the contextual information.

6. The intelligent curtain control method based on control pyramid and multimodal interaction according to claim 5 is characterized in that: When analyzing and converting control instructions, it includes: Determine whether the control command is a text command or a voice command. If it is a text command, it is directly converted into a third-level curtain control feature. If it is a voice command, it is converted into a third-level curtain control feature after lexical and grammatical analysis. When performing lexical and grammatical analysis, this includes: The voice command string is scanned one by one by the lexical analyzer to form a lexical unit sequence. The grammatical analyzer performs recursive descent grammatical analysis on the input lexical unit sequence based on the grammar tree nodes and according to the strategy grammar to obtain a grammatical analysis tree, and generates the second or third level curtain control features based on the grammatical analysis tree.

7. The intelligent curtain control method based on control pyramid and multimodal interaction according to any one of claims 1 to 6, characterized in that: When obtaining the combined features, it also includes: The first, second and third level curtain control features are mapped through the transformation matrix to obtain the shared features among the three levels of curtain control features. , and the three levels of curtains have their own unique independent features , , , and concatenate them to get the combined features ; When constructing the regression coefficient matrix and forming the curtain control pyramid, it also includes: Get the corresponding shared features , independent features , , The first, second, third, and fourth weight coefficients , , , , and construct the regression coefficient matrix ; Based on the combined features and the regression coefficient matrix, the curtain control pyramid is formed, as shown in Formula 15: （15）； In the formula, Control pyramid for curtains.

8. The intelligent curtain control method based on control pyramid and multimodal interaction according to claim 7 is characterized in that: When determining control intent, include: Using self-learning The curtain control pyramid is learned and transformed in a layer to obtain an initial effective feature set; Filling the initial valid feature set with blank vectors until the maximum length of the sentence is reached to obtain a final valid feature set; Based on The established bidirectional semantic extraction network performs bidirectional extraction on the final effective feature set to obtain a vector group expression of the curtain control pyramid in the feature space and a high-dimensional semantic vector representation; The control intention is determined according to the vector group expression and the high-dimensional semantic vector representation.

9. The intelligent curtain control method based on control pyramid and multimodal interaction according to claim 8 is characterized in that: When performing fine opening and closing state control of curtains, it includes: Get the armature voltage of the curtain DC motor , and place the DC motor in Input voltage at the moment Fuzzy for the armature voltage As a fuzzy controller Output: The control intention is combined with historical data to determine the target light intensity value of the current control area, which is compared with the real-time light intensity value contained in the first-level curtain control feature to obtain the real-time light intensity difference, which is used as the fuzzy controller The target input voltage is obtained. The real-time input voltage of the DC motor is collected and compared with the target input voltage to obtain the input voltage difference. The input voltage difference is used as the fuzzy Controller input, and then determine Control the proportional gain change, integral gain change, differential gain change, and output The pulse signal is adjusted instantly until the input voltage difference is less than the preset threshold.

10. The intelligent curtain control method based on control pyramid and multimodal interaction according to claim 9 is characterized in that: When making dynamic adjustments, it includes: Combining the control intention with historical data to determine the target travel amount of the curtain; The stroke amount obtained by real-time monitoring is compared with the target stroke amount to obtain the stroke deviation amount, and further obtain the stroke deviation change rate; The stroke deviation and stroke deviation change rate are taken as input and passed through The established fuzzy control table is Control the proportional gain change, integral gain change, and differential gain change for dynamic adjustment.

Images