Control system and controlled device

Abstract

The invention discloses a control system and a controlled device, and relates to the field of intelligent control, and the control system comprises an acquisition module, a controlled module, an automatic control module and an execution module. The acquisition module senses urine contact to generate a high-level signal, and acquires a sound digital signal sequence and a video digital signal sequence; the controlled module senses touch or receives a remote control signal to generate a control instruction, and generates a prompt instruction for baby crying or human language selection based on a sound digital signal sequence or analyzes voice content through a voice recognition technology to generate a control instruction; the automatic control module periodically intercepts a video digital signal sequence, recognizes the sleep state of the baby through transcoding, face recognition and optical flow detection, automatically generates a control instruction or a prompt instruction, and continuously generates the prompt instruction during the high-level signal receiving period; and the execution module executes the control instruction, receives the video digital signal sequence and the prompt instruction and uploads the video digital signal sequence and the prompt instruction to the cloud platform, thereby realizing remote and intelligent cooperative control of the cradle.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control, and in particular to a control system and a controlled device. Background Art

[0002] Traditional baby cradles have a single function and can only achieve simple rocking movements, which cannot meet the needs of modern families for comprehensive baby care and real-time monitoring.

[0003] Existing electric cradles have the following defects: the movement mode is single and cannot simulate the complex movement characteristics of natural embrace; the intelligence level is low, relying on preset programs or close human intervention, and lacking interactive capabilities and behavior recognition capabilities. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the present invention proposes a control system and a controlled device to realize an intelligent baby cradle with four control modes: button, remote control, voice control and automatic control.

[0005] The technical solutions for achieving the purpose of the present invention are:

[0006] The control system includes an acquisition module, a controlled module, an automatic control module, and an execution module;

[0007] The acquisition module continuously sends a high-level signal y when urine is present h To the automatic control module, collect the sound analog signal x raw , generate the sound digital signal sequence y through filtering and analog-to-digital conversion and transmit it to the controlled module, continuously obtain the video digital signal sequence Y vid And transmit to the automatic control module and execution module;

[0008] The controlled module senses user touch or receives a remote control signal to generate a control instruction, converts the sound digital signal sequence y into Mel-frequency cepstral coefficients MF, and uses a progressive binary classification model to identify whether the sound digital signal sequence y is a baby crying or human language. If it is a baby crying, a prompt instruction is generated and sent to the execution module. If it is human language, a deep hidden Markov model is combined with a context-constrained word segmentation model to convert the Mel-frequency cepstral coefficients MF into speech content z and split and reorganize it into a speech phrase sequence Ph based on semantic dependencies. The word skipping model is used to convert the trigger phrases, function phrases, and speech phrases in the speech phrase sequence Ph into vector representations and perform matching verification. When the trigger phrase matches successfully and at least one function phrase matches successfully, a control instruction is generated based on the matched function phrase. In other cases, the sound digital signal sequence y is ignored. After the control instruction is generated, it is transmitted to the execution module;

[0009] The automatic control module periodically intercepts the video digital signal sequence Y vid And transcode it into time-lapse video Identify the baby's sleeping state through face recognition and optical flow detection, generate control instructions or prompt instructions based on the rule engine and transmit them to the execution module, and h During the reception period, prompt instructions are periodically sent to the execution module;

[0010] The execution module executes the control instruction and receives the video digital signal sequence Y vid And upload to the cloud platform synchronously, receive prompt instructions and send them to all paired devices through the cloud platform. The paired devices are mobile devices registered in the cloud platform and bound to the control system.

[0011] Furthermore, the collection module includes a urine sensing unit, a sound sensing unit, and a video monitoring unit;

[0012] The urine sensing unit senses urine contact based on the capacitive sensing principle and the conduction characteristics of the NMOS tube, and generates a high-level signal y when urine contacts h And transmit it to the automatic control module, and stop generating high-level signal y when urine is cleared h ;

[0013] The sound perception unit collects sound analog signals x raw , which is converted into a sound digital signal sequence y through the filter bank, analog-to-digital converter and adaptive filtering algorithm and transmitted to the controlled module;

[0014] The video surveillance unit collects ambient light intensity in real time and adaptively determines the fill light brightness, continuously shooting to generate surveillance video X vid And use H.265 encoding algorithm to convert it into video digital signal sequence Y vid , synchronously send video digital signal sequence Y vid To the controlled module and the execution module.

[0015] Furthermore, the urine sensing unit senses urine contact based on the capacitive sensing principle and the NMOS tube conduction characteristics, including the following specific steps:

[0016] Connect the capacitor to the bias circuit, and connect the gate and source of the NMOS tube to the bias circuit and ground respectively. The bias circuit includes a resistor R and a DC power supply V cc ;

[0017] When there is no urine contact, the normal current I0 of the bias circuit is equal to the DC power supply V cc Divide by the resistance R and the normal capacitance The sum of normal capacitive reactance It is inversely proportional to the normal capacitance C0. The normal capacitance C0 is the capacitance when the capacitor medium is air. The normal gate-source voltage of the NMOS tube is the DC power supply V ccAdd normal capacitive reactance The product of the normal current I0 is less than or equal to the breakdown voltage V th , the NMOS tube is in the cut-off state;

[0018] When urine comes into contact with the capacitor, the capacitor medium becomes urine and the abnormal capacitive reactance of the capacitor Less than normal capacitance The abnormal current I1 of the bias circuit is greater than the normal current I0, and the abnormal gate-source voltage of the NMOS tube is Equal to the DC power supply V cc Add abnormal capacitive reactance The product of the abnormal current I1 is greater than the breakdown voltage V th , the NMOS tube is in the on state, generating a drain-source current I ds ;

[0019] The drain-source current I ds Converted into output signal voltage V out And connected to the comparator circuit, the comparator outputs a signal voltage V out Greater than the reference voltage V ref When a high level signal y is generated h And transmit it to the automatic control module, otherwise the output signal voltage V out ;

[0020] When the urine is cleared, the medium of the capacitor returns to air, the NMOS tube turns to the cut-off state, and stops outputting the high-level signal y h .

[0021] Furthermore, the conversion and generation of the sound digital signal sequence y by the filter bank, analog-to-digital converter and adaptive filtering algorithm includes the following specific steps:

[0022] Convert the sound analog signal x raw Input filter bank, which is less than the minimum human voice frequency f min and greater than the maximum human voice frequency f max The frequency range is filtered multiple times to generate a filtered analog signal x hp ;

[0023] The analog-to-digital converter is used to filter the analog signal x hp Perform discrete sampling to generate the initial digital signal sequence y of the sound ori ;

[0024] Initialize the initial digital signal y of the sound at the first sampling moment ori (1) weight coefficient vector w(1) and positive definite symmetric matrix P(1);

[0025] Generate reference noise signal sequence d based on discretization of prior white noise model ref , where d ref (n) is the reference noise signal at the nth sampling moment, n = 1,…,N, where N is the total number of sampling moments;

[0026] According to the initial digital signal y of the sound at the nth sampling moment ori The initial digital signal of the sound at the first M-1 sampling moments of (n) constructs the initial input vector y of the sound at the nth sampling moment ori (n), if there are less than M-1 sampling moments before the n-th sampling moment, the missing part will be padded with 0;

[0027] The weight coefficient vector w(n) at the nth sampling moment is combined with the initial sound input vector y ori The inner product of (n) is the sound digital signal y(n) at the nth sampling moment;

[0028] The error signal e(n) at the nth sampling moment is used to reflect the effect of adaptive filtering and is equal to the sound digital signal y(n) plus the reference noise signal d ref (n) Subtract the initial digital signal y ori (n);

[0029] The gain vector g(n) at the n-th sampling moment is calculated based on the forgetting factor λ and the positive definite symmetric matrix P(n-1) at the n-1-th sampling moment. The weight coefficient vector w(n) at the n-th sampling moment is adjusted in combination with the error signal e(n) to generate the weight coefficient vector w(n+1) at the n+1-th sampling moment.

[0030] Update the positive definite symmetric matrix P(n-1) at the n-1th sampling moment based on the forgetting factor λ and the gain vector g(n) at the n-th sampling moment to obtain the positive definite symmetric matrix P(n) at the n-th sampling moment;

[0031] The weighted square sum J of the error signal is minimized through multiple rounds of iterations, and a sound digital signal sequence y is generated when the algorithm converges. Each round of iteration updates the weight coefficient vector and the positive definite symmetric matrix at N sampling moments.

[0032] Furthermore, the controlled module includes a touch control unit, a remote control unit and a voice control unit;

[0033] The touch control unit generates a touch distribution network based on parasitic capacitance sensing, verifies the overlap rate with the key layout network to generate control instructions, provide vibration feedback and send them to the execution module;

[0034] The remote control unit receives the remote control signal, generates the corresponding control instruction and forwards it to the execution module;

[0035] The voice control unit filters and performs logarithmic operations on the sound digital signal sequence y with a Mel filter bank in the frequency domain, converts it to the cepstral domain to generate Mel-frequency cepstral coefficients MF, and uses a progressive binary classification model to determine whether the sound digital signal sequence y is a baby's cry or human language based on the Mel-frequency cepstral coefficients MF. If it is a baby's cry, a prompt instruction is generated and sent to the execution module. If it is human language, a deep hidden Markov model is used to convert the Mel-frequency cepstral coefficients MF into speech content z. The context-constrained word segmentation model is used to split the speech content z into a word segmentation sequence Wo based on semantic dependencies, and a speech phrase sequence Ph is constructed by a translation sliding window combination. The word skipping model is used to match and verify the trigger phrases, function phrases and speech phrases in the speech phrase sequence Ph. If the trigger phrase and at least one function phrase match the speech phrase sequence Ph, the corresponding control instruction is generated according to the matched function phrases. In other cases, the sound digital signal sequence y is ignored.

[0036] Furthermore, generating a touch distribution network based on parasitic capacitance sensing and verifying the overlap rate with the button layout network includes the following specific steps:

[0037] Obtain the local charge change peak value caused by the parasitic capacitance formed by coupling when the user touches the touch and determine whether it is greater than the effective touch peak value;

[0038] If the local charge peak is less than the effective touch peak, ignore the local charge change. If the local charge peak is greater than or equal to the effective touch peak, mark the grid in the touch distribution network that overlaps with the local charge change area as the active grid.

[0039] Call the button layout network and perform overlap verification with the touch distribution network to confirm the overlap rate between the grid area of ​​each button number in the button layout network and the activated grid in the touch distribution network, and determine whether the maximum overlap rate is greater than or equal to the overlap rate threshold;

[0040] If the maximum overlap rate is less than the overlap rate threshold, the current touch is ignored. If the maximum overlap rate is greater than or equal to the overlap rate threshold, a corresponding control instruction is generated according to the key number of the maximum overlap rate.

[0041] Furthermore, the sound digital signal sequence y is filtered and logarithmically operated with the Mel filter bank in the frequency domain, and converted to the cepstral domain to generate the Mel frequency cepstral coefficient MF, which includes the following specific steps:

[0042] The sound digital signal sequence y is processed by discrete Fourier transform to generate a spectrum set F, which includes spectrum sequences at N sampling moments and spectrum sequence f at the nth sampling moment. n It includes frequency domain signals at K frequencies, where N and K are the total number of sampling moments and the total number of frequencies respectively;

[0043] According to the Mel transform equation, the frequency range of the sound digital signal sequence y is converted into a Mel frequency range, and the Mel frequency range is further divided into Q Mel frequency segments. The center Mel frequency of each Mel frequency segment is determined, and Q is the total number of Mel frequency segments;

[0044] According to the center Mel frequency of the q-1th Mel frequency segment The center Mel frequency of the qth Mel frequency segment and the center Mel frequency of the q+1th Mel frequency segment Set the triangular filter for the qth Mel frequency band, q = 1, ..., Q;

[0045] The spectrum sequence of N sampling moments in the spectrum set F is multiplied by Q triangular filters and the logarithm is taken to generate a logarithmic domain signal set LF. The logarithmic domain signal set LF includes the logarithmic domain signal sequence of N sampling moments and the logarithmic domain signal sequence lf at the nth sampling moment. n It includes Q logarithmic domain signals, the logarithmic domain signal sequence lf n The qth logarithmic domain signal lf n (q) is the spectral sequence f n The sum of the products of the frequency domain signal of each frequency in and the frequency response of the triangular filter of the qth Mel frequency band at that frequency;

[0046] The logarithmic domain signal set LF is converted to the cepstral domain by discrete cosine transform to generate Mel-frequency cepstral coefficients MF, which include Mel-frequency cepstral feature vectors at N sampling moments.

[0047] Specifically, the progressive binary classification model includes a first support vector machine and a second support vector machine. The first support vector machine classifies the Mel-frequency cepstral coefficient MF into baby crying or the first residual class in the high-dimensional space. When the classification result of the first support vector machine is the first residual class, the second support vector machine further classifies the Mel-frequency cepstral coefficient MF into human language or the second residual class in the high-dimensional space. The first residual class is equal to the union of human language and the second residual class, and the baby crying is mutually exclusive with the first residual class, and the human language is mutually exclusive with the second residual class.

[0048] Specifically, deep hidden Markov models include deep networks and hidden Markov models;

[0049] The input of the deep network is the Mel-frequency cepstral coefficient MF, which is linearly modulated multiple times by the linear layer to adjust to a certain dimension. The posterior probability distribution of the state vector is generated by the Softmax function, which records the posterior probability of each state in the state vector.

[0050] The hidden Markov model defines a state vector based on phonemes. The state vector includes a silent state, a phonation start state, a stable phonation state, and a phonation end state. A state transition probability matrix is ​​defined to describe the probability of transitioning from one state to another. The observation probability distribution is defined as the posterior probability distribution of the state vector, reflecting the probability of observing the Mel-frequency cepstral coefficient MF in a single state. The initial state probability distribution of the initial time step is defined. At each time step, the probability of transitioning from the current state to the next state and the probability of observing the Mel-frequency cepstral coefficient MF are determined based on the state transition probability matrix and the observation probability distribution. The Viterbi algorithm is used to calculate the maximum probability path from the current state to the remaining states and record the path information. The optimal state sequence is obtained by backtracking. The phonemes in the optimal state sequence are combined according to the pronunciation dictionary to generate the speech content z.

[0051] Specifically, the context-constrained word segmentation model is used to split the speech content z into the word sequence Wo. The context-constrained word segmentation model includes a bidirectional long short-term memory network layer and a conditional random field;

[0052] The bidirectional long short-term memory network layer includes a forward LSTM and a backward LSTM. The forward LSTM and the backward LSTM process the speech content z in forward and reverse order, respectively. Through the input gate and the forget gate, they learn the influence between adjacent Chinese characters, capture the forward and reverse semantic associations of the speech content z, and generate semantic features.

[0053] The conditional random field calculates the probability that the feature value of each dimension in the semantic feature belongs to a single part-of-speech tag based on the state feature function to obtain all candidate word segmentation sequences. The candidate word segmentation sequences are constrained by focusing on the transfer relationship between part-of-speech tags through the transfer feature function, and the conditional probability of each candidate word segmentation sequence is calculated. The optimal part-of-speech tag output order is found through the Viterbi algorithm and the speech content z is annotated for word segmentation to generate the word segmentation sequence Wo.

[0054] Specifically, the construction of the speech phrase sequence Ph depends on a sliding window. The sliding window width is the maximum phrase length among the trigger phrase and all functional phrases. The starting edge of the sliding window is placed at the first segmentation in the segmentation sequence Wo. The segmentations within the sliding window width are combined into speech phrases. Then, the sliding window is shifted backward by one segmentation and combined again to generate a new speech phrase. When the end edge of the sliding window reaches the last segmentation, the last speech phrase is generated. All speech phrases are constructed into the speech phrase sequence Ph according to the generation order.

[0055] Furthermore, the trigger phrase, function phrase, and speech phrase in the speech phrase sequence Ph are converted into vector representations using the skip-gram model and matched and verified, including the following specific steps:

[0056] The skip-gram model extracts the central participle of the input phrase and determines the co-occurrence probability of the central participle of the input phrase and the context participles. The central participle is converted into a one-hot encoding based on the corpus. The probability distribution of the input phrase is generated through two linear layers and a softmax function. The weights of the linear layer are adjusted using backpropagation to minimize the error between the probability distribution of the input phrase and the co-occurrence probability. The weights of the first linear layer are used as the vector representation of the input phrase, where the input phrase includes a trigger phrase, a function phrase, and a speech phrase in the speech phrase sequence Ph. The vector representation includes a trigger vector representation, a function vector representation, and a speech vector representation.

[0057] Calculate the trigger cosine similarity between the trigger vector representation and each speech vector representation, and calculate the function cosine similarity between each function vector representation and each speech vector representation, and compare them with the similarity threshold in turn;

[0058] If both the trigger cosine similarity and the function cosine similarity are greater than or equal to the similarity threshold, a corresponding control instruction is generated according to all function phrases whose function cosine similarity is greater than or equal to the similarity threshold;

[0059] If there is no trigger cosine similarity or function cosine similarity greater than or equal to the similarity threshold, the sound digital signal sequence y is ignored.

[0060] Furthermore, the self-control module includes a bed-wetting reminder unit and a video monitoring unit;

[0061] The bed-wetting reminder unit receives a high-level signal y h And record the receiving time, periodically generate and send prompt instructions to the execution module until no high-level signal y is received h Clear the receiving time;

[0062] The video monitoring unit periodically captures the video digital signal sequence Y after the control system is turned on. vid And convert and generate time period video according to H.265 encoding algorithm Analyzing period videos with facial recognition and optical flow detection Identify the infant's sleeping state, generate control instructions or prompt instructions based on the rule engine and transmit them to the execution module. The infant's sleeping state includes deep sleep, light sleep, semi-awake, awake and abnormal.

[0063] Furthermore, face recognition and optical flow detection are used to analyze the time period video. Identifying a baby's sleep state involves the following specific steps:

[0064] Sampling period video The sampling frame images of T frames in the image are used to construct a sampling frame image sequence

[0065] Use Dlib's face detector and 68 facial landmark models to locate the sampled frame image sequence The coordinates of the key points around the baby's face and eyes in each sampling frame image are calculated, and the vertical distance between the midpoint of the upper eyelid and the midpoint of the lower eyelid is calculated based on the coordinates of the key points around the eyes in each sampling frame image and averaged. The average vertical distance is obtained and divided by the maximum vertical distance to generate the eye opening ratio;

[0066] Extract the sample frame image sequence through the initial convolutional layer of the MobileNetV3 model The feature map of each sampling frame image in the image is constructed by using the spatial attention mechanism to stitch the average pooling results of the channel sub-maps of the feature map on multiple channels and the global pooling results, and the spatial attention map of each sampling frame image is generated through convolution and Sigmoid function;

[0067] The spatial attention map is connected to the depthwise separable convolution layer of the MobileNetV3 model. Point-by-point convolution, depth-wise convolution, and point-by-point convolution are used to achieve dimensionality increase, feature extraction, and dimensionality reduction of the spatial attention map. After global average pooling, a fully connected layer, and a softmax function, the probability distribution of the degree of expression of each sampled frame image is generated and the average probability distribution of the degree of expression is calculated. The degree of expression includes calm, moderate, and rich.

[0068] Use optical flow method to detect the sample frame image sequence Based on the temporal changes of the pixels in the image, a brightness identity is defined to describe the constant brightness of the corresponding pixels between adjacent frames. Based on the assumption that the offset of the corresponding pixels between adjacent frames is not large, the image sequence describing the sampled frames is found through TV-L1 iterative optimization. The optical flow feature sequence in the optical flow feature sequence describes the movement trajectory and speed of the pixel points between the frame images of adjacent frames over time;

[0069] The optical flow feature sequence is input into the temporal convolution layer. The temporal convolution layer captures the temporal changes of the optical flow features through two identical residual blocks. Then, the fully connected layer reduces the dimension and outputs an aggregated optical flow feature map with the same dimension as the optical flow feature map. The residual block includes dilated convolution and residual structure. Dilated convolution inserts 0 elements between convolution kernels to expand the receptive field. The residual structure superimposes the output of the dilated convolution with the input.

[0070] The aggregated optical flow feature map is reduced in dimension and mapped through a fully connected layer and a Softmax function to generate a probability distribution of motion amplitudes, which include small, medium, and large.

[0071] Based on the scores of each level of expression degree and movement amplitude, the expression score and movement score are generated by weighted summing up the probability distribution of expression degree and movement amplitude respectively. According to the reference weight, the eye-opening ratio, expression score and movement score are weighted summed again to generate a comprehensive sleep score. The infant's sleep state is determined according to the comprehensive score range in which the comprehensive sleep score is located. The infant's sleep state includes deep sleep, light sleep, semi-awake, awake and abnormal.

[0072] Specifically, the rule engine establishes a correspondence between the baby's sleeping state and the target working state of the variable-speed synchronous motor and the speaker. After determining the baby's sleeping state, it generates corresponding control instructions or prompt instructions in combination with the current working state of the variable-speed synchronous motor and the speaker to make the variable-speed synchronous motor and the speaker reach the target working state. The control instructions include power on instructions, power off instructions, swing start instructions, swing stop instructions, swing acceleration instructions, swing deceleration instructions, music play instructions, music stop instructions, story play instructions and story stop instructions.

[0073] Furthermore, the execution module includes a power on / off unit, a variable speed synchronous motor execution unit, a sound playing unit, and a cloud communication unit;

[0074] The power switch unit receives a power on instruction or a power off instruction and opens or closes the circuit switch;

[0075] The variable speed synchronous motor execution unit receives a swing start instruction or a swing stop instruction, starts the variable speed synchronous motor according to the previous motor gear position or records the current motor gear position and turns off the variable speed synchronous motor, receives a swing acceleration instruction or a swing deceleration instruction, and increases or decreases the motor gear position by one gear within the allowable motor gear range, and the motor gear positions include low gear, medium gear and high gear;

[0076] The sound playback unit receives a music playback instruction or a story playback instruction, obtains the previously played music or story, plays the next music or story through the speaker, receives a music stop instruction or a story stop instruction, records the currently played music or story and turns off the speaker;

[0077] The cloud communication unit receives the video digital signal sequence Y vid The control system and the paired device are connected to the cloud platform for remote communication.

[0078] A controlled device, controlled by the control system, comprising a cradle body, a base, a driving device, a universal ball bearing, a universal rocking plate, a cradle body connecting flange, a limiting rubber belt, a urine sensor, and an integrated processing device;

[0079] The cradle body is used to accommodate the baby to sleep;

[0080] The base is used to increase stability;

[0081] The driving device includes a variable speed synchronous motor and a variable speed synchronous motor rocker arm. When the variable speed synchronous motor starts, the variable speed synchronous motor rocker arm is driven to swing. The variable speed synchronous motor rocker arm is fixedly connected to the universal swing plate.

[0082] Universal ball bearings are used to support the universal swing plate;

[0083] The universal swing plate and the rocker arm of the variable speed synchronous motor keep swinging in the same direction;

[0084] The cradle body connecting flange enables the cradle body and the universal swing plate to swing in the same direction;

[0085] The limiting rubber belt prevents the universal swing plate from derailing;

[0086] The urine sensor is used to sense urine contact and generate a high-level signal;

[0087] The integrated processing device is used to collect sound digital signal sequences and monitor and obtain video digital signal sequences, providing four methods of user touch control, remote control, voice control and automatic control to control the opening and closing of the drive device and gear adjustment, as well as the playback and stopping of music and stories, and sending video digital signal sequences and prompt instructions to the cloud platform.

[0088] Furthermore, the integrated processing device includes a key screen, a rear camera, a microphone array, a speaker, a wireless communication module and a processor;

[0089] The key screen displays keys for the user to touch;

[0090] The rear camera monitors and records the cradle;

[0091] The microphone array is used to collect sound analog signals and convert them into sound digital signal sequences;

[0092] The speakers are controlled by the processor to play or stop the music or story;

[0093] The wireless communication module receives remote control signals and transmits them to the processor, which receives video digital signal sequences and prompt instructions and uploads them to the cloud platform;

[0094] The processor generates prompt instructions during the reception of high-level signals, generates control instructions when identifying user-touched buttons, generates control instructions when receiving remote control signals, receives and determines whether the sound digital signal sequence is a baby crying or human language. If it is a baby crying, a prompt instruction is generated. If it is human language, it is converted into voice content and matched and verified to decide whether to generate a control instruction. The video digital signal sequence is periodically intercepted and transcoded into time-segment video to identify the baby's sleeping state. Control instructions or prompt instructions are generated based on the rule engine. The control instructions are used to control the drive device and the speaker, and the prompt instructions are forwarded to the wireless communication module.

[0095] Compared with the existing technology, the present invention senses urine contact based on the capacitive sensing principle and the conduction characteristics of the NMOS tube to generate a high-level signal, and continuously generates prompt instructions during the generation of the high-level signal, thereby realizing automatic detection of infant bedwetting; based on the traditional key touch, the controlled device and the paired device are bound by establishing a cloud platform, and remote control is realized based on the remote control signal; the sound digital signal sequence obtained by the acquisition module is converted into Mel-frequency cepstral coefficients, and prompt instructions are generated for infant crying or human language selection based on the recognition results of the progressive binary classification model, or voice processing technologies such as deep hidden Markov model, context-constrained word segmentation model and skipping word model are used to analyze the voice content to generate corresponding control instructions, thereby realizing automatic detection of infant crying and effective voice control; the video digital signal is periodically intercepted, and the infant's sleeping state is identified through transcoding, facial recognition and optical flow detection. Control instructions are automatically generated according to the rule engine, realizing adaptive intelligent control based on the infant's sleeping state, realizing a four-in-one collaborative control mode of touch, remote control, voice and automation, and improving the intelligence level of the infant cradle. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 Schematic diagram of the control system in the present invention;

[0097] Figure 2 This is a schematic diagram of the overlap rate verification in the present invention;

[0098] Figure 3 A schematic diagram of identifying an infant's sleeping state through facial recognition and optical flow detection analysis in the present invention;

[0099] Figure 4 This is a structural diagram of the controlled device in the present invention.

[0100] Figure numerals: 1. cradle body; 2. base; 3. driving device; 31. variable speed synchronous motor; 32. motor rocker arm 32; 4. universal ball bearing; 5. universal rocking plate; 6. cradle body connecting flange; 7. limiting rubber belt; 8. urine sensor; 9. integrated processing device. DETAILED DESCRIPTION

[0101] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0102] Example 1

[0103] like Figure 1 As shown, the present invention discloses a control system, including an acquisition module, a controlled module, an automatic control module and an execution module;

[0104] The acquisition module continuously sends a high-level signal y during the presence of urine hTo the automatic control module, collect the sound analog signal x raw , after filtering and analog-to-digital conversion, the sound digital signal sequence y is generated and transmitted to the controlled module, and the video digital signal sequence Y is continuously monitored and obtained vid And transmit to the automatic control module and execution module;

[0105] The controlled module senses user touch, identifies touch buttons and generates control instructions, receives remote control signals sent by the cloud platform and converts them into control instructions, receives sound digital signal sequence y and converts them into Mel-frequency cepstral coefficients MF, adopts a progressive binary classification model, and analyzes the Mel-frequency cepstral coefficients MF twice through support vector machines to identify whether the sound digital signal sequence y is a baby crying or human language. If it is a baby crying, a prompt instruction is generated and sent to the execution module. If it is human language, the Mel-frequency cepstral coefficients MF are converted into speech content z through a deep hidden Markov model. The context-constrained word segmentation model splits and reorganizes the speech content z into a speech phrase sequence Ph according to the semantic dependency relationship of the speech content z, and uses a skip-word model to separate the trigger phrase, each functional phrase and each speech short phrase in the speech phrase sequence Ph. The language is converted into a vector representation and matched based on similarity. If the trigger phrase matches the voice phrase sequence Ph and there is at least one function phrase that matches the voice phrase sequence Ph, the corresponding control instruction is generated according to the matched function phrase retrieval instruction code table. If the trigger phrase fails to match or all function phrases fail to match, the sound digital signal sequence y is ignored. After the control instruction is generated, it is transmitted to the execution module, wherein the control instruction is binary coded and uniquely marks the corresponding function, including power on instruction, power off instruction, swing start instruction, swing stop instruction, swing acceleration instruction, swing deceleration instruction, music play instruction, music stop instruction, story play instruction and story stop instruction. The instruction code table stores the function phrase, key number, remote control signal and control instruction corresponding to each function respectively;

[0106] The automatic control module is in the high level signal y h During the reception period, prompt instructions are periodically sent to the execution module, nodes are periodically generated, and the video digital signal sequence Y is intercepted at each node. vid The video digital signal from the previous node to the current node is transcoded into a time period video Identify the baby's sleeping state through facial recognition and optical flow detection, generate control instructions or prompt instructions based on the rule engine and transmit them to the execution module;

[0107] The execution module executes the control instruction and receives the video digital signal sequence Y vidAnd synchronously upload it to the video database of the cloud platform, receive prompt instructions and send them to all paired devices through the cloud platform, where the paired devices are mobile devices registered by the user on the cloud platform and bound to the control system. The control system can pair multiple successfully registered mobile devices at the same time.

[0108] Furthermore, the collection module includes a urine sensing unit, a sound sensing unit, and a video monitoring unit;

[0109] The urine sensing unit senses urine contact based on the capacitive sensing principle and the conduction characteristics of the NMOS tube. When urine contacts the urine sensor, the medium between the capacitor electrodes changes from air to urine, the current of the bias circuit increases, causing the NMOS tube to turn on, generating a high-level signal y h And transmit it to the automatic control module. When the urine is cleared, the NMOS tube turns to the cut-off state, and the high-level signal y h Stop producing;

[0110] The sound perception unit collects sound analog signals x through the microphone array raw , a filter bank is used to perform the initial filtering on the sound analog signal, and the residual noise is further suppressed by the adaptive filtering algorithm after the analog-to-digital converter, and the sound digital signal sequence y is generated and transmitted to the controlled module;

[0111] The video monitoring unit uses a digital optical sensor to collect the ambient light intensity in real time, determine the light intensity range of the ambient light intensity and the corresponding infrared fill light brightness, and continuously shoots the baby's status in the cradle through a high-definition camera to generate monitoring video X vid , using H.265 encoding algorithm to convert surveillance video X vid Converted into video digital signal sequence Y vid And it is sent synchronously to the controlled module and the execution module. The light intensity range is pre-divided, and each light intensity range is calibrated according to the corresponding fill light brightness based on the experiment. The wavelength of the infrared fill light is 850nm, which is invisible to the human eye and will not affect the baby's sleep. The H.265 encoding algorithm is the existing video encoding and transmission algorithm.

[0112] Furthermore, the urine sensing unit senses urine contact based on the capacitive sensing principle and the NMOS tube conduction characteristics, including the following specific steps:

[0113] Connect the capacitor to the bias circuit to form a complete circuit loop, connect the gate of the NMOS tube to the bias circuit, and ground the source. The bias circuit includes resistor R and DC power supply V cc ;

[0114] When there is no urine contact, the medium between the capacitor electrodes is air, the capacitance is the normal capacitance C0, and the normal current I0 of the bias circuit should be equal to the DC power supply Vcc Divide by the resistance R and the normal capacitance The sum is as follows:

[0115]

[0116] Among them, normal capacitance It is inversely proportional to the normal capacitance C0, as follows:

[0117]

[0118] Among them, f cc is the operating frequency of the power supply;

[0119] Normal gate-source voltage of NMOS tube is the DC power supply V cc Normal voltage of capacitor Specifically: Normal gate-source voltage Less than or equal to the breakdown voltage V th , the source and drain of the NMOS tube are in the cut-off state;

[0120] When urine comes into contact, the medium between the capacitor electrodes becomes urine, and the capacitance becomes the abnormal capacitance C1. Since the capacitor capacitance is equal to the product of the electrode plate area and the dielectric constant divided by the electrode plate distance, and the urine dielectric constant is greater than the air dielectric constant, the abnormal capacitance C1 is greater than the normal capacitance C0, and the abnormal capacitive reactance Less than normal capacitance The abnormal current I1 is greater than the normal current I0, and the abnormal voltage of the capacitor Greater than the normal voltage of the capacitor Abnormal gate-source voltage of NMOS tube Greater than the breakdown voltage V th , the source and drain of the NMOS tube are in the on state, generating a drain-source current I ds ;

[0121] The drain-source current I ds Converted into output signal voltage V out And connected to the comparator circuit, the comparator will output the signal voltage V out With the preset reference voltage V ref For comparison, if the output signal voltage V out Greater than the reference voltage V ref , generating a high level signal y h And transmit to the automatic control module, if the output signal voltage V out Less than or equal to the reference voltage V ref , then ignore the output signal voltage V outCompared with directly judging based on the level signal generated after the NMOS tube is turned on, the comparator circuit effectively avoids misjudgment caused by noise interference or environmental electromagnetic interference. Moreover, the level signal generated after the NMOS tube is turned on does not meet the logic level standard and still requires additional circuit conversion before transmission. The high level signal y h Comply with logic level standards and have good compatibility with any chip or circuit architecture;

[0122] When the urine is cleared, the medium of the capacitor gradually returns to air, and the capacitance gradually recovers from the abnormal capacitance C1 to the normal capacitance C0. The drain-source current I ds Gradually decreases to 0, when the output signal voltage V out Less than or equal to the reference voltage V ref When the comparator stops outputting high level signal y h .

[0123] Furthermore, the filter bank, analog-to-digital converter, and adaptive filtering algorithm are used to filter, convert, and refilter the sound analog signal to generate the sound digital signal sequence y, which includes the following specific steps:

[0124] Convert the sound analog signal x raw Input filter group, the filter group includes band pass filter, low pass filter and high pass filter. The band pass filter allows [f min ,f max ] frequency range of the signal through, where f min and f max They are the minimum human voice frequency and the maximum human voice frequency respectively, and low-pass filter and high-pass filter are used to filter the frequencies below the minimum human voice frequency f min and greater than the maximum human voice frequency f max The frequency range is further filtered to generate the filtered analog signal x hp In this embodiment, considering the need to identify infant crying or human language, the minimum human voice frequency f min and the maximum vocal frequency f max 80Hz and 8kHz respectively;

[0125] The analog-to-digital converter is used to filter the analog signal x hp Discrete sampling is performed in time to generate the initial digital signal sequence y of the sound ori ={y ori (1),…,y ori (n),…,y ori (N)},y ori (n) is the initial digital signal of the sound at the nth sampling moment, n = 1, ..., N, N is the total number of sampling moments, the sampling frequency f sSatisfy Nyquist sampling theorem, that is, the sampling frequency f s Greater than the maximum human voice frequency f max 2 times;

[0126] Initialize and generate the initial digital signal y of the sound at the first sampling moment ori The weight coefficient vector of (1) is w(1) = [w1(1),…,w m (1),…,w M (1)] and the positive definite symmetric matrix P(1) = δ -1 I, where w m (1) is the initial digital signal y of the sound at the first sampling moment ori The mth weight coefficient of (1), m = 1, ..., M, δ is a very small positive number, I is the unit matrix with dimension M × M, and M is the reference number;

[0127] The reference noise signal sequence d is generated by discretizing the prior white noise model that obeys Gaussian distribution. ref ={d ref (1),…,d ref (n),…,d ref (N)}, where d ref (n) is the reference noise signal at the nth sampling moment, n = 1,…,N, where N is the total number of sampling moments;

[0128] Get the initial digital signal y of the sound at the nth sampling moment according to the reference number M ori (n) The initial digital signal y of the sound at the previous M-1 sampling moments ori (n-1),…,y ori (n-M+1), construct the initial sound input vector y at the nth sampling moment ori (n)=[y ori (n),…,y ori (n-M+1)], where if n < M-1, that is, the initial digital sound signal of the M-1 sampling moments cannot be obtained before the nth sampling moment, the insufficient part is supplemented with 0. Taking n = 2 and M = 3 as an example, the initial sound input vector y ori (2)=[y ori (2),y ori (1),0];

[0129] The weight coefficient vector w(n) = [w1(n),…,w m (n),…,w M (n)] and the initial sound input vector y ori The inner product of (n) is the sound digital signal y(n) at the nth sampling moment after adaptive filtering. The specific formula is:

[0130] Calculate the error signal e(n)=y(n)+d at the nth sampling moment ref (n)-y ori (n), the error signal e(n) reflects the effect of adaptive filtering. The smaller the error signal e(n), the smaller the adaptive filtering will be. ori The more complete the noise filtering in (n);

[0131] The gain vector g(n) at the n-th sampling moment is calculated based on the forgetting factor λ and the positive definite symmetric matrix P(n-1) at the n-1-th sampling moment. The specific formula is as follows:

[0132]

[0133] The weight coefficient vector w(n) at the nth sampling moment is adjusted according to the gain vector g(n) and the error signal e(n) at the nth sampling moment to generate the weight coefficient vector w(n+1) at the n+1th sampling moment. Specifically, w(n+1)=w(n)+g(n)e(n). That is, the larger the error signal e(n), the larger the adjustment amplitude.

[0134] To ensure convergence, the positive definite symmetric matrix P(n) at the nth sampling moment is updated and obtained. The specific formula is as follows:

[0135]

[0136] The ultimate goal of the adaptive filtering algorithm is to minimize the weighted sum of squares of the error signal through multiple rounds of iterations. When the algorithm converges, a sound digital signal sequence y is generated, wherein the weight coefficient vectors and the positive definite symmetric matrix at N sampling moments are updated in sequence in each round of iteration.

[0137] Furthermore, the controlled module includes a touch control unit, a remote control unit and a voice control unit;

[0138] The touch control unit is implemented based on a capacitive touch sensor. Based on the principle of capacitance parasitics, it senses the user's touch position, generates a touch distribution network, and verifies the overlap rate with the key layout network. If and only if the overlap rate between the activated grid in the touch distribution network and the grid area of ​​a single key is greater than or equal to the overlap rate threshold, the unit retrieves the instruction encoding table based on the key number, generates the corresponding control instruction, and sends it to the execution module. It also provides vibration feedback to indicate that the touch is valid. In all other cases, the touch is ignored.

[0139] The remote control unit wirelessly receives the remote control signal sent by the cloud platform, retrieves the instruction encoding table, generates a control instruction matching the remote control signal, and forwards it to the execution module. The remote control signal is sent by the user through the paired device. The cloud platform receives the remote control signal sent by the paired device, determines the paired control system based on the registration information, and sends it;

[0140] The voice control unit receives the sound digital signal sequence y, converts it to the frequency domain, and constructs a Mel filter bank for filtering. After performing a logarithmic operation, it is converted to the cepstral domain to generate the Mel-frequency cepstral coefficient MF. A progressive binary classification model is used to analyze the Mel-frequency cepstral coefficient MF. The first support vector machine is used to determine whether the sound digital signal sequence y is a baby crying. If it is a baby crying, a prompt instruction is generated and sent to the execution module. If it is not a baby crying, the second support vector machine is used to determine whether the sound digital signal sequence y is human language. If it is not human language, the sound digital signal sequence y is ignored. If it is human language, the deep hidden Markov model is used to convert the Mel-frequency cepstral coefficient MF into speech. The content z is extracted by the context-constrained word segmentation model and the semantic dependencies are learned. The speech content z is split into a word segmentation sequence Wo. The sliding window is translated in the word segmentation sequence Wo to combine the continuous word segments in the sliding window into speech phrases, and a speech phrase sequence Ph is constructed. The trigger phrase, each functional phrase and each speech phrase in the speech phrase sequence Ph are converted into corresponding vector representations using the skip-word model and matched and verified by similarity. If the trigger phrase matches the speech phrase sequence Ph and there is at least one functional phrase that matches the speech phrase sequence Ph, the corresponding control instruction is generated according to the matching functional phrase retrieval instruction encoding table.

[0141] by Figure 2 For example, further, based on the capacitance parasitic principle, sensing the user's touch position to generate a touch distribution network and verifying the overlap rate with the button layout network includes the following specific steps:

[0142] Detecting a local charge change, obtaining a local charge peak, and determining whether the local charge peak is greater than a touch effective peak. The local charge change is caused by a user touch. When a user touches a key area, coupling is formed with the capacitive touch sensor below the key area. A parasitic capacitance in parallel with the parallel capacitance is generated near the contact position, causing an increase in the local charge. The parallel capacitance is a stable capacitance uniformly distributed within the capacitive touch sensor when no touch is applied.

[0143] If the local charge peak is less than the effective touch peak, it is determined to be an invalid touch and the local charge change is ignored. If the local charge peak is greater than or equal to the effective touch peak, it is determined to be a valid touch and the grid in the touch distribution network that overlaps with the local charge change area is recorded as an active grid ( Figure 2The touch distribution network is generated by dividing the key area into grids. When there is no user touch, all grids in the touch distribution network are silent grids by default. When the user touches, the distribution of active grids in the touch distribution network can effectively reflect the user's touch position in the key area.

[0144] Call the button layout network and verify the overlap with the touch distribution network. The button layout network uses grids to divide the button area in the same way as the touch distribution network. For a single grid in the button layout network, if it is covered by a button, the grid value corresponds to the button number. If there is no button covered, the grid value is empty.

[0145] Determine the overlap ratio between the grid area of ​​each button number in the button layout network and the activated grid in the touch distribution network, and determine whether the maximum overlap ratio is greater than or equal to the overlap ratio threshold, where the overlap ratio is equal to the number of grids in the grid area of ​​a single button number that overlap with the activated grid divided by the total number of grids for the single button number;

[0146] If the maximum overlap rate is less than the overlap rate threshold, it indicates that the user's touch position does not overlap with any key, and the touch is considered invalid and ignored.

[0147] If the maximum overlap rate is greater than or equal to the overlap rate threshold, the command code table is searched according to the key number of the maximum overlap rate (in Figure 2 The middle button number is 2), and a control instruction corresponding to the button number is generated.

[0148] Furthermore, the sound digital signal sequence y is converted into the frequency domain and a Mel filter bank is constructed for filtering. After performing logarithmic operation, the signal is converted into the cepstral domain to generate the Mel frequency cepstral coefficient MF, which includes the following specific steps:

[0149] The sound digital signal sequence y={y(1),…,y(n),…,y(N)} is converted into the frequency domain by discrete Fourier transform to generate a spectrum set F={f1,…,f n ,…,f N}, where f n ={f n (0),…,f n (k),…,f n (K-1)} is the sound digital signal y(n) at the nth sampling moment, which is converted into the corresponding spectrum sequence through discrete Fourier transform, f n (k) is the sound digital signal y(n) at the kth frequency f k The frequency domain signal on is calculated as follows:

[0150]

[0151] j is an imaginary number, j 2=-1,y(n) l is the frame signal of the sound digital signal y(n) at the lth frame, l = 0, ..., L-1, L is the frame length of each sound digital signal, k is the frequency index, k = 0, ..., K-1, K is the total number of frequencies, and the discrete Fourier transform has symmetry, that is, the total number of frequencies K in the frequency domain is equal to the frame length L in the time domain;

[0152] Since the human auditory system perceives sound frequencies in a similar way to the Mel scale, it is known that the sound digital signal sequence y is filtered by the filter bank, and the frequency range is [f min ,f max ], f min and f max The minimum and maximum human voice frequencies are based on the Mel frequency f. m The Mel transform equation of the sum frequency f determines the Mel frequency range and are the minimum Mel frequency and the maximum Mel frequency respectively. The Mel transform equation is as follows:

[0153]

[0154] Mel frequency range Divide into Q Mel frequency segments and determine the center Mel frequency of each Mel frequency segment is the center Mel frequency of the qth Mel frequency segment, q = 1,…,Q, where Q is the total number of Mel frequency segments;

[0155] A triangular filter is set for each Mel frequency band, and the frequency response H of the triangular filter of the qth Mel frequency band at frequency f is q (f) as follows:

[0156]

[0157] in, and are the q-1th center Mel frequencies respectively qth center Mel frequency and the q+1th center Mel frequency The q-1th center frequency, qth center frequency and q+1th center frequency are obtained by reverse deduction of the Mel transform equation;

[0158] Each spectrum sequence in the spectrum set F is multiplied by Q triangular filters in turn and the logarithm is taken to obtain the logarithmic domain signal set LF = {lf1,…,lf n ,…,lf N}, lf n ={lfn (1),…,lf n (q),…,lf n (Q)} is the spectrum sequence f at the nth sampling moment n The logarithmic domain signal sequence is obtained by multiplying Q triangular filters in sequence and taking the logarithm, where lf n (q) is the spectrum sequence f at the nth sampling moment n The logarithmic domain signal obtained by multiplying with the qth triangular filter and taking the logarithm is as follows: Among them, f n (k) is the sound digital signal y(n) at the kth frequency f k The frequency domain signal on q (f k ) is the triangular filter of the qth Mel frequency band at frequency f k Frequency response at ;

[0159] The logarithmic domain signal set LF={lf1,…,lf n ,…,lf N}Convert to the cepstral domain to generate Mel frequency cepstral coefficients MF = [mf1,…,mf n ,…,mf N ] T , mf n The Mel-frequency cepstrum feature vector generated by converting the nth logarithmic domain signal to the cepstrum domain is as follows:

[0160] mf n =[mf n (1),…,mf n (u),…,mf n (U)],

[0161] Among them, mf n (u) is the nth Mel-frequency cepstrum eigenvector mf n The specific calculation formula for the u-th Mel frequency cepstrum feature is as follows:

[0162]

[0163] Wherein, u=1,…,U, where U is the dimension of the Mel-frequency cepstrum feature vector.

[0164] Specifically, the first support vector machine and the second support vector machine need to be pre-trained, and a large number of baby crying sounds, human language sounds and other types of sounds are collected in advance, all of which are converted into Mel-frequency cepstral coefficients MF and labeled as class 1, class 2 and class 3 respectively. When training the first support vector machine, the goal is to distinguish class 1 and the first remaining class as much as possible in the high-dimensional space to realize the judgment of baby crying. When training the second support vector machine, all Mel-frequency cepstral coefficients MF of class 1 are eliminated, and the Mel-frequency cepstral coefficients MF belonging to class 2 and the second remaining class are distinguished as much as possible in the high-dimensional space, and the judgment of human language is realized under the premise that it is clear that it is not baby crying. Compared with the synchronous judgment of the two support vector machines, the progressive judgment can reduce the training overhead and accelerate the convergence of the model. The first remaining class is the union of class 2 and class 3, class 1 and the first remaining class are mutually exclusive, the second remaining class is class 3, and class 2 and the second remaining class are mutually exclusive.

[0165] Specifically, deep hidden Markov models include deep networks and hidden Markov models;

[0166] The input of the deep network is the Mel-frequency cepstral coefficient MF. After multiple layers of linear weights, the Mel-frequency cepstral coefficient MF is linearly modulated and the dimension is changed to the number of states of the hidden Markov model. The complex relationship between the Mel-frequency cepstral feature vectors in the Mel-frequency cepstral coefficient MF is learned through the Softmax function to generate the posterior probability distribution of the state vector. The posterior probability distribution records the posterior probability of each state of the hidden Markov model.

[0167] The hidden Markov model defines a state vector based on phonemes. The state vector includes a silent state, a vocalization start state, a stable vocalization state, and a vocalization end state. A state transition probability matrix is ​​defined to describe the probability of transitioning from one state to another. The state transition probability matrix is ​​a square matrix whose dimension is directly related to the number of states in the state vector. An observation probability distribution is defined to represent the probability of observing the Mel-frequency cepstral coefficient MF in a single state. It is generally replaced by the posterior probability distribution output by a deep network. The initial state probability distribution is defined as the distribution consisting of the probabilities of being in each state at the initial time step. At each time step, the probability of transitioning from the current state to the next state and the probability of observing the Mel-frequency cepstral coefficient MF are determined based on the state transition probability matrix and the observation probability distribution. The Viterbi algorithm is used to calculate the maximum probability path from the current state to the remaining states through dynamic programming and record the path information. The optimal state sequence is obtained by backtracking. A predetermined pronunciation dictionary is queried and the speech content z is generated based on the phoneme combination in the optimal state sequence number. The Viterbi algorithm is an existing algorithm.

[0168] Furthermore, the speech content z is split into speech phrases using a context-constrained word segmentation model and recombined into a speech phrase sequence Ph using a translation sliding window. The trigger phrase, all functional phrases, and all speech phrases in the speech phrase sequence Ph are converted into vector representations using a skip-gram model and matched and verified using similarity. The following specific steps are involved:

[0169] Input the speech content z into the context-constrained word segmentation model, which includes a bidirectional long short-term memory network layer and a conditional random field;

[0170] The bidirectional long short-term memory network layer includes a forward LSTM and a backward LSTM. The forward LSTM processes the speech content z from the beginning to the end, learning the influence of the previous Chinese character on the next Chinese character through the input gate and the forget gate, capturing the forward semantic association of the speech content z. The backward LSTM processes the speech content z from the end to the beginning, capturing the reverse semantic association of the speech content z. The bidirectional long short-term memory network layer can simultaneously capture the semantic information of the front and back of the Chinese characters, generating semantic features that help understand the contextual dependencies of the speech content z.

[0171] The conditional random field calculates the probability that the feature value of each dimension in the semantic feature belongs to a single part-of-speech tag based on the state feature function to obtain all possible candidate word segmentation sequences. The candidate word segmentation sequence is constrained by focusing on the transfer relationship between the part-of-speech tags through the transfer feature function, and the conditional probability of each candidate word segmentation sequence is calculated. The optimal part-of-speech tag output order is found through dynamic programming using the Viterbi algorithm, and the speech content z is annotated in turn for word segmentation to obtain the word segmentation sequence Wo. Among them, the conditional random field is an existing algorithm. The part-of-speech tags include nouns, verbs, adjectives and adverbs, and the transfer feature function is essentially the constraint relationship between the part-of-speech tags. For example, the sentence structure of "noun + verb + adjective + noun" is generally reasonable, while the sentence structure of "noun + noun + adverb" is obviously unreasonable.

[0172] The sliding window width is set according to the maximum phrase length among the trigger phrase and all functional phrases. The sliding window width refers to the number of segmentations that can be accommodated between the starting edge and the ending edge of the sliding window. The starting edge is placed at the first segmentation in the segmentation sequence Wo. The segmentations within the sliding window width are combined into the first voice phrase. The starting edge is shifted to the second segmentation and combined to generate the second voice phrase. When the ending edge reaches the last segmentation in the segmentation sequence Wo, the last voice phrase is generated. All voice phrases are sequentially generated into a voice phrase sequence Ph.

[0173] The skip-gram model is used to convert the input phrase into a vector representation. The skip-gram model extracts the central participle in the input phrase and determines the co-occurrence probability of the central participle of the input phrase and the context participles. The central participle is converted into a one-hot encoding based on a predetermined corpus. The one-hot encoding is first expanded and then reduced in dimension through two linear layers so that the dimension is equal to the number of participles in the input phrase. The probability distribution of the input phrase is generated through the Softmax function. The weights of the linear layer are adjusted using backpropagation to minimize the error between the probability distribution of the input phrase and the co-occurrence probability. Finally, the weights of the first linear layer are used as the vector representation of the input phrase. Among them, the trigger phrase is converted into a trigger vector representation through the skip-gram model, each function phrase is converted into a corresponding function vector representation, and each speech phrase in the speech phrase sequence Ph is converted into a corresponding speech vector representation;

[0174] Calculate the trigger cosine similarity between the trigger vector representation and each speech vector representation, and determine whether there is a trigger cosine similarity greater than or equal to a similarity threshold;

[0175] If it does not exist, it indicates that there is no trigger phrase in the voice phrase sequence Ph, the trigger phrase fails to match the voice phrase sequence Ph, and the sound digital signal sequence y is ignored, wherein the trigger phrase is a preset specific phrase used to determine the validity of the voice phrase sequence Ph and prevent daily conversations from causing erroneous control of the control system;

[0176] If it exists, it indicates that there is a trigger phrase in the speech phrase sequence Ph, and the trigger phrase matches the speech phrase sequence Ph. Further, each function vector representation and each speech vector representation are calculated with a function cosine similarity to determine whether there is a function cosine similarity greater than or equal to the similarity threshold;

[0177] If it does not exist, it indicates that there is no functional phrase in the voice phrase sequence Ph, all functional phrases fail to match the voice phrase sequence Ph, and the sound digital signal sequence y is ignored;

[0178] If so, all functional phrases whose functional cosine similarity is greater than or equal to the similarity threshold are considered to match the voice phrase sequence Ph, and the instruction coding table is searched to obtain the control instructions corresponding to all matching functional phrases.

[0179] Furthermore, the self-control module includes a bed-wetting reminder unit and a video monitoring unit;

[0180] The bed-wetting reminder unit receives a high-level signal y h , record the receiving time and immediately generate a prompt instruction and send it to the execution module. Starting from the receiving time, every time a prompt interval passes, it is determined whether the high level signal y is still being received. h , if it is still receiving high level signal y hThen repeatedly generate prompt instructions and send them to the execution module. If no high level signal y is received h Then clear the recorded receiving time;

[0181] After the control system is turned on, the video monitoring unit periodically generates nodes according to the monitoring interval and intercepts the video digital signal sequence Y at each node. vid All video digital signals from the previous node to the current node are reversely converted into time-segment video based on the H.265 encoding algorithm Analyzing period videos with facial recognition and optical flow detection The system detects the facial and body features of the infant, identifies the infant's sleeping state, generates control instructions or prompt instructions based on the rule engine and transmits them to the execution module. The infant's sleeping state includes deep sleep, light sleep, semi-awake, awake and abnormal.

[0182] like Figure 3 As shown, further, the time period video is analyzed through face recognition and optical flow detection Identifying a baby's sleep state involves the following specific steps:

[0183] Time period video Perform equal-interval sampling to obtain T frames of sampling frame images to construct a sampling frame image sequence in, is the sampling frame image of the tth frame;

[0184] Combined with Dlib library to sample frame image sequence To identify the eye state, use Dlib's face detector to locate the baby's face area in each sample frame image. Use Dlib's pre-trained 68-facial landmark model to obtain the coordinates of the key points around the eyes in each sample frame image. The face detector and 68-facial landmark model are existing models.

[0185] Calculate the vertical distance between the midpoint of the upper eyelid and the midpoint of the lower eyelid in each sampling frame image and take the average to obtain the sampling frame image sequence The average vertical distance of the eyes was taken as the ratio of the average vertical distance to the maximum vertical distance, where the maximum vertical distance was determined by averaging the vertical distances of a large number of infants with fully open eyes. The eyes-opening ratio was always in the range of 0 to 1.

[0186] Based on the lightweight MobileNetV3 model, the spatial attention mechanism is introduced to improve the accuracy of expression recognition. The initial convolutional layer in the MobileNetV3 model extracts the sample frame image sequence. For each sample frame image in the feature map, the spatial attention mechanism performs average pooling and global pooling on the channel sub-maps of the three RGB channels of the feature map, splices the two pooling results, and learns the attention weights of different spatial positions in the sample frame image through convolution operation, and obtains the spatial attention map through Sigmoid function;

[0187] The spatial attention map is connected to the depthwise separable convolution layer of the MobileNetV3 model. The spatial attention map is sequentially upgraded, feature extracted, and reduced through point-by-point convolution, depth-wise convolution, and point-by-point convolution. It is then converted into a spatial attention vector through global average pooling. After linear modulation through a fully connected layer, the softmax function is used to generate the probability distribution of the expression level of the sampled frame image. The average probability distribution of the expression level of all sampled frames is calculated. The expression levels are classified as calm, moderate, and rich.

[0188] Use optical flow method to detect the sample frame image sequence The change of the pixel points in the time sequence defines the brightness identity, which describes the frame image of the tth frame The pixel point with pixel coordinates (α1, α2) in The brightness Lg(α1,α2,t) of the image is Lg(α1,α2,t). The pixel coordinates are offset from the tth frame to the t+1th frame. The offset is (Δα1,Δα2). In the t+1th frame image, The corresponding pixel point in the The brightness Lg(α1+Δα1,α2+Δα2,t+1) of the t+1 frame is still the same as the brightness Lg(α1,α2,t), that is, Lg(α1,α2,t)=Lg(α1+Δα1,α2+Δα2,t+1), and based on the assumption of spatiotemporal continuity, the offset (Δα1,Δα2) will not be too large. The image sequence describing the sampled frame is found through TV-L1 iterative optimization. The optical flow feature sequence, the optical flow feature map in the optical flow feature sequence and the sampling frame image sequence The sampling frame image of the t-th frame corresponds one to one, and the optical flow feature map of the t-th frame is a two-dimensional vector field, which describes the frame image from the t-1th frame. Frame image to frame t The movement trajectory and speed of pixels over time are optimized by TV-L1 iteratively.

[0189] The optical flow feature sequence is aggregated through a temporal convolution layer. The temporal convolution layer captures the temporal changes of the optical flow features in the optical flow feature sequence through two identical residual blocks. Each residual block is composed of a dilated convolution and a residual structure. The dilated convolution inserts 0 elements between the convolution kernels to expand the receptive field. The residual structure superimposes the output of the dilated convolution with the input. After passing through two identical residual blocks, the fully connected layer reduces the dimension and outputs an aggregated optical flow feature map of the same dimension as any optical flow feature map in the optical flow feature sequence.

[0190] The aggregated optical flow feature map is linearly reduced in dimension through a fully connected layer, and the Softmax function is used again to generate the probability distribution of the action amplitude, which includes small, medium and large amplitudes.

[0191] Set the evaluation score, define calm, moderate, and rich expression levels as 0, 0.5, and 1 respectively, define small, medium, and large movement amplitudes as 0, 0.5, and 1 respectively, and calculate the expression score by weighted summation based on the expression level probability distribution. Calculate the movement score by weighted summation based on the movement amplitude probability distribution. Both the expression score and the movement score are within the range of 0-1.

[0192] The eye-opening ratio, expression score, and movement score are weighted and summed again based on the reference weights of eye-opening, expression, and movement to obtain a comprehensive sleep score. The infant's sleep state is determined based on the comprehensive score interval in which the comprehensive sleep score falls. The reference weight and comprehensive score interval are predefined. In this embodiment, the reference weight is 0.3:0.3:0.4, and the comprehensive score interval is as follows:

[0193] If the comprehensive score is less than 0.1, the baby's sleep state is deep sleep;

[0194] A comprehensive score greater than or equal to 0.1 and less than 0.35 indicates that the infant's sleep state is light sleep;

[0195] A comprehensive score greater than or equal to 0.35 and less than 0.65 corresponds to the infant's sleep state being semi-awake;

[0196] A combined score greater than or equal to 0.65 and less than 0.9 corresponds to the infant's sleep state being awake;

[0197] A comprehensive score greater than or equal to 0.9 indicates that the infant's sleep state is abnormal.

[0198] Specifically, the rule engine generates corresponding control instructions or prompt instructions based on the baby's sleep state, including:

[0199] If the baby is in deep sleep, a series of control instructions are generated in sequence according to the current working status of the variable-speed synchronous motor and the speaker and sent to the execution module to stop the variable-speed synchronous motor and stop the speaker from playing music and stories. For example, if the variable-speed synchronous motor is currently started and the speaker is only playing music, only the swing stop instruction and the music stop instruction need to be generated;

[0200] If the baby is in light sleep, control instructions are generated in sequence based on the current operating states of the variable-speed synchronous motor and the speaker to cause the variable-speed synchronous motor to operate at a low gear and the speaker to stop playing music and stories. For example, if the variable-speed synchronous motor is not currently started and the previous motor gear was high, and the speaker is currently started and playing music and stories, a swing start instruction, two swing deceleration instructions, a music stop instruction, and a story stop instruction are generated in sequence;

[0201] If the baby is semi-awake, control instructions are generated in sequence based on the current operating states of the variable-speed synchronous motor and the speaker, causing the variable-speed synchronous motor to operate at a medium gear and the speaker to play only music. For example, if the variable-speed synchronous motor is already started and the current motor gear is at a medium gear, and the speaker is playing music and a story simultaneously, only a story stop instruction is generated.

[0202] If the baby's sleep state is awake, control instructions are generated in sequence according to the current working states of the variable-speed synchronous motor and the speaker, so that the variable-speed synchronous motor operates at high speed and the speaker plays music and stories at the same time. For example, if the variable-speed synchronous motor is currently started and the current motor gear is at low speed, and the speaker is currently not started, then two swing acceleration instructions, a music playing instruction, and a story playing instruction are generated in sequence;

[0203] If the baby's sleeping state is abnormal, a prompt instruction is directly generated and sent to the execution module.

[0204] Furthermore, the execution module includes a power on / off unit, a variable speed synchronous motor execution unit, a sound playing unit, and a cloud communication unit;

[0205] The power on / off unit receives a power on instruction or a power off instruction, opens or closes a circuit switch to control the start and stop of the control system and the controlled device;

[0206] The variable speed synchronous motor execution unit receives a swing start instruction, starts the variable speed synchronous motor according to the previous motor gear, receives a swing stop instruction, records the current motor gear and turns off the variable speed synchronous motor, receives a swing acceleration instruction, determines whether the current motor gear is high, if so, ignores it, if not, adjusts the motor gear up by one gear, receives a swing deceleration instruction, determines whether the current motor gear is low, if so, ignores it, if not, adjusts the motor gear down by one gear, wherein the motor gear includes low, medium and high gears;

[0207] The sound playback unit receives a music playback instruction or a story playback instruction, and sequentially calls the next built-in music or story based on the previously played music or story and plays it through the speaker. When receiving a music stop instruction or a story stop instruction, it records the currently playing music or story and turns off the speaker.

[0208] The cloud communication unit receives the video digital signal sequence Y vid And synchronously upload it to the video database of the cloud platform, receive prompt instructions and send them to all paired devices through the cloud platform, wherein the cloud platform allocates different storage areas according to the number of the controlled device controlled by the control system and establishes a corresponding video database. Only paired devices can view the video database of the corresponding control system through the cloud platform.

[0209] Example 2

[0210] like Figure 4 As shown, the present invention further discloses a controlled device, which is controlled by the control system and includes a cradle body 1, a base 2, a driving device 3, a universal ball bearing 4, a universal rocking plate 5, a cradle body connecting flange 6, a limiting rubber belt 7, a urine sensor 8 and an integrated processing device 9;

[0211] The cradle body 1 is used to accommodate a baby to sleep;

[0212] The base 2 is used to increase the stability of the controlled device, and there is a compartment inside the base 2;

[0213] The driving device 3 is installed in the compartment of the base 2 and includes a variable speed synchronous motor 31 and a variable speed synchronous motor rocker arm 32. When the variable speed synchronous motor 31 is started, it drives the variable speed synchronous motor rocker arm 32 to swing in the left and right directions and the front and back directions. The variable speed synchronous motor rocker arm 32 is fixedly connected to the universal swing plate 5.

[0214] The universal ball bearing 4 is used to support the universal swing plate 5, reducing the friction between the universal swing plate 5 and the base 2, making the swing of the universal swing plate 5 smoother;

[0215] The universal swing plate 5 and the variable speed synchronous motor rocker arm 32 keep swinging in the same direction;

[0216] The cradle body connecting flange 6 connects the universal swing plate 5 and the cradle body 1, so that the cradle body 1 and the universal swing plate 5 keep swinging in the same direction;

[0217] The limiting rubber belt 7 connects the base 2 and the universal swing plate 5 to prevent the universal swing plate 5 from derailing after being hit by external force;

[0218] The urine sensor 8 is installed at the bottom of the cradle body 1 and is used to sense urine contact and generate a high-level signal y h ;

[0219] The integrated processing device 9 is used to collect the sound digital signal sequence y and continuously monitor and obtain the video digital signal sequence Y vid , providing four modes of user touch control, remote control, voice control and automatic control to control the start, stop and gear adjustment of the drive device 3 as well as the play and stop of music and stories, and uploading the video digital signal sequence Y to the cloud platform vid and prompt instructions.

[0220] Furthermore, the integrated processing device 9 includes a key screen, a rear camera, a microphone array, a speaker, a wireless communication module and a processor;

[0221] The key screen is located on the front of the integrated processing device 9, displays keys, and provides touch control for the user;

[0222] The rear camera is fixed to the back of the integrated processing device 9 and monitors and records the interior of the cradle body 1 at a specific angle;

[0223] Microphone array is used to collect sound analog signals x raw And generate a sound digital signal sequence y through filtering and analog-to-digital conversion;

[0224] The speakers are controlled by the processor to play or stop the music or story;

[0225] The wireless communication module receives the remote control signal and transmits it to the processor, which receives the video digital signal sequence Y vid and prompt instructions and upload to the cloud platform;

[0226] The processor is at high level signal y h During the reception period, prompt instructions are continuously generated, user touch buttons are recognized to generate control instructions, remote control signals are received to generate control instructions, and a sound digital signal sequence y is received and judged to be a baby crying or human language. If it is a baby crying, a prompt instruction is generated. If it is human language, it is converted into voice content z and matched and verified with the trigger phrase and all functional phrases. When the trigger phrase is successfully matched, the instruction coding table is retrieved according to the matched functional phrase to generate the corresponding control instruction. The video digital signal sequence Y is received. vid And periodically capture and transcode into time-segment videos The baby's sleeping state is identified through facial recognition and optical flow detection, and control instructions or prompt instructions are generated based on the rule engine. The control instructions are used to control the start, stop and gear adjustment of the drive device 3 and the playing and stopping of music and stories of the speaker, and the prompt instructions are transferred to the wireless communication module.

[0227] The present invention discloses a control system and a controlled device, including an acquisition module, a controlled module, an automatic control module and an execution module; the acquisition module continuously sends a high-level signal when urine is present, collects sound analog signals and processes them to generate a sound digital signal sequence, and continuously obtains a video digital signal sequence; the controlled module senses user touch or receives a remote control signal to generate a control instruction, converts the sound digital signal sequence into a Mel-frequency cepstral coefficient, and identifies whether the sound digital signal sequence is a baby crying or human language based on a progressive binary classification model. If it is a baby crying, a prompt instruction is generated; if it is human language, a deep hidden Markov model is used in combination with a context-constrained word segmentation model to convert the Mel-frequency cepstral coefficient into speech content and split and reorganize it into a speech phrase sequence based on semantic dependency, and a skip-gram model is used to identify whether the sound digital signal sequence is a baby crying or human language. The model converts the trigger phrases, function phrases and voice phrases in the voice phrase sequence into vector representations and performs matching verification. When the trigger phrase matches successfully and at least one function phrase matches successfully, a control instruction is generated based on the matched function phrase. In other cases, the sound digital signal sequence is ignored. The automatic control module periodically intercepts the video digital signal sequence and transcodes it into a time period video, identifies the baby's sleeping state through face recognition and optical flow detection, generates control instructions or prompt instructions based on the rule engine, and periodically generates prompt instructions during the high-level signal reception period. The execution module executes the control instruction, receives the video digital signal sequence and prompt instructions and uploads them to the cloud platform synchronously, realizing the four-in-one collaborative control mode of touch, remote, voice and automation, and improving the intelligence level of the baby cradle.

[0228] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A control system, characterized in that Including controlled module and automatic control module; The controlled module senses user touch or receives a remote control signal to generate a control instruction, receives a sound digital signal sequence and converts it into Mel-frequency cepstral coefficients, identifies whether the sound digital signal sequence is a baby crying or human language based on a progressive binary classification model, and generates a prompt instruction if it is a baby crying; if it is human language, uses a deep hidden Markov model and a context-constrained word segmentation model to analyze the Mel-frequency cepstral coefficients to generate speech content and splits and reorganizes it into a speech phrase sequence based on semantic dependencies, uses a skip-gram model to match and verify the speech phrase sequence to detect whether it matches a trigger phrase and a function phrase, and generates a control instruction based on the matched function phrase when the trigger phrase and at least one function phrase are matched, and ignores the sound digital signal sequence in all other cases; The self-control module periodically intercepts the video digital signal sequence and identifies the baby's sleeping state through transcoding, face recognition and optical flow detection, generates control instructions or prompt instructions according to the rule engine, and continuously generates prompt instructions during the high-level signal reception period.

2. The control system according to claim 1, wherein: The controlled module includes a voice control unit; The voice control unit filters and performs logarithmic operations on the sound digital signal sequence with the Mel filter bank in the frequency domain, converts it to the cepstral domain to generate Mel frequency cepstral coefficients, adopts a progressive binary classification model based on the Mel frequency cepstral coefficients to determine in sequence whether the sound digital signal sequence is a baby crying or human language, generates a prompt instruction if it is a baby crying, and adopts a deep hidden Markov model to convert the Mel frequency cepstral coefficients into speech content if it is human language, adopts a context-constrained word segmentation model to split the speech content into word segmentation sequences based on semantic dependency, and constructs a speech phrase sequence through a translation sliding window combination, utilizes a skip-word model to match and verify the trigger phrases, function phrases and speech phrases in the speech phrase sequence, and generates corresponding control instructions according to the matched function phrases if the trigger phrase and at least one function phrase match the speech phrase sequence. In other cases, the sound digital signal sequence is ignored.

3. The control system according to claim 2, wherein: The method of filtering and performing logarithmic operations on the sound digital signal sequence with the Mel filter bank in the frequency domain and converting the signal to the cepstral domain to generate Mel-frequency cepstral coefficients includes the following specific steps: Processing the sound digital signal sequence through discrete Fourier transform to generate a spectrum set, the spectrum set includes spectrum sequences at N sampling moments, the spectrum sequence at the nth sampling moment includes frequency domain signals of K frequencies, where N and K are the total number of sampling moments and the total number of frequencies respectively; According to the Mel transform equation, the frequency range of the sound digital signal sequence is converted into a Mel frequency range and divided into Q Mel frequency segments, and the center Mel frequencies of the Q Mel frequency segments are determined respectively, where Q is the total number of Mel frequency segments; A triangular filter is set in the middle Mel frequency segment according to the center Mel frequency of three adjacent Mel frequency segments, and a total of Q triangular filters are obtained. The triangular filters correspond to the Mel frequency segments one by one. Multiply the spectrum sequences at N sampling moments in the spectrum set by Q triangular filters and take their logarithms to generate a logarithmic domain signal set. The logarithmic domain signal set includes the logarithmic domain signal sequences at N sampling moments. The logarithmic domain signal sequence at the nth sampling moment includes Q logarithmic domain signals, where the qth logarithmic domain signal is the sum of the products of the frequency response of the qth triangular filter and the corresponding frequency signal for each frequency in the spectrum sequence at the nth sampling moment, q = 1,…,Q; The logarithmic domain signal set is converted to the cepstral domain by discrete cosine transform to generate Mel-frequency cepstral coefficients, which include Mel-frequency cepstral feature vectors at N sampling moments.

4. The control system according to claim 2, wherein: The deep hidden Markov model includes a deep network and a hidden Markov model; The deep network linearly modulates the Mel-frequency cepstral coefficients through a linear layer and a Softmax function and maps them to generate a posterior probability distribution of the state vector, which records the posterior probability of each state in the state vector; The hidden Markov model defines a state vector based on phonemes, where the state vector includes a silent state, a phonation start state, a stable phonation state, and a phonation end state. A state transition probability matrix is ​​defined to describe the probability of transitioning from one state to another. The posterior probability distribution of the state vector is defined to reflect the probability of observing Mel-frequency cepstral coefficients in a single state. The initial state probability distribution of the initial time step is defined. At each time step, the probability of transitioning from the current state to the next state is confirmed based on the state transition probability matrix, and the probability of observing Mel-frequency cepstral coefficients is determined based on the posterior probability distribution of the state vector. The Viterbi algorithm is used to calculate the maximum probability path from the current state to the remaining states and record the path information. The optimal state sequence is obtained by backtracking, and the phonemes in the optimal state sequence are combined according to the pronunciation dictionary to generate speech content.

5. The control system according to claim 2, wherein: The context-constrained word segmentation model includes a bidirectional long short-term memory network layer and a conditional random field; The bidirectional long short-term memory network layer uses forward LSTM and backward LSTM respectively. Through the input gate and forget gate, it learns the influence between adjacent Chinese characters in the speech content in two different orders, capturing the semantic associations in the forward and reverse order to generate semantic features. The conditional random field calculates the probability that each feature value in the semantic feature belongs to a single part-of-speech tag based on the state feature function to obtain all candidate word segmentation sequences. The candidate word segmentation sequences are constrained by focusing on the transfer relationship between part-of-speech tags through the transfer feature function, and the conditional probability of each candidate word segmentation sequence is calculated. The optimal part-of-speech tag output order is found through the Viterbi algorithm, and the speech content is annotated for word segmentation to generate a word segmentation sequence.

6. The control system according to claim 2, wherein: The method of using the word skipping model to match and verify the trigger phrase, the function phrase, and the voice phrase in the voice phrase sequence includes the following specific steps: The skip-gram model calculates the co-occurrence probability of the central word and context words of the input phrase. Based on the corpus, the central word is converted into a one-hot encoding and the probability distribution of the input phrase is generated through two linear layers and a softmax function. Backpropagation is used to adjust the weights of the linear layer to minimize the error between the probability distribution of the input phrase and the co-occurrence probability. The weights of the first linear layer are used as the vector representation of the input phrase, where the input phrase includes a trigger phrase, a function phrase, and a speech phrase in the speech phrase sequence. The vector representation includes a trigger vector representation, a function vector representation, and a speech vector representation. Calculate the trigger cosine similarity between the trigger vector representation and each speech vector representation, as well as the function cosine similarity between each function vector representation and each speech vector representation, and compare them with the similarity threshold in turn; If both the trigger cosine similarity and the function cosine similarity are greater than or equal to the similarity threshold, a corresponding control instruction is generated according to all function phrases whose function cosine similarities are greater than or equal to the similarity threshold; if neither the trigger cosine similarity nor the function cosine similarity is greater than or equal to the similarity threshold, the sound digital signal sequence is ignored.

7. The control system according to claim 1, wherein: The automatic control module includes a bed-wetting prompt unit and a video monitoring unit; The bed-wetting prompt unit receives the high-level signal and records the receiving time, periodically generates a prompt instruction, and clears the receiving time when no high-level signal is received; After the control system is turned on, the video monitoring unit periodically intercepts the video digital signal sequence and converts it into a time period video based on the H.265 encoding algorithm. The time period video is analyzed through facial recognition and optical flow detection to identify the infant's sleeping state. Based on the correspondence between the infant's sleeping state and the target working state in the rule engine, a control instruction or prompt instruction is generated in combination with the current working state to adjust the current working state to the target working state. The infant's sleeping state includes deep sleep, light sleep, semi-awake, awake and abnormal.

8. The control system according to claim 7, wherein: The method of identifying the sleeping state of an infant by analyzing a period video through face recognition and optical flow detection comprises the following specific steps: The sampling period video was used to construct a sampling frame image sequence. A face detector and a 68-facial key point model were used to locate the coordinates of the baby's facial region and key points around the eyes in each sampling frame image in the sampling frame image sequence. The vertical distance between the midpoint of the upper eyelid and the midpoint of the lower eyelid was calculated. The average vertical distance was obtained and divided by the maximum vertical distance to generate the eye-opening ratio. The MobileNetV3 model extracts the feature map of each sample frame image in the sample frame image sequence through the convolution layer, uses the spatial attention mechanism to splice the average pooling results and global pooling results of the feature map on multiple channels, and generates the corresponding spatial attention map through convolution and sigmoid function; The MobileNetV3 model performs dimensionality upscaling, feature extraction, and dimensionality reduction on the spatial attention map of each sampled frame image. It then uses global average pooling, a fully connected layer, and a Softmax function to generate the corresponding probability distribution of expression levels and calculate the average probability distribution of expression levels. Expression levels include calm, moderate, and rich. The optical flow method is used to detect the temporal changes of pixels in the sample frame image sequence. A brightness identity is defined to describe the invariant brightness of corresponding pixels between adjacent frames. Based on the assumption of spatiotemporal continuity, the optical flow feature sequence describing the sample frame image sequence is found through TV-L1 iterative optimization. The optical flow feature sequence is input into the temporal convolution layer. The temporal convolution layer captures the temporal changes of the optical flow features through two identical residual blocks. Dimensionality reduction and mapping are performed through multiple fully connected layers and the Softmax function to generate a probability distribution of motion amplitude. The residual block includes dilated convolution and residual structure. Dilated convolution inserts 0 elements between convolution kernels to expand the receptive field. The residual structure superimposes the output of the dilated convolution with the input. The motion amplitude includes small, medium, and large. Based on the scores of each level of expression degree and movement amplitude, the expression score and movement score are generated by weighted summing up the probability distribution of expression degree and movement amplitude respectively. According to the reference weight, the eye-opening ratio, expression score and movement score are weighted summed again to generate a comprehensive sleep score. The infant's sleep state is determined based on the comprehensive score range in which the comprehensive sleep score is located.

9. The control system according to claim 1, wherein: The control system also includes an acquisition module and an execution module; The acquisition module includes a urine sensing unit, a sound sensing unit, and a video monitoring unit. The urine sensing unit senses urine contact based on the capacitive sensing principle and the conduction characteristics of the NMOS tube. When urine contacts, it generates a high-level signal and transmits it to the automatic control module. When the urine is cleared, it stops generating the high-level signal. The sound sensing unit collects the sound analog signal, sequentially passes it through a filter group, an analog-to-digital converter, and an adaptive filtering algorithm, converts it into a sound digital signal sequence, and transmits it to the controlled module. The video monitoring unit collects the ambient light intensity in real time and adaptively determines the fill light brightness, continuously shoots and generates monitoring video, uses the H.265 encoding algorithm to convert it into a video digital signal sequence, and synchronously sends it to the controlled module and the execution module. The execution module executes the opening and closing of the circuit switch, the opening, closing and gear adjustment of the variable-speed synchronous motor, the playing and stopping of music, and the playing and stopping of the story based on the control instructions, and sends the received video digital signal sequence and prompt instructions to the cloud platform, which is used to establish remote communication between the control system and the paired device.

10. A controlled device, controlled by the control system according to any one of claims 1 to 9, comprising a cradle body for accommodating a sleeping baby, a base for increasing stability, a universal rocking plate for swinging, a cradle body connecting flange for ensuring that the cradle body and the universal rocking plate swing in the same direction, and a limiting rubber belt for limiting the universal rocking plate from swinging off the track, characterized in that: The controlled device further comprises a driving device, a universal ball bearing, a urine sensor and an integrated processing device; The driving device is controlled by an integrated processing device and is connected to the universal swing plate. When started, the driving device swings left and right and front and back to drive the universal swing plate to keep swinging in the same direction. The universal ball bearing is used to support the universal swing plate and reduce the friction between the universal swing plate and the base; The urine sensor is installed at the bottom of the cradle body and is used to sense urine contact and generate a high-level signal; The integrated processing device is used to collect sound digital signal sequences and continuously monitor and obtain video digital signal sequences, providing four control modes: user touch control, remote control, voice control and automatic control. It is used to control the start, stop and gear adjustment of the drive device, as well as the playback and stop of music and stories, and at the same time upload video digital signal sequences and prompt instructions to the cloud platform.

Images