Control method, system, device and storage medium of pan / tilt camera
By combining voiceprint recognition and gesture control technology, using sequence-to-sequence model and attention mechanism for intelligent control of gimbal cameras, the problems of cumbersome operation and insufficient security in the existing technology are solved, and efficient, personalized and Safe gimbal camera control experience.
Patent Information
- Application Number
- CN202410644580.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-23
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-05-23
AI Technical Summary
The control methods of existing gimbal cameras are cumbersome, real-time and security are poor, and personalized customization is not possible, user operation experience is low, and voiceprint recognition and gesture control technology are accurate and stable in complex scenarios. insufficient.
Using voiceprint recognition and gesture control technology, by obtaining preset sound passwords and gesture control action data from the owner, using the sequence-to-sequence model and attention mechanism to generate gesture control features and signal mapping relationships, and obtaining environmental audio data in real time for identity Verify and control signal prediction to realize intelligent and personalized control of the gimbal camera.
Improve the security and reliability of the control certification of the gimbal camera, improve the intelligence and adaptability of gesture control, enhance the system's adaptability to multiple sound environments, and realize a real-time, coherent and adaptive control experience. And improves safety and energy efficiency through automatic shutdown function.
Smart Images

Figure CN118233737B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent control, and in particular to a control method, system, device and storage medium for a pan / tilt camera. Background Art
[0002] At present, with the continuous development of smart home and smart security market, PTZ cameras are widely used in smart monitoring systems. At present, the market generally uses mobile phone APP or PC client to control PTZ cameras, but there are problems such as cumbersome operation, poor real-time performance and security, inability to perform personalized customization, and low user operation experience.
[0003] Existing voiceprint recognition technology still faces certain challenges in terms of recognition accuracy and robustness. Although gesture control technology has made certain progress in motion recognition and tracking, it still has problems such as high latency, poor accuracy and stability for complex gestures and scenes with high real-time requirements. Researchers have been committed to exploring how to combine advanced artificial intelligence technology to further optimize gimbal cameras, making them more intelligent, personalized and highly adaptable. Summary of the invention
[0004] The invention provides a control method, system, equipment and storage medium of a pan / tilt camera.
[0005] A first aspect of the present invention provides a method for controlling a pan-tilt camera, the method comprising: obtaining first voiceprint data of a voice password preset by a camera owner;
[0006] Extracting features from the first voiceprint data to generate sample voiceprint features;
[0007] Get the gesture control action data preset by the device owner;
[0008] Extracting features from the gesture control action data to generate gesture control features;
[0009] Based on a preset sequence-to-sequence model and an attention mechanism, a first mapping relationship between gesture control features and gesture control signals is generated;
[0010] Acquire environmental audio data in real time, identify the voice password contained in the environmental audio data, and generate a second voiceprint feature;
[0011] Performing voiceprint verification on the second voiceprint feature according to the sample voiceprint feature to generate an identity authentication result;
[0012] According to the identity verification result, the pan / tilt camera activates or disables the gesture control function;
[0013] After the gesture control function is activated, the pan-tilt camera obtains current gesture data in real time;
[0014] According to the first mapping relationship, predictive modeling is performed on the current gesture data to generate an optimal control signal for the pan-tilt camera.
[0015] Optionally, in a first implementation of the first aspect of the present invention, performing feature extraction on the first voiceprint data to generate sample voiceprint features includes:
[0016] Performing time-frequency analysis on the first voiceprint data to generate a first Mel spectrum graph;
[0017] Perform convolution operation on the first Mel spectrum to extract the sample voiceprint features;
[0018] The formula for the convolution operation is:
[0019] ;
[0020] in, is the output result of the convolution operation, which is expressed at position The convolution result at ; Represents the weight parameter of the convolution kernel; Indicates the position of the first Mel spectrum The pixel value at ; b is the bias term; ∑ represents the summation operation.
[0021] Optionally, in a second implementation of the first aspect of the present invention, the step of extracting features from the gesture control action data to generate gesture control features includes:
[0022] Extracting time series features from the gesture control action data to generate gesture control features;
[0023] The cyclic calculation formula for extracting the time series features is:
[0024] = ( + + );
[0025] = + + + );
[0026] =tanh( + );
[0027] = * + * ;
[0028] = ( );
[0029] *tanh ;
[0030] in, , , , and Represent the forget gate, input gate, cell state, current cell state, output gate and hidden state respectively; represents the Sigmoid activation function; tanh represents the hyperbolic tangent activation function; Represents the input of the current time step; represents the hidden state of the previous time step; represents the cell state at the previous time step; W represents the weight parameter, and b represents the bias term.
[0031] Optionally, in a third implementation of the first aspect of the present invention, generating a first mapping relationship between a gesture control feature and a gesture control signal based on a preset sequence-to-sequence model and an attention mechanism includes:
[0032] The sequence-to-sequence model is provided with an encoder and a decoder;
[0033] Based on the attention mechanism, the attention weights of the decoder’s current state and the encoder’s time steps are calculated to adjust the decoder’s attention to different input positions.
[0034] Attention Weight The calculation formula is:
[0035] ;
[0036] in, represents the attention weight of the decoder at time step t to the jth time step of the encoder; Represents the attention score To index; is the sum of the attention scores of all encoder time steps; T is the length of the encoder sequence;
[0037] Among them, the attention score The calculation formula is: ; Indicates the current state of the decoder and the encoder state at the jth time step The inner product of
[0038] The gesture control feature and the gesture control signal are encoded to calculate a context vector, wherein the calculation formula of the context vector is:
[0039] ;
[0040] in, is the context vector of the decoder at time step t, which is the weighted average of the encoder’s states at each time step;
[0041] The context vector is used to express a first mapping relationship between the gesture control feature and the gesture control signal.
[0042] Optionally, in a fourth implementation of the first aspect of the present invention, obtaining environmental audio data in real time, identifying a voice password contained in the environmental audio data, and generating a second voiceprint feature; performing voiceprint verification on the second voiceprint feature according to the sample voiceprint feature, and generating an identity verification result, includes:
[0043] Performing a reconstruction error operation on the sample voiceprint feature and the environmental audio data to obtain a first reconstruction error;
[0044] Comparing the first reconstruction error with a first preset threshold, identifying a real-time voice password contained in the environmental audio data, and generating a second voiceprint feature according to the real-time voice password;
[0045] The voiceprint verification includes performing a secondary reconstruction error operation on the sample voiceprint feature and the second voiceprint feature to obtain a second reconstruction error;
[0046] comparing the second reconstruction error with a second preset threshold to generate an identity authentication result;
[0047] The reconstruction error operation is: based on the mean square error as the loss function, the reconstruction error is minimized. The calculation formula of the reconstruction error is:
[0048] Loss(x,g(f(x)))=||xg(f(x))||²;
[0049] Where x is the input data, including sample voiceprint features, environmental audio data and second voiceprint features; g(f(x)) is the reconstructed data; ||xg(f(x))||² represents the Euclidean distance between the input data x and the reconstructed data.
[0050] Optionally, in a fifth implementation of the first aspect of the present invention, predictive modeling is performed on the current gesture data according to the first mapping relationship to generate an optimal control signal for the pan-tilt camera, including:
[0051] Extracting time series features from the current gesture data to generate current gesture features;
[0052] Perform predictive modeling based on the first mapping relationship and current gesture features to obtain a predicted gesture control strategy;
[0053] According to the predicted gesture control strategy, by minimizing the cross entropy loss function as the optimization goal, the prediction model parameters are updated to generate the optimal control signal;
[0054] The calculation formula of the cross entropy loss function is:
[0055] ;
[0056] in, represents the true label value of the i-th label, It represents the predicted probability of the model for the i-th label, and K is the number of labels.
[0057] Optionally, in a sixth implementation of the first aspect of the present invention, the method for controlling a pan-tilt camera further includes:
[0058] According to the first mapping relationship and the current gesture feature, the duration and frequency of the gesture operation are analyzed to determine whether there is an abnormal operation or no operation for a long time. If so, the gesture control function is automatically turned off.
[0059] A second aspect of the present invention provides a control system for a pan-tilt camera, the control system for the pan-tilt camera comprising: an acquisition module, a processing module, a control module and a self-protection module; each module is electrically connected.
[0060] Optionally, in a first implementation of the second aspect of the present invention, the acquisition module is specifically used to: acquire first voiceprint data of a voice password preset by the machine owner; acquire gesture control action data preset by the machine owner; acquire ambient audio data in real time; and acquire current gesture data in real time after activating the gesture control function;
[0061] Optionally, in a second implementation of the second aspect of the present invention, the processing module is specifically used to: extract sample voiceprint features and gesture control features; generate a first mapping relationship between gesture control features and gesture control signals; identify the voice password contained in the environmental audio data to generate a second voiceprint feature; perform voiceprint verification on the second voiceprint feature to generate an identity verification result, and generate an optimal control signal for the pan-tilt camera;
[0062] Optionally, in a third implementation of the second aspect of the present invention, the control module is used to activate or deactivate the gesture control function of the pan-tilt camera and execute the above-mentioned pan-tilt camera control method according to the identity authentication result.
[0063] Optionally, in a fourth implementation of the second aspect of the present invention, the self-protection module is used to determine whether there is an abnormal operation or no operation for a long time, and if so, automatically shut down the gesture control function.
[0064] The third aspect of the present invention provides a pan-tilt camera device, comprising: a microphone, a camera, a memory and at least one processor, wherein the memory stores instructions; the microphone is used to obtain first voiceprint data and environmental audio data of a voice command preset by the owner;
[0065] The camera is used to obtain the gesture control action data preset by the owner and the current gesture data;
[0066] The at least one processor calls the instructions in the memory so that the pan-tilt camera device executes the above-mentioned pan-tilt camera control method.
[0067] A fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned method for controlling a pan-tilt camera.
[0068] Beneficial effects: In the technical solution provided by the present invention, the control method of the pan-tilt camera provided by the present invention combines voiceprint technology, gesture control technology and various artificial intelligence technologies, and has multiple beneficial effects. First, identity authentication is performed based on voiceprint biometrics to ensure the security and reliability of control authentication. Secondly, based on the combination of sequence-to-sequence model and attention mechanism, the learning and matching of gesture control features and gesture control signals are effectively realized, and the intelligence and adaptability of the gesture control system are improved. In addition, by intelligently identifying the sound commands in the environmental audio data, not only the environmental adaptability is improved, but also the adaptability of the system to various sound environments is enhanced. Furthermore, by combining the current gesture data obtained on site for predictive modeling, the next control signal is predicted, and the optimal control signal is accurately generated, the real-time, coherent and adaptable nature of gesture control is achieved, providing users with a more convenient, customizable and personalized intelligent pan-tilt camera control experience. Finally, by automatically identifying abnormal operations or automatically turning off the gesture control function for a long time without operation, the safety and energy efficiency of the system are improved. This comprehensive control method not only expands the functional boundaries of the intelligent pan-tilt camera, but also improves the user experience and system performance, and has extremely high practical and application promotion value. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] Figure 1 A schematic diagram of an embodiment of a method for controlling a pan-tilt camera according to an embodiment of the present invention;
[0070] Figure 2 A schematic diagram of another embodiment of a method for controlling a pan-tilt camera according to an embodiment of the present invention;
[0071] Figure 3 A schematic diagram of an embodiment of a control system for a pan-tilt camera in an embodiment of the present invention; DETAILED DESCRIPTION
[0072] Embodiments of the present invention provide a control method, system, device and storage medium for a pan-tilt camera. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0073] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the control method of the pan-tilt camera in the embodiment of the present invention includes:
[0074] Step 101, obtaining the first voiceprint data of the voice password preset by the owner;
[0075] It is understandable that the execution subject of the present invention may be a control device of a PTZ camera, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.
[0076] Specifically, the owner can upload a custom voice command in advance through the mobile terminal or the host control terminal. The content of the command can be freely set, such as "Please start the gesture control mode" or "Turn on the control" or even sing a line of lyrics, etc. The uploaded audio file can be the owner's own recorded audio file, or it can be recorded and collected on-site through the pan-tilt camera system, or it can be recorded and collected remotely online, etc., and the specifics are not limited here.
[0077] Step 102: extract features from the first voiceprint data to generate sample voiceprint features;
[0078] Specifically, performing time-frequency analysis on the first voiceprint data to generate a first Mel spectrum graph;
[0079] Perform convolution operation on the first Mel spectrum to extract the sample voiceprint features;
[0080] The formula for the convolution operation is:
[0081] ;
[0082] in, is the output result of the convolution operation, which is expressed at position The convolution result at ; Represents the weight parameter of the convolution kernel; Indicates the position of the first Mel spectrum The pixel value at ; b is the bias term; ∑ represents the summation operation.
[0083] More specifically, Mel spectrograms are time-frequency images converted from the waveform data of voiceprint sample data. Through normalization, time-frequency images can visualize sound signals in time and frequency dimensions, making it easier to learn and extract features of sound data through operations such as convolution, pooling, and the introduction of nonlinearity.
[0084] Step 103: Acquire gesture control action data preset by the device owner;
[0085] Specifically, the owner can upload custom control gestures for each control content in advance through the mobile terminal or host control terminal. The control content includes the movement, rotation, pitch, zoom, scaling, shutter, function mode switching, etc. of the camera in all directions. The control gestures can be freely set, such as a set of continuous gestures, fixed gestures that last for a certain period of time, and combinations of single and double-hand gestures. The uploaded gestures can be the gesture video files recorded by the owner himself, or they can be recorded and collected on-site through the pan-tilt camera system, or they can be recorded and collected remotely online, etc., and the specifics are not limited here.
[0086] Step 104: extracting features from the gesture control action data to generate gesture control features;
[0087] Specifically, extracting time series features from the gesture control action data to generate gesture control features;
[0088] The cyclic calculation formula for extracting the time series features is:
[0089] = ( + + );
[0090] = + + + );
[0091] =tanh( + );
[0092] = * + * ;
[0093] = ( );
[0094] *tanh ;
[0095] in, , , , and Represent the forget gate, input gate, cell state, current cell state, output gate and hidden state respectively; represents the Sigmoid activation function; tanh represents the hyperbolic tangent activation function; Represents the input of the current time step; represents the hidden state of the previous time step; represents the cell state at the previous time step; W represents the weight parameter, and b represents the bias term.
[0096] More specifically, through the memory function of recurrent neurons, the timing information of gesture control action data is learned, and the timing dependency between gesture control action features is captured.
[0097] The time series sequence of gesture control action data is used as the input sequence, and the time series information in the sequence is learned through the time step cycle; the forget gate, input gate and output gate in the cycle control the transmission and forgetting of information, the cell state stores and transmits information, and the hidden state outputs the final result; the input of each time step is actually the time-frequency characteristics of the gesture control action at a moment, and its hidden state is gradually updated according to these inputs to learn the long-term dependencies in the sequence.
[0098] Step 105: Generate a first mapping relationship between gesture control features and gesture control signals based on a preset sequence-to-sequence model and an attention mechanism;
[0099] Specifically, the sequence-to-sequence model is provided with an encoder and a decoder;
[0100] Based on the attention mechanism, the attention weights of the decoder’s current state and the encoder’s time steps are calculated to adjust the decoder’s attention to different input positions.
[0101] Attention Weight The calculation formula is:
[0102] ;
[0103] in, represents the attention weight of the decoder at time step t to the jth time step of the encoder; Represents the attention score To index; is the sum of the attention scores of all encoder time steps; T is the length of the encoder sequence;
[0104] Among them, the attention score The calculation formula is: ; Indicates the current state of the decoder and the encoder state at the jth time step The inner product of is the state of the decoder at time step t, representing the current state of the model when generating the sequence; is the state of the encoder at time step j, representing the encoder’s representation of different parts of the input sequence. It can measure the similarity between the current state of the decoder and the state of the encoder at different time steps.
[0105] The gesture control feature and the gesture control signal are encoded to calculate a context vector, wherein the calculation formula of the context vector is:
[0106] ;
[0107] in, is the context vector of the decoder at time step t, which is the weighted average of the encoder’s states at each time step;
[0108] The context vector is a weighted average of the encoder states at each time step, weighted by the attention weights.
[0109] The context vector is used to express a first mapping relationship between the gesture control feature and the gesture control signal.
[0110] More specifically, the sequence-to-sequence (Seq2Seq) model is composed of an encoder and a decoder, and is used to map one sequence to another sequence, that is, the first mapping relationship used to map the gesture control feature to the gesture control signal in this embodiment.
[0111] The encoder encodes the input sequence (gesture control feature sequence) into a fixed-length context vector, and the decoder generates an output sequence (gesture control signal sequence) based on the context vector.
[0112] The encoder calculates the hidden state of each time step by calculating the current element of the input sequence and the hidden state of the previous time step. When the entire input sequence is processed, the hidden state of the final time step can be regarded as a context vector, which stores the overall information of the input sequence. The context vector will be passed to the decoder to generate the output sequence, realizing sequence-to-sequence mapping and conversion. Through the gating mechanism and state update, the encoder can effectively encode the input sequence and generate a context vector for subsequent decoder use.
[0113] The attention mechanism enables the model to focus on different positions of the input sequence when generating the output sequence, and improves the attention to important information. The attention weights of the current state of the decoder and each time step of the encoder are calculated to adjust the attention of the decoder to different input positions. By learning the attention weights, the model can dynamically select the part to focus on based on the input sequence and the current decoding state. In general, the Seq2Seq model based on the attention mechanism calculates the attention score and weight based on the current state of the decoder and the state of the encoder at each decoder time step, and finally obtains the context vector of the current time step through weighted averaging. This context vector can dynamically capture important information at different positions in the input sequence, helping the decoder to generate a more accurate and precise output sequence. Through the attention mechanism, the Seq2Seq model can better cope with long sequence problems and improve model performance.
[0114] By focusing on key information through the attention mechanism, the real-time monitoring and analysis of the owner's gestures can be achieved, the precise control of the gimbal camera movement can be improved, and the owner's gestures can be accurately mapped to the corresponding camera control signals, achieving more precise gesture control and flexible control of the gimbal camera. This method can better capture the relationship and attention in the gesture sequence, and improve the reliability and accuracy of the system.
[0115] Step 106: Acquire environmental audio data in real time, identify the voice password contained in the environmental audio data, and generate a second voiceprint feature;
[0116] Specifically, a reconstruction error operation is performed on the sample voiceprint feature and the environmental audio data to obtain a first reconstruction error;
[0117] Comparing the first reconstruction error with a first preset threshold, identifying a real-time voice password contained in the environmental audio data, and generating a second voiceprint feature according to the real-time voice password;
[0118] The first reconstruction error operation is: based on the mean square error as the loss function, the first reconstruction error is minimized. The calculation formula of the first reconstruction error is:
[0119] Loss(x,g(f(x)))=||xg(f(x))||²;
[0120] Among them, x is the input data, including sample voiceprint features and environmental audio data; g(f(x)) is the reconstructed data; ||xg(f(x))||² represents the Euclidean distance between the input data x and the reconstructed data.
[0121] More specifically, the reconstruction error is used to determine whether the input data is abnormal, that is, to identify whether it contains the preset voice password. The reconstruction error of abnormal passwords is usually larger than that of normal passwords, which can be used as a sign of recognition. The core idea is to use normal data for training and learn the feature representation of these normal data, namely the sample voiceprint features. The feature representation of normal data is rich in the general pattern and structure of the data, so that it can reconstruct the normal data (voice passwords in the ambient audio data) more accurately. When abnormal data (non-voice passwords in the ambient audio data) is input, since the abnormal data does not conform to the feature representation of normal data, it is impossible to reconstruct these abnormal data well, resulting in a large reconstruction error. This reconstruction error can be used to capture the features that distinguish normal data (voice passwords in the ambient audio data) from abnormal data (non-voice passwords in the ambient audio data), thereby realizing the recognition of the voice passwords contained in the ambient audio data.
[0122] By analyzing the reconstruction error distribution of the sample voiceprint features, we can set a threshold (the first preset threshold). Reconstruction errors smaller than this threshold are identified as normal data (voice passwords in environmental audio data).
[0123] Therefore, when the user speaks a preset voice command within the sound pickup range of the PTZ camera and its system, the PTZ camera and its system will automatically collect and identify the correctness of the voice command.
[0124] Step 107: Perform voiceprint verification on the second voiceprint feature according to the sample voiceprint feature to generate an identity authentication result;
[0125] Specifically, the voiceprint verification includes performing a secondary reconstruction error operation on the sample voiceprint feature and the second voiceprint feature to obtain a second reconstruction error;
[0126] comparing the second reconstruction error with a second preset threshold to generate an identity authentication result;
[0127] The second reconstruction error operation is: based on the mean square error as the loss function, the second reconstruction error is minimized, and the calculation formula of the second reconstruction error is:
[0128] Loss(x,g(f(x)))=||xg(f(x))||²;
[0129] Where x is the input data, including the sample voiceprint features and the second voiceprint features; g(f(x)) is the reconstructed data; ||xg(f(x))||² represents the Euclidean distance between the input data x and the reconstructed data.
[0130] More specifically, reconstruction error is used to determine whether the input data is abnormal, that is, whether the recognition meets the preset voiceprint features. The reconstruction error of abnormal voiceprints is usually larger than that of normal voiceprints, which can be used as a sign of recognition. The core idea is to use normal data (sample voiceprint features) for training and learn the feature representation of these normal data (sample voiceprint features). The feature representation of normal data (sample voiceprint features) is rich in the general pattern and structure of the data, so that it can reconstruct normal data (voiceprints of legal identities) more accurately. When abnormal data (voiceprints of illegal identities) are input, since the abnormal data (voiceprints of illegal identities) do not conform to the feature representation of normal data, these abnormal data cannot be well reconstructed, resulting in a large reconstruction error. This reconstruction error can be used to capture the features that distinguish normal data (voiceprints of legal identities) from abnormal data (voiceprints of illegal identities), thereby realizing voiceprint verification and generating identity authentication results.
[0131] By analyzing the reconstruction error distribution of the sample voiceprint features, we can set a threshold (the second preset threshold). Reconstruction errors smaller than this threshold are identified as normal data (voiceprints of legitimate identities), and the identity verification result is output. Reconstruction errors larger than this threshold are identified as abnormal data (voiceprints of illegal identities), and the identity verification result is output.
[0132] Therefore, only when the owner is within the sound pickup range of the gimbal camera and its system and speaks the preset voice command, the system will trigger the gesture control function. Through dual recognition, the security and privacy of the gimbal camera control can be guaranteed.
[0133] Step 108: According to the identity verification result, the PTZ camera activates or disables the gesture control function;
[0134] Specifically,
[0135] The system recognizes the specific password spoken by a specific owner. Only when the owner speaks the correct password will the system activate the gesture control function to ensure the safety of operation.
[0136] Step 109: After the PTZ camera activates the gesture control function, current gesture data is acquired in real time;
[0137] Specifically, once the system recognizes the owner, gestures can control the movement of the gimbal camera, achieving convenient and fast operation.
[0138] Step 110: According to the first mapping relationship, predictive modeling is performed on the current gesture data to generate an optimal control signal for the pan-tilt camera.
[0139] Specifically, extracting time series features from the current gesture data to generate current gesture features;
[0140] Perform predictive modeling based on the first mapping relationship and current gesture features to obtain a predicted gesture control strategy;
[0141] According to the predicted gesture control strategy, by minimizing the cross entropy loss function as the optimization goal, the prediction model parameters are updated to generate the optimal control signal;
[0142] The calculation formula of the cross entropy loss function is:
[0143] ;
[0144] in, represents the true label value of the i-th label, It represents the predicted probability of the model for the i-th label, and K is the number of labels.
[0145] Specifically, through this loss function, we can measure the difference between the probability distribution predicted by the model and the true label. The closer the model's prediction is to the true label, the smaller the cross entropy loss, and vice versa.
[0146] More specifically, the encoder encodes the input sequence to obtain the context vector, and the decoder predicts the current output at each time step based on the context vector and the previous output. The encoder and decoder transmit and calculate information through LSTM or other RNN structures, and finally generate the probability distribution of the output sequence. Such a structure can realize predictive modeling of sequence data.
[0147] The cross entropy loss function measures the prediction performance of the model by the difference between the probability distribution of the actual label and the model prediction. Minimizing the value of the loss function, that is, minimizing the cross entropy loss between the actual label and the model prediction, enables the model to give more accurate prediction results.
[0148] By calculating the similarity between the current real-time gesture action and the preset gesture action, the most matching gesture control instruction is determined. According to the user's current gesture data and historical data, the user's possible next gesture action is predicted. According to the prediction results of the model, the system can prepare corresponding operations in advance to reduce operation delays and optimize the system's response speed, improve the accuracy, sensitivity and fluency of gesture control, and thus improve the user experience.
[0149] In the embodiment of the present invention, the beneficial effect The control method of the pan-tilt camera provided by the present invention combines voiceprint technology, gesture control technology and various artificial intelligence technologies, and has multiple beneficial effects. First, identity authentication is performed based on voiceprint biometrics to ensure the security and reliability of control authentication. Secondly, based on the combination of sequence-to-sequence model and attention mechanism, the learning and matching of gesture control features and gesture control signals are effectively realized, and the intelligence and adaptability of the gesture control system are improved. In addition, by intelligently identifying the sound commands in the environmental audio data, not only the environmental adaptability is improved, but also the adaptability of the system to various sound environments is enhanced. Furthermore, by combining the current gesture data obtained on site for predictive modeling, the next control signal is predicted, and the optimal control signal is accurately generated, the real-time, coherent and adaptable nature of gesture control is achieved, and a more convenient, customizable and personalized intelligent pan-tilt camera control experience is provided to users. Finally, the gesture control function is automatically turned off by automatically identifying abnormal operations or long-term no operation, thereby improving the security and energy efficiency of the system. This comprehensive control method not only expands the functional boundaries of the intelligent pan-tilt camera, but also improves the user experience and system performance, and has extremely high practical and application promotion value.
[0150] See also Figure 2 Another embodiment of the control method of the pan-tilt camera in the embodiment of the present invention includes:
[0151] Step 201: Analyze the duration and frequency of gesture operation according to the first mapping relationship and the current gesture feature to determine whether there is abnormal operation or no operation for a long time. If so, automatically turn off the gesture control function.
[0152] Specifically, the anomaly detection technology based on time series data is used to monitor the mode and frequency of user gesture operations to determine whether there is abnormal behavior or long-term inactivity. Combining user operation behavior and time factors, it is intelligently determined whether the user is in a real operation state. When the user gesture control operation ends or there is no operation for a long time, the control state is automatically triggered to reduce the risk of misoperation and ensure device safety.
[0153] In the embodiments of the present invention, the beneficial effects are reducing the risk of misoperation and ensuring equipment safety, and improving the safety and energy efficiency of the control system.
[0154] The control method of the pan-tilt camera in the embodiment of the present invention is described above. The control system of the pan-tilt camera in the embodiment of the present invention is described below. Figure 3 In one embodiment of the present invention, a control system for a pan-tilt camera includes: an acquisition module, a processing module, a control module and a self-protection module; each module is electrically connected.
[0155] Specifically, the acquisition module is specifically used to: acquire the first voiceprint data of the voice password preset by the machine owner; acquire the gesture control action data preset by the machine owner; acquire the ambient audio data in real time; and acquire the current gesture data in real time after activating the gesture control function;
[0156] Specifically, the processing module is specifically used to: extract sample voiceprint features and gesture control features; generate a first mapping relationship between gesture control features and gesture control signals; identify the voice password contained in the environmental audio data to generate a second voiceprint feature; perform voiceprint verification on the second voiceprint feature to generate an identity verification result, and generate an optimal control signal for the pan-tilt camera;
[0157] Specifically, the control module is used to activate or deactivate the gesture control function of the pan-tilt camera and execute the above-mentioned pan-tilt camera control method according to the identity authentication result.
[0158] Specifically, the self-protection module is used to determine whether there is an abnormal operation or no operation for a long time, and if so, automatically shut down the gesture control function.
[0159] In the embodiment of the present invention, the beneficial effects are as follows: first, identity authentication is performed based on voiceprint biometrics, which ensures the security and reliability of control authentication. Secondly, based on the combination of sequence-to-sequence model and attention mechanism, the learning and matching of gesture control features and gesture control signals are effectively realized, and the intelligence and adaptability of gesture control system are improved. In addition, by intelligently identifying the voice commands in the environmental audio data, not only the environmental adaptability is improved, but also the adaptability of the system to various sound environments is enhanced. Furthermore, by combining the current gesture data obtained on site for predictive modeling, the next control signal is predicted, and the optimal control signal is accurately generated, the real-time, coherent and adaptable nature of gesture control is achieved, providing users with a more convenient, customizable and personalized intelligent pan-tilt camera control experience. Finally, by automatically identifying abnormal operations or automatically turning off the gesture control function after a long period of no operation, the safety and energy efficiency of the system are improved. This comprehensive control system not only expands the functional boundaries of intelligent pan-tilt cameras, but also improves user experience and system performance, and has extremely high practical and application promotion value.
[0160] The present invention also provides a pan-tilt camera device, the pan-tilt camera device comprising: a microphone, a camera, a memory and at least one processor, wherein the memory stores instructions;
[0161] The microphone is used to obtain the first voiceprint data and the ambient audio data of the voice password preset by the machine owner;
[0162] The camera is used to obtain the gesture control action data preset by the owner and the current gesture data;
[0163] The at least one processor calls the instructions in the memory so that the pan-tilt camera device executes the steps of the pan-tilt camera control method in the above embodiments.
[0164] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the above-mentioned pan-tilt camera control method.
[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0166] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0167] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for controlling a pan-tilt camera, characterized in that: The control method of the pan / tilt camera comprises: Acquire the first voiceprint data of the voice password preset by the device owner; Extracting features from the first voiceprint data to generate sample voiceprint features; Get the gesture control action data preset by the device owner; Extracting features from the gesture control action data to generate gesture control features; Based on a preset sequence-to-sequence model and an attention mechanism, a first mapping relationship between gesture control features and gesture control signals is generated; Acquire environmental audio data in real time, identify the voice password contained in the environmental audio data, and generate a second voiceprint feature; Performing voiceprint verification on the second voiceprint feature according to the sample voiceprint feature to generate an identity authentication result; According to the identity verification result, the pan / tilt camera activates or disables the gesture control function; After the gesture control function is activated, the pan-tilt camera obtains current gesture data in real time; According to the first mapping relationship, predictive modeling is performed on the current gesture data to generate an optimal control signal for the pan-tilt camera.
2. The control method of the PTZ camera according to claim 1, characterized in that: The real-time acquisition of environmental audio data, identification of the voice password contained in the environmental audio data, and generation of a second voiceprint feature; Performing voiceprint verification on the second voiceprint feature according to the sample voiceprint feature to generate an identity authentication result includes: Performing a reconstruction error operation on the sample voiceprint feature and the environmental audio data to obtain a first reconstruction error; Comparing the first reconstruction error with a first preset threshold, identifying a real-time voice password contained in the environmental audio data, and generating a second voiceprint feature according to the real-time voice password; The voiceprint verification includes performing a secondary reconstruction error operation on the sample voiceprint feature and the second voiceprint feature to obtain a second reconstruction error; comparing the second reconstruction error with a second preset threshold to generate an identity authentication result; The reconstruction error operation is: based on the mean square error as the loss function, the reconstruction error is minimized. The calculation formula of the reconstruction error is: Loss(x,g(f(x)))=||xg(f(x))||²; Where x is the input data, including sample voiceprint features, environmental audio data and second voiceprint features; g(f(x)) is the reconstructed data; ||xg(f(x))||² represents the Euclidean distance between the input data x and the reconstructed data.
3. The control method of the PTZ camera according to claim 2, characterized in that: The step of performing predictive modeling on the current gesture data according to the first mapping relationship to generate an optimal control signal for the pan / tilt camera includes: Extracting time series features from the current gesture data to generate current gesture features; Perform predictive modeling based on the first mapping relationship and current gesture features to obtain a predicted gesture control strategy; According to the predicted gesture control strategy, by minimizing the cross entropy loss function as the optimization goal, the prediction model parameters are updated to generate the optimal control signal; The calculation formula of the cross entropy loss function is: ; in, represents the true label value of the i-th label, It represents the predicted probability of the model for the i-th label, and K is the number of labels.
4. The control method of the PTZ camera according to claim 3, characterized in that: The control method of the PTZ camera also includes: According to the first mapping relationship and the current gesture feature, the duration and frequency of the operation are analyzed to determine whether there is an abnormal operation or no operation for a long time. If so, the gesture control function is automatically turned off.
5. A control system for a pan-tilt camera, characterized in that: The control system of the pan / tilt camera includes: An acquisition module is used to acquire the first voiceprint data of the voice password preset by the machine owner; acquire the gesture control action data preset by the machine owner; acquire the ambient audio data in real time; and acquire the current gesture data in real time after activating the gesture control function; A processing module is used to extract sample voiceprint features and gesture control features; generate a first mapping relationship between gesture control features and gesture control signals; identify the voice password contained in the environmental audio data to generate a second voiceprint feature; perform voiceprint verification on the second voiceprint feature to generate an identity verification result, and generate an optimal control signal for the pan-tilt camera; A control module, used to activate or deactivate the gesture control function of the pan-tilt camera and execute the control method of the pan-tilt camera according to any one of claims 1 to 4 according to the identity authentication result; Self-protection module: used to determine whether there is abnormal operation or no operation for a long time. If so, the gesture control function is automatically turned off.
6. A pan-tilt camera device, characterized in that: The pan-tilt camera device comprises: a microphone, a camera, a memory and at least one processor, wherein the memory stores instructions; The microphone is used to obtain the first voiceprint data and the ambient audio data of the voice password preset by the machine owner; The camera is used to obtain the gesture control action data preset by the owner and the current gesture data; The at least one processor calls the instruction in the memory so that the pan-tilt camera device executes the pan-tilt camera control method as described in any one of claims 1-4.
7. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by the processor, the control method of the pan-tilt camera according to any one of claims 1 to 4 is implemented.
Citation Information
Patent Citations
Intelligent awakening method and system
CN105501121A
Application encryption method and mobile terminal
CN106960161A
User authorization addition method and system based on car-mounted terminal, medium and car-mounted terminal
CN110816470A
Model training method, gesture control method and device, medium and electronic equipment
CN111124108A
Parameter setting method and device of industrial control terminal, industrial control terminal and storage medium
CN112541175A
