Equipment control method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an equipment control method and device, electronic equipment and a storage medium, and the method comprises the steps: detecting visual information and sound information when the equipment is in a child lock mode and receives an operation instruction; performing image recognition according to the visual information, and determining a first user feature; performing voiceprint recognition according to the sound information, and determining a second user feature; determining a target user feature in combination with the first user feature and the second user feature; and when the target user feature is a non-child user, executing the operation instruction. According to the embodiment of the invention, the device can be effectively prevented from being operated by mistake in the child lock mode. And the safety of the user is effectively protected under the condition that the user experience is not lost.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment safety control, and in particular to an equipment control method, an equipment control device, an electronic device and a storage medium. Background Art

[0002] To prevent accidents caused by children misusing their devices, parents often set a child lock mode on their devices. However, existing child locks typically require two buttons to be pressed simultaneously, making it easy for children to accidentally unlock them. This design makes it difficult for child locks to fully protect children in practice. Furthermore, some devices even require a single button to disable the child lock, further challenging child safety. This significantly reduces the effectiveness and reliability of the child lock function, making it difficult to protect children. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a device control method, a device control apparatus, an electronic device, and a storage medium that overcome the above problems or at least partially solve the above problems.

[0004] In order to solve the above problems, in a first aspect of the present invention, an embodiment of the present invention discloses a device control method, comprising:

[0005] When the device is in child lock mode and receives an operation instruction, detecting visual information and sound information;

[0006] performing image recognition based on the visual information to determine a first user feature;

[0007] Performing voiceprint recognition based on the sound information to determine a second user feature;

[0008] Determining target user characteristics by combining the first user characteristics and the second user characteristics;

[0009] When the target user characteristic is a non-child user, the operation instruction is executed.

[0010] Optionally, the step of determining target user characteristics by combining the first user characteristics and the second user characteristics includes:

[0011] determining a first weight corresponding to the first user feature;

[0012] determining a second weight corresponding to the first user feature;

[0013] The first user feature, the second user feature, the first weight, and the second weight are used to determine a target user feature.

[0014] Optionally, the step of determining target user features by using the first user features, the second user features, the first weight, and the second weight includes:

[0015] combining the first user feature with the first weight to determine a first intermediate value;

[0016] combining the second user feature with the second weight to determine a second intermediate value;

[0017] The first intermediate value and the second intermediate value are added together to obtain target user features.

[0018] Optionally, the step of performing image recognition based on the visual information to determine the first user feature includes:

[0019] extracting image features from the visual information;

[0020] The image features are classified to determine a first user feature.

[0021] Optionally, the step of classifying the image features and determining the first user feature includes:

[0022] Inputting the image features into a preset image classification model, wherein the preset image classification model is configured to output a first probability value based on the image features;

[0023] The first probability value is determined to be a first user feature.

[0024] Optionally, the step of performing voiceprint recognition based on the sound information to determine the second user feature includes:

[0025] extracting voiceprint features from the sound information;

[0026] The voiceprint features are classified to determine a second user feature.

[0027] Optionally, the step of classifying the voiceprint features to determine the second user features includes:

[0028] Inputting the voiceprint feature into a preset sound classification model, wherein the preset sound classification model is configured to output a second probability value based on the voiceprint feature;

[0029] The second probability value is determined as a second user feature.

[0030] Optionally, the method comprises:

[0031] When the target user characteristic is not greater than a preset threshold, determining that the target user characteristic is a child user;

[0032] When the target user characteristic is greater than a preset threshold, it is determined that the target user characteristic is a non-child user.

[0033] Optionally, the method comprises:

[0034] When the target user is characterized as a child user, an alarm signal is issued.

[0035] In a second aspect of the present invention, an embodiment of the present invention discloses a device control apparatus, comprising:

[0036] a detection module, configured to detect visual and sound information when receiving an operation instruction while the device is in child lock mode;

[0037] a first recognition module, configured to perform image recognition based on the visual information and determine a first user feature;

[0038] a second recognition module, configured to perform voiceprint recognition based on the voice information to determine a second user feature;

[0039] a combining module, configured to combine the first user characteristics and the second user characteristics to determine target user characteristics;

[0040] The first execution module is configured to execute the operation instruction when the target user characteristic is a non-child user.

[0041] In the third aspect of the present invention, an embodiment of the present invention discloses an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the motor fault detection method described above when executed by the processor.

[0042] In a fourth aspect of the present invention, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the motor fault detection method described above are implemented.

[0043] The embodiments of the present invention include the following advantages:

[0044] This embodiment of the present invention detects visual and audio information when an operation instruction is received while the device is in child lock mode; performs image recognition based on the visual information to determine a first user characteristic; performs voiceprint recognition based on the audio information to determine a second user characteristic; combines the first and second user characteristics to determine a target user characteristic; and executes the operation instruction if the target user characteristic indicates a non-child user. By determining the identity of the user initiating the operation instruction based on both visual and audio information when the child lock is enabled, the relevant operation instruction can be directly executed if the user is not a child, effectively preventing accidental operation of the device in child lock mode. This effectively protects user safety without compromising user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flowchart of steps of an embodiment of a device control method of the present invention;

[0046] Figure 2 is a flowchart of steps of another device control method embodiment of the present invention;

[0047] Figure 3 is a flowchart of an example of a device control method of the present invention;

[0048] Figure 4 is a structural block diagram of an embodiment of a device control device of the present invention;

[0049] Figure 5 This is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0050] Figure 6 This is a structural block diagram of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Reference Figure 1 , shows a flowchart of a device control method embodiment of the present invention, the device control method may specifically include the following steps:

[0053] Step 101: When the device is in child lock mode and receives an operation instruction, detect visual information and sound information;

[0054] The devices used in the embodiments of the present invention can be various electronic or mechanical devices with a child lock function. For example, washing machines are equipped with a child lock function, which is typically activated or deactivated by simultaneously pressing two buttons (such as "Water Volume" and "Timer" or "Wash" and "Rinse"). When the child lock function is activated, other operating buttons are temporarily disabled to prevent children from operating the device accidentally. Televisions also provide a child lock function to restrict children from viewing inappropriate content or preventing accidental operation. The child lock option is typically found in the TV settings menu. Refrigerators also have a child lock function, which is activated or deactivated by long pressing or simultaneously pressing two buttons. When activated, the child lock function prevents children from arbitrarily adjusting the refrigerator temperature or opening the refrigerator or freezer door. The child lock function of microwave ovens is typically activated or deactivated by pressing the "Pause / Cancel / Power Save" button. Once activated, the control panel enters a locked state to prevent children from accidentally operating the device. The child lock function of gas stoves is typically designed to require a firm press and rotation to start the ignition, preventing children from accidentally operating the device. Air conditioner remote controls also have a child lock function, which is activated or deactivated by pressing a specific button combination. The car door is also equipped with a child lock function, usually located on the rear door lock of the car. Open the rear door, there is a small pull rod (safety mechanism) under the door lock, push it to the end with the child icon, and then close the door. At this time, the door cannot be opened from inside the car and can only be opened from outside.

[0055] After the child lock function is enabled on a device, it enters child lock mode. While in child lock mode, the user can initiate corresponding commands to the device through relevant operations. When the device receives the command, it can detect visual and audio information; visual information refers to the visual content captured by the device for user authentication. Audio information refers to the audio content captured by the device for user authentication.

[0056] Step 102: performing image recognition based on the visual information to determine a first user feature;

[0057] After the visual information is acquired, image recognition can be performed on the visual information to identify the content related to the user identity in the visual information and determine a first user feature. The first user feature represents the user identity feature in the visual information.

[0058] Step 103: Perform voiceprint recognition based on the voice information to determine a second user feature;

[0059] After acquiring the voice information, voiceprint recognition can be performed on the voice information to identify the content related to the user's identity in the voice information and determine the second user feature. The second user feature represents the user's identity feature in the voice information.

[0060] Step 104: Determine target user characteristics by combining the first user characteristics and the second user characteristics;

[0061] The first user feature and the second user feature are fused to determine a target user feature. The target user feature represents a feature of the user identity that combines the sound information and the visual information.

[0062] Step 105: When the target user characteristic is a non-child user, execute the operation instruction.

[0063] When it is determined that the target user is not a child user, the operation instruction can be directly executed without unlocking the child lock.

[0064] This embodiment of the present invention detects visual and audio information when an operation instruction is received while the device is in child lock mode; performs image recognition based on the visual information to determine a first user characteristic; performs voiceprint recognition based on the audio information to determine a second user characteristic; combines the first and second user characteristics to determine a target user characteristic; and executes the operation instruction if the target user characteristic indicates a non-child user. By determining the identity of the user initiating the operation instruction based on both visual and audio information when the child lock is enabled, the relevant operation instruction can be directly executed if the user is not a child, effectively preventing accidental operation of the device in child lock mode. This effectively protects user safety without compromising user experience.

[0065] Reference Figure 2 , shows a flowchart of another device control method embodiment of the present invention, the device control method may specifically include the following steps:

[0066] Step 201: When the device is in child lock mode and receives an operation instruction, detect visual information and sound information;

[0067] When users need to operate and control a device, they can initiate operation instructions to the device through the Internet of Things platform or smart home system, through the device's application or software, or through physical buttons, remote controls, operation panels, etc.

[0068] The method and process of issuing operation instructions to devices through the IoT platform or smart home system are as follows:

[0069] Device access platform:

[0070] The device needs to be connected to the IoT platform or smart home system first, which usually involves device registration, authentication, and connection.

[0071] User login platform:

[0072] Users log in to the IoT platform or smart home system through mobile applications, web interfaces, or other methods.

[0073] Initiate control instructions:

[0074] The user selects the target device on the platform or system interface and enters or selects the corresponding control instructions.

[0075] After receiving the instruction, the platform or system performs necessary processing, such as verifying user permissions and parsing instructions.

[0076] Command transmission:

[0077] The platform or system transmits control instructions to the device through appropriate communication protocols, such as HTTP (HyperText Transfer Protocol), MQTT (Message Queuing Telemetry Transport), etc.

[0078] The method and process of initiating operation instructions through the device application or software are as follows:

[0079] Download and install the app:

[0080] Users need to download and install the corresponding application or software on their mobile devices.

[0081] Device connection:

[0082] Users connect to devices through applications or software, which usually involves steps such as device pairing and authentication.

[0083] Initiate control instructions:

[0084] The user selects the target device on the application or software interface and enters or selects the corresponding control instruction.

[0085] Command transmission:

[0086] The application or software transmits the control instructions to the target device through communication methods such as Bluetooth and Wi-Fi (mobile hotspot network).

[0087] The method and process of initiating operation commands through physical buttons, remote control, operation panel, etc. are as follows:

[0088] Device configuration:

[0089] It needs to be controlled through physical buttons, remote control, and operation panel.

[0090] These devices are usually equipped with corresponding physical buttons, remote controls, and operation panel functions.

[0091] Initiate control instructions:

[0092] The user directly presses a physical button, remote control, or corresponding button on the operation panel. The button can be a physical button or a virtual button.

[0093] When the device is in child lock mode and receives an operation instruction, the visual information can be detected by the visual recognition system on the device, and the sound information can be detected by the voice recognition system.

[0094] Specifically, the camera in the visual recognition system is usually in low-power mode to reduce the energy consumption of the device. In this mode, the camera takes a low-quality image every 5 seconds and saves photos of no more than one minute as a temporary record. When a user operation is detected, the camera switches from a low-power state to an active mode, and takes a high-quality image every 0.5 seconds after the operation is triggered, for a total of two photos. The visual recognition system combines the captured images and uses the twenty-two photos taken during the two previous operations as visual information. By combining multiple photos as visual information, it helps to improve the accuracy of recognition and ensure that the judgment of user identity is more reliable.

[0095] The microphone in the voice recognition system is also in low-power mode, normally collecting user voice information at a low frequency. Under normal circumstances, the microphone collects one second of voice data every ten seconds and stores up to one minute of audio recordings as backup information. When a user action is detected, the microphone immediately enters active mode and continuously collects two seconds of voice data. The voice recognition system combines the collected audio data—that is, the seven segments of audio information collected during the previous and subsequent actions—as the voice information.

[0096] Step 202: performing image recognition based on the visual information to determine a first user feature;

[0097] Image recognition may be performed on the visual information to determine the identity of the user represented in the visual information and to determine the first user feature.

[0098] In an optional embodiment of the present invention, the step of performing image recognition based on the visual information to determine the first user feature includes: extracting image features from the visual information; and classifying the image features to determine the first user feature.

[0099] A variety of image features may be extracted from the visual information, the image features may be classified, and the first user feature may be determined.

[0100] Specifically, the step of classifying the image features and determining the first user feature includes: inputting the image features into a preset image classification model, the preset image classification model is used to output a first probability value based on the image features; and determining the first probability value as the first user feature.

[0101] A preset image classification model can be used to assign image features and determine a probability value, i.e., a first probability value, that the image is not a child. The first probability value is used as the first user feature.

[0102] For example, a large amount of image data can be used as a training set. This image data comes from real pictures and is divided into adults and children (physical age, special populations are not considered). It is continuously increased during the testing and development process to achieve a relatively accurate recognition rate. Then, an appropriate deep learning model is selected, such as convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory network (LSTM), Transformer, etc., and the most suitable learning model is found through continuous testing to obtain a preset image classification model. The preset image classification model then extracts key features from the captured visual information, compares them with the database, and determines the probability that the user is an adult. If the probability exceeds 50%, it is judged as an adult.

[0103] The process of obtaining an image classification model based on image training based on models such as convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory networks (LSTM), and Transformer is as follows:

[0104] 1. Data Preparation Phase

[0105] Image data collection: Collect a large amount of image data from various sources, such as public datasets, the Internet, or specific projects. Make sure that this data covers all the categories that need to be classified, such as animals, plants, buildings, vehicles, etc.

[0106] Data preprocessing: Preprocess the images, including resizing, normalizing pixel values, and data augmentation (such as rotation, scaling, flipping, etc.) to increase data diversity. The preprocessing step aims to improve the generalization ability of the model and prevent overfitting.

[0107] Labeled data: Each image is assigned a correct category label, which will be used for supervised learning during training.

[0108] 2. Model Selection and Construction

[0109] Convolutional Neural Network (CNN): CNN is the model of choice for image classification tasks. It uses convolutional layers to extract local features in an image, such as edges, textures, and shapes. By stacking multiple convolutional and pooling layers, CNN is able to capture high-level features in an image.

[0110] (Optional) Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTMs): Although RNNs and LSTMs excel at processing sequential data, they are not commonly used in image classification tasks. Image data is static and does not have the characteristics of a time series, so these models are usually not the best choice for image classification.

[0111] Transformer: In recent years, the Transformer model has also achieved remarkable results in image classification tasks. Through the self-attention mechanism, the Transformer can capture the relationship between different regions in the image, thereby achieving more accurate classification.

[0112] Model construction: Based on the selected model type (mainly CNN or Transformer), the corresponding image classification model is constructed. The model usually includes an input layer (for receiving image data), a feature extraction layer (such as a convolutional layer, a Transformer block), a fully connected layer, and an output layer (for generating category predictions).

[0113] 3. Model Training and Optimization

[0114] Set the loss function: Choose an appropriate loss function (such as cross entropy loss) to measure the difference between the model prediction results and the true labels.

[0115] Choose an optimizer: Choose a suitable optimizer (such as Adam, SGD, etc.) to update the weights of the model to minimize the loss function.

[0116] Training the model: Preprocessed image data is fed into the model, and the model weights are continuously adjusted using optimization methods such as backpropagation and gradient descent. During training, metrics such as loss and accuracy need to be monitored to ensure that the model is learning effective feature representations.

[0117] Model evaluation and adjustment: Evaluate the model's performance on the validation set and adjust the model structure, hyperparameters, or data augmentation strategy based on the evaluation results. This process may require multiple iterations until the model achieves satisfactory performance on the validation set.

[0118] 4. Model Deployment and Application

[0119] Model export: Export the trained model to a deployable format, such as TensorFlow SavedModel, PyTorch model file, etc.

[0120] Integrate into applications: Integrate the exported model into image classification applications to implement real-time or offline image classification capabilities.

[0121] Continuous optimization: In practical applications, new image data is continuously collected and used for continuous optimization and updating of the model to improve the accuracy and generalization ability of the model.

[0122] Step 203: Perform voiceprint recognition based on the voice information to determine a second user feature;

[0123] The voiceprint recognition can be performed on the sound information to determine the identity of the user represented in the sound information and determine the second user feature.

[0124] In an optional embodiment of the present invention, the step of performing voiceprint recognition based on the sound information to determine the second user feature includes: extracting voiceprint features from the sound information; and classifying the voiceprint features to determine the second user feature.

[0125] A variety of voiceprint features may be extracted from the sound information, the voiceprint features may be classified, and the second user feature may be determined.

[0126] Specifically, the step of classifying the voiceprint features and determining the second user feature includes: inputting the voiceprint features into a preset sound classification model, the preset sound classification model is used to output a second probability value based on the voiceprint features; and determining the second probability value as the second user feature.

[0127] The preset voice classification model can be used to assign voiceprint features and determine a probability value (i.e., a second probability value) that the voice is not a child's identity. The second probability value is used as the second user feature.

[0128] For example, a large amount of audio data can be used as a training set. This data comes from real audio, divided into adults and children, and is continuously increased during the test and development process to achieve a relatively accurate recognition rate. Then, an appropriate deep learning model is selected, such as a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), a Transformer, etc., and the appropriate learning model is found through continuous testing to obtain a preset sound classification model. The preset sound classification model then extracts key features from the collected audio, compares them with the database, and determines the probability that the user is an adult. If the probability exceeds 50%, it is judged as an adult.

[0129] The process of obtaining a sound classification model based on audio training for models such as convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory networks (LSTM), and Transformer is as follows:

[0130] 1. Data Preparation

[0131] Collect audio data: Collect a large amount of audio data from various sources, which should cover the various sound categories that need to be classified.

[0132] Data preprocessing: The collected audio data is preprocessed, including denoising, standardization, segmentation into appropriate segments, etc., to ensure the quality and consistency of the data.

[0133] Labeled data: Label each piece of audio data with the correct category label for use in the training process.

[0134] 2. Model Selection

[0135] Convolutional Neural Networks (CNNs): CNNs have achieved remarkable success in image recognition and can also be applied to audio feature extraction. Through multi-layer convolution and pooling operations, CNNs can capture local features in audio data, such as frequency and rhythm.

[0136] Recurrent Neural Networks (RNNs): RNNs are particularly well-suited for processing sequential data, such as audio signals. They are able to capture temporal dependencies in audio data, i.e., the relationship between previous and subsequent audio segments. However, standard RNNs can suffer from vanishing or exploding gradients when processing long sequences.

[0137] Long Short-Term Memory (LSTM): LSTM is a variant of RNN. By introducing a "gate" mechanism (input gate, forget gate, output gate) and cell states, it effectively alleviates the vanishing gradient problem of RNN. LSTM can memorize information in a sequence for a long time, making it excellent at processing long audio signals.

[0138] Transformer: The Transformer model processes sequential data through self-attention and positional encoding, without relying on recurrent connections. It has achieved great success in natural language processing and is gradually being applied to audio processing.

[0139] 3. Model Training

[0140] Build the model: Based on the selected model type (CNN, RNN, LSTM, Transformer, etc.), build the corresponding sound classification model. The model usually includes an input layer, a feature extraction layer (such as a convolutional layer or a recurrent layer), a fully connected layer, and an output layer.

[0141] Setting the loss function and optimizer: Choose an appropriate loss function (such as cross entropy loss) and optimizer (such as Adam optimizer) to train the model. The loss function measures the difference between the model's predictions and the true labels, while the optimizer updates the model's weights to minimize the loss.

[0142] Training model: Input the preprocessed audio data into the model, and continuously adjust the model weights through optimization methods such as backpropagation algorithm and gradient descent until the model performance on the validation set reaches the best.

[0143] Model evaluation: Evaluate the model's performance on the test set, including metrics such as accuracy, recall, and F1 score. Further optimize the model's performance by adjusting the model structure and hyperparameters.

[0144] 4. Model Deployment and Application

[0145] Model export: Export the trained model to a deployable format, such as TensorFlow SavedModel, ONNX, etc.

[0146] Integrate into applications: Integrate the exported model into audio classification applications to implement real-time or offline sound classification capabilities.

[0147] Continuous optimization: In practical applications, new audio data is continuously collected and used for continuous optimization and updating of the model to improve the accuracy and generalization ability of the model.

[0148] Step 204: Determine target user characteristics by combining the first user characteristics and the second user characteristics;

[0149] The first user feature and the second user feature may be combined to determine a target user feature, and the target user feature may be used to represent the identity of the user.

[0150] In an optional embodiment of the present invention, the step of determining the target user feature by combining the first user feature and the second user feature includes: determining a first weight corresponding to the first user feature; determining a second weight corresponding to the first user feature; and determining the target user feature by combining the first user feature, the second user feature, the first weight, and the second weight.

[0151] A first weight corresponding to the first user feature can be determined, as can a second weight corresponding to the first user feature. The first weight and the second weight can be pre-set and dynamically adjusted during subsequent use. The specific values ​​of the first weight and the second weight, as well as the relationship between the first weight and the second weight, are not specifically limited in this embodiment of the present invention.

[0152] For example, because audio information cannot always accurately reflect whether the user is standing in front of the device, and sometimes a child's voice may be similar to that of an adult, the system gives a higher weight to visual recognition. Visual recognition is the main basis for judgment, while voice recognition is used as an auxiliary to ensure that the system can more accurately determine the user type.

[0153] The first user feature, the second user feature, the first weight, and the second weight may be fused to determine the target user feature.

[0154] Specifically, the step of determining the target user feature by combining the first user feature, the second user feature, the first weight, and the second weight includes: combining the first user feature with the first weight to determine a first intermediate value; combining the second user feature with the second weight to determine a second intermediate value; and adding the first intermediate value and the second intermediate value to obtain the target user feature.

[0155] The first user feature is multiplied by the first weight, and the resulting product is the first intermediate value; the second user feature is multiplied by the second weight, and the resulting product is the second intermediate value; then the first intermediate value and the second intermediate value are added together to obtain the target user feature.

[0156] Step 205: When the target user characteristic is greater than a preset threshold, determining that the target user characteristic is a non-child user;

[0157] When the target user feature is greater than a preset threshold, it can be determined that the target user feature is a non-child user. The value of the preset threshold can be determined according to actual conditions and is not limited in the embodiment of the present invention.

[0158] Step 206: When the target user characteristic is a non-child user, execute the operation instruction;

[0159] When the target user is not a child user, it means that the current operation instruction is valid. The device can interpret the operation instruction and perform the corresponding operation. The execution result may also be fed back to the user, so that the user can visualize the execution result of the instruction.

[0160] Step 207: When the target user characteristic is not greater than a preset threshold, determining that the target user characteristic is a child user;

[0161] When the target user characteristic is greater than a preset threshold, it can be determined that the target user characteristic is a child user.

[0162] Step 208: When the target user is characterized as a child user, an alarm signal is issued.

[0163] When the target user is characterized as a child user, it means that the current operation instruction is invalid, and the device can refuse to perform the operation corresponding to the operation instruction; and issue an alarm signal. That is, when it is identified that a child user is performing an operation, an alarm signal can be triggered. The alarm signal can include various forms such as sound, light, and text prompts to attract the attention of other people in a timely manner. After receiving the alarm signal, other people can immediately pay attention to the child's behavior, and can also check and confirm the device. If it is confirmed whether there are other abnormal conditions caused by the child's operation, the relevant personnel need to take appropriate measures to eliminate the alarm signal and restore the normal operation of the device in the mode before the alarm. In the presence of other abnormal conditions, further corresponding repair measures may be needed to troubleshoot the device.

[0164] In addition, in order to more accurately identify the user, it is possible to make a judgment based on the addition of click habits. Since children's erroneous operations are to frequently initiate operation instructions in a short period of time, the frequency of the corresponding operation instructions initiated by the user during the operation can be pre-recorded. For example, the user initiates an operation instruction by clicking. The third user feature can be determined by comparing the frequency of the user initiating the operation instruction in advance with the frequency of the current operation instruction. The third user feature, the first user feature and the second user feature can be combined based on the first user feature and the second user feature, and a third weight can be introduced to calculate the target user feature. For example, when the frequency of initiating operation instructions within a certain period of time does not match the pre-recorded frequency, the probability of a child user in the third user feature is higher.

[0165] In an embodiment of the present invention, when a device is in child lock mode and receives an operation instruction, the device detects visual and audio information; performs image recognition based on the visual information to determine a first user characteristic; performs voiceprint recognition based on the audio information to determine a second user characteristic; combines the first and second user characteristics to determine a target user characteristic; when the target user characteristic is greater than a preset threshold, determines that the target user characteristic is a non-child user; executes the operation instruction if the target user characteristic is a non-child user; when the target user characteristic is not greater than the preset threshold, determines that the target user characteristic is a child user; and issues an alarm signal if the target user characteristic is a child user. By jointly determining the identity of the user initiating the operation instruction based on visual and audio information when the child lock is enabled, the relevant operation instruction can be directly executed when the user is not a child user, effectively preventing the device from being accidentally operated in child lock mode. This effectively protects user safety without compromising user experience.

[0166] In order to make the embodiments of the present invention clear to those skilled in the art, the following reference is made to Figure 3 , to illustrate with an example:

[0167] 1. The current system is in child lock mode.

[0168] 2. The user performs an operation on the smart central control device, triggering the device's user identification function, and the system begins to collect and analyze user information.

[0169] 3. The system uses the camera and microphone to collect the user's visual and audio information, respectively, as a basis for determining the user's identity.

[0170] 4. The device processes the collected images and audio through its visual and voice recognition systems to determine the type of user operating the device. The visual recognition system uses image features, while the voice recognition system analyzes voiceprint and sound characteristics to comprehensively determine whether the user is a child.

[0171] 5. The operation will only be allowed if the system recognizes that the user is not a child; otherwise, the device will refuse to execute the user's instructions and continue to maintain child lock mode to ensure that children cannot bypass the protection mechanism.

[0172] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0173] Reference Figure 4 , shows a structural block diagram of an embodiment of a device control device of the present invention, the device control device may specifically include the following modules:

[0174] A detection module 401 is configured to detect visual and audio information when receiving an operation instruction while the device is in child lock mode;

[0175] A first recognition module 402 is configured to perform image recognition based on the visual information to determine a first user feature;

[0176] A second recognition module 403 is configured to perform voiceprint recognition based on the voice information to determine a second user feature;

[0177] A combining module 404 is configured to combine the first user characteristics and the second user characteristics to determine target user characteristics;

[0178] The first execution module 405 is configured to execute the operation instruction when the target user characteristic is a non-child user.

[0179] In an optional embodiment of the present invention, the combining module 404 includes:

[0180] A first weight determination submodule, configured to determine a first weight corresponding to the first user feature;

[0181] A second weight determination submodule, configured to determine a second weight corresponding to the first user feature;

[0182] The feature determination submodule is configured to determine a target user feature by using the first user feature, the second user feature, the first weight, and the second weight.

[0183] In an optional embodiment of the present invention, the feature determination submodule includes:

[0184] a first calculation unit, configured to combine the first user feature with the first weight to determine a first intermediate value;

[0185] a second calculation unit, configured to combine the second user feature with the second weight to determine a second intermediate value;

[0186] The third calculation unit is configured to add the first intermediate value and the second intermediate value to obtain a target user feature.

[0187] In an optional embodiment of the present invention, the first identification module 402 includes:

[0188] a first extraction submodule, configured to extract image features from the visual information;

[0189] The first classification submodule is used to classify the image features and determine a first user feature.

[0190] In an optional embodiment of the present invention, the first classification submodule includes:

[0191] a first classification unit, configured to input the image features into a preset image classification model, wherein the preset image classification model is configured to output a first probability value based on the image features;

[0192] A first determining unit is configured to determine the first probability value as a first user feature.

[0193] In an optional embodiment of the present invention, the second identification module 403 includes:

[0194] A second extraction submodule is used to extract voiceprint features from the sound information;

[0195] The second classification submodule is used to classify the voiceprint feature and determine the second user feature.

[0196] In an optional embodiment of the present invention, the second classification submodule includes:

[0197] a second classification unit, configured to input the voiceprint feature into a preset sound classification model, wherein the preset sound classification model is configured to output a second probability value based on the voiceprint feature;

[0198] The second determining unit is configured to determine the second probability value as a second user feature.

[0199] In an optional embodiment of the present invention, the device further comprises:

[0200] A first comparison module is configured to determine that the target user characteristic is a child user when the target user characteristic is not greater than a preset threshold;

[0201] The second comparison module is configured to determine that the target user feature is a non-child user when the target user feature is greater than a preset threshold.

[0202] In an optional embodiment of the present invention, the device further comprises:

[0203] The second execution module is configured to send an alarm signal when the target user is characterized as a child user.

[0204] This embodiment of the present invention detects visual and audio information when an operation instruction is received while the device is in child lock mode; performs image recognition based on the visual information to determine a first user characteristic; performs voiceprint recognition based on the audio information to determine a second user characteristic; combines the first and second user characteristics to determine a target user characteristic; and executes the operation instruction if the target user characteristic indicates a non-child user. By determining the identity of the user initiating the operation instruction based on both visual and audio information when the child lock is enabled, the relevant operation instruction can be directly executed if the user is not a child, effectively preventing accidental operation of the device in child lock mode. This effectively protects user safety without compromising user experience.

[0205] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0206] Reference Figure 5 , an embodiment of the present invention further provides an electronic device, including:

[0207] A processor 501 and a storage medium 502, wherein the storage medium 502 stores a computer program executable by the processor 501. When the electronic device is controlled to operate, the processor 501 executes the computer program to implement the device control method as described in any embodiment of the present invention. The device control method includes:

[0208] When the device is in child lock mode and receives an operation instruction, detecting visual information and sound information;

[0209] performing image recognition based on the visual information to determine a first user feature;

[0210] Performing voiceprint recognition based on the sound information to determine a second user feature;

[0211] Determining target user characteristics by combining the first user characteristics and the second user characteristics;

[0212] When the target user characteristic is a non-child user, the operation instruction is executed.

[0213] Optionally, the step of determining target user characteristics by combining the first user characteristics and the second user characteristics includes:

[0214] determining a first weight corresponding to the first user feature;

[0215] determining a second weight corresponding to the first user feature;

[0216] The first user feature, the second user feature, the first weight, and the second weight are used to determine a target user feature.

[0217] Optionally, the step of determining target user features by using the first user features, the second user features, the first weight, and the second weight includes:

[0218] combining the first user feature with the first weight to determine a first intermediate value;

[0219] combining the second user feature with the second weight to determine a second intermediate value;

[0220] The first intermediate value and the second intermediate value are added together to obtain target user features.

[0221] Optionally, the step of performing image recognition based on the visual information to determine the first user feature includes:

[0222] extracting image features from the visual information;

[0223] The image features are classified to determine a first user feature.

[0224] Optionally, the step of classifying the image features and determining the first user feature includes:

[0225] Inputting the image features into a preset image classification model, wherein the preset image classification model is configured to output a first probability value based on the image features;

[0226] The first probability value is determined to be a first user feature.

[0227] Optionally, the step of performing voiceprint recognition based on the sound information to determine the second user feature includes:

[0228] extracting voiceprint features from the sound information;

[0229] The voiceprint features are classified to determine a second user feature.

[0230] Optionally, the step of classifying the voiceprint features to determine the second user features includes:

[0231] Inputting the voiceprint feature into a preset sound classification model, wherein the preset sound classification model is configured to output a second probability value based on the voiceprint feature;

[0232] The second probability value is determined as a second user feature.

[0233] Optionally, the method comprises:

[0234] When the target user characteristic is not greater than a preset threshold, determining that the target user characteristic is a child user;

[0235] When the target user characteristic is greater than a preset threshold, it is determined that the target user characteristic is a non-child user.

[0236] Optionally, the method comprises:

[0237] When the target user is characterized as a child user, an alarm signal is issued.

[0238] This embodiment of the present invention detects visual and audio information when an operation instruction is received while the device is in child lock mode; performs image recognition based on the visual information to determine a first user characteristic; performs voiceprint recognition based on the audio information to determine a second user characteristic; combines the first and second user characteristics to determine a target user characteristic; and executes the operation instruction if the target user characteristic indicates a non-child user. By determining the identity of the user initiating the operation instruction based on both visual and audio information when the child lock is enabled, the relevant operation instruction can be directly executed if the user is not a child, effectively preventing accidental operation of the device in child lock mode. This effectively protects user safety without compromising user experience.

[0239] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0240] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0241] Reference Figure 6 The embodiment of the present invention further provides a computer-readable storage medium 601, on which a computer program is stored. The computer program is executed by a processor to execute the device control method as described in any one of the embodiments of the present invention. The device control method includes:

[0242] When the device is in child lock mode and receives an operation instruction, detecting visual information and sound information;

[0243] performing image recognition based on the visual information to determine a first user feature;

[0244] Performing voiceprint recognition based on the sound information to determine a second user feature;

[0245] Determining target user characteristics by combining the first user characteristics and the second user characteristics;

[0246] When the target user characteristic is a non-child user, the operation instruction is executed.

[0247] Optionally, the step of determining target user characteristics by combining the first user characteristics and the second user characteristics includes:

[0248] determining a first weight corresponding to the first user feature;

[0249] determining a second weight corresponding to the first user feature;

[0250] The first user feature, the second user feature, the first weight, and the second weight are used to determine a target user feature.

[0251] Optionally, the step of determining target user features by using the first user features, the second user features, the first weight, and the second weight includes:

[0252] combining the first user feature with the first weight to determine a first intermediate value;

[0253] combining the second user feature with the second weight to determine a second intermediate value;

[0254] The first intermediate value and the second intermediate value are added together to obtain target user features.

[0255] Optionally, the step of performing image recognition based on the visual information to determine the first user feature includes:

[0256] extracting image features from the visual information;

[0257] The image features are classified to determine a first user feature.

[0258] Optionally, the step of classifying the image features and determining the first user feature includes:

[0259] Inputting the image features into a preset image classification model, wherein the preset image classification model is configured to output a first probability value based on the image features;

[0260] The first probability value is determined to be a first user feature.

[0261] Optionally, the step of performing voiceprint recognition based on the sound information to determine the second user feature includes:

[0262] extracting voiceprint features from the sound information;

[0263] The voiceprint features are classified to determine a second user feature.

[0264] Optionally, the step of classifying the voiceprint features to determine the second user features includes:

[0265] Inputting the voiceprint feature into a preset sound classification model, wherein the preset sound classification model is configured to output a second probability value based on the voiceprint feature;

[0266] The second probability value is determined as a second user feature.

[0267] Optionally, the method comprises:

[0268] When the target user characteristic is not greater than a preset threshold, determining that the target user characteristic is a child user;

[0269] When the target user characteristic is greater than a preset threshold, it is determined that the target user characteristic is a non-child user.

[0270] Optionally, the method comprises:

[0271] When the target user is characterized as a child user, an alarm signal is issued.

[0272] This embodiment of the present invention detects visual and audio information when an operation instruction is received while the device is in child lock mode; performs image recognition based on the visual information to determine a first user characteristic; performs voiceprint recognition based on the audio information to determine a second user characteristic; combines the first and second user characteristics to determine a target user characteristic; and executes the operation instruction if the target user characteristic indicates a non-child user. By determining the identity of the user initiating the operation instruction based on both visual and audio information when the child lock is enabled, the relevant operation instruction can be directly executed if the user is not a child, effectively preventing accidental operation of the device in child lock mode. This effectively protects user safety without compromising user experience.

[0273] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0274] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, embodiments of the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0275] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0276] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0277] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0278] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0279] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0280] The above is a detailed introduction to a device control method, a device control device, an electronic device and a storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A device control method, characterized in that: include: When the device is in child lock mode and receives an operation instruction, detecting visual information and sound information; Performing image recognition based on the visual information to determine a first user feature; Performing voiceprint recognition based on the sound information to determine the second user's characteristics; Determine target user characteristics by combining the first user characteristics and the second user characteristics; When the target user characteristic is a non-child user, the operation instruction is executed.

2. The method according to claim 1, characterized in that The step of determining target user characteristics by combining the first user characteristics and the second user characteristics comprises: Determining a first weight corresponding to the first user feature; Determining a second weight corresponding to the first user feature; The first user feature, the second user feature, the first weight, and the second weight are used to determine a target user feature.

3. The method according to claim 2, characterized in that The step of determining a target user feature by using the first user feature, the second user feature, the first weight, and the second weight comprises: combining the first user feature with the first weight to determine a first intermediate value; combining the second user feature with the second weight to determine a second intermediate value; The first intermediate value and the second intermediate value are added together to obtain the target user feature.

4. The method according to claim 1, characterized in that The step of performing image recognition based on the visual information to determine the first user feature includes: extracting image features from the visual information; The image features are classified to determine a first user feature.

5. The method according to claim 4, characterized in that The step of classifying the image features and determining the first user feature comprises: Inputting the image feature into a preset image classification model, wherein the preset image classification model is used to output a first probability value based on the image feature; The first probability value is determined as a first user feature.

6. The method according to claim 1, characterized in that The step of performing voiceprint recognition based on the sound information to determine the second user feature includes: Extracting voiceprint features from the sound information; The voiceprint features are classified to determine a second user feature.

7. The method according to claim 6, characterized in that The step of classifying the voiceprint features to determine the second user features includes: Inputting the voiceprint feature into a preset sound classification model, wherein the preset sound classification model is used to output a second probability value based on the voiceprint feature; The second probability value is determined as a second user feature.

8. The method according to claim 1, characterized in that The method comprises: When the target user characteristic is not greater than a preset threshold, determining that the target user characteristic is a child user; When the target user characteristic is greater than a preset threshold, it is determined that the target user characteristic is a non-child user.

9. The method according to claim 1, characterized in that: The method comprises: When the target user characteristic is a child user, an alarm signal is issued.

10. A device control device, characterized in that: include: A detection module, used for detecting visual information and sound information when receiving an operation instruction when the device is in a child lock mode; A first recognition module, configured to perform image recognition based on the visual information to determine a first user feature; A second recognition module, configured to perform voiceprint recognition based on the sound information to determine a second user feature; A combining module, configured to combine the first user feature and the second user feature to determine a target user feature; The first execution module is used to execute the operation instruction when the target user feature is a non-child user.

11. An electronic device, characterized in that: The device comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the device control method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the device control method according to any one of claims 1 to 9 are implemented.

Images