Control method and control system of intelligent equipment

Abstract

The invention provides a control method and a control system of intelligent equipment. The control method comprises the steps of obtaining an image of a target area; performing face recognition on the image, and determining whether there is a registered user in the image; when it is determined that there is a registered user in the image, performing gesture recognition on the image according to a preset gesture set; determining a corresponding control instruction from a preset gesture instruction set according to a gesture recognition result; determining an intelligent device corresponding to the control instruction; and controlling the corresponding intelligent equipment to execute the control instruction. By introducing face recognition and gesture recognition, a registered user can control the corresponding intelligent equipment through a made gesture, so that people can control the intelligent equipment more easily and fully enjoy the functions of the intelligent equipment, and the convenience of controlling the intelligent equipment is improved. According to the method, people appearing in the target area can be discriminated through face recognition, it is ensured that only registered users can control the intelligent device, and the control safety is improved.

Description

Technical Field

[0001] The present invention generally relates to the technical field of device control, and in particular to a control method and control system for intelligent devices. Background Art

[0002] With the rapid development of computer technology and mobile networks, the intelligent devices that people need to operate are no longer limited to personal computers, desktop devices, and smartphones, but also include smart TVs, smart wearable devices, smart homes, etc. People expect to use these intelligent devices more conveniently at home, on the road, or in the office. However, the technology for controlling intelligent devices usually focuses on voice output or APP operation implementation, and the control requirements for sign language input of deaf-mute people have not been fully met, which makes them feel restricted when interacting with intelligent devices and unable to fully enjoy the functions of the devices.

[0003] The content in the background art section is only the technology known to the inventor and does not necessarily represent the prior art in this field. Summary of the Invention

[0004] In view of one or more defects in the prior art, the present invention provides a control method for intelligent devices, including:

[0005] Obtaining an image of a target area;

[0006] Performing face recognition on the image to determine whether there is a registered user in the image;

[0007] When it is determined that there is a registered user in the image, gesture recognition is performed on the image according to a preset set of gestures;

[0008] Determining a corresponding control instruction from a preset set of gesture instructions according to the result of the gesture recognition;

[0009] Determining the intelligent device corresponding to the control instruction; and

[0010] Controlling the corresponding intelligent device to execute the control instruction.

[0011] According to one aspect of the present invention, the step of performing face recognition on the image to determine whether there is a registered user in the image includes:

[0012] Performing frame change detection on the image;

[0013] When frame change is detected, performing human detection on the image;

[0014] When it is detected that there is a human form in the image, performing face detection on the image;

[0015] When a human face is detected, the detected human face is matched with the pre-stored human face information of registered users.

[0016] If the matching is successful, it is determined that there is a registered user in the image.

[0017] If the matching fails, it is determined that there is no registered user in the image.

[0018] According to one aspect of the present invention, the step of determining a corresponding control instruction from a preset set of gesture instructions according to the result of the gesture recognition includes:

[0019] When a preset gesture is recognized, a corresponding control instruction is determined from the preset set of gesture instructions.

[0020] According to one aspect of the present invention, when a corresponding control instruction cannot be determined from the preset set of gesture instructions, the result of the gesture recognition is uploaded to the upper-layer control device, and a corresponding control instruction is determined by the upper-layer control device.

[0021] According to one aspect of the present invention, the control method is implemented by a camera device;

[0022] The step of controlling the corresponding intelligent device to execute the control instruction includes:

[0023] When the corresponding intelligent device is the camera device, control the camera device to execute the control instruction;

[0024] When the corresponding intelligent device is not the camera device, control the corresponding intelligent device to execute the control instruction through the upper-layer control device.

[0025] According to one aspect of the present invention, the intelligent device is configured to output feedback information according to the execution result of the control instruction;

[0026] The control method further includes: receiving the feedback information, and making corresponding prompts in the feedback unit of the camera device according to the feedback information.

[0027] According to one aspect of the present invention, the upper-layer control device is a cloud server or a gateway.

[0028] The present invention also provides an intelligent control system, including:

[0029] A first intelligent device, the first intelligent device includes:

[0030] An image acquisition unit configured to acquire an image of a target area;

[0031] A detection and recognition unit configured to perform face recognition on the image, determine whether there is a registered user in the image, and when it is determined that there is a registered user in the image, perform gesture recognition on the image according to a preset set of gestures; and

[0032] A control unit configured to determine a corresponding control instruction from a preset set of gesture instructions according to the result of the gesture recognition, and is also configured to determine the intelligent device corresponding to the control instruction, and when the corresponding intelligent device is the first intelligent device, control the first intelligent device to execute the control instruction.

[0033] According to one aspect of the present invention, the detection and recognition unit is configured to:

[0034] Perform screen change detection on the image;

[0035] When a screen change is detected, perform human detection on the image;

[0036] When it is detected that there is a human form in the image, perform face detection on the image;

[0037] When a face is detected, match the detected face with the face information of the pre-stored registered user,

[0038] If the match is successful, it is determined that there is a registered user in the image;

[0039] If the match fails, it is determined that there is no registered user in the image.

[0040] According to one aspect of the present invention, the first intelligent device further includes a face information management unit, the face information management unit is coupled to the detection and recognition unit, and the face information management unit stores the face information of the registered user.

[0041] According to one aspect of the present invention, the intelligent control system further includes:

[0042] An interaction interface configured to provide a graphical user interface, receive modification input according to the user's interaction operation on the graphical user interface and output corresponding modification information, and the modification input includes information and photos for adding and deleting registered users;

[0043] An upper control device communicating with the interaction interface and the first intelligent device respectively, the upper control device is configured to store the photo according to the modification information and output a registration instruction to the first intelligent device, or delete the photo and output a deletion instruction to the first intelligent device;

[0044] The first intelligent device is configured to call a corresponding photo from the upper control device according to a registration instruction, and save the face information of the corresponding registered user in the user face information management unit according to the photo.

[0045] The first intelligent device is also configured to delete the face information of the corresponding registered user in the user face information management unit according to a deletion instruction.

[0046] According to one aspect of the present invention, the control unit is configured to determine a corresponding control instruction from a preset set of gesture instructions when a preset gesture is recognized.

[0047] According to one aspect of the present invention, the intelligent control system further includes an upper control device, and the first intelligent device communicates with the upper control device.

[0048] The control unit is configured to: when a corresponding control instruction cannot be determined from the preset set of gesture instructions, upload the result of the gesture recognition to the upper control device.

[0049] The upper control device is configured to determine a corresponding control instruction according to the result of the gesture recognition.

[0050] According to one aspect of the present invention, the intelligent control system further includes one or more second intelligent devices, and the second intelligent devices communicate with the upper control device.

[0051] The upper control device is configured to determine the second intelligent device corresponding to the control instruction and control the second intelligent device to execute the control instruction.

[0052] According to one aspect of the present invention, the first intelligent device further includes a gesture command management unit, the gesture command management unit is coupled to the control unit, and the gesture command management unit stores a set of gestures and a set of gesture instructions.

[0053] According to one aspect of the present invention, the second intelligent device is configured to output feedback information according to the execution result of the control instruction.

[0054] The upper control device is configured to receive the feedback information output by the second intelligent device and forward the feedback information to the first intelligent device.

[0055] The first intelligent device further includes a feedback unit, and the feedback unit is configured to make corresponding prompts according to the feedback information.

[0056] According to one aspect of the present invention, the upper control device is a cloud or a gateway.

[0057] Compared with the prior art, the embodiments of the present invention provide a control method and a control system for intelligent devices. By introducing face recognition and gesture recognition, registered users can control corresponding intelligent devices by making gestures, providing an intuitive and personalized way for people (especially the deaf and mute population) to interact with intelligent devices, enabling people to more easily control intelligent devices, fully enjoy the functions of intelligent devices, and improving the convenience of controlling intelligent devices. Among them, face recognition can identify the people appearing in the target area to ensure that only registered users can control the intelligent devices, improving the security of control. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, but do not constitute a limitation to the present invention. In the accompanying drawings:

[0059] Figure 1 shows a flowchart of a control method for an intelligent device according to an embodiment of the present invention;

[0060] Figure 2 shows a flowchart of face recognition according to an embodiment of the present invention;

[0061] Figure 3 shows a schematic diagram of some gestures included in a gesture set according to an embodiment of the present invention;

[0062] Figure 4 shows a schematic diagram of an intelligent control system according to an embodiment of the present invention;

[0063] Figure 5 shows an interaction flowchart for modifying a registered user according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] In the following, only some exemplary embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present invention. Therefore, the drawings and the description are considered to be exemplary in nature rather than restrictive.

[0065] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality" means two or more, unless otherwise specifically defined.

[0066] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "mounted", "connected" and "coupled" shall be construed broadly. For example, it may be a fixed connection, a detachable connection, or an integral connection: it may be a mechanical connection, an electrical connection or may communicate with each other; it may be directly connected, or indirectly connected through an intermediate medium, and it may be the internal communication of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0067] In the present invention, unless otherwise clearly specified and defined, the first feature being "on" or "under" the second feature may include the direct contact between the first and second features, or may include the situation where the first and second features are not in direct contact but in contact through additional features therebetween. Moreover, the first feature being "above", "over" and "on top of" the second feature includes that the first feature is directly above and obliquely above the second feature, or merely means that the horizontal height of the first feature is higher than that of the second feature. The first feature being "under", "beneath" and "underneath" the second feature includes that the first feature is directly below and obliquely below the second feature, or merely means that the horizontal height of the first feature is lower than that of the second feature.

[0068] The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the present invention. In addition, the present invention may repeat reference numerals and / or reference letters in different examples. Such repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides examples of various specific processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0069] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not used to limit the present invention.

[0070] Figure 1 The flowchart of the control method of the intelligent device according to an embodiment of the present invention is shown. The following will be described in detail Figure 1 herein.

[0071] The control method can be implemented by a camera device, which refers to an intelligent device with a camera function, such as a smart phone, a smart screen (intelligent screen), a smart speaker, a smart access control, a camera, etc. The control method can also be implemented by a controller of the camera device or a gateway (intelligent gateway) in the home.

[0072] As Figure 1 shown, the control method includes the following steps, which will be described in detail below respectively.

[0073] In step S110, an image of the target area is acquired.

[0074] The camera device can take a picture of the target area at preset intervals (such as 0.5 seconds, 1 second, 2 seconds, etc.) to obtain an image of the target area at the corresponding time. This image can be stored and transmitted in picture formats such as YUV, PNG, GIF, JPG, BMP, etc. In some embodiments, the camera device can also continuously record the target area and then intercept pictures from the recording at preset time intervals (or frames) to obtain an image of the target area.

[0075] In step S120, face recognition is performed on the image to determine whether there is a registered user in the image.

[0076] Through face recognition, people and objects appearing in the target area can be identified to ensure that only registered users can control the intelligent device, thereby improving the security of control.

[0077] Figure 2The flowchart of face recognition according to an embodiment of the present invention is shown. As Figure 2 shown, in step S210, scene change detection is performed on the image. Among them, scene change detection can be to compare the currently acquired image with the image acquired previously (for example, at the previous time node). If the scenes of the two images are basically the same, it can be considered that the scene has not changed. If the scenes of the two images are basically different, it can be considered that the scene has changed. Specifically, the gray values or color values of the corresponding pixels in the two images can be compared, and based on this, the similarity of the two images can be calculated. When the similarity of the two images reaches a preset threshold (such as 90%, 95%, 98% or 100%), the scenes of the two images are basically the same, and further it can be considered that the scene has not changed (small changes can be ignored), and at this time, face recognition can be stopped; when the similarity of the two images is lower than the preset threshold, the scenes of the two images are basically different, and it can be considered that the scene has changed, and at this time, step S212 can be entered. Among them, by directly comparing the gray values or color values of the two images, scene change detection can be performed simply and quickly, without complex preprocessing or feature extraction of the image, which can save computing power and reduce power consumption.

[0078] In step S213, human detection is performed on the image. Specifically, human detection can be based on a visual human pose recognition algorithm. The visual human pose recognition algorithm can recognize and analyze the key points, poses, actions, etc. of the human body to determine whether there is a human body in the target area. When no human form is detected in the image, face recognition can be stopped; when a human form is detected in the image, step S213 can be entered. In addition, a deep model (Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), etc.) can be used to train the visual human pose recognition algorithm to improve the accuracy of human detection. In this step, by performing human detection on the image, it can be determined whether there is a human body in the target area, excluding scene changes caused by animals, toys, robots, etc., saving computing power and reducing power consumption.

[0079] In step S213, face detection is performed on the image. By performing face detection on the image, it can be determined whether there is a face in the image (target area). When there is no face in the image, face recognition can be stopped; when there is a face in the image, the face can be extracted and step S214 can be entered. Specifically, the face detection can be a face detection algorithm based on Haar features, a face detection algorithm based on deep learning, a face detection algorithm based on skin color segmentation, or a face detection algorithm based on contours. Among them, the face detection algorithm based on Haar features uses Haar features to describe the shape and texture of the face, and trains a classifier to recognize and detect the face; the face detection algorithm based on deep learning uses a deep neural network to learn the features of the face, and trains a model to recognize and detect the face; the face detection algorithm based on contours extracts the contour of the face by performing contour detection on the image, and then uses other algorithms to further confirm and extract the face; the face detection algorithm based on skin color segmentation extracts the candidate area of the face by performing skin color segmentation on the image, and then uses other algorithms to further confirm and extract the face.

[0080] In step S214, the detected face is matched with the face information of the pre-stored registered users. Among them, a face matching algorithm can be used for matching. When the matching is successful, it can be determined that there is a registered user in the image; when the matching fails, it can be determined that there is no registered user in the image. The face matching algorithm can be a face matching algorithm based on geometric features, which has a small amount of calculation and high speed; the face matching algorithm can also be a face matching algorithm based on deep learning, which has high accuracy and good robustness and is not very sensitive to changes in face pose, illumination, etc.

[0081] During the process of performing face recognition on the image, scene change detection, human detection, face detection, and face matching are sequentially performed on the image, and face recognition is stopped when it is detected that the scene has not changed, when it is detected that there is no human form in the image, and when it is detected that there is no face in the image, which can save computing power and reduce power consumption.

[0082] In step S130, when it is determined that there is a registered user in the image, gesture recognition is performed on the image according to a preset set of gestures.

[0083] In the specific implementation manner, the set of gestures is pre-stored in the camera device, and the set of gestures can include pictures or features of one or more gestures, such as Figure 3 shown, the gestures can be, for example, single-handed heart, confirmation, like, step on, love you, victory, rock, shoot, flick, fist, index finger, middle finger, little finger, palm, number 3, number 4, number 6, etc. Figure 3 Some single-handed gestures are shown in, but the present invention is not limited thereto, and the gestures can also be two-handed gestures, such as two-handed heart, etc.

[0084] In a specific implementation manner, the image can be matched with the gesture set to determine whether the registered user makes a gesture in the gesture set and which gesture in the gesture set the registered user makes.

[0085] In step S140, according to the result of the gesture recognition, a corresponding control instruction is determined from a preset gesture instruction set.

[0086] In step S150, the intelligent device corresponding to the control instruction is determined.

[0087] In a specific implementation manner, the gesture instruction set is pre-stored in the imaging device. The gesture instruction set can be the corresponding relationship between some gestures in the gesture set and the control instructions for controlling the imaging device. For example, the index finger corresponds to starting video recording, the middle finger corresponds to stopping video recording, and stepping corresponds to hibernation. The above examples are only for illustration, and the present invention is not limited thereto. When a preset gesture is recognized (that is, a gesture in the gesture set is recognized), a corresponding control instruction can be determined from the gesture instruction set. When the control instruction is determined from the gesture instruction set, the intelligent device corresponding to the control instruction is the imaging device. When the corresponding control instruction cannot be determined from the gesture instruction set, the recognized gesture can be uploaded to the upper control device, and the upper control device determines the corresponding control instruction. Specifically, the upper control device can be a cloud server or a gateway. The upper control device and the imaging device can communicate with each other. In the upper control device, a corresponding table of another part of the gestures in the gesture set and the control instructions for controlling other intelligent devices can be stored. The upper control device can determine the control instruction corresponding to the gesture and the intelligent device corresponding to the control instruction according to the corresponding table.

[0088] In step S160, the corresponding intelligent device is controlled to execute the control instruction.

[0089] In a specific implementation manner, when the intelligent device corresponding to the control instruction is the imaging device, the imaging device is controlled to execute the control instruction. When the intelligent device corresponding to the control instruction is not the imaging device, the upper control device controls the corresponding intelligent device to execute the control instruction. Specifically, the upper control device sends the control instruction to the corresponding intelligent device, and the intelligent device executes the control instruction.

[0090] The control method further includes: receiving feedback information and making corresponding prompts in the feedback unit of the imaging device according to the feedback information.

[0091] Specifically, the intelligent device is configured to output feedback information according to the execution result of a control instruction. The feedback information can be, for example, execution success and execution failure. The upper-layer control device is configured to receive the feedback information output by the intelligent device and forward the feedback information to the imaging device. The feedback unit of the imaging device can be a display screen. When receiving the feedback information, the imaging device can display gestures representing execution success or failure through the display screen; the feedback unit of the imaging device can also be an indicator light. When receiving the feedback information, the imaging device can prompt execution success or failure by lighting indicator lights of different colors, or can also prompt execution success or failure by controlling the indicator light to flash according to a preset rule (such as Morse code).

[0092] The above describes a control method for an intelligent device that can be executed by an imaging device. By introducing face recognition and gesture recognition, registered users can control the corresponding intelligent device by making gestures, providing an intuitive and personalized way for people (especially the deaf and mute population) to interact with the intelligent device, enabling people to control the intelligent device more easily, fully enjoying the functions of the intelligent device, and improving the convenience of controlling the intelligent device. For the same purpose, the following provides an intelligent control system.

[0093] Figure 4 The schematic diagram of an intelligent control system 200 according to an embodiment of the present invention is shown, as Figure 4 shown, the intelligent control system 200 may include a first intelligent device 210. The first intelligent device 210 includes an image acquisition unit 211, a detection and recognition unit 212, and a control unit 213. Among them, the image acquisition unit 211 is coupled to the detection and recognition unit 212, and the detection and recognition unit 212 is coupled to the control unit 213. The image acquisition unit 211 is configured to acquire an image of a target area. The detection and recognition unit 212 is configured to perform face recognition on the image to determine whether there is a registered user in the image. When it is determined that there is a registered user in the image, gesture recognition is performed on the image according to a preset set of gestures. The control unit 213 is configured to determine a corresponding control instruction from a preset set of gesture instructions according to the result of the gesture recognition; the control unit 213 is also configured to determine the intelligent device corresponding to the control instruction, and when the control instruction corresponds to the first intelligent device 210, control the first execution device to execute the control instruction.

[0094] According to an embodiment of the present invention, as Figure 4As shown, the image acquisition unit 211 is configured to take a picture of the target area every preset time (such as 0.5 seconds, 1 second, 2 seconds, etc.) to obtain an image of the target area at the corresponding time. In some embodiments, the image acquisition unit 211 can also be configured to continuously record the target area and intercept pictures from the recording at a preset time interval (or number of frames) to obtain an image of the target area. The above-mentioned images can be stored and transmitted in picture formats such as YUV, PNG, GIF, JPG, BMP, etc.

[0095] According to an embodiment of the present invention, as Figure 4 As shown, the detection and recognition unit 212 is configured to perform picture change detection on the image. Specifically, the detection and recognition unit 212 can compare the currently acquired image with the previously (such as the previous time node) acquired image. If the pictures of the two images are basically the same, it can be considered that the picture has not changed. If the pictures of the two images are basically different, it can be considered that the picture has changed. The detection and recognition unit 212 is also configured to stop face recognition when it detects that the picture has not changed, and perform human detection on the image when it detects a picture change to determine whether there is a human body in the target area. The detection and recognition unit 212 is also configured to stop face recognition when it detects that there is no human form in the image, and perform face detection on the image when it detects that there is a human form in the image to determine whether there is a face in the target area. The detection and recognition unit 212 is also configured to stop face recognition when it detects that there is no face in the image, and match the detected face with the face information of the pre-registered user when it detects that there is a face in the image. If the match is successful, it is determined that there is a registered user in the image; if the match fails, it is determined that there is no registered user in the image. Among them, performing picture change detection, human detection, and face detection on the image in sequence, and stopping face recognition when it is detected that the picture has not changed, when it is detected that there is no human form in the image, and when it is detected that there is no face in the image, can save the computing power of the detection and recognition unit 212 and reduce the operating power consumption of the detection and recognition unit 212.

[0096] According to an embodiment of the present invention, as Figure 4 As shown, the first intelligent device 210 further includes a face information management unit 214. The face information management unit 214 is coupled to the detection and recognition unit 212, and the face information management unit 214 stores the face information of the registered user. When performing face recognition on the image, the detection and recognition unit 212 can retrieve the face information of the registered user from the face recognition unit.

[0097] According to an embodiment of the present invention, as Figure 4As shown, the intelligent control system 200 further includes an interaction interface 220 and an upper-layer control device 230. Among them, the upper-layer control device 230 communicates with the interaction interface 220 and the first intelligent device 210 respectively. The interaction interface 220 can be provided by a smart phone, or by the first intelligent device 210 or other devices. The present invention is not limited thereto. Figure 5 The interaction flow chart for modifying registered users according to an embodiment of the present invention is shown, as Figure 5 As shown, the interaction interface 220 is configured to provide a graphical user interface, receive modification inputs according to the user's interaction operations on the graphical user interface, and output corresponding modification information. The modification information includes information and photos of adding and deleting registered users. The upper-layer control device 230 is configured to receive the modification information, and according to the modification information, store the photos and output a registration instruction to the first intelligent device 210, or delete the photos and output a deletion instruction to the first intelligent device 210. The first intelligent device 210 is configured to call the corresponding photos from the upper-layer control device 230 according to the registration instruction, and save the face information of the corresponding registered users in the user face information management unit 214 according to the photos; the first intelligent device 210 is further configured to delete the face information of the corresponding registered users in the user face information management unit 214 according to the deletion instruction.

[0098] Specifically, as Figure 4 shown, when adding a registered user is needed, the user can perform relevant operations on the graphical user interface provided by the interaction interface 220 (such as clicking the "Add Registered User" button, uploading or taking a photo); the interaction interface 220 sends modification information (including an instruction to add a registered user and a photo) to the upper-layer control device 230 according to the relevant operations of the user on the graphical user interface; after receiving the modification information, the upper-layer control device 230 stores the photo and outputs a registration instruction to the first intelligent device 210; after receiving the registration instruction, the first intelligent device 210 adds a registered user and requests to call the photo from the upper-layer control device 210; after receiving the request of the first intelligent device 210, the upper-layer control device 210 sends the photo to the first intelligent device 210; after receiving the photo, the first intelligent device 210 performs face detection on the photo, extracts and saves the face information (feature values) in the photo. When deleting a registered user is needed, the user can perform relevant operations on the graphical user interface provided by the interaction interface 220 (such as clicking the "Delete Registered User" button); the interaction interface 220 sends modification information (the modification information includes an instruction to delete the information and photo of the registered user) to the upper-layer control device 230 according to the relevant operations of the user on the graphical user interface; after receiving the modification information, the upper-layer control device 230 deletes the stored photo and outputs a deletion instruction to the first intelligent device 210; after receiving the deletion instruction, the first intelligent device 210 deletes the face information of the stored registered users.

[0099] According to an embodiment of the present invention, as Figure 4 shown, the first intelligent device 210 further includes a gesture command management unit 215. The gesture command management unit 215 is coupled to the control unit 213. The gesture command management unit 215 stores a set of gestures, which may include pictures or feature values of one or more gestures. As Figure 3 shown, the gestures may be, for example, a single-handed heart, confirmation, like, dislike, love you, victory, rock, shooting, flicking, fist, index finger, middle finger, little finger, palm, number 3, number 4, number 6, etc. Figure 3 Only some single-handed gestures are shown in , but the present invention is not limited thereto. The gestures may also be two-handed gestures, such as two-handed heart, etc. The control unit 213 is configured to match the image with the set of gestures, determine whether the registered user has made a gesture in the set of gestures, and determine which gesture in the set of gestures the registered user has made.

[0100] According to an embodiment of the present invention, as Figure 4 shown, the gesture command management unit 215 also stores a set of gesture instructions. The set of gesture instructions may be the corresponding relationship between some gestures in the set of gestures and the control instructions for controlling the first intelligent device 210. For example, the index finger corresponds to starting video recording, the middle finger corresponds to stopping video recording, and dislike corresponds to hibernation. The above examples are only for illustration, and the present invention is not limited thereto. The control unit 213 is configured to determine the corresponding control instruction from the set of gesture instructions when recognizing a preset gesture (i.e., recognizing a gesture in the set of gestures). When the control instruction is determined from the set of gesture instructions, the intelligent device corresponding to the control instruction is the first intelligent device 210.

[0101] According to an embodiment of the present invention, as Figure 4 shown, the control system further includes one or more second intelligent devices 240 (three are taken as examples in the present invention). The second intelligent devices 240 communicate with the upper-layer control device 230. The control unit 213 is further configured to upload the recognized gesture to the upper-layer control device 230 when the corresponding control instruction cannot be determined from the set of gesture instructions. The upper-layer control device 230 is configured to determine the corresponding control instruction according to the result of gesture recognition. Specifically, the upper-layer control device 230 may be a cloud server or a gateway. In the upper-layer control device 230, a correspondence table between another part of the gestures in the set of gestures and the control instructions for controlling the one or more second intelligent devices 240 may be stored. The upper-layer control device 230 is configured to determine the control instruction corresponding to the gesture and the second intelligent device 240 corresponding to the control instruction according to the correspondence table. Through the above settings, the registered user can control more intelligent devices through gestures, not only the first intelligent device 210.

[0102] According to an embodiment of the present invention, as Figure 4 shown, the first intelligent device 210 further includes a feedback unit 216. The feedback unit 216 is coupled to the control unit 213. The feedback unit 216 is configured to give corresponding prompts according to the execution result of the control instruction of the first intelligent device 210. The second intelligent device 240 is configured to output feedback information according to the execution result of the control command. The feedback information can be, for example, execution success and execution failure. The upper-layer control device 230 is configured to receive the feedback information output by the second intelligent device 240 and forward the feedback information to the first intelligent device 210. The feedback unit 216 of the first intelligent device 210 is further configured to give corresponding prompts according to the feedback information. Specifically, the feedback unit 216 can be a display screen. When giving the prompt, the display screen can display gestures representing execution success or failure. In some embodiments, the feedback unit 216 can also be an indicator light. When giving the prompt, the indicator light can flash according to a preset rule (such as Morse code) to indicate execution success or failure.

[0103] Compared with the prior art, the embodiments of the present invention provide a control method and control system for intelligent devices. By introducing face recognition and gesture recognition, registered users can control corresponding intelligent devices by making gestures, providing an intuitive and personalized way for people (especially the deaf-mute population) to interact with intelligent devices, enabling people to control intelligent devices more easily, fully enjoying the functions of intelligent devices, and improving the convenience of controlling intelligent devices. Among them, face recognition can identify the people appearing in the target area to ensure that only registered users can control the intelligent devices, improving the security of control.

[0104] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements on some of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A control method for an intelligent device, comprising: Obtaining an image of a target area; Performing face recognition on the image to determine whether there is a registered user in the image; When it is determined that there is a registered user in the image, performing gesture recognition on the image according to a preset set of gestures; Determining a corresponding control instruction from a preset set of gesture instructions according to the result of the gesture recognition; Determining the intelligent device corresponding to the control instruction; And Controlling the corresponding intelligent device to execute the control instruction.

2. The control method according to claim 1, wherein The step of performing face recognition on the image to determine whether there is a registered user in the image comprises: Performing frame change detection on the image; When frame change is detected, performing human detection on the image; When a human form is detected in the image, performing face detection on the image; When a face is detected, matching the detected face with the face information of a pre-stored registered user, If the match is successful, determining that there is a registered user in the image; If the match fails, determining that there is no registered user in the image.

3. The control method according to claim 1, wherein The step of determining a corresponding control instruction from a preset set of gesture instructions according to the result of the gesture recognition comprises: When a preset gesture is recognized, determining a corresponding control instruction from a preset set of gesture instructions.

4. The control method according to claim 1, wherein When a corresponding control instruction cannot be determined from a preset set of gesture instructions, uploading the result of the gesture recognition to the upper-layer control device, and determining a corresponding control instruction through the upper-layer control device.

5. The control method according to claim 4, wherein, The control method is implemented by a camera device; The step of controlling the corresponding intelligent device to execute the control instruction comprises: When the corresponding intelligent device is the camera device, controlling the camera device to execute the control instruction; When the corresponding intelligent device is not the camera device, controlling the corresponding intelligent device to execute the control instruction through the upper-layer control device.

6. The control method according to claim 5, wherein, The intelligent device is configured to output feedback information according to the execution result of the control instruction; The control method further comprises: receiving the feedback information and making corresponding prompts in the feedback unit of the camera device according to the feedback information.

7. The control method according to any one of claims 4-6, wherein, The upper-layer control device is a cloud server or a gateway.

8. An intelligent control system, comprising: A first intelligent device, the first intelligent device comprising: An image acquisition unit configured to obtain an image of a target area; A detection and recognition unit configured to perform face recognition on the image to determine whether there is a registered user in the image, and when it is determined that there is a registered user in the image, performing gesture recognition on the image according to a preset set of gestures; and A control unit configured to determine a corresponding control instruction from a preset set of gesture instructions according to the result of the gesture recognition, further configured to determine the intelligent device corresponding to the control instruction, and when the corresponding intelligent device is the first intelligent device, controlling the first intelligent device to execute the control instruction.

9. The intelligent control system according to claim 8, wherein, The detection and recognition unit is configured to: Perform frame change detection on the image; When frame change is detected, perform human detection on the image; When a human form is detected in the image, face detection is performed on the image; When a face is detected, the detected face is matched with the face information of the pre-stored registered users; If the match is successful, it is determined that there is a registered user in the image; If the match fails, it is determined that there is no registered user in the image.

10. The intelligent control system according to claim 9, wherein, The first intelligent device further includes a face information management unit, which is coupled to the detection and recognition unit, and the face information management unit stores the face information of the registered users.

11. The intelligent control system according to claim 10 further includes: An interaction interface configured to provide a graphical user interface, receive modification inputs according to the user's interaction operations on the graphical user interface, and output corresponding modification information, where the modification inputs include adding and deleting information of registered users and their photos; An upper-layer control device that communicates with the interaction interface and the first intelligent device respectively, and the upper-layer control device is configured to store the photo and output a registration instruction to the first intelligent device according to the modification information, or delete the photo and output a deletion instruction to the first intelligent device; The first intelligent device is configured to call the corresponding photo from the upper-layer control device according to the registration instruction, and save the face information of the corresponding registered user in the user face information management unit according to the photo; The first intelligent device is further configured to delete the face information of the corresponding registered user in the user face information management unit according to the deletion instruction.

12. The intelligent control system according to claim 8, wherein, The control unit is configured to determine the corresponding control instruction from a preset set of gesture instructions when a preset gesture is recognized.

13. The intelligent control system according to claim 8 further includes an upper-layer control device, and the first intelligent device communicates with the upper-layer control device; The control unit is configured to: when the corresponding control instruction cannot be determined from the preset set of gesture instructions, upload the result of the gesture recognition to the upper-layer control device; The upper-layer control device is configured to determine the corresponding control instruction according to the result of the gesture recognition.

14. The intelligent control system according to claim 13 further includes one or more second intelligent devices, and the second intelligent devices communicate with the upper-layer control device; The upper-layer control device is configured to determine the second intelligent device corresponding to the control instruction and control the second intelligent device to execute the control instruction.

15. The intelligent control system according to claim 12, wherein, The first intelligent device further includes a gesture command management unit, which is coupled to the control unit, and the gesture command management unit stores a set of gestures and a set of gesture instructions.

16. The intelligent control system according to claim 14, wherein, The second intelligent device is configured to output feedback information according to the execution result of the control instruction; The upper-layer control device is configured to receive the feedback information output by the second intelligent device and forward the feedback information to the first intelligent device; The first intelligent device further includes a feedback unit, which is configured to give corresponding prompts according to the feedback information.

17. The intelligent control system according to any one of claims 13-16, wherein, The upper-layer control device is a cloud or a gateway.

Images