Device control method, device, electronic device and storage medium

Abstract

The embodiments of the present application disclose a device control method, apparatus, vehicle and storage medium, and relate to the field of data processing technology. The method includes: identifying the body posture of the target object in the video frame of the real-time video stream; if at least two target body postures are identified from the continuous video frames of a preset number of frames, triggering the acquisition of a control strategy matching the at least two target body postures, so as to send a target control instruction to the corresponding target device based on the control strategy. The present application can reduce the occurrence of inaccurate control strategies caused by accidentally touching the target posture, improve the accuracy of the control strategy, and thus improve the accuracy of device control.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a device control method, apparatus, electronic device and storage medium. Background Art

[0002] With the advancement of technology, cameras are steadily developing in an increasingly intelligent direction. Functions such as face recognition, human shape detection, crying detection, and pet recognition have become standard features of cameras. These functions can be used to filter out effective data from massive video data.

[0003] In the field of device control, a camera can be used to capture a video stream of a human body, identify the human body shape from the video stream, obtain control instructions corresponding to the human body shape, and then control the device through the control instructions. However, this method of device control is prone to cause the device to be falsely triggered and controlled, and the accuracy of device control is low. Summary of the invention

[0004] The present application proposes a device control method, apparatus, electronic device and storage medium for improving the accuracy of device control.

[0005] In a first aspect, an embodiment of the present application provides a device control method, the method comprising: identifying the body posture of a target object in a video frame of a real-time video stream; if at least two target body postures are identified from a preset number of consecutive video frames, triggering acquisition of a control strategy matching the at least two target body postures, so as to send a target control instruction to the corresponding target device based on the control strategy.

[0006] In the second aspect, an embodiment of the present application provides a device control device, which includes: an identification module for identifying the body posture of a target object in a video frame of a real-time video stream; a control module for triggering the acquisition of a control strategy matching the at least two target body postures if at least two target body postures are identified from a preset number of consecutive video frames, so as to send a target control instruction to the corresponding target device based on the control strategy.

[0007] Optionally, the control module is further configured to trigger acquisition of a control strategy matching the at least two first target body postures if at least two first target body postures are identified from any video frame of the preset number of consecutive video frames.

[0008] Optionally, the control module is also used to trigger acquisition of a control strategy that matches the combination of the at least two first target body postures and the at least one second target body posture if at least two first target body postures are identified from a first video frame in the preset number of consecutive video frames, and at least one second target body posture is identified from a second video frame in the preset number of consecutive video frames.

[0009] Optionally, the control module is also used to trigger acquisition of a control strategy that matches the combination of the at least one third target body posture and the at least one fourth target body posture if at least one third target body posture is identified from a third video frame in the preset number of consecutive video frames, and at least one fourth target body posture is identified from a fourth video frame in the preset number of consecutive video frames.

[0010] Optionally, the control module is also used to trigger acquisition of a control strategy that matches the at least two target body postures if a specified body posture is identified in the fifth video frame in the real-time video stream, and at least one target body posture is identified in a preset number of continuous frames of video captured after the fifth video frame.

[0011] Optionally, the device also includes a candidate body posture determination module, which is used to determine at least one candidate body posture from the multiple body postures based on the proportion of each body posture in the one video frame if multiple body postures are identified in a frame of the real-time video stream; and judge whether there is a specified body posture or whether there is at least one target body posture among the at least one candidate body posture.

[0012] Optionally, the candidate body posture determination module is also used to, if multiple body postures are identified in a frame of video in the real-time video stream, determine one body posture with the largest proportion from the multiple body postures as a candidate body posture based on the proportion of each body posture in the one frame of video.

[0013] Optionally, the candidate body posture determination module is also used to select each body posture in the one frame of image with an anchor frame of a preset shape according to the position of each body posture among the multiple body postures in the one frame of video and the preset feature value corresponding to each body posture among the multiple body postures; and determine the body posture selected by at least one anchor frame with the largest area among the anchor frames corresponding to the body postures as a candidate body posture.

[0014] Optionally, the candidate body posture determination module is also used to determine the image area occupied by each of the multiple body postures in the one video frame based on all pixel points contained in each of the multiple body postures; and determine at least one body posture with the largest image area occupied by each of the multiple body postures as a candidate body posture.

[0015] Optionally, the device also includes: an acquisition module, used to acquire at least two preset body postures and at least one preset control strategy of at least one preset object, the at least two preset body postures including at least two preset standard body postures; establish and store a mapping relationship between the at least two preset standard body postures of at least one preset object acquired and the at least one preset control strategy; a control module, further used to determine that the at least two body postures are target body postures if at least two body postures are identified as preset standard body postures from a preset number of continuous video frames. The control module triggers the acquisition of a control strategy matching the at least two target body postures according to the mapping relationship.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above method.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, wherein the above method is executed when the program code is executed by a processor.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above method.

[0019] The embodiments of the present application provide a device control method, apparatus, electronic device and storage medium, which identify at least two target body postures from a preset number of continuous video frames in a video stream, and determine a control strategy that matches the at least two target body postures. The control strategy is determined based on the at least two target body postures, which reduces the occurrence of inaccurate control strategies caused by accidentally touching the target body postures, improves the accuracy of the control strategy, and thereby improves the accuracy of device control. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is shown that a device control system is provided according to an embodiment of the present application;

[0022] Figure 2 A flow chart of a device control method proposed in one embodiment of the present application is shown;

[0023] Figure 3 A schematic diagram showing a target body posture in an embodiment of the present application is shown;

[0024] Figure 4 A schematic diagram showing another target body posture in an embodiment of the present application is shown;

[0025] Figure 5 A schematic diagram showing another target body posture in an embodiment of the present application is shown;

[0026] Figure 6 A schematic diagram showing another target body posture in an embodiment of the present application is shown;

[0027] Figure 7 A flow chart of a device control method proposed in another embodiment of the present application is shown;

[0028] Figure 8 A flow chart of a device control method proposed in another embodiment of the present application is shown;

[0029] Fig. 9 A schematic diagram showing a gesture recognition process in an embodiment of the present application is shown;

[0030] Fig.10 A block diagram of a device control apparatus proposed in one embodiment of the present application is shown;

[0031] Fig.11 A structural block diagram of an electronic device for executing a device control method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. According to the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0035] Currently, a human body shape is identified from the video stream, and then a control instruction corresponding to the human body shape is obtained, and then the device is controlled by the control instruction. However, the human body shape identified in the video stream may be a human body shape that is touched by mistake, and it is not the human body shape sent by the user based on demand, which leads to the control instruction obtained according to the human body shape being inaccurate, resulting in low accuracy of device control.

[0036] Therefore, the inventor proposes a device control method, apparatus, electronic device and storage medium. By identifying the body posture of the target object in the video frame of the real-time video stream; if at least two target body postures are identified from the continuous video frames of a preset number of frames, a control strategy matching the at least two target body postures is triggered to send a target control instruction to the corresponding target device based on the control strategy. Since the control strategy is determined based on at least two target postures, the occurrence of inaccurate control strategies due to mis-touch of the target postures is reduced, the accuracy of the control strategy is improved, and the accuracy of device control is thereby improved.

[0037] Figure 1 It is shown that a device control system is provided according to an embodiment of the present application. Figure 1 As shown, the device control system includes a camera 101, a server (or cloud) 102, a terminal 103 and a smart device 104. The camera 101 and each smart device 104 usually need to transmit data to the server 102 or the terminal 103 through a network device 105 (such as a router or a gateway);.

[0038] The camera 101 may be a high-definition camera, an infrared camera, a color camera, etc. The camera is used to capture a real-time video stream of a target object.

[0039] Server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0040] Terminal 103 can be a smart phone, tablet computer, laptop computer, desktop computer, smart home appliance, vehicle-mounted terminal, aircraft, wearable device terminal, virtual reality device and other terminal devices that can play videos. The terminal can run a video playback application or other applications that can call the video playback application (such as instant messaging applications, shopping applications, search applications, game applications, forum applications, map and transportation applications, etc.).

[0041] The smart device 104 may be a smart home device or a smart office device, such as a refrigerator, a printer, a television, an air conditioner, etc.

[0042] As an implementation mode, the camera 101 can shoot a real-time video stream of the target object, and the real-time video stream is sent to the terminal 103 through the server 102. The terminal 103 processes the real-time video stream sent by the camera 101, obtains a control strategy, and obtains a control instruction according to the control strategy. Then the terminal 103 sends the control instruction to the smart device 104 so that the smart device 104 executes the control instruction.

[0043] As another implementation, the camera 101 can capture a real-time video stream of the target object and send the real-time video stream to the server 102. The server 102 processes the real-time video stream sent by the camera 101, obtains a control strategy, and obtains a control instruction based on the control strategy. Then the server 102 sends the control instruction to the smart device 104 so that the smart device 104 executes the control instruction.

[0044] In some implementations, the camera 101 itself can process the captured real-time video stream to obtain a control strategy, and obtain a control instruction based on the control strategy. The camera 101 then sends the control instruction to the smart device 104 so that the smart device 104 executes the control instruction.

[0045] For the convenience of description, in the following embodiments of the present application, the camera 101, the server 102 and the terminal 103 that can execute the solution of the present application are all referred to as electronic devices.

[0046] See also Figure 2 , Figure 2A flow chart of a device control method proposed in one embodiment of the present application is shown. The method can be executed by an electronic device. For example, the electronic device can be Figure 1 The server in Figure 1 The terminal in the embodiment may also have a camera device with a camera function, such as Figure 1 The method may specifically include the following steps:

[0047] S110: Recognize the body posture of the target object in the video frame of the real-time video stream.

[0048] In this embodiment, the target object can be photographed in real time by a camera to obtain a real-time video stream. The electronic device obtains the real-time video stream from the camera and recognizes the shape and posture of the target object in the video frame of the real-time video stream.

[0049] Among them, the target object refers to the object that needs to be identified, and specifically may be a specific object in the video frame that needs to be identified. For example, the target object may be a human body, including adults, minors, and teenagers. The target object may be the entire human body, or a local part of the human body, such as the head, hands, etc.

[0050] It can be understood that the body posture of the target object may refer to the external appearance posture of the target object, which may be the overall external appearance posture or the local external appearance posture of the target object. For example, the body posture may include the whole body posture of the human body and / or the posture of a part of the human body, and the posture of a part of the human body includes the head posture of the human body and / or the limb posture of the human body, and the limb posture of the human body includes the hand gestures of the human body.

[0051] When the body posture is a whole-body posture, one target object corresponds to one body posture; when the body posture is a head posture, one target object corresponds to one body posture; when the body posture is a gesture, one gesture corresponds to one body posture. If a person makes gestures with both hands, then there are two corresponding body postures.

[0052] As an implementation method, each video frame of the real-time video stream may be recognized through a deep learning model (eg, a trained body posture recognition model) to determine the body posture of the target object in each video frame.

[0053] The training process of the deep learning model may include: inputting sample images including body postures into a preset algorithm model for training. During the training process, the predicted value output by the preset algorithm model in each round may be compared with the sample label of the body posture to obtain the corresponding loss, and then the parameters of the preset algorithm model may be adjusted according to the corresponding loss to continuously converge the preset algorithm model; the training is stopped until the conditions are met, thereby obtaining a trained deep learning model.

[0054] The real-time video stream can be a 1080P high-definition video. Each frame of the real-time video stream can be buffered in a buffer, and then the cached current video frame is obtained from the buffer, and the current video frame is converted into a 512×512 resolution video frame to be processed. Then, the shape and posture of the video frame to be processed are recognized through a deep learning model.

[0055] S120: If at least two target body postures are identified from a preset number of consecutive video frames, a control strategy matching the at least two target body postures is triggered to send a target control instruction to a corresponding target device based on the control strategy.

[0056] The preset frame number may be set based on demand and actual scenarios, for example, the preset frame number is 100 or 120.

[0057] The control strategy may refer to a control method or control process for controlling a target device. The control strategy may include a device identifier of the target device and a target control instruction that the target device needs to execute. The target device may be any controllable device, such as a smart home device. The device identifier may be the name or number of the target device, etc., so that the target control instruction is sent to the target device through the device identifier. The target control instruction may be a specific instruction that the target device needs to execute. For example, if the target device is an air conditioner, the target control instruction may be to turn on the air conditioner and set the temperature of the air conditioner to 26 degrees Celsius.

[0058] The target body posture may refer to a body posture in a video frame that matches a preset body posture. The electronic device may store multiple preset body posture pairs input by a preset object, and preset control strategies corresponding to multiple preset body posture pairs, each preset body posture pair includes at least two preset body postures, the preset body posture may refer to a pre-stored preset standard body posture, the preset body posture pair may be stored in the form of a preset combination result, the preset combination result may include each preset body posture in the preset body posture pair and the positional relationship between each preset body posture, and the preset combination result may also include each preset body posture in the preset body posture pair, and the time sequence between each preset body posture.

[0059] It can be understood that the target object in the recognized target shape and posture correspondence and the preset object of the input preset shape and posture can be the same object or different objects. The recognized target shape and posture are used as the basis for obtaining the control strategy, and the face information and shape information of the object itself can not be used as the basis for obtaining the control strategy.

[0060] For each video frame, all preset body postures included in multiple preset body posture pairs are compared with the body posture in the video frame to determine whether there is a preset body posture matching the body posture in the video frame among all the preset body postures included in the multiple preset body posture pairs. If so, it is determined that the body posture in the video frame is the target body posture. If not, it is determined that the body posture in the video frame is not the target body posture, and the next video frame is obtained to continue the above-mentioned target body posture judgment process.

[0061] For example, there are 10 preset body posture pairs corresponding to 10 preset control strategies, each preset body posture pair includes 2 preset body postures, and the 20 preset body postures included in the 10 preset body posture pairs are compared with the body posture in the video frame to determine whether the 20 preset body postures match the body posture in the video frame. When there is a preset body posture among the 20 preset body postures that matches the body posture in the video frame, the body posture in the video frame is determined to be the target body posture.

[0062] It may be possible to capture multiple preset body posture pairs of at least one preset object through a camera, and obtain multiple preset control strategies entered by the preset object, wherein a preset body posture pair includes at least two preset standard body postures; then establish and store a mapping relationship between the obtained multiple preset body posture pairs and the multiple preset control strategies, thereby obtaining the multiple preset body posture pairs mentioned above and the preset control strategies corresponding to the multiple preset body posture pairs.

[0063] The matching of the body posture with the preset body posture may mean that the body posture is the same as the preset body posture. The matching of the body posture with the preset body posture may also mean that the similarity between the body posture and the preset body posture reaches a similarity threshold. The similarity threshold may be 80% or 88%, etc.

[0064] When there is one target object, the target body posture in one video frame refers to the target body posture of the target object; when there are multiple target objects, the target body posture in one video frame refers to the target body posture of at least one target object.

[0065] For example, there are three target objects, and the first frame of the real-time video stream includes the target body posture of one target object, then the target body posture in the video frame is the one target body posture; the tenth frame of the real-time video stream includes the target body postures of three target objects, then the target body postures in the video frame are the three target body postures.

[0066] As an implementation mode, if each video frame in the preset number of consecutive video frames includes at most one target body posture, when at least two video frames in the preset number of consecutive video frames include the target body posture, then the acquisition of control strategies matching the at least two target body postures is triggered. When at least two video frames in the preset number of consecutive video frames do not include the target body posture, the acquisition of control strategies matching the at least two target body postures is not triggered.

[0067] For example, if the preset number of frames is 100, and the first video frame of the continuous 100 video frames includes the target body posture, and the 78th frame includes the target body posture, then the acquisition of the control strategy matching the at least two target body postures is triggered. For another example, if the preset number of frames is 100, and the first video frame of the continuous video frames includes the target body posture, and the 120th frame includes the target body posture, then it is determined that the continuous 100 frames do not include at least two target body postures, and the acquisition of the control strategy matching the at least two target body postures is not triggered.

[0068] As another implementation, if there is a video frame including at least two target body postures in the preset number of consecutive video frames, the acquisition of the control strategy matching the at least two target body postures is triggered. For example, the preset number of frames is 100, and the 56th video frame in the 100 consecutive video frames includes at least two target body postures, which triggers the acquisition of the control strategy matching the at least two target body postures.

[0069] The control strategy matching at least two target body postures may refer to a preset control strategy corresponding to a preset body posture pair matching at least two target body postures. The target device refers to the device pointed to by the preset control strategy matching at least two target body postures, and the target control instruction refers to the instruction required to be executed by the target device in the control strategy matching the at least two target body postures. The control strategy may include a device identifier (such as a device name or device number, etc.) of the controlled device and the specific content of the control instruction.

[0070] After the electronic device obtains the control strategy, it can send the target control instruction in the control strategy to the target device according to the device identification in the control strategy. After receiving the target control instruction, the target device executes the target control instruction, thereby realizing control over the target device.

[0071] As an implementation manner, the matching of the above-mentioned preset body posture pair with at least two target body postures may refer to the matching of a preset combination result corresponding to the preset body posture pair with a combination result of at least two target body postures, wherein the combination result of at least two target body postures may refer to the result obtained by combining at least two target body postures into one image according to their position information in the video frame, and the combination result of at least two target body postures may also be the result obtained by arranging at least two target body postures according to the chronological order of the video frames to which they belong.

[0072] For example, the preset body posture pair includes three body postures A1, B1 and C1. In the preset combination result corresponding to the preset body posture pair, A1 is at the upper left of B1, and B1 is at the upper left of C1. The identified target body postures include three, namely A2, B2 and C2. A2, B2 and C2 are combined into one image according to their position information in the video frame. When it is determined that A1 is the same as A2, B1 is the same as B2, and C1 is the same as C2, and in the combined image, A2 is at the upper left of B2 and B2 is at the upper left of C2, it is determined that the body posture pair matches the three target body postures.

[0073] For another example, the preset body posture pair includes three body postures A3, B3 and C3. In the preset combination result corresponding to the preset body posture pair, A3 is before B3, and B3 is before C3. The identified target body postures include three, namely A4, B4 and C4. A4, B4 and C4 are arranged according to the time sequence of the video frames in which they are located. When it is determined that A3 is the same as A4, B3 is the same as B4, C3 is the same as C4, and A4 is before B4, and B4 is before C4, it is determined that the body posture pair matches the three target body postures.

[0074] In some embodiments, after identifying at least two target body postures from a preset number of consecutive video frames, the at least two target body postures are positioned in a restored image (the restored image is the combination result of the at least two target body postures described above) having the same resolution as the video frame according to the positional relationship of the at least two target body postures in their respective video frames, and then a posture area corresponding to the at least two target body postures is framed from the restored image, and the posture area is compressed into a comparison image with a resolution of 112×112, and then the comparison image is compared with a preset combination result corresponding to a plurality of pre-stored preset body posture pairs.

[0075] Optionally, as an implementation, S120 may include: if at least two first target body postures are identified from any video frame of the preset number of continuous video frames, triggering acquisition of a control strategy matching the at least two first target body postures, so as to send a target control instruction to the corresponding target device based on the control strategy.

[0076] The at least two first target body postures may refer to at least two target body postures in the same video frame, and the at least two first target body postures may be the same or different. For example, the at least two first target body postures may be two gestures respectively made by two hands of the same target object, or the at least two first target body postures may be four gestures respectively made by two hands of two target objects.

[0077] When it is determined that a certain video frame includes at least two first target body postures, a target body posture pair matching the at least two first target body postures is obtained from a plurality of preset body posture pairs, and a preset control strategy corresponding to the target body posture pair is used as a control strategy matching the at least two first target body postures.

[0078] The matching of a preset body posture pair with at least two first target body postures may refer to the matching of a preset combination result corresponding to the preset body posture pair with a video frame corresponding to at least two first target body postures, wherein the matching of the preset combination result with the video frame may refer to the preset combination result and the video frame including the same body posture, and the positional relationship between the body postures is the same.

[0079] For example, the preset body posture pair includes two body postures A3 and B3. In the preset combination result corresponding to the preset body posture pair, A3 is on the left side of B3. The identified first target body postures include two, A4 and B4. In the video frame corresponding to the first target body posture, A4 is on the left side of B4. When it is determined that A3 is the same as A4 and B3 is the same as B4, it is determined that the body posture pair matches the two first target body postures.

[0080] like Figure 3 As shown, Figure 3 The recognition results of the four video frames a, b, c and d in the figure are as follows. The gestures in each video frame are the target body postures. Each video frame includes at least two target body postures. Figure 3 In a, the five-finger gesture is on the left side of the two-finger gesture. Figure 3 In b, the OK gesture of the left hand is on the left side of the OK gesture of the right hand. Figure 3 In c, the two-finger gesture of the right hand is on the far left, the two-finger gesture of the left hand is in the middle, and the OK gesture of the right hand is on the far right. Figure 3In d, the four-finger gestures of the right hand are on the far left and far right, and the four-finger gestures of the left hand are in the middle; at least two gestures in each video frame correspond to a control strategy, for example, Figure 3 The control strategy corresponding to a in is to play the next song. Figure 3 The control strategy corresponding to b in FIG. 1 is to turn on the air conditioner.

[0081] Optionally, S120 may also include: if at least two first target body postures are identified from the first video frame in the preset number of continuous video frames, and at least one second target body posture is identified from the second video frame in the preset number of continuous video frames, then triggering acquisition of a control strategy that matches the combination of the at least two first target body postures and the at least one second target body posture, so as to send a target control instruction to the corresponding target device based on the control strategy.

[0082] The first acquisition time for acquiring the first video frame is earlier than the second acquisition time for acquiring the second video frame.

[0083] The first target body posture may refer to the target body posture of the target object in the first video frame, the second body posture refers to the target body posture of the target object in the second video frame, the first video frame may refer to a video frame including at least two target body postures that appears for the first time in a preset number of consecutive video frames, and the second video frame may refer to a video frame including at least one target body posture that appears for the first time after the first video frame.

[0084] At least two first target body postures and the at least one second target body posture can be combined to obtain a first body posture pair to be identified, and a target body posture pair matching the first body posture pair to be identified can be determined from a plurality of stored preset body posture pairs, and then a preset control strategy corresponding to the target body posture pair is obtained as a control strategy matching the combination of the at least two first target body postures and the at least one second target body posture.

[0085] The matching of the preset body posture pair with the first body posture pair to be identified may be that the preset body posture pair only includes the first preset body posture that matches at least two first target body postures and the second preset body posture that matches at least one second target body posture, and the first preset body posture is arranged in front of the second preset body posture. The matching of the first preset body posture with at least two first target body postures may mean that the body posture included in the at least two first target body postures is the same as the body posture included in the first preset body posture, and the positional relationship of each body posture in the at least two first target body postures is the same as the positional relationship of each body posture in the first preset body posture. The matching of the second preset body posture with at least one second target body posture may mean that the body posture included in the at least one second target body posture is the same as the body posture included in the second preset body posture, and the positional relationship of each body posture in the at least one second target body posture is the same as the positional relationship of each body posture in the second preset body posture.

[0086] like Figure 4 As shown, Figure 4 In a, there are two first target body postures, wherein the two first target body postures are a left hand five-finger gesture and a right hand two-finger gesture, and the left hand five-finger gesture is on the left of the right hand two-finger gesture. Figure 4 b in the figure is a second target body posture, and the second target body posture is the right hand OK gesture. Figure 4 a and b in are combined as a combined result, and Figure 4 The order of a and b in is a first, and the combination result corresponds to a control strategy, for example, turning on the TV.

[0087] Optionally, S120 may also include: if at least one third target body posture is identified from the third video frame in the preset number of consecutive video frames, and at least one fourth target body posture is identified from the fourth video frame in the preset number of consecutive video frames, then triggering acquisition of a control strategy that matches the combination of the at least one third target body posture and the at least one fourth target body posture.

[0088] The third target body posture may refer to the target body posture of the target object in the third video frame, the fourth body posture refers to the target body posture of the target object in the fourth video frame, the third video frame may refer to the first video frame including at least one target body posture among a preset number of consecutive video frames, and the fourth video frame may refer to the first video frame including at least one target body posture after the third video frame.

[0089] At least one third target body posture and the at least one fourth target body posture can be combined to obtain a second body posture pair to be identified, and a target body posture pair matching the second body posture pair to be identified can be determined from a plurality of stored preset body posture pairs, and then a preset control strategy corresponding to the target body posture pair is obtained as a control strategy matching the combination of the at least one third target body posture and the at least one fourth target body posture.

[0090] The matching of the preset body posture pair with the second body posture pair to be identified may be that the preset body posture pair only includes a third preset body posture that matches at least one third target body posture and a fourth preset body posture that matches at least one fourth target body posture, and the third preset body posture is arranged in front of the fourth preset body posture. The matching of the third preset body posture with at least one third target body posture may mean that the body posture included in at least one third target body posture is the same as the body posture included in the third preset body posture, and the positional relationship of each body posture in at least one third target body posture is the same as the positional relationship of each body posture in the third preset body posture. The matching of the fourth preset body posture with at least one fourth target body posture may mean that the body posture included in at least one fourth target body posture is the same as the body posture included in the fourth preset body posture, and the positional relationship of each body posture in at least one fourth target body posture is the same as the positional relationship of each body posture in the fourth preset body posture.

[0091] like Figure 5 As shown, Figure 5 a in the figure is a third target body posture, and the third target body posture is the right hand OK gesture. Figure 5 b in the figure are two fourth target body postures, the two fourth target body postures are respectively a left hand OK gesture and a right hand OK gesture, and the left hand OK gesture is on the left side of the right hand OK gesture; Figure 5 a and b in are combined as a combined result, and Figure 5 The order of a and b in is a first, and the combination result corresponds to a control strategy, for example, turning on the bedroom light.

[0092] Optionally, S120 may also include: if a specified body posture is identified in the fifth video frame in the real-time video stream, and at least one target body posture is identified in a preset number of continuous frames of video captured after the fifth video frame, triggering acquisition of a control strategy that matches the at least two target body postures.

[0093] The fifth video frame may refer to a video frame in which a specified body posture of the target object exists, and the specified body posture may be any body posture. The specified body posture may be a body posture set by a user for triggering, and the specified body posture may be one or more. When any one of the specified body postures exists in the video frame, the video frame is determined to be the fifth video frame.

[0094] After the designated body posture is recognized in the fifth video frame, the designated body posture is used as a target body posture, and if at least one target body posture is recognized in the preset number of frames of continuous video collected after the fifth video frame, the designated body posture and at least one subsequent target body posture are used as at least two target body postures. If at least one target body posture is not recognized in the preset number of frames of continuous video collected after the fifth video frame, the fifth video frame including the designated body posture is re-recognized.

[0095] The preset body posture pair matches the combination result of the at least two target body postures, and the preset body posture pair only includes the fifth preset body posture that matches the specified body posture and the sixth preset body posture that matches the at least one target body posture, and the fifth preset body posture is arranged in front of the sixth preset body posture. The fifth preset body posture matches the specified body posture, and the body posture included in the specified body posture is the same as the body posture included in the fifth preset body posture, and the positional relationship of each body posture in the specified body posture is the same as the positional relationship of each body posture in the fifth preset body posture. The sixth preset body posture matches at least one target body posture, and the body posture included in at least one target body posture is the same as the body posture included in the sixth preset body posture, and the positional relationship of each body posture in at least one target body posture is the same as the positional relationship of each body posture in the sixth preset body posture.

[0096] like Figure 6 As shown, Figure 6 a in the formula is the specified body posture, which includes 1 gesture. The specified body posture is the right hand OK gesture. Figure 6 b in Figure 6 The target body posture corresponding to a in the figure is: Figure 6 b in the figure includes three target body postures, namely, a right hand four-finger gesture, a left hand four-finger gesture, and a right hand four-finger gesture, wherein the left hand four-finger gesture is between the two right hand four-finger gestures; Figure 6 a and b in are combined as a combined result, and Figure 6 The order of a and b in is a first, and the combination result corresponds to a control strategy, for example, turning on a humidifier.

[0097] In this embodiment, at least two target body postures are identified from a preset number of consecutive video frames in the video stream, and a control strategy matching the at least two target body postures is determined. The control strategy is determined based on the at least two target body postures, thereby reducing the occurrence of inaccurate control strategies caused by accidentally touching the target body postures, improving the accuracy of the control strategy, and thereby improving the accuracy of device control.

[0098] At the same time, if at least two first target body postures are identified from the first video frame in the continuous video frames of the preset number of frames, and at least one second target body posture is identified from the second video frame in the continuous video frames of the preset number of frames, then a control strategy that matches the combination of the at least two first target body postures and the at least one second target body posture is triggered to be acquired, and the control strategy is determined by multiple target body postures in the first video frame and the second video frame, thereby further reducing the occurrence of inaccurate control strategy determined due to accidental touching of the target posture, thereby further improving the accuracy of the control strategy and the accuracy of device control.

[0099] A specified body posture can also be identified in the fifth video frame in the real-time video stream, and at least one target body posture can be identified in a preset number of continuous video frames captured after the fifth video frame, thereby triggering the acquisition of a control strategy that matches the at least two target body postures. By using the specified video frame as the basis for triggering the acquisition of the control strategy, the occurrence of inaccurate control strategies caused by accidentally touching the target posture is further reduced, thereby further improving the accuracy of the control strategy and the accuracy of device control.

[0100] See also Figure 7 , Figure 7 A flowchart of a device control method proposed in another embodiment of the present application is shown. The method can be executed by an electronic device. For example, the electronic device can be Figure 1 The server in Figure 1 The terminal in the embodiment may also have a camera device with a camera function, such as Figure 1 The method may specifically include the following steps:

[0101] S210: Identify the body posture of the target object in the video frame of the real-time video stream.

[0102] The description of S210 refers to the description of S110 above, and will not be repeated here.

[0103] S220: If a plurality of body postures are identified in a video frame of the real-time video stream, at least one candidate body posture is determined from the plurality of body postures according to the proportion of each body posture in the video frame.

[0104] When multiple body postures are recognized in a video frame of the real-time video stream, the proportion of each body posture in the video frame is determined, and a body posture with the largest proportion is determined from the multiple body postures as a candidate body posture.

[0105] As an implementation method, each body posture can be framed in the one frame of image with an anchor frame of a preset shape according to the position of each body posture among the multiple body postures in the one frame of video and the preset feature value corresponding to each body posture among the multiple body postures; and the body posture framed by at least one anchor frame with the largest area among the anchor frames corresponding to the body postures is determined as a candidate body posture.

[0106] The user can set a preset feature value for each body posture based on the needs. The preset feature value of the more important body posture can be larger, and the preset feature value of the less important body posture can be smaller. The size of the preset feature value determines the size of the anchor box area. For body postures of the same size, the larger the preset feature value, the larger the anchor box obtained.

[0107] The size of the anchor frame can be determined according to the preset feature value of the body posture, and the position of the anchor frame can be determined according to the position of the body posture. Then, the area within each anchor frame is calculated, and the body posture selected by at least one anchor frame with the largest area is determined as the candidate body posture.

[0108] The anchor frame of each body posture is determined by the preset feature value of each body posture, and the candidate body posture is determined by the size of the anchor frame area, filtering out unimportant body postures with small anchor frame areas in the video frame, reducing the interference of unimportant body postures with small anchor frame areas in the video frame, and improving the accuracy of body posture recognition. At the same time, it is not necessary to recognize all body postures, which improves the recognition efficiency of body postures.

[0109] As another embodiment, the image area occupied by each of the multiple body postures in the one video frame can be determined based on all pixel points contained in each of the multiple body postures; and among the image areas occupied by the body postures, at least one body posture with the largest image area occupied is determined as a candidate body posture.

[0110] The area composed of all pixels contained in each body posture in the video frame can be used as the image area, and the area composed of pixels not contained in the body posture in the video frame can be used as the non-image area. The area of ​​each image area is calculated, and at least one body posture that occupies the largest image area is determined as a candidate body posture.

[0111] S230: Determine whether there is a specified body posture or at least one target body posture in the at least one candidate body posture.

[0112] For each video frame, it is determined whether there is a designated body posture or at least one target body posture in at least one candidate body posture determined in the video frame. If there is, it is determined that the designated body posture or at least one target body posture is recognized, and S240 is executed. If not, it is determined that the designated body posture and at least one target body posture are not recognized, and the above steps S210-S230 are continued for the next video frame.

[0113] S240: If at least two target body postures are identified from a preset number of consecutive video frames, a control strategy matching the at least two target body postures is triggered to send a target control instruction to a corresponding target device based on the control strategy.

[0114] The description of S240 refers to the description of S120 above and will not be repeated here.

[0115] In this embodiment, when a video frame includes multiple body postures, at least one candidate body posture is determined, and unimportant body postures that account for a small proportion in the video frame are filtered out, thereby reducing the interference of unimportant body postures that account for a small proportion in the video frame and improving the accuracy of body posture recognition. At the same time, it is not necessary to recognize all body postures, thereby improving the recognition efficiency of body postures.

[0116] See also Figure 8 , Figure 8 A flowchart of a device control method proposed in another embodiment of the present application is shown. The method can be executed by an electronic device. For example, the electronic device can be Figure 1 The server in Figure 1 The terminal in the embodiment may also have a camera device with a camera function, such as Figure 1 The method may specifically include the following steps:

[0117] S310: Acquire at least two preset body postures and at least one preset control strategy of at least one preset object, wherein the at least two preset body postures include at least two preset standard body postures.

[0118] The preset object may be an object for which a preset body posture is recorded. The preset object may be the same as the target object, or may be different from the target object. The preset body posture may refer to a preset standard body posture recorded by the preset object, and at least two preset standard body postures may be used as a preset body posture pair. The description of the preset body posture pair is referred to the above description and will not be repeated here.

[0119] S320: Establish and store a mapping relationship between at least two preset standard body postures of the at least one preset object obtained and the at least one preset control strategy.

[0120] A mapping relationship is established between at least two preset standard body postures and at least one preset control strategy, so as to index the corresponding preset control strategy through the at least two preset standard body postures and the mapping relationship.

[0121] As an implementation method, at least two preset standard body postures can be combined into a preset body posture pair, and then a mapping relationship between at least one preset body posture pair and at least one preset control strategy can be established and stored, so as to index the corresponding preset control strategy through the mapping relationship and the preset body posture pair.

[0122] S330: Identify the body posture of the target object in the video frame of the real-time video stream.

[0123] The description of S330 refers to the description of S110 above and will not be repeated here.

[0124] S340: If at least two body postures are identified as preset standard body postures from a preset number of continuous video frames, it is determined that the at least two body postures are target body postures, and a control strategy matching the at least two target body postures is acquired according to the mapping relationship, so as to send a target control instruction to the corresponding target device based on the control strategy.

[0125] When a body posture in a video frame is identified as one of at least two preset standard body postures that have been stored, the body posture in the video frame is determined to be a target body posture. Similarly, if at least two body postures are identified as preset standard body postures from a preset number of consecutive video frames, it is determined that at least two body postures are both target body postures.

[0126] Among them, the step of triggering to obtain a control strategy matching the at least two target body postures according to the mapping relationship and sending a target control instruction to the corresponding target device based on the control strategy refers to the description of S120 above and will not be repeated here.

[0127] In this embodiment, a mapping relationship between at least two preset standard body postures of at least one preset object and at least one preset control strategy is established and stored, so that the corresponding control instructions can be directly indexed through the mapping relationship, thereby improving the speed of acquiring the control instructions and thus improving the control efficiency of the device.

[0128] In order to more conveniently understand the solution of the present application, the device control method of the present application will be explained in conjunction with a specific application scenario. In this scenario, the body posture is a hand gesture, the resolution of the camera is 1080P (1920×1080), the camera itself serves as the main body for executing the device control method, and the smart device is a refrigerator.

[0129] like Fig. 9 As shown, firstly, the gesture recognition algorithm of the camera is started (that is, the device control method of the present application. In this scenario, the body posture required to be recognized by the device control method is a gesture), and the camera captures the video stream of the target object in real time.

[0130] The camera extracts the latest frame of YUV format ("Y" represents brightness, that is, grayscale value, and "U" and "V" represent chrominance) from the video stream, and stores the video frame with a resolution of 1080P in the buffer.

[0131] The camera compresses the video frames in the buffer into video frames to be processed with a resolution of 512×512, matches the preprocessed video frames with the algorithm model (that is, the preset standard body posture pre-stored in the above embodiment, which is the preset standard gesture in this scenario), and determines whether the preset standard gesture that matches the video frame to be processed is matched.

[0132] If no match is found, a new current frame is extracted from the video stream and the above matching steps are performed again.

[0133] If a match is found, the gesture in the video frame to be processed is used as the target gesture, and the gesture in the preprocessed image is used as the target gesture. If there are at least two target gestures, the at least two target gestures are inversely positioned to the coordinates of the original image of 1080P in YUV format to obtain a restored image. The gesture area on the restored image is then compressed into a gesture image with a resolution of 112×112. The gesture image is then compared with multiple pre-stored gesture combinations (the gesture combination is the preset body posture pair in the above embodiment).

[0134] Determine whether there is a gesture combination matching the gesture image in the pre-stored gesture combinations; if so, obtain the control strategy corresponding to the gesture combination matching the gesture image (the control strategy corresponding to the gesture combination is that of the preset body posture pair in the above embodiment); if not, re-extract a new current frame from the video stream and re-execute the above steps.

[0135] The control strategy obtained is "set the refrigerator's cold storage compartment temperature to 0 degrees Celsius". According to the control strategy, the camera obtains the control instruction "set the cold storage compartment temperature to 0 degrees Celsius". The camera then sends the control instruction to the refrigerator, and the refrigerator executes the control instruction "set the cold storage compartment temperature to 0 degrees Celsius".

[0136] See also Fig.10 , Fig.10 A block diagram of a device control apparatus proposed in an embodiment of the present application is shown, wherein the apparatus 1100 includes:

[0137] The recognition module 1110 is used to recognize the body posture of the target object in the video frame of the real-time video stream;

[0138] The control module 1120 is used to trigger the acquisition of a control strategy matching the at least two target body postures if at least two target body postures are identified from a preset number of consecutive video frames, so as to send a target control instruction to the corresponding target device based on the control strategy.

[0139] Optionally, the control module 1120 is further configured to trigger acquisition of a control strategy matching the at least two first target body postures if at least two first target body postures are identified from any video frame of the preset number of consecutive video frames.

[0140] Optionally, the control module 1120 is also used to trigger acquisition of a control strategy that matches the combination of the at least two first target body postures and the at least one second target body posture if at least two first target body postures are identified from the first video frame in the preset number of consecutive video frames, and at least one second target body posture is identified from the second video frame in the preset number of consecutive video frames.

[0141] Optionally, the control module 1120 is further used to trigger acquisition of a control strategy that matches the combination of the at least one third target body posture and the at least one fourth target body posture if at least one third target body posture is identified from the third video frame in the preset number of consecutive video frames, and at least one fourth target body posture is identified from the fourth video frame in the preset number of consecutive video frames.

[0142] Optionally, the control module 1120 is also used to trigger acquisition of a control strategy that matches the at least two target body postures if a specified body posture is identified in the fifth video frame in the real-time video stream, and at least one target body posture is identified in a preset number of continuous frames of video captured after the fifth video frame.

[0143] Optionally, the device also includes a candidate body posture determination module, which is used to determine at least one candidate body posture from the multiple body postures based on the proportion of each body posture in the one video frame if multiple body postures are identified in a frame of the real-time video stream; and judge whether there is a specified body posture or whether there is at least one target body posture among the at least one candidate body posture.

[0144] Optionally, the candidate body posture determination module is also used to, if multiple body postures are identified in a frame of video in the real-time video stream, determine one body posture with the largest proportion from the multiple body postures as a candidate body posture based on the proportion of each body posture in the one frame of video.

[0145] Optionally, the candidate body posture determination module is also used to select each body posture in the one frame of image with an anchor frame of a preset shape according to the position of each body posture among the multiple body postures in the one frame of video and the preset feature value corresponding to each body posture among the multiple body postures; and determine the body posture selected by at least one anchor frame with the largest area among the anchor frames corresponding to the body postures as a candidate body posture.

[0146] Optionally, the candidate body posture determination module is also used to determine the image area occupied by each of the multiple body postures in the one video frame based on all pixel points contained in each of the multiple body postures; and determine at least one body posture with the largest image area occupied by each of the multiple body postures as a candidate body posture.

[0147] Optionally, the device also includes: an acquisition module, used to acquire at least two preset body postures and at least one preset control strategy of at least one preset object, the at least two preset body postures including at least two preset standard body postures; establish and store a mapping relationship between the at least two preset standard body postures of at least one preset object acquired and the at least one preset control strategy; a control module, further used to determine that the at least two body postures are target body postures if at least two body postures are identified as preset standard body postures from a preset number of continuous video frames. The control module triggers the acquisition of a control strategy matching the at least two target body postures according to the mapping relationship.

[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0149] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0150] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0151] Fig.11 The structure block diagram of the electronic device used to execute the device control method according to the embodiment of the present application is shown. It should be noted that: Fig.11 The computer system 1200 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0152] like Fig.11 As shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1202 or the program loaded from the storage part 1208 to the random access memory (RAM) 1203, such as executing the method in the above embodiment. In RAM 1203, various programs and data required for system operation are also stored. CPU 1201, ROM 1202 and RAM 1203 are connected to each other through bus 1204. Input / output (I / O) interface 1205 is also connected to bus 1204.

[0153] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed so that a computer program read therefrom is installed into the storage section 1208 as needed.

[0154] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1209, and / or installed from a removable medium 1211. When the computer program is executed by a central processing unit (CPU) 1201, various functions defined in the system of the present application are executed.

[0155] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0156] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0157] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0158] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries computer-readable instructions, and when the computer-readable storage instructions are executed by a processor, the method in any of the above embodiments is implemented.

[0159] According to one aspect of the present application, an electronic device is also provided, which includes: a processor; a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, the method in any of the above embodiments is implemented.

[0160] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method in any of the above embodiments.

[0161] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0162] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0163] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. It should be understood that the present application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A device control method, characterized in that: The method comprises: Recognize the shape and posture of the target object in the video frame of the real-time video stream; If multiple body postures are identified in a video frame of the real-time video stream, each body posture is framed in the video frame with an anchor frame of a preset shape according to a position of each body posture in the video frame and a preset feature value corresponding to each body posture in the multiple body postures; Determine the body posture selected by at least one anchor box with the largest area among the anchor boxes corresponding to the body postures as a candidate body posture; Determine whether there is a specified body posture or at least one target body posture in the at least one candidate body posture; If at least two target body postures are identified from a preset number of consecutive video frames, a control strategy matching the at least two target body postures is triggered to obtain a target control instruction to the corresponding target device based on the control strategy.

2. The method according to claim 1, characterized in that If at least two target body postures are identified from the preset number of consecutive video frames, triggering acquisition of a control strategy matching the at least two target body postures includes: If at least two first target body postures are identified from any video frame of the preset number of consecutive video frames, a control strategy matching the at least two first target body postures is acquired.

3. The method according to claim 2, characterized in that If at least two target body postures are identified from the preset number of consecutive video frames, triggering acquisition of a control strategy matching the at least two target body postures includes: If at least two first target body postures are identified from the first video frame of the continuous video frames of the preset number of frames, and at least one second target body posture is identified from the second video frame of the continuous video frames of the preset number of frames, then a control strategy that matches the combination of the at least two first target body postures and the at least one second target body posture is triggered; wherein, the first video frame refers to the video frame including at least two target body postures that appears for the first time in the continuous video frames of the preset number of frames, and the second video frame refers to the video frame including at least one target body posture that appears for the first time after the first video frame; the first target body posture refers to the target body posture identified in the first video frame; and the second target body posture refers to the target body posture identified in the second video frame.

4. The method according to claim 1, characterized in that: If at least two target body postures are identified from the preset number of consecutive video frames, triggering acquisition of a control strategy matching the at least two target body postures includes: If at least one third target body posture is identified from the third video frame among the continuous video frames of the preset number of frames, and at least one fourth target body posture is identified from the fourth video frame among the continuous video frames of the preset number of frames, then a control strategy that matches the combination of the at least one third target body posture and the at least one fourth target body posture is triggered; wherein the third video frame refers to the video frame including at least one target body posture that appears for the first time among the continuous video frames of the preset number of frames, and the fourth video frame refers to the video frame including at least one target body posture that appears for the first time after the third video frame; the third target body posture refers to the target body posture identified in the third video frame; and the fourth target body posture refers to the target body posture identified in the fourth video frame.

5. The method according to claim 1, characterized in that If at least two target body postures are identified from the preset number of consecutive video frames, triggering acquisition of a control strategy matching the at least two target body postures includes: If a specified body posture is identified in the fifth video frame in the real-time video stream, and at least one target body posture is identified in a preset number of consecutive video frames collected after the fifth video frame, then a control strategy matching the at least two target body postures is triggered; wherein the fifth video frame refers to a video frame in which the specified body posture of the target object exists.

6. The method according to claim 1, characterized in that Before the step of identifying the body posture of the target object in the video frame of the real-time video stream, the method further includes: Acquire at least two preset body postures and at least one preset control strategy of at least one preset object, wherein the at least two preset body postures include at least two preset standard body postures; Establishing and storing a mapping relationship between at least two preset standard body postures of at least one preset object acquired and at least one preset control strategy; If at least two target body postures are identified from the preset number of consecutive video frames, triggering acquisition of a control strategy matching the at least two target body postures includes: If at least two body postures are identified as preset standard body postures from a preset number of consecutive video frames, the at least two body postures are determined to be target body postures, and a control strategy matching the at least two target body postures is acquired according to the mapping relationship.

7. The method according to any one of claims 1 to 6, characterized in that The body posture of the target object includes the whole body posture of the human body and / or the body part posture of the human body, the body part posture of the human body includes the head posture of the human body and / or the limb posture of the human body, and the limb posture of the human body includes the hand gestures of the human body.

8. A device control device, characterized in that: The device comprises: A recognition module is used to recognize the body posture of the target object in the video frame of the real-time video stream; if multiple body postures are recognized in a video frame of the real-time video stream, then according to the position of each body posture in the one video frame and the preset feature value corresponding to each body posture in the multiple body postures, each body posture is framed in the one video frame with an anchor frame of a preset shape; among the anchor frames corresponding to the body postures, the body posture framed by at least one anchor frame with the largest area is determined as a candidate body posture; and it is determined whether there is a specified body posture or whether there is at least one target body posture in the at least one candidate body posture; The control module is used to trigger the acquisition of a control strategy matching the at least two target body postures if at least two target body postures are identified from a preset number of consecutive video frames, so as to send a target control instruction to the corresponding target device based on the control strategy.

9. An electronic device, characterized in that: include: a memory for storing one or more computer programs; One or more processors, configured to call and run the one or more computer programs from the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images