Air conditioning control method, device, storage medium and electronic equipment

Abstract

The present invention discloses an air conditioner control method, device, storage medium, and electronic device. The method comprises: detecting the status of an air conditioner control terminal; when the control terminal is in an abnormal state, activating the air conditioner's depth camera to obtain a depth image of a preset area of ​​the air conditioner; analyzing the target motion of a target object in the depth image; generating a control instruction for the air conditioner based on the target motion; and controlling the air conditioner using the control instruction. This invention solves the technical problem of inconvenience in air conditioner control in related technologies.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to an air conditioning control method, device, storage medium and electronic equipment. Background Art

[0002] With the development of technology, smart appliances are becoming widely used in daily life. In addition to traditional remote control, many air conditioners also support various intelligent control functions such as related apps and voice assistants. However, app and voice assistant control still have some inconveniences. For example, app control cannot completely free the hands. While voice assistants can free the hands, they cannot effectively recognize voices in situations such as sudden throat discomfort or noisy family gatherings, resulting in poor control.

[0003] Currently, no effective solution has been found for the above-mentioned problems existing in the related technologies. Summary of the Invention

[0004] Embodiments of the present invention provide an air-conditioning control method, device, storage medium, and electronic device to solve the technical problem of inconvenient air-conditioning control in related technologies.

[0005] According to one aspect of an embodiment of the present application, an air-conditioning control method is provided, including: detecting the status of an air-conditioning control terminal; when the status of the control terminal is abnormal, turning on the depth camera of the air-conditioning to obtain a depth image of a preset area of ​​the air-conditioning; parsing the target action of the target object in the depth image; generating a control instruction of the air-conditioning according to the target action; and controlling the air-conditioning using the control instruction.

[0006] Furthermore, parsing the target action of the target object in the depth image includes: obtaining multiple consecutive frames of depth images; inputting the multiple consecutive frames of depth images into a neural network model, and outputting the target action of the target object in the depth image, wherein the target action includes a static posture and / or a dynamic action, and each target action corresponds to a control instruction.

[0007] Furthermore, parsing the target action of the target object in the depth image includes: identifying the target object in the depth image, obtaining a set of coordinate values ​​of multiple feature points of a preset position of the target object in each frame of the depth image; and determining the target action of the target object based on the set of coordinate values.

[0008] Furthermore, determining the target action of the target object based on the coordinate value set includes: for the coordinate value set in each frame of the depth image, connecting each coordinate point in the coordinate value set according to the morphological characteristics of the preset part to generate a vascular diagram of the preset part; judging whether the vascular diagram conforms to the preset form; if the vascular diagram conforms to the first preset form, determining that the target action of the target object is a first static posture, wherein the first static posture is used to indicate turning on the air conditioner; if the vascular diagram conforms to the second preset form, determining that the target action of the target object is a second static posture, wherein the second static posture is used to indicate turning off the air conditioner.

[0009] Furthermore, determining the target action of the target object based on the coordinate value set also includes: obtaining multiple coordinate value sets in continuous multi-frame depth images; calculating the relative displacement of each feature point of the preset part in the continuous multi-frame depth images based on the multiple coordinate value sets, wherein the relative displacement includes direction and / or distance; and determining the target action of the target object based on the relative displacement.

[0010] Further, determining the target action of the target object based on the relative displacement includes: if the distance that each feature point moves upward in the vertical direction is greater than or equal to a first preset distance, then determining that the target action of the target object is moving upward; if the distance that each feature point moves downward in the vertical direction is greater than or equal to a second preset distance, then determining that the target action of the target object is moving downward; if the distance that each feature point moves to the left in the horizontal direction is greater than or equal to a third preset distance, then determining that the target action of the target object is moving to the left; if the distance that each feature point moves to the right in the horizontal direction is greater than or equal to a fourth preset distance, then determining that the target action of the target object is moving to the right.

[0011] Furthermore, generating control instructions for the air conditioner based on the target action includes: if the target action is moving upward, generating a first control instruction for instructing the air conditioner to increase the temperature; if the target action is moving downward, generating a second control instruction for instructing the air conditioner to cool down; if the target action is moving to the left, generating a third control instruction for instructing the air conditioner to decrease the wind speed; if the target action is moving to the right, generating a fourth control instruction for instructing the air conditioner to increase the wind speed.

[0012] According to another aspect of the embodiment of the present application, an air-conditioning control device is also provided, including: a detection module for detecting the status of an air-conditioning control terminal; a start-up module for turning on the depth camera of the air-conditioning to obtain a depth image of a preset area of ​​the air-conditioning when the status of the control terminal is abnormal; a parsing module for parsing the target action of the target object in the depth image; a generation module for generating a control instruction of the air-conditioning according to the target action; and a control module for controlling the air-conditioning using the control instruction.

[0013] Furthermore, the parsing module includes: a first acquisition unit, which acquires multiple consecutive frames of depth images; an input

[0014] A unit for inputting the continuous multi-frame depth image into a neural network model; an output unit for inputting the continuous multi-frame depth image into a neural network model, and outputting the target image in the depth image

[0015] The target action of the target object includes a static posture and / or a dynamic action, and each target action corresponds to a control instruction.

[0016] Furthermore, the parsing module further includes: an identification unit for identifying the target object in the depth image; a second acquisition unit for acquiring a set of coordinate values ​​of a plurality of feature points of a preset part of the target object in each frame of the depth image; a determination unit for determining the target object according to the set of coordinate values.

[0017] Target action for the target object.

[0018] Furthermore, the determining unit includes: a generating unit for connecting each coordinate point in the coordinate value set according to the morphological features of the preset part for each coordinate value set in the depth image frame, and generating

[0019] a judgment unit for judging whether the venous map conforms to a preset shape; a first determining subunit for determining the target region if the venous map conforms to a first preset shape;

[0020] The target action of the target object is a first static posture, wherein the first static posture is used to indicate turning on the air conditioner; the second determination subunit is used to determine that the target action of the target object is a second static posture if the vascular diagram conforms to a second preset form, wherein the second static posture is used to indicate turning off the air conditioner.

[0021] 0 Further, the determining unit further includes: a third acquiring unit for acquiring a continuous multi-frame depth image

[0022] a plurality of said coordinate value sets; a calculation unit, for calculating the relative displacement of each feature point of the preset part in the said continuous multi-frame depth image according to the plurality of said coordinate value sets, wherein the relative displacement includes direction and / or distance; a third determination subunit, for determining the target action of the target object according to the relative displacement.

[0023] Furthermore, the third determining subunit is configured to determine that the target action of the target object is upward movement if the distance each feature point moves upward in the vertical direction is greater than or equal to a first preset distance; determine that the target action of the target object is downward movement if the distance each feature point moves downward in the vertical direction is greater than or equal to a second preset distance; determine that the target action of the target object is leftward movement if the distance each feature point moves leftward in the horizontal direction is greater than or equal to a third preset distance; and determine that the target action of the target object is rightward movement if the distance each feature point moves rightward in the horizontal direction is greater than or equal to a fourth preset distance.

[0024] Furthermore, the generation module includes: a first generation unit, which is used to generate a first control instruction for instructing the air conditioner to increase the temperature if the target action is to move upward; a second generation unit, which is used to generate a second control instruction for instructing the air conditioner to lower the temperature if the target action is to move downward; a third generation unit, which is used to generate a third control instruction for instructing the air conditioner to decrease the wind speed if the target action is to move to the left; and a fourth generation unit, which is used to generate a fourth control instruction for instructing the air conditioner to increase the wind speed if the target action is to move to the right.

[0025] The present invention detects the status of the air conditioning control terminal. When the air conditioning control terminal is abnormal, the depth camera of the air conditioner is turned on to obtain a depth image of a preset area, analyzes the target action of the target object in the depth image, and generates control instructions for the air conditioner based on the target action to control the air conditioner. When the air conditioning control terminal is abnormal, the depth image is obtained by the depth camera to control the air conditioner, which makes up for the defect that the air conditioner cannot be used due to the abnormal control terminal. The use of motion recognition to control the air conditioner eliminates the need for the user to use the control terminal, freeing the user's hands and making air conditioning control more convenient. Image recognition avoids the influence of environmental noise factors, making air conditioning control more accurate and efficient. In addition, the present invention only turns on the depth camera when the control terminal is unavailable, which also avoids the problem of real-time image capture invading user privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of the present invention;

[0028] Figure 2 is a flow chart of an air conditioning control method according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the air conditioning control process in the implementation scenario of the first embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of data communication in an implementation scenario of the first embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of the workflow of the instruction recognition terminal in the implementation scenario of the first embodiment of the present invention;

[0032] Figure 6 4 is a structural block diagram of an air conditioning control device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the technical personnel in this technical field better understand the present application, the following will be combined with the drawings in the present application embodiment to clearly and completely describe the technical solution in the present application embodiment. Obviously, the described embodiment is only a part of the embodiment of the present application, not all embodiments.

[0034] The embodiments in this application and all other embodiments obtained by persons of ordinary skill in the art without creative work shall fall within the scope of protection of this application.

[0035] In case of conflict, the embodiments of this application and the features therein may be combined with each other.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or priority.

[0037] It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in sequences other than those illustrated or described herein.

[0038] The terms "comprise," "comprises," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, product, or apparatus that comprises a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0039] 5 Example 1

[0040] The method embodiment provided in the first embodiment of the present application can be executed in a server, a computer, a mobile phone, or a similar computing device. Taking running on a computer as an example, Figure 1 This is a hardware structure diagram of a computer according to an embodiment of the present invention. Figure 1 As shown, the computer may include one or more ( Figure 1

[0041] Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and the like) and a memory 104 for storing data. Optionally,

[0042] The computer may further include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer. For example, the computer may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0043] The memory 104 can be used to store computer programs, such as software programs of application software and modules.

[0044] Block, such as a computer program corresponding to an air conditioning control method in an embodiment of the present invention, the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory and may also include

[0045] Non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory remotely located relative to processor 102, which may be connected to the computer via a network. Examples of the aforementioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0046] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by a computer's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0047] In this embodiment, an air conditioning control method is provided. Figure 2 is a flow chart of an air conditioning control method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0048] S1, detect the status of the air conditioning control terminal;

[0049] The air conditioner control terminal is generally an air conditioner remote control, and can also be a smart terminal associated with the air conditioner and used to control the air conditioner, such as a mobile phone, smart speaker, etc.

[0050] S2, when the control terminal is in an abnormal state, turning on the depth camera of the air conditioner to obtain a depth image of a preset area of ​​the air conditioner;

[0051] The air conditioner in this embodiment is provided with a depth camera, wherein the depth camera is generally provided at the upper left or upper right position of the indoor unit so as to capture the image directly in front of the air conditioner.

[0052] When the air conditioner control terminal is in an abnormal state, such as a remote control failure or an unusable mobile phone, this embodiment turns on the air conditioner's depth camera to obtain a depth image of a preset area, and controls the air conditioner by analyzing the depth image information. The preset area is generally a limited area in front of the air conditioner, which can be determined by adjusting the shooting angle of the depth camera.

[0053] S3, analyzing the target action of the target object in the depth image;

[0054] The target object is the user of the air conditioner, which can be any person or a specific person. The target action can include a hand gesture, such as gestures 1, 2, and 3, or waving up or waving down. In some embodiments, the target action can also be a head movement, such as shaking the head or nodding, or a body movement, such as making a specific gesture with the body.

[0055] S4, generating a control instruction for the air conditioner according to the target action;

[0056] Each target action corresponds to an air conditioning control command. For example, raising an index finger represents gesture 1, while raising both the index and ring fingers represents gesture 2, turning the air conditioner on and off, respectively. Waving upwards indicates increasing the temperature when the air conditioner is running, waving downwards indicates cooling, waving left indicates decreasing the air speed, and waving right indicates increasing the air speed.

[0057] In a specific implementation, the user can also customize actions and control instructions corresponding to each action.

[0058] S5, controlling the air conditioner using the control instruction.

[0059] By analyzing the target actions of the target objects in the collected depth image information, corresponding control instructions are generated to control the operation of the air conditioner.

[0060] In the embodiment of the present invention, the air conditioner is equipped with a depth camera. When the air conditioner control terminal is abnormal, the depth camera is used to obtain the user's depth image, and the depth image is recognized to control the air conditioner, which makes up for the defect that the air conditioner cannot be used due to the abnormality of the control terminal. There is no need to use the control terminal, and the air conditioner is controlled by the action of the target object, thereby freeing the user's hands, making the air conditioner control more convenient, and avoiding the problem of real-time image capture infringing on user privacy.

[0061] In one implementation of this embodiment, parsing the target action of the target object in the depth image, S3, includes: acquiring multiple consecutive frames of depth images; inputting the multiple consecutive frames of depth images into a neural network model, and outputting the target action of the target object in the depth image, wherein the target action includes a static posture and / or a dynamic action, and each target action corresponds to a control instruction.

[0062] In this example, a convolutional neural network is used to identify depth images. First, the training data is serialized and packaged into a tfrecords file format for the neural network model to read. During the recognition process, the processed image is input into the trained neural network structure to output the recognition result.

[0063] In one embodiment, analyzing the target motion of the target object in the depth image, S3, includes:

[0064] S31, identifying a target object in the depth image, and obtaining a set of coordinate values ​​of a plurality of feature points of a preset part of the target object in each frame of the depth image;

[0065] The preset part may be the hand or head of the target object, or the entire body of the target object.

[0066] The feature points are key points that can characterize the shape, position and other features of the preset parts.

[0067] For example, if the preset part is the hand, the multiple feature points are the locations corresponding to the knuckles, fingertips, and base of the palm of each finger. If the preset part is the head, the multiple feature points are the locations corresponding to the eyes, nose, mouth, eyebrows, etc. If the preset part is the entire body, the multiple feature points are the locations corresponding to the head, torso, limbs, and joints. In a specific embodiment, a Kinect sensor can be used to obtain the coordinate positions of the key feature points of the hand in space.

[0068] S32: Determine a target action of the target object according to the coordinate value set.

[0069] Taking gesture recognition to control an air conditioner as an example, image recognition technology is first used to determine whether the depth image contains the target object. If the depth image contains the target object, the key feature points of the fingers are visually detected to obtain the coordinate values ​​of the target object's knuckles, fingertips, and palm base. The position information of the key feature points of the fingers is used to determine the complex movements of the five fingers, and then the type of gesture is determined. For example, the angle between the upper and lower joints of the finger joints is calculated, and relevant thresholds are set to determine whether the fingers are bent or straight. The gesture is then identified based on the amount of bending of the five fingers.

[0070] This embodiment determines the target action through the coordinates of key feature points. Compared with voice control, it avoids the influence of environmental noise factors and can control the air conditioner more accurately.

[0071] In one embodiment, determining the target action of the target object according to the coordinate value set, S32, includes:

[0072] S321, for each frame of the depth image, connecting each coordinate point in the coordinate value set according to the morphological characteristics of the preset part to generate a venation map of the preset part;

[0073] Because the shape of a human hand differs from that of other objects, hand detection based on its shape features is an effective method. Taking the hand as an example, these features include various geometric features of the hand, such as the hand's outline, finger articulation, and the length, width, and aspect ratio of the fingers and palm.

[0074] A set of coordinate values ​​of multiple feature points of the hand is extracted from the depth image, and the coordinate value set includes at least the coordinates of the fingertips, knuckles and base of the palm of each finger of the hand. A hand generally has 20 coordinates, among which the thumb includes 3 feature points (fingertips and two knuckles), the four fingers from the index finger to the little finger each include 4 (fingertips and three knuckles), a total of 16 feature points, and 1 feature point at the base of the palm). It can be understood that in a specific embodiment, those skilled in the art can set other numbers or other positions of key feature points as needed.

[0075] After obtaining the coordinates of the 20 feature points of the hand, the coordinate points are connected according to the hand's morphological characteristics. For example, the fingertips of each finger are connected to each knuckle, and then converged at the coordinate point of the palm base to obtain the gesture vein map of the hand.

[0076] S322, determining whether the venation diagram conforms to a preset form;

[0077] S323: If the context diagram conforms to a first preset form, determining that the target action of the target object is a first static gesture, wherein the first static gesture is used to indicate turning on the air conditioner;

[0078] S324: If the context diagram conforms to the second preset form, determine the target movement 5 of the target object as a second static posture, wherein the second static posture is used to indicate turning off the air conditioner.

[0079] The preset form in this example is a static gesture, such as the gestures for indicating numbers 0, 1, 2, 3, etc. If the obtained gesture context diagram is the first preset form of gesture 1 with the index finger straight and the other four fingers bent, then the target action of the target object is determined to be turning on the air conditioner. If the obtained gesture context diagram is

[0080] The second preset form of the gesture 2 in which the index finger and the middle finger are straight and the other three fingers are bent determines that the target action of the target object 0 is to turn off the air conditioner.

[0081] This embodiment generates a context map by connecting the coordinates of each feature point of a preset part, and determines the corresponding static posture according to the context map, thereby realizing the recognition of the static posture of the target object.

[0082] In another embodiment, the target action of the target object is determined according to the coordinate value set.

[0083] S32 also includes: obtaining multiple coordinate value sets in the continuous multi-frame depth image; calculating the relative displacement of each feature point of the preset part in the continuous multi-frame depth image based on the multiple coordinate value sets, wherein the relative displacement includes direction and / or distance; and determining the target action of the target object based on the relative displacement.

[0084] In this example, the target action is a dynamic action. By capturing multiple frames of depth images, we can obtain

[0085] The gesture sends a series of continuous movements, and obtains the coordinate position of the key feature point 0 of the hand in each frame of the depth image in real time. By calculating the relative displacement between the key feature points and the angle and distance between the fingers

[0086] And direction and other data to achieve basic recognition function of hand movements.

[0087] In an embodiment of the present invention, determining the target action of the target object according to the relative displacement includes: if the distance each feature point moves upward in the vertical direction is greater than or equal to

[0088] If the distance that each feature point moves to the left in the horizontal direction is greater than or equal to a third preset distance, the target action of the target object is determined to be moving left; if the distance that each feature point moves to the left in the horizontal direction is greater than or equal to a third preset distance, the target action of the target object is determined to be moving left;

[0089] If the distance that each feature point moves to the right in the horizontal direction is greater than or equal to a fourth preset distance, it is determined that the target action of the target object is moving to the right.

[0090] In this embodiment, the relative displacement of each feature point in the continuous multi-frame depth image is calculated, and the target action is determined according to the relative displacement. The relative displacement of each feature point between two adjacent frames of depth image can be calculated first, and then the displacement of multiple frames of image is accumulated to determine the relative displacement of each feature point in the continuous multi-frame image.

[0091] The distance and direction of the relative displacement in the frame depth image are used to judge the action according to the direction and distance of the movement. 5 Considering that if the target object is relatively still or has a small movement amplitude, the two adjacent frame images

[0092] The position change of the feature points between two adjacent frames of depth images is not large, and if the target object has a large movement amplitude, the position change of the feature points between two adjacent frames of images is also correspondingly large. Therefore, in some embodiments, the relative displacement of each feature point between two adjacent frames of depth images can also be judged to determine the target movement of the target object.

[0093] In one embodiment, generating the control instruction of the air conditioner according to the target action includes:

[0094] If the target action is to move upward, a control instruction is generated to instruct the air conditioner to increase the temperature; if the target action is to move downward, a control instruction is generated to instruct the air conditioner to lower the temperature; if the target action is to move left, a control instruction is generated to instruct the air conditioner to decrease the wind speed; if the target action is to move right, a control instruction is generated to instruct the air conditioner to increase the wind speed.

[0095] 5 In this embodiment, the target action is a dynamic action, including moving the preset position upward, moving the preset position downward,

[0096] For example, the hand movements of waving up, waving down, waving to the left, and waving to the right correspond to controlling the air conditioner to increase the temperature, cool down, reduce the air conditioner wind speed, and increase the air conditioner wind speed respectively.

[0097] Set as needed, generally 1° or 1 gear. For example, when the user waves his hand upward, the air conditioner set temperature is adjusted up 1° on the current basis. When the user waves his hand to the left, the air conditioner wind speed gear is adjusted up.

[0098] Lower one level, etc.

[0099] In addition, if the depth image includes multiple target objects and multiple target objects perform valid target actions at the same time, in one embodiment, a priority can be set for each target object, and the air conditioner preferentially selects the action execution control of the target object with a higher level.

[0100] 5 The following fully explains the embodiment of the present invention through an implementation scenario.

[0101] This implementation scenario provides a method for controlling an air conditioner through gesture recognition based on a depth camera. The depth camera is used as the image input device, and the effect of controlling an intelligent air conditioner through gesture recognition is achieved through convolutional neural networks and command-based interaction.

[0102] Reference Figure 3 , Figure 3 This is a schematic diagram of the air conditioning control process in the implementation scenario of the first embodiment of the present invention. Figure 3 As shown, the process includes:

[0103] Step 1: Initialize device information;

[0104] Step 2: Get the depth image;

[0105] Step 3: Perform gesture recognition;

[0106] Step 4: Send interactive information;

[0107] Step 5: Get the command response;

[0108] Step 6: Air conditioning generates interaction.

[0109] This implementation scenario uses a depth camera to collect depth images, and uses a filtering algorithm to optimize the depth image data for noise problems such as pixel jitter and holes.

[0110] The gestures in this implementation scenario include static gesture commands and dynamic gesture commands. Static gestures 1 and 2 correspond to turning the air conditioner on and off respectively. Dynamic gestures of waving the hand up, waving the hand down, waving the hand to the left, and waving the hand to the right correspond to heating up, cooling down, reducing the wind speed, and increasing the air conditioner respectively.

[0111] Dynamic commands are a series of continuous movements generated by deep-seated gesture capture. The air conditioner recognizes and responds to these gestures. The design and recognition of dynamic commands requires describing dynamic gesture commands by determining the angle, distance, and direction between fingers. The training data is serialized and packaged into the TFRecords file format for the neural network model to read. During the recognition process, the processed image is input into the designed neural network structure to obtain the gesture recognition results.

[0112] In this implementation scenario, static gesture commands are combined with dynamic gesture commands, and a collaborative command interaction method is designed, which can effectively increase the number of gesture interaction commands and avoid overly complex commands.

[0113] The data interaction in this implementation scenario involves information exchange between two processes (i.e., instruction recognition and interactive operation process), refer to Figure 4 , Figure 4 This is a schematic diagram of data communication in an implementation scenario of the first embodiment of the present invention, as shown in FIG. Figure 4 As shown, the command recognition end and the interactive operation end are implemented using a long socket connection. The command recognition end and the interactive operation end create sockets respectively and bind them to the corresponding ports. The interactive operation end sends the byte stream and data length of the depth image to the command recognition end. The command recognition end receives the data and decodes the data to obtain image data information. After calculation, the recognition result is sent back to the interactive operation end. The interactive operation end receives the recognition result and responds to the command.

[0114] Reference Figure 5 , Figure 5 This is a schematic diagram of the workflow of the instruction recognition terminal in the implementation scenario of the first embodiment of the present invention. Figure 5 As shown, the command recognition end first binds the socket to the IP port, and the socket monitors the IP port to determine whether there is data. If so, the data is identified and processed to obtain the result, and sent back to the interactive end. If the interactive operation end is closed, the command recognition end is disconnected.

[0115] In the related technology, APP control of the air conditioner cannot completely free your hands (because APP control still requires carrying a mobile phone with you, which is no different from the air conditioner remote control). The voice assistant may be affected by environmental noise factors, resulting in low recognition rate and the air conditioner control cannot achieve the expected effect. Traditional gesture recognition is easily affected by environmental factors and has low recognition efficiency. However, this example controls the air conditioner through gesture recognition, which can free your hands, facilitate life, avoid the impact of environmental noise factors, and control the air conditioner more accurately and efficiently. The depth image of the gesture is obtained by the depth camera equipped with the air conditioner to avoid the impact of environmental factors such as light and angle on the effect of gesture recognition.

[0116] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0117] Example 2

[0118] This embodiment also provides an air conditioning control device for implementing the above-mentioned embodiments and preferred embodiments. Details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0119] Figure 6 is a structural block diagram of an air conditioning control device according to an embodiment of the present invention. Figure 6 As shown, the device includes: a detection module 100, an opening module 200, an analysis module 300, a generation module 400, and a control module 500, wherein:

[0120] Detection module 100, used to detect the status of the air conditioning control terminal;

[0121] The activation module 200 is configured to activate the depth camera of the air conditioner to acquire a depth image of a preset area of ​​the air conditioner when the control terminal is in an abnormal state;

[0122] An analysis module 300 is used to analyze the target action of the target object in the depth image;

[0123] A generating module 400 is configured to generate a control instruction for the air conditioner according to the target action;

[0124] The control module 500 is configured to control the air conditioner using the control instruction.

[0125] Optionally, the parsing module includes: a first acquisition unit for acquiring continuous multi-frame depth images; an input unit for inputting the continuous multi-frame depth images into a neural network model; and an output unit for outputting the target action of the target object in the depth image, wherein the target action includes a static posture and / or a dynamic action, and each target action corresponds to a control instruction.

[0126] Optionally, the parsing module also includes: an identification unit, used to identify the target object in the depth image, and obtain a set of coordinate values ​​of multiple feature points of a preset position of the target object in each frame of the depth image; and a determination unit, used to determine the target action of the target object based on the coordinate value set.

[0127] Optionally, the determination unit includes: a generation unit, for connecting each coordinate point in the coordinate value set in each frame of the depth image according to the morphological characteristics of the preset part to generate a vascular map of the preset part; a judgment unit, for judging whether the vascular map conforms to the preset form; a first determination subunit, for determining that the target action of the target object is a first static posture if the vascular map conforms to the first preset form, wherein the first static posture is used to indicate turning on the air conditioner; a second determination subunit, for determining that the target action of the target object is a second static posture if the vascular map conforms to the second preset form, wherein the second static posture is used to indicate turning off the air conditioner.

[0128] Optionally, the determination unit also includes: a third acquisition unit, used to acquire multiple coordinate value sets in the continuous multi-frame depth image; a calculation unit, used to calculate the relative displacement of each feature point of the preset part in the continuous multi-frame depth image based on the multiple coordinate value sets, wherein the relative displacement includes direction and / or distance; a third determination subunit, used to determine the target action of the target object based on the relative displacement.

[0129] Optionally, the third determination subunit is used to determine that the target action of the target object is moving upward if the distance that each feature point moves upward in the vertical direction is greater than or equal to a first preset distance; determine that the target action of the target object is moving downward if the distance that each feature point moves downward in the vertical direction is greater than or equal to a second preset distance; determine that the target action of the target object is moving left if the distance that each feature point moves to the left in the horizontal direction is greater than or equal to a third preset distance; and determine that the target action of the target object is moving right if the distance that each feature point moves to the right in the horizontal direction is greater than or equal to a fourth preset distance.

[0130] Optionally, the generating module includes: a first generating unit, configured to generate a first control instruction for instructing the air conditioner to increase the temperature if the target action is upward movement; and a second generating unit, configured to generate a second control instruction for instructing the air conditioner to decrease the temperature if the target action is downward movement;

[0131] The third generating unit is used to generate a third control instruction for instructing the air conditioner 5 to reduce the wind speed if the target action is to move to the left; the fourth generating unit is used to generate a third control instruction for instructing the air conditioner 5 to reduce the wind speed if the target action is to move to the right.

[0132] A fourth control instruction is generated to instruct the air conditioner to increase the wind speed.

[0133] It should be noted that each of the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited thereto: the above modules are all located in the same processor;

[0134] Alternatively, the above modules are located in different processors in any combination.

[0135] 0 Example 3

[0136] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0137] Optionally, in this embodiment, the storage medium may be configured to store a computer program for executing the following five steps:

[0138] S1, detect the status of the air conditioning control terminal;

[0139] S2, when the control terminal is in an abnormal state, the depth camera of the air conditioner is turned on to obtain the air conditioner

[0140] Adjust the depth image of the preset area;

[0141] S3, analyzing the target action of the target object in the depth image;

[0142] 0S4, generating a control instruction for the air conditioner according to the target action;

[0143] S5: Control the air conditioner using the control instruction.

[0144] Optionally, in this embodiment, the storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM),

[0145] Various media that can store computer programs, such as RAM, mobile hard disk, magnetic disk or optical disk, etc. 5 The embodiment of the present invention also provides an electronic device, including a memory and a processor, the memory

[0146] A computer program is stored in the processor, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0147] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0148] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0149] S1, detect the status of the air conditioning control terminal;

[0150] S2, when the control terminal is in an abnormal state, turning on the depth camera of the air conditioner to obtain a depth image of a preset area of ​​the air conditioner;

[0151] S3, analyzing the target action of the target object in the depth image;

[0152] S4, generating a control instruction for the air conditioner according to the target action;

[0153] S5: Control the air conditioner using the control instruction.

[0154] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0155] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0156] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0158] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0159] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0160] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0161] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An air conditioning control method, characterized in that: The method comprises: Detect the status of the air conditioning control terminal; When the control terminal is in an abnormal state, the depth camera of the air conditioner is turned on to obtain a depth image of the preset area of ​​the air conditioner; parsing a target motion of a target object in the depth image; generating a control instruction for the air conditioner according to the target action; controlling the air conditioner using the control instruction; Parsing the target action of the target object in the depth image includes: Identify the target object in the depth image, and obtain a set of coordinate values ​​of multiple feature points of a preset position of the target object in each frame of the depth image, wherein the set of coordinate values ​​is obtained by visual detection; determining a target action of the target object according to the coordinate value set; Determining the target action of the target object according to the coordinate value set includes: For each coordinate value set in the depth image frame, connecting each coordinate point in the coordinate value set according to the morphological characteristics of the preset part to generate a venation map of the preset part; Determining whether the venation diagram conforms to a preset form; If the context diagram conforms to a first preset form, determining that the target action of the target object is a first static gesture, wherein the first static gesture is used to indicate turning on the air conditioner; If the context diagram conforms to a second preset form, the target action of the target object is determined to be a second static gesture, wherein the second static gesture is used to instruct to turn off the air conditioner.

2. The air conditioning control method according to claim 1, wherein: Parsing the target action of the target object in the depth image includes: Acquire continuous multi-frame depth images; The continuous multi-frame depth images are input into a neural network model, and the target action of the target object in the depth image is output, wherein the target action includes a static posture and / or a dynamic action, and each target action corresponds to a control instruction.

3. The air conditioning control method according to claim 1, wherein: Determining the target action of the target object according to the coordinate value set further includes: Acquire a plurality of coordinate value sets in a continuous plurality of depth image frames; Calculating a relative displacement of each feature point of the preset part in the continuous multi-frame depth image according to the plurality of coordinate value sets, wherein the relative displacement includes direction and / or distance; A target action of the target object is determined according to the relative displacement.

4. The air conditioning control method according to claim 3, wherein: Determining the target action of the target object according to the relative displacement includes: If the distance that each feature point moves upward in the vertical direction is greater than or equal to a first preset distance, determining that the target action of the target object is moving upward; If the distance each feature point moves downward in the vertical direction is greater than or equal to a second preset distance, determining that the target action of the target object is moving downward; If the distance that each feature point moves to the left in the horizontal direction is greater than or equal to a third preset distance, determining that the target action of the target object is moving to the left; If the distance that each feature point moves to the right in the horizontal direction is greater than or equal to a fourth preset distance, it is determined that the target action of the target object is moving to the right.

5. The air conditioning control method according to claim 1, wherein: Generating the control instruction of the air conditioner according to the target action includes: If the target motion is upward movement, generating a first control instruction for instructing the air conditioner to increase the temperature; If the target motion is downward movement, generating a second control instruction for instructing the air conditioner to lower the temperature; If the target action is to move left, generating a third control instruction for instructing the air conditioner to reduce the wind speed; If the target action is moving rightward, a fourth control instruction is generated for instructing the air conditioner to increase the wind speed.

6. An air conditioning control device, characterized in that: The device comprises: Detection module, used to detect the status of the air conditioning control terminal; An activation module, configured to activate the depth camera of the air conditioner to acquire a depth image of a preset area of ​​the air conditioner when the control terminal is in an abnormal state; A parsing module, configured to parse a target action of a target object in the depth image; A generating module, configured to generate a control instruction for the air conditioner according to the target action; A control module, configured to control the air conditioner using the control instruction; an identification unit, configured to identify a target object in the depth image, and obtain a set of coordinate values ​​of a plurality of feature points of a preset portion of the target object in each frame of the depth image, wherein the set of coordinate values ​​is obtained by visual detection; and a determination unit, configured to determine a target action of the target object based on the set of coordinate values; The determination unit includes: a generation unit, for connecting each coordinate point in the coordinate value set in each frame of the depth image according to the morphological characteristics of the preset part to generate a vascular map of the preset part; a judgment unit, for judging whether the vascular map conforms to the preset form; a first determination subunit, for determining that the target action of the target object is a first static posture if the vascular map conforms to the first preset form, wherein the first static posture is used to indicate turning on the air conditioner; a second determination subunit, for determining that the target action of the target object is a second static posture if the vascular map conforms to the second preset form, wherein the second static posture is used to indicate turning off the air conditioner.

7. A storage medium, characterized in that: The storage medium includes a stored program, wherein the program executes the steps of the method according to any one of claims 1 to 5 when executed.

8. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein: The processor, communication interface, and memory communicate with each other via a communication bus; wherein: Memory for storing computer programs; A processor, configured to execute the steps of the method according to any one of claims 1 to 5 by running a program stored in a memory.

Images