Equipment control method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an equipment control method and device, electronic equipment and a storage medium, and relates to the technical field of equipment control, and the method comprises the steps: determining an infrared image collected by an infrared camera; acquiring a temperature value corresponding to each pixel point in the infrared image and temperature constraint information for the infrared image, and identifying the infrared image according to the temperature values of the pixel points and the temperature constraint information to obtain a target candidate box set corresponding to the infrared image; performing human body detection according to the target candidate frame set, screening out a human body candidate frame from the target candidate frame set, and obtaining a human body temperature corresponding to the human body candidate frame; and executing the equipment operation corresponding to the human body temperature, thereby improving the accuracy and precision of human body temperature identification.

Description

Technical Field

[0001] The present invention relates to the field of equipment control technology, and in particular to a equipment control method, an equipment control device, an electronic device and a computer-readable storage medium. Background Art

[0002] With the continuous development of science and technology, smart home has become an indispensable part of people's lives. In the process of controlling smart home devices, users can control smart home devices through voice commands, terminal devices, remote controls and physical control keys, and these control methods are all triggered by users. In practice, users hope that smart home devices can automatically perform corresponding operations according to user needs, such as automatically turning on lights, air conditioners and other equipment, so as to improve the intelligence and automation of the equipment and improve the user's life comfort and convenience. However, even if smart home devices can automatically perform corresponding operations according to user needs, there is still the problem of inaccurate recognition. At the same time, in order to improve the recognition accuracy, additional sensors need to be added, which increases costs. Summary of the invention

[0003] The embodiments of the present invention provide a device control method, apparatus, electronic device and computer-readable storage medium to solve or partially solve the problems of inaccurate detection and low detection accuracy in the device control process of smart home devices.

[0004] An embodiment of the present invention discloses a device control method, which is applied to a smart home device, wherein the smart home device is equipped with an infrared camera, and the method includes:

[0005] Determining the infrared image collected by the infrared camera;

[0006] Acquire a temperature value corresponding to each pixel in the infrared image and temperature constraint information for the infrared image, identify the infrared image according to the temperature value of the pixel and the temperature constraint information, and obtain a set of target candidate frames corresponding to the infrared image;

[0007] Performing human body detection according to the target candidate frame set, screening out a human body candidate frame from the target candidate frame set, and obtaining a human body temperature corresponding to the human body candidate frame;

[0008] A device operation corresponding to the human body temperature is performed.

[0009] In some feasible implementations, the identifying the infrared image according to the temperature value of the pixel point and the temperature constraint information to obtain a set of target candidate frames corresponding to the infrared image includes:

[0010] Inputting the temperature value and the number of pixels corresponding to the pixel points into the target model to recognize the infrared image, and obtaining a current candidate frame set corresponding to the infrared image and a confidence level corresponding to each candidate frame in the current candidate frame set;

[0011] A target candidate frame set is filtered out from the current candidate frame set according to the temperature constraint information and the confidence level.

[0012] In some feasible implementations, the filtering out a target candidate frame set from the current candidate frame set according to the temperature constraint information and the confidence level includes:

[0013] Select the candidate box with the highest confidence from the current candidate box set as the first current candidate box;

[0014] The first current candidate frame is removed from the current candidate frame set, and added to the initial candidate frame set, and a first intersection-and-union ratio between the first current candidate frame and other candidate frames in the current candidate frame set is calculated;

[0015] Compare the first intersection-to-union ratio with a first preset threshold, and filter the current candidate box set according to the first comparison result to obtain a new current candidate box set;

[0016] Select the candidate box with the highest confidence from the new current candidate box set as the second current candidate box;

[0017] The second current candidate box is removed from the current candidate box set, and added to the initial candidate box set, and a second intersection-and-union ratio between the second current candidate box and other candidate boxes in the current candidate box set is calculated;

[0018] Compare the second intersection-over-union ratio with the first preset threshold, and filter the current candidate box set according to the second comparison result to obtain a new current candidate box set;

[0019] Return to the step of selecting the candidate box with the highest confidence from the new current candidate box set as the second current candidate box, until no candidate box exists in the current candidate box set;

[0020] The temperature constraint information is used to screen the initial candidate frame set to obtain a corresponding target candidate frame set.

[0021] In some feasible implementations, comparing the first intersection-to-union ratio with a preset threshold, filtering the current candidate frame set according to the first comparison result, and obtaining a new current candidate frame set includes:

[0022] The candidate boxes whose first intersection-over-union ratio is greater than a first preset threshold are removed from the current candidate box set, and a new current candidate box set is obtained based on the removal result.

[0023] In some feasible implementations, the temperature constraint information includes at least a first temperature threshold and a second temperature threshold, the initial candidate frame set includes a first candidate frame, and the using the temperature constraint information to filter the initial candidate frame set to obtain a corresponding target candidate frame set includes:

[0024] Based on the temperature values ​​corresponding to the pixels, determine a first number of first target pixels in the first candidate frame that are greater than the first temperature threshold, and a second number of second target pixels in the first candidate frame that are less than the second temperature value;

[0025] The first candidate frames whose sum of the first number and the second number is greater than the second preset threshold are eliminated, and the corresponding target candidate frame set is obtained based on the elimination result.

[0026] In some feasible implementations, the target candidate frame set includes a second candidate frame, and performing human body detection according to the target candidate frame set and filtering out a human body candidate frame from the target candidate frame set includes:

[0027] Obtaining coordinate information and size information corresponding to each of the second candidate boxes;

[0028] Tracking the second candidate frame according to the coordinate information and the size information to obtain a target position of the second candidate frame in the next frame of infrared image;

[0029] Determining a relative moving speed corresponding to the second candidate frame based on a target position of the same second candidate frame in a plurality of consecutive infrared image frames;

[0030] The second candidate frame is filtered using the relative moving speed, and a human body candidate frame is selected from the target candidate frame set.

[0031] In some feasible implementations, tracking the second candidate frame according to the coordinate information and the size information to obtain a target position of the second candidate frame in the next frame of infrared image includes:

[0032] Determine candidate frame parameters of the second candidate frame by using the coordinate information and the size information, the candidate frame parameters including a first central horizontal coordinate, a first central vertical coordinate, a first height, and a first width;

[0033] Construct a corresponding state quantity based on the first central horizontal coordinate, the first central vertical coordinate, the first height, and the first width, and substitute the state quantity into a basic formula of Kalman filtering to obtain a corresponding state prediction equation;

[0034] Constructing a corresponding observation equation based on the state prediction equation and the state quantity;

[0035] Acquire Kalman gain information, and construct a corresponding state estimation equation based on the Kalman gain information, the observation equation, and an iterative formula of the Kalman filter;

[0036] The first center horizontal coordinate, the first center vertical coordinate, the first height and the first width are input into the state prediction equation and the state estimation equation to track the second candidate box to obtain the target position of the second candidate box in the next frame of infrared image.

[0037] In some feasible implementations, the filtering the second candidate frame by using the relative moving speed to select a human candidate frame from the target candidate frame set includes:

[0038] The second candidate frames whose relative moving speed is greater than or equal to the preset speed threshold are eliminated, and the remaining second candidate frames are used as human body candidate frames.

[0039] In some feasible implementations, the smart home device includes at least one of an air conditioner, a humidifier, a dehumidifier, a lighting device, a fan, and a health monitoring device, and the performing of the device operation corresponding to the human body temperature includes at least one of the following:

[0040] Adjust the temperature and wind speed of the air conditioner according to the human body temperature;

[0041] adjusting the humidity of the humidifier according to the human body temperature;

[0042] adjusting the humidity of the dehumidifier according to the human body temperature;

[0043] Adjust the brightness and / or color temperature of the lighting device according to the human body temperature;

[0044] adjusting the wind speed of the fan according to the human body temperature;

[0045] If the human body temperature is greater than or equal to a preset temperature threshold, a corresponding prompt message is issued through the health monitoring device.

[0046] The embodiment of the present invention further discloses a device control apparatus, which is applied to a smart home device, wherein the smart home device is equipped with an infrared camera, and the device comprises:

[0047] An image determination module, used to determine the infrared image collected by the infrared camera;

[0048] an identification module, used to obtain a temperature value corresponding to each pixel in the infrared image and temperature constraint information for the infrared image, identify the infrared image according to the temperature value of the pixel and the temperature constraint information, and obtain a target candidate frame set corresponding to the infrared image;

[0049] A detection module, configured to perform human body detection according to the target candidate frame set, filter out a human body candidate frame from the target candidate frame set, and obtain a human body temperature corresponding to the human body candidate frame;

[0050] An execution module is used to execute a device operation corresponding to the human body temperature.

[0051] In some feasible implementations, the identification module is specifically used to:

[0052] Inputting the temperature value and the number of pixels corresponding to the pixel points into the target model to recognize the infrared image, and obtaining a current candidate frame set corresponding to the infrared image and a confidence level corresponding to each candidate frame in the current candidate frame set;

[0053] A target candidate frame set is filtered out from the current candidate frame set according to the temperature constraint information and the confidence level.

[0054] In some feasible implementations, the identification module is specifically used to:

[0055] Select the candidate box with the highest confidence from the current candidate box set as the first current candidate box;

[0056] The first current candidate frame is removed from the current candidate frame set, and added to the initial candidate frame set, and a first intersection-and-union ratio between the first current candidate frame and other candidate frames in the current candidate frame set is calculated;

[0057] Compare the first intersection-to-union ratio with a first preset threshold, and filter the current candidate box set according to the first comparison result to obtain a new current candidate box set;

[0058] Select the candidate box with the highest confidence from the new current candidate box set as the second current candidate box;

[0059] The second current candidate box is removed from the current candidate box set, and added to the initial candidate box set, and a second intersection-and-union ratio between the second current candidate box and other candidate boxes in the current candidate box set is calculated;

[0060] Compare the second intersection-over-union ratio with the first preset threshold, and filter the current candidate box set according to the second comparison result to obtain a new current candidate box set;

[0061] Return to the step of selecting the candidate box with the highest confidence from the new current candidate box set as the second current candidate box, until no candidate box exists in the current candidate box set;

[0062] The temperature constraint information is used to screen the initial candidate frame set to obtain a corresponding target candidate frame set.

[0063] In some feasible implementations, the identification module is specifically used to:

[0064] The candidate boxes whose first intersection-over-union ratio is greater than a first preset threshold are removed from the current candidate box set, and a new current candidate box set is obtained based on the removal result.

[0065] In some feasible implementations, the temperature constraint information includes at least a first temperature threshold and a second temperature threshold, the initial candidate frame set includes a first candidate frame, and the identification module is specifically configured to:

[0066] Based on the temperature values ​​corresponding to the pixels, determine a first number of first target pixels in the first candidate frame that are greater than the first temperature threshold, and a second number of second target pixels in the first candidate frame that are less than the second temperature value;

[0067] The first candidate frames whose sum of the first number and the second number is greater than the second preset threshold are eliminated, and the corresponding target candidate frame set is obtained based on the elimination result.

[0068] In some feasible implementations, the target candidate box set includes a second candidate box, and the detection module is specifically configured to:

[0069] Obtaining coordinate information and size information corresponding to each of the second candidate boxes;

[0070] Tracking the second candidate frame according to the coordinate information and the size information to obtain a target position of the second candidate frame in the next frame of infrared image;

[0071] Determining a relative moving speed corresponding to the second candidate frame based on a target position of the same second candidate frame in a plurality of consecutive infrared image frames;

[0072] The second candidate frame is filtered using the relative moving speed, and a human body candidate frame is selected from the target candidate frame set.

[0073] In some feasible implementations, the detection module is specifically used to:

[0074] Determine candidate frame parameters of the second candidate frame by using the coordinate information and the size information, the candidate frame parameters including a first central horizontal coordinate, a first central vertical coordinate, a first height, and a first width;

[0075] Construct a corresponding state quantity based on the first central horizontal coordinate, the first central vertical coordinate, the first height, and the first width, and substitute the state quantity into a basic formula of Kalman filtering to obtain a corresponding state prediction equation;

[0076] Constructing a corresponding observation equation based on the state prediction equation and the state quantity;

[0077] Acquire Kalman gain information, and construct a corresponding state estimation equation based on the Kalman gain information, the observation equation, and an iterative formula of the Kalman filter;

[0078] The first center horizontal coordinate, the first center vertical coordinate, the first height and the first width are input into the state prediction equation and the state estimation equation to track the second candidate box to obtain the target position of the second candidate box in the next frame of infrared image.

[0079] In some feasible implementations, the detection module is specifically used to:

[0080] The second candidate frames whose relative moving speed is greater than or equal to the preset speed threshold are eliminated, and the remaining second candidate frames are used as human body candidate frames.

[0081] In some feasible implementations, the smart home device includes at least one of an air conditioner, a humidifier, a dehumidifier, a lighting device, a fan, and a health monitoring device, and the execution module is specifically used to execute one of the following:

[0082] Adjust the temperature and wind speed of the air conditioner according to the human body temperature;

[0083] adjusting the humidity of the humidifier according to the human body temperature;

[0084] adjusting the humidity of the dehumidifier according to the human body temperature;

[0085] Adjust the brightness and / or color temperature of the lighting device according to the human body temperature;

[0086] adjusting the wind speed of the fan according to the human body temperature;

[0087] If the human body temperature is greater than or equal to a preset temperature threshold, a corresponding prompt message is issued through the health monitoring device.

[0088] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0089] The memory is used to store computer programs;

[0090] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0091] The embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enables the processors to execute the method described in the embodiment of the present invention.

[0092] The embodiments of the present invention include the following advantages:

[0093] In the embodiment of the present invention, it can be applied to smart home devices. The smart home devices can be configured with corresponding infrared cameras. The corresponding infrared images can be collected through the infrared cameras. Then, during the operation of the smart home devices, the temperature values ​​corresponding to each pixel in the infrared image and the temperature constraint information for the infrared image can be obtained, and the infrared image can be identified according to the temperature values ​​of the pixel points and the temperature constraint information to obtain a target candidate frame set corresponding to the infrared image, and then human body detection is performed according to the target candidate frame set, and human body candidate frames are screened out from the target candidate frame set, and the human body temperature corresponding to the human body candidate frames is obtained, and then the device operation corresponding to the human body temperature is executed, so that the temperature constraint information is added to the identified candidate frame set for human body detection, and the accuracy of identifying the human body candidate frame from the candidate frame is improved, so that the smart home device can accurately identify the human body temperature, and then automatically execute the corresponding device operation based on the human body temperature, thereby improving the intelligence and automation of the device and improving the user's life comfort and convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 is a flowchart of a method for controlling a device provided in an embodiment of the present invention;

[0095] Figure 2 is a schematic diagram of data collection provided in an embodiment of the present invention;

[0096] Figure 3 is a schematic diagram of human body detection and tracking provided in an embodiment of the present invention;

[0097] Figure 4 is a structural block diagram of a control device of a device provided in an embodiment of the present invention;

[0098] Figure 5It is a block diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0099] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0100] As an example, with the continuous development of science and technology, smart home has become an indispensable part of people's lives. As an important part of smart home, infrared technology can better protect the privacy and security of users, and its application is becoming more and more extensive. Infrared technology can sense the temperature of human body and objects through infrared sensors, so as to realize the automatic control of smart home. However, even if smart home devices can automatically perform corresponding operations according to user needs, there is still the problem of inaccurate recognition. At the same time, in order to improve the recognition accuracy, additional sensors need to be added, which increases the cost.

[0101] In this regard, in the present invention, it is applied to smart home devices. The smart home devices can be configured with corresponding infrared cameras, and corresponding infrared images can be collected through the infrared cameras. Then, during the operation of the smart home devices, the temperature values ​​corresponding to each pixel in the infrared image and the temperature constraint information for the infrared image can be obtained, and the infrared image can be identified according to the temperature values ​​of the pixel points and the temperature constraint information to obtain a target candidate frame set corresponding to the infrared image, and then human body detection is performed according to the target candidate frame set, and human body candidate frames are screened out from the target candidate frame set, and the human body temperature corresponding to the human body candidate frames is obtained, and then the device operation corresponding to the human body temperature is executed, so that the temperature constraint information is added to the identified candidate frame set for human body detection, and the accuracy of identifying the human body candidate frame from the candidate frame is improved, so that the smart home device can accurately identify the human body temperature, and then automatically execute the corresponding device operation based on the human body temperature, thereby improving the intelligence and automation of the device and improving the user's life comfort and convenience.

[0102] Reference Figure 1 , shows a flow chart of the steps of a control method of a device provided in an embodiment of the present invention, which is applied to a smart home device, wherein the smart home device is equipped with an infrared camera, and specifically may include the following steps:

[0103] Step 101, determining the infrared image collected by the infrared camera;

[0104] Optionally, the smart home device may be a home appliance equipped with an infrared camera, which can collect infrared images of the environment so as to detect human body temperature based on the infrared images. It should be noted that the infrared camera may be built into the home appliance or external to the home appliance, and establish a communication connection with the home appliance so as to instruct the home appliance to perform corresponding device operations based on the collection results of the infrared camera.

[0105] For example, taking the example of an infrared camera externally mounted on a home appliance as an example, the infrared camera can be placed in a bedroom or living room at a height of 2.4 meters and an inclination of 30 degrees to simulate the installation of smart home devices such as air conditioners, and record the bedroom or living room in real time, and extract the collected video data into thermal imaging image data, and then use the collected data to identify the human body to identify the human body temperature, and then perform corresponding equipment operations based on the human body temperature. Optionally, the infrared camera can be an infrared thermal imager that can collect 80*62 pixels, and the present invention is not limited to this.

[0106] Step 102, obtaining a temperature value corresponding to each pixel in the infrared image and temperature constraint information for the infrared image, identifying the infrared image according to the temperature value of the pixel and the temperature constraint information, and obtaining a set of target candidate frames corresponding to the infrared image;

[0107] For infrared images, each pixel on the image can correspond to a corresponding temperature value. The smart home device can then identify the infrared image based on the temperature values ​​corresponding to each pixel in the infrared image, and obtain a set of target candidate frames corresponding to the infrared image, so as to identify areas in the image that are suspected to be human bodies.

[0108] In some feasible implementations, the smart home appliance device can input the temperature value corresponding to the pixel point and the number of pixels into the target model to identify the infrared image, obtain the current candidate frame set corresponding to the infrared image, and the confidence corresponding to each candidate frame in the current candidate frame set, and then filter out the target candidate frame set from the current candidate frame set according to the temperature constraint information and the confidence, so as to identify the infrared image through the target model, obtain the corresponding target candidate frame set, preliminarily identify the area suspected to be a human body in the image, and at the same time, add the temperature constraint information to the identified candidate frame set for human body detection, improve the accuracy of identifying the human body candidate frame from the candidate frame, and ensure the accuracy of subsequent data processing. Among them, the target model can be used for human body recognition in infrared images, and the corresponding candidate frame can be displayed in the infrared image according to the recognition result. In the case of accurate recognition, one candidate frame can correspond to one user, so that the human body problem of the user in the candidate frame can be identified through each candidate frame.

[0109] In the process of screening the candidate frames based on confidence, the candidate frame with the highest confidence can be first selected from the current candidate frame set as the first current candidate frame, and then the first current candidate frame is removed from the current candidate frame set and added to the initial candidate frame set, and a first intersection-and-union ratio between the first current candidate frame and other candidate frames in the current candidate frame set is calculated, and the first intersection-and-union ratio is compared with a first preset threshold value, and the current candidate frame set is filtered according to the first comparison result to obtain a new current candidate frame set. After the current candidate frame set is updated, the candidate frame with the highest confidence can be selected from the new current candidate frame set as the second current candidate frame, and then the second current candidate frame is removed from the current candidate frame set. Remove and add to the initial candidate frame set, and calculate the second intersection-and-union ratio between the second current candidate frame and other candidate frames in the current candidate frame set, and then compare the second intersection-and-union ratio with the first preset threshold, filter the current candidate frame set according to the second comparison result, obtain a new current candidate frame set, return to the step of selecting the candidate frame with the highest confidence from the new current candidate frame set as the second current candidate frame, and so on, until there is no candidate frame in the current candidate frame set, and then use the temperature constraint information to filter the initial candidate frame set to obtain the corresponding target candidate frame set, so as to identify the infrared image through the target model, obtain the corresponding target candidate frame set, and preliminarily identify the area suspected to be a human body in the image.

[0110] Optionally, the intersection-and-union ratio can be used to characterize the degree of overlap between two candidate boxes. In the process of filtering the current candidate box set, the candidate boxes whose first intersection-and-union ratio is greater than a first preset threshold can be removed from the current candidate box set, and a new current candidate box set is obtained based on the removal result; and the candidate boxes whose second intersection-and-union ratio is greater than the first preset threshold can be removed from the current candidate box set, and a new current candidate box set is obtained based on the removal result.

[0111] In addition, in order to improve the correlation between the candidate frame and the human body, it is necessary to exclude the thermal interference source in the candidate frame. Specifically, the temperature constraint information includes at least a first temperature threshold and a second temperature threshold. The initial candidate frame set includes the first candidate frame. Then, based on the temperature value corresponding to the pixel point, the first number of first target pixel points in the first candidate frame that are greater than the first temperature threshold and the second number of second target pixel points in the first candidate frame that are less than the second temperature value can be determined. Then, the first candidate frame whose sum of the first number and the second number is greater than the second preset threshold is eliminated, and the corresponding target candidate frame set is obtained based on the elimination result, so as to add the temperature constraint information to the identified candidate frame set for human body detection, thereby improving the accuracy of identifying the human body candidate frame from the candidate frame and ensuring the accuracy of subsequent data processing.

[0112] Through the above process, based on the confidence level corresponding to the candidate frames, a set of target candidate frames that may be human candidate frames can be continuously screened out to ensure the data accuracy of subsequent data processing, and the candidate frames can be filtered in combination with the temperature constraint information to eliminate thermal interference sources higher than the human body temperature and thermal interference sources lower than the human body temperature, thereby ensuring the accuracy of the data and improving the accuracy of subsequent human detection and tracking.

[0113] It should be noted that, for the image recognition process, the infrared image can be processed by a corresponding target model, and the target model can be trained according to corresponding training data, for example, referring to Figure 2 , showing a schematic diagram of data collection provided in an embodiment of the present invention, the infrared camera module (infrared module) can be placed in the bedroom and the living room, 2.4 meters high and tilted at 30°, to simulate the installation of smart homes such as air conditioners, such as Figure 2 As shown in the data collection diagram, the video is recorded in real time, and the collected video data is framed into thermal imaging image data (in the upper right corner of the figure). The collected data is annotated with the human body (i.e., the human body is framed, the green frame in the figure), and different thermal interference sources are added, such as hot water cups, electric heaters, lights, and sunlight. Then about 10,000 pictures are collected for model training, and the corresponding target model is obtained based on the model training results. In actual applications, the infrared image can be recognized based on the target model to obtain the candidate frame set corresponding to the infrared image.

[0114] In one example, a target model can be used for human body detection. When an infrared camera collects an infrared image corresponding to the environment, a temperature value T={T1, T2, ..., T m}, m = 80 * 62 (i.e., the number of pixels in the image), and a set of candidate boxes B = {B1, B2, ..., B n},n∈R, the corresponding confidence score is S={S1,S2,...,S n}, set the overlap threshold to τ, and finally predict the preliminary candidate box to be D, which can then be processed according to the following process:

[0115] (1) First select the candidate box B with the highest confidence score max , and remove it from B and add it to the final candidate list D (initially D is empty).

[0116] (2) Compare the candidate box in (1) with other candidate boxes in B, that is, calculate the intersection-over-union ratio of this candidate box and each other candidate box. If the intersection-over-union ratio is greater than the threshold τ, remove the candidate box from B.

[0117] (3) Select the candidate box with the highest confidence from the remaining candidate boxes in B again, remove it from B and add it to D.

[0118] (4) Calculate the intersection-and-union ratio of the candidate box in (3) and all candidates in B again, and eliminate those bounding boxes whose intersection-and-union ratio is higher than the threshold τ.

[0119] (5) Repeat this process until there are no more candidate boxes in B, then D′={d1,d2,...,d p}, p∈R is the target candidate box set predicted by the target model.

[0120] (6) Since there may be heat sources in the room that are higher or lower than the human body temperature, such as hot water cups, electric heaters, windows, etc., a temperature constraint condition can be added to the prediction algorithm of the target model. The candidate box obtained in step (5) is used to judge the temperature of the area where the box is located, that is, the number of pixels N in the candidate box whose temperature is greater than 40° and less than 30° pixel If the number exceeds the specified number num, the corresponding box will be removed, and finally a preliminary candidate box D will be obtained.

[0121] Step 103, performing human body detection according to the target candidate frame set, screening out a human body candidate frame from the target candidate frame set, and obtaining a human body temperature corresponding to the human body candidate frame;

[0122] After obtaining the corresponding target candidate frame set, human body detection and tracking can be performed based on the target candidate frame set to further filter out human body candidate frames from the candidate frames, and then determine the body temperature of the identified user based on the human body candidate frames.

[0123] In some feasible implementations, the target candidate frame set includes a second candidate frame. The coordinate information and size information corresponding to each second candidate frame can be obtained, and then the second candidate frame can be tracked according to the coordinate information and size information to obtain the target position of the second candidate frame in the next frame of infrared image. Then, based on the target position of the same second candidate frame in multiple consecutive frames of infrared images, the relative moving speed corresponding to the second candidate frame is determined, and then the second candidate frame is filtered by the relative moving speed to screen out the human candidate frame from the target candidate frame set. Thus, by setting corresponding speed detection conditions, the candidate frames that are not related to the human body can be further eliminated, the data accuracy can be improved, and the human candidate frame in the infrared image can be effectively identified.

[0124] It should be noted that the coordinate information may include the center coordinates of the candidate frame (the horizontal coordinate and the vertical coordinate corresponding to the center point), and the size information may include the height and width of the candidate frame, etc., then the coordinate information and the size information may be used to determine the candidate frame parameters of the second candidate frame, the candidate frame parameters include the first center horizontal coordinate, the first center vertical coordinate, the first height and the first width, and the corresponding state quantity is constructed based on the first center horizontal coordinate, the first center vertical coordinate, the first height and the first width, and the state quantity is substituted into the basic formula of the Kalman filter to obtain the corresponding state prediction equation, and then the corresponding observation equation is constructed based on the state prediction equation and the state quantity, and the Kalman gain information is obtained. The corresponding state estimation equation is constructed based on the Kalman gain information, the observation equation and the iterative formula of the Kalman filter, and then the first center horizontal coordinate, the first center vertical coordinate, the first height and the first width are input into the state prediction equation and the state estimation equation to track the second candidate frame to obtain the target position of the second candidate frame in the next frame of infrared image.

[0125] After obtaining the target position in the next frame of infrared image, the relative moving speed corresponding to the target selected by the candidate frame can be calculated based on the target position corresponding to the same candidate frame in multiple consecutive frames of infrared images, and then the relative moving speed is compared with the preset speed threshold, and the second candidate frame whose relative moving speed is greater than or equal to the preset speed threshold is eliminated, and the remaining second candidate frames are used as human candidate frames, so that the candidate frames unrelated to the human body are further eliminated through the screening conditions related to the speed, thereby improving the accuracy of the data and effectively identifying the human body candidate frames in the infrared image.

[0126] In one example, after the candidate box suspected to be a human body is detected in each frame of infrared image through the target model, the position of the current candidate box in the next frame of image can be predicted through the human body Kalman filter algorithm. The coordinates and size of the candidate box that can be calculated based on the center point coordinates and size information are used as the observation quantity of the Kalman filter, that is, z k =[x k ,y k ,a k ,h k ] T , the state quantity is set to where x k and k Represents the center coordinates of the human body frame, a k and h k represents the width and height of the human body frame, and dx, dy, da, dh are the corresponding first-order differences. Then, according to the basic formula of Kalman filtering, the state prediction equation can be obtained as:

[0127] Among them, F kis the state transfer matrix. In this proposal, we only need to predict the coordinate frame of the next frame, so the state transfer matrix can be set to be relatively simple, as shown below:

[0128]

[0129] Among them, B k is the control input matrix, u k is the control vector, set to 0, w k is the process noise.

[0130] Then, we can get the observation equation:

[0131] Among them, v k To observe the noise, since each variable in the state is independent of each other, the covariance is 0, so the covariance matrix P k It can be expressed as a diagonal matrix, H k is the observation matrix, that is:

[0132]

[0133] Finally, according to the iterative formula of Kalman filtering, the state estimation equation can be obtained:

[0134]

[0135] Among them, K k is the Kalman gain. Through the above formula, the estimation of the state quantity of the next frame can be realized, so as to realize human body tracking and determine the relative position of the candidate frame in the continuous multi-frame image. Since the walking speed of a person is usually between 1-2 meters per second, the running speed may be between 3-5 meters per second, and the acceleration of the person is usually small, especially when walking, the change is relatively gentle. Therefore, the corresponding speed detection conditions can be set to eliminate abnormal infection sources to the greatest extent. For example, if the speed of the target is between 1-5 meters per second, it can be considered that the target may be a person. Specifically, the Euclidean distance of the target selected by the candidate frame can be calculated based on the position of the candidate frame in the multi-frame image obtained in the aforementioned process. If the Euclidean distance is less than the specified threshold κ, it is considered to be a person, and the corresponding candidate frame is used as a human candidate frame, so that according to the above conditions, the unqualified frames are eliminated, and the remaining frames are the final candidate frames, and then through the screening conditions related to speed, the candidate frames that are not related to the human body are further eliminated, the accuracy of the data is improved, and the human candidate frame in the infrared image is effectively identified.

[0136] Step 104, executing device operations corresponding to the human body temperature.

[0137] In an embodiment of the present invention, the smart home device includes at least one of an air conditioner, a humidifier, a dehumidifier, a lighting device, a fan and a health monitoring device, and the device operation corresponding to the human body temperature is performed, including at least one of the following: adjusting the temperature and wind speed of the air conditioner according to the human body temperature; adjusting the humidity of the humidifier according to the human body temperature; adjusting the humidity of the dehumidifier according to the human body temperature; adjusting the brightness and / or color temperature of the lighting device according to the human body temperature; adjusting the wind speed of the fan according to the human body temperature; if the human body temperature is greater than or equal to the preset temperature threshold, the health monitoring device sends a corresponding prompt message, so that for the smart home device, it can collect the corresponding infrared image and identify the infrared image, add temperature constraint information to the identified candidate frame set to perform human body detection, improve the accuracy of identifying the human body candidate frame from the candidate frame, so that the smart home device can accurately identify the human body temperature, and then automatically perform the corresponding device operation based on the human body temperature, thereby improving the intelligence and automation of the device and improving the user's life comfort and convenience.

[0138] It should be noted that the embodiments of the present invention include but are not limited to the above examples. It is understandable that those skilled in the art can also make settings according to actual needs under the guidance of the ideas of the embodiments of the present invention, and the present invention is not limited to this.

[0139] In the embodiment of the present invention, it can be applied to smart home devices. The smart home devices can be configured with corresponding infrared cameras. The corresponding infrared images can be collected through the infrared cameras. Then, during the operation of the smart home devices, the temperature values ​​corresponding to each pixel in the infrared image and the temperature constraint information for the infrared image can be obtained, and the infrared image can be identified according to the temperature values ​​of the pixel points and the temperature constraint information to obtain a target candidate frame set corresponding to the infrared image, and then human body detection is performed according to the target candidate frame set, and human body candidate frames are screened out from the target candidate frame set, and the human body temperature corresponding to the human body candidate frames is obtained, and then the device operation corresponding to the human body temperature is executed, so that the temperature constraint information is added to the identified candidate frame set for human body detection, and the accuracy of identifying the human body candidate frame from the candidate frame is improved, so that the smart home device can accurately identify the human body temperature, and then automatically execute the corresponding device operation based on the human body temperature, thereby improving the intelligence and automation of the device and improving the user's life comfort and convenience.

[0140] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following examples are used for exemplary description:

[0141] Reference Figure 3, shows a schematic diagram of human body detection and tracking provided in an embodiment of the present invention. The smart home device can collect the corresponding data set through the infrared camera, and identify the data set through the target model to obtain a preliminary prediction frame. Then, the preliminary prediction frame can be processed by the human body frame prediction algorithm, and the number of pixels N with a temperature greater than 40° and less than 30° in the candidate frame is eliminated. pixel For candidate frames exceeding the specified number num, the corresponding target candidate frames are obtained, and then the candidate frames are further filtered through the Kalman filtering algorithm and the anomaly detection module (based on the speed detection condition) to obtain the human frame (also known as the human candidate frame). After the infrared image is recognized, the temperature constraint information is added to the recognized candidate frame set for human detection, so as to improve the accuracy of identifying the human candidate frame from the candidate frame, so that the smart home device can accurately identify the human body temperature, and then automatically perform the corresponding device operation based on the human body temperature, thereby improving the intelligence and automation of the device and improving the user's life comfort and convenience.

[0142] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0143] Reference Figure 4 , shows a structural block diagram of a control device of a device provided in an embodiment of the present invention, which is applied to a smart home device, wherein the smart home device is equipped with an infrared camera and may specifically include the following modules:

[0144] An image determination module 401 is used to determine the infrared image collected by the infrared camera;

[0145] The recognition module 402 is used to obtain the temperature value corresponding to each pixel in the infrared image and the temperature constraint information for the infrared image, recognize the infrared image according to the temperature value of the pixel and the temperature constraint information, and obtain a set of target candidate frames corresponding to the infrared image;

[0146] The detection module 403 is used to perform human body detection according to the target candidate frame set, filter out a human body candidate frame from the target candidate frame set, and obtain a human body temperature corresponding to the human body candidate frame;

[0147] The execution module 404 is used to execute the device operation corresponding to the human body temperature.

[0148] In some feasible implementations, the identification module 402 is specifically used to:

[0149] Inputting the temperature value and the number of pixels corresponding to the pixel points into the target model to recognize the infrared image, and obtaining a current candidate frame set corresponding to the infrared image and a confidence level corresponding to each candidate frame in the current candidate frame set;

[0150] A target candidate frame set is filtered out from the current candidate frame set according to the temperature constraint information and the confidence level.

[0151] In some feasible implementations, the identification module 402 is specifically used to:

[0152] Select the candidate box with the highest confidence from the current candidate box set as the first current candidate box;

[0153] The first current candidate frame is removed from the current candidate frame set, and added to the initial candidate frame set, and a first intersection-and-union ratio between the first current candidate frame and other candidate frames in the current candidate frame set is calculated;

[0154] Compare the first intersection-to-union ratio with a first preset threshold, and filter the current candidate box set according to the first comparison result to obtain a new current candidate box set;

[0155] Select the candidate box with the highest confidence from the new current candidate box set as the second current candidate box;

[0156] The second current candidate box is removed from the current candidate box set, and added to the initial candidate box set, and a second intersection-and-union ratio between the second current candidate box and other candidate boxes in the current candidate box set is calculated;

[0157] Compare the second intersection-over-union ratio with the first preset threshold, and filter the current candidate box set according to the second comparison result to obtain a new current candidate box set;

[0158] Return to the step of selecting the candidate box with the highest confidence from the new current candidate box set as the second current candidate box, until no candidate box exists in the current candidate box set;

[0159] The temperature constraint information is used to screen the initial candidate frame set to obtain a corresponding target candidate frame set.

[0160] In some feasible implementations, the identification module 402 is specifically used to:

[0161] The candidate boxes whose first intersection-over-union ratio is greater than a first preset threshold are removed from the current candidate box set, and a new current candidate box set is obtained based on the removal result.

[0162] In some feasible implementations, the temperature constraint information includes at least a first temperature threshold and a second temperature threshold, the initial candidate box set includes a first candidate box, and the identification module 402 is specifically configured to:

[0163] Based on the temperature values ​​corresponding to the pixels, determine a first number of first target pixels in the first candidate frame that are greater than the first temperature threshold, and a second number of second target pixels in the first candidate frame that are less than the second temperature value;

[0164] The first candidate frames whose sum of the first number and the second number is greater than the second preset threshold are eliminated, and the corresponding target candidate frame set is obtained based on the elimination result.

[0165] In some feasible implementations, the target candidate box set includes a second candidate box, and the detection module 403 is specifically used to:

[0166] Obtaining coordinate information and size information corresponding to each of the second candidate boxes;

[0167] Tracking the second candidate frame according to the coordinate information and the size information to obtain a target position of the second candidate frame in the next frame of infrared image;

[0168] Determining a relative moving speed corresponding to the second candidate frame based on a target position of the same second candidate frame in a plurality of consecutive infrared image frames;

[0169] The second candidate frame is filtered using the relative moving speed, and a human body candidate frame is selected from the target candidate frame set.

[0170] In some feasible implementations, the detection module 403 is specifically used to:

[0171] Determine candidate frame parameters of the second candidate frame by using the coordinate information and the size information, the candidate frame parameters including a first central horizontal coordinate, a first central vertical coordinate, a first height, and a first width;

[0172] Construct a corresponding state quantity based on the first central horizontal coordinate, the first central vertical coordinate, the first height, and the first width, and substitute the state quantity into a basic formula of Kalman filtering to obtain a corresponding state prediction equation;

[0173] Constructing a corresponding observation equation based on the state prediction equation and the state quantity;

[0174] Acquire Kalman gain information, and construct a corresponding state estimation equation based on the Kalman gain information, the observation equation, and an iterative formula of the Kalman filter;

[0175] The first center horizontal coordinate, the first center vertical coordinate, the first height and the first width are input into the state prediction equation and the state estimation equation to track the second candidate box to obtain the target position of the second candidate box in the next frame of infrared image.

[0176] In some feasible implementations, the detection module 403 is specifically used to:

[0177] The second candidate frames whose relative moving speed is greater than or equal to the preset speed threshold are eliminated, and the remaining second candidate frames are used as human body candidate frames.

[0178] In some feasible implementations, the smart home device includes at least one of an air conditioner, a humidifier, a dehumidifier, a lighting device, a fan, and a health monitoring device, and the execution module 404 is specifically used to execute one of the following:

[0179] Adjust the temperature and wind speed of the air conditioner according to the human body temperature;

[0180] adjusting the humidity of the humidifier according to the human body temperature;

[0181] adjusting the humidity of the dehumidifier according to the human body temperature;

[0182] Adjust the brightness and / or color temperature of the lighting device according to the human body temperature;

[0183] adjusting the wind speed of the fan according to the human body temperature;

[0184] If the human body temperature is greater than or equal to a preset temperature threshold, a corresponding prompt message is issued through the health monitoring device.

[0185] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0186] In addition, an embodiment of the present invention further provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the control method embodiment of the above-mentioned device are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0187] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the control method embodiment of the above-mentioned device is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0188] Figure 5 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0189] The electronic device 500 includes but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, a processor 510, and a power supply 511. Those skilled in the art will appreciate that the electronic device structure involved in the embodiments of the present invention does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently. In the embodiments of the present invention, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal, a wearable device, and a pedometer.

[0190] It should be understood that in the embodiment of the present invention, the radio frequency unit 501 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving downlink data from the base station, it is sent to the processor 510 for processing; in addition, uplink data is sent to the base station. Generally, the radio frequency unit 501 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 501 can also communicate with the network and other devices through a wireless communication system.

[0191] The electronic device provides users with wireless broadband Internet access through the network module 502, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0192] The audio output unit 503 can convert the audio data received by the RF unit 501 or the network module 502 or stored in the memory 509 into an audio signal and output it as sound. Moreover, the audio output unit 503 can also provide audio output related to a specific function performed by the electronic device 500 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 503 includes a speaker, a buzzer, a receiver, etc.

[0193] The input unit 504 is used to receive audio or video signals. The input unit 504 may include a graphics processor (GPU) 5041 and a microphone 5042. The graphics processor 5041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 506. The image frame processed by the graphics processor 5041 can be stored in the memory 509 (or other storage medium) or sent via the radio frequency unit 501 or the network module 502. The microphone 5042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 501 in the case of a telephone call mode.

[0194] The electronic device 500 also includes at least one sensor 505, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 5061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 5061 and / or the backlight when the electronic device 500 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 505 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0195] The display unit 506 is used to display information input by the user or information provided to the user. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0196] The user input unit 507 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 507 includes a touch panel 5071 and other input devices 5072. The touch panel 5071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 5071 or near the touch panel 5071 using any suitable object or accessory such as a finger, stylus, etc.). The touch panel 5071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the contact point coordinates, and then sends it to the processor 510, receives the command sent by the processor 510 and executes it. In addition, the touch panel 5071 can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic waves. In addition to the touch panel 5071, the user input unit 507 may also include other input devices 5072. Specifically, other input devices 5072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail here.

[0197] Further, the touch panel 5071 may be covered on the display panel 5061. When the touch panel 5071 detects a touch operation on or near it, it is transmitted to the processor 510 to determine the type of the touch event, and then the processor 510 provides a corresponding visual output on the display panel 5061 according to the type of the touch event. It can be understood that in one embodiment, the touch panel 5071 and the display panel 5061 are used as two independent components to implement the input and output functions of the electronic device, but in some embodiments, the touch panel 5071 and the display panel 5061 can be integrated to implement the input and output functions of the electronic device, which is not limited here.

[0198] The interface unit 508 is an interface for connecting an external device to the electronic device 500. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 508 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic device 500 or may be used to transmit data between the electronic device 500 and an external device.

[0199] The memory 509 can be used to store software programs and various data. The memory 509 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 509 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0200] The processor 510 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 509 and calling data stored in the memory 509, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 510 may include one or more processing units; preferably, the processor 510 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 510.

[0201] The electronic device 500 may also include a power supply 511 (such as a battery) for supplying power to each component. Preferably, the power supply 511 may be logically connected to the processor 510 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0202] In addition, the electronic device 500 includes some functional modules not shown, which will not be described in detail here.

[0203] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0204] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0205] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

[0206] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the embodiments of the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0207] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0208] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0211] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0212] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for controlling a device, characterized in that: Applied to a smart home device, the smart home device is equipped with an infrared camera, and the method comprises: Determining the infrared image collected by the infrared camera; Acquire a temperature value corresponding to each pixel in the infrared image and temperature constraint information for the infrared image, identify the infrared image according to the temperature value of the pixel and the temperature constraint information, and obtain a set of target candidate frames corresponding to the infrared image; Performing human body detection according to the target candidate frame set, screening out a human body candidate frame from the target candidate frame set, and obtaining a human body temperature corresponding to the human body candidate frame; A device operation corresponding to the human body temperature is performed.

2. The method according to claim 1, characterized in that The step of identifying the infrared image according to the temperature value of the pixel point and the temperature constraint information to obtain a set of target candidate frames corresponding to the infrared image includes: Inputting the temperature value and the number of pixels corresponding to the pixel points into the target model to recognize the infrared image, and obtaining a current candidate frame set corresponding to the infrared image and a confidence level corresponding to each candidate frame in the current candidate frame set; A target candidate frame set is filtered out from the current candidate frame set according to the temperature constraint information and the confidence level.

3. The method according to claim 2, characterized in that The step of selecting a target candidate frame set from the current candidate frame set according to the temperature constraint information and the confidence level includes: Select the candidate box with the highest confidence from the current candidate box set as the first current candidate box; The first current candidate frame is removed from the current candidate frame set, and added to the initial candidate frame set, and a first intersection-and-union ratio between the first current candidate frame and other candidate frames in the current candidate frame set is calculated; Compare the first intersection-to-union ratio with a first preset threshold, and filter the current candidate box set according to the first comparison result to obtain a new current candidate box set; Select the candidate box with the highest confidence from the new current candidate box set as the second current candidate box; The second current candidate box is removed from the current candidate box set, and added to the initial candidate box set, and a second intersection-and-union ratio between the second current candidate box and other candidate boxes in the current candidate box set is calculated; Compare the second intersection-over-union ratio with the first preset threshold, and filter the current candidate box set according to the second comparison result to obtain a new current candidate box set; Return to the step of selecting the candidate box with the highest confidence from the new current candidate box set as the second current candidate box, until no candidate box exists in the current candidate box set; The temperature constraint information is used to screen the initial candidate frame set to obtain a corresponding target candidate frame set.

4. The method according to claim 3, characterized in that The step of comparing the first intersection-over-union ratio with a preset threshold, filtering the current candidate frame set according to the first comparison result, and obtaining a new current candidate frame set includes: The candidate boxes whose first intersection-over-union ratio is greater than a first preset threshold are removed from the current candidate box set, and a new current candidate box set is obtained based on the removal result.

5. The method according to claim 3, characterized in that: The temperature constraint information includes at least a first temperature threshold and a second temperature threshold, the initial candidate frame set includes a first candidate frame, and the initial candidate frame set is screened by using the temperature constraint information to obtain a corresponding target candidate frame set, including: Based on the temperature values ​​corresponding to the pixels, determine a first number of first target pixels in the first candidate frame that are greater than the first temperature threshold, and a second number of second target pixels in the first candidate frame that are less than the second temperature value; The first candidate frames whose sum of the first number and the second number is greater than the second preset threshold are eliminated, and the corresponding target candidate frame set is obtained based on the elimination result.

6. The method according to claim 1, characterized in that The target candidate frame set includes a second candidate frame, and performing human body detection according to the target candidate frame set and filtering out a human body candidate frame from the target candidate frame set includes: Obtaining coordinate information and size information corresponding to each of the second candidate boxes; Tracking the second candidate frame according to the coordinate information and the size information to obtain a target position of the second candidate frame in the next frame of infrared image; Determining a relative moving speed corresponding to the second candidate frame based on a target position of the same second candidate frame in a plurality of consecutive infrared image frames; The second candidate frame is filtered using the relative moving speed, and a human body candidate frame is selected from the target candidate frame set.

7. The method according to claim 6, characterized in that The tracking of the second candidate frame according to the coordinate information and the size information to obtain a target position of the second candidate frame in the next frame of infrared image includes: Determine candidate frame parameters of the second candidate frame by using the coordinate information and the size information, the candidate frame parameters including a first central horizontal coordinate, a first central vertical coordinate, a first height, and a first width; Construct a corresponding state quantity based on the first central horizontal coordinate, the first central vertical coordinate, the first height, and the first width, and substitute the state quantity into a basic formula of Kalman filtering to obtain a corresponding state prediction equation; Constructing a corresponding observation equation based on the state prediction equation and the state quantity; Acquire Kalman gain information, and construct a corresponding state estimation equation based on the Kalman gain information, the observation equation, and an iterative formula of the Kalman filter; The first center horizontal coordinate, the first center vertical coordinate, the first height and the first width are input into the state prediction equation and the state estimation equation to track the second candidate box to obtain the target position of the second candidate box in the next frame of infrared image.

8. The method according to claim 6, characterized in that The step of filtering the second candidate frame by using the relative moving speed to select a human body candidate frame from the target candidate frame set includes: The second candidate frames whose relative moving speed is greater than or equal to the preset speed threshold are eliminated, and the remaining second candidate frames are used as human body candidate frames.

9. The method according to claim 1, characterized in that: The smart home device includes at least one of an air conditioner, a humidifier, a dehumidifier, a lighting device, a fan, and a health monitoring device, and the performing of the device operation corresponding to the human body temperature includes at least one of the following: Adjust the temperature and wind speed of the air conditioner according to the human body temperature; adjusting the humidity of the humidifier according to the human body temperature; adjusting the humidity of the dehumidifier according to the human body temperature; Adjust the brightness and / or color temperature of the lighting device according to the human body temperature; adjusting the wind speed of the fan according to the human body temperature; If the human body temperature is greater than or equal to a preset temperature threshold, a corresponding prompt message is issued through the health monitoring device.

10. A control device for an equipment, characterized in that: Applied to a smart home device, the smart home device is equipped with an infrared camera, and the device comprises: An image determination module, used to determine the infrared image collected by the infrared camera; an identification module, used to obtain a temperature value corresponding to each pixel in the infrared image and temperature constraint information for the infrared image, identify the infrared image according to the temperature value of the pixel and the temperature constraint information, and obtain a target candidate frame set corresponding to the infrared image; A detection module, configured to perform human body detection according to the target candidate frame set, filter out a human body candidate frame from the target candidate frame set, and obtain a human body temperature corresponding to the human body candidate frame; An execution module is used to execute a device operation corresponding to the human body temperature.

11. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 10 when executing the program stored in the memory.

12. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 10.

Images