Air conditioner control method and device, electronic device, storage medium and program product

Abstract

The present invention provides an air conditioner control method and device, electronic device, storage medium, and program product, relating to the field of air conditioner control. The method comprises: acquiring an infrared image of an indoor space using a single infrared camera; the infrared image comprises: a historical infrared image and a current infrared image; inputting the infrared image into a YOLO network to obtain a first prediction frame containing a person in the current infrared image; when the number of first prediction frames is greater than or equal to 1, obtaining motion information of the person from the infrared image; inputting the motion information into a motion model to obtain a second prediction frame containing the person in the current infrared image; determining a first position of the person in the infrared image based on the first prediction frame and the second prediction frame; converting the first position into a second position of the person in the indoor space based on a homography matrix; and controlling the air conditioner to supply air to the second position according to a preset air supply strategy. This application reduces costs and avoids the problem of air supply lag.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of air conditioning control, and in particular to a method and device for controlling an air conditioning, an electronic device, a storage medium, and a program product. Background Art

[0002] With the advancement of technology, people's demand for air conditioning has changed, and their pursuit of health and comfort has become increasingly higher, so the intelligent air supply mode has emerged accordingly.

[0003] Currently, air conditioners generally rely on hardware such as multi-sensor fusion or high-cost depth cameras to achieve intelligent air supply.

[0004] However, the intelligent air supply mode of existing air conditioners is highly dependent on hardware, and the cost of the above hardware is high, resulting in high air conditioning costs and complex wiring. Summary of the Invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide a method and apparatus for controlling an air conditioner, an electronic device, a storage medium, and a program product that overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, an embodiment of the present application discloses a method for controlling an air conditioner, comprising:

[0007] Acquire an infrared image of an indoor space through a single infrared camera; the infrared image includes: a historical infrared image and a current infrared image;

[0008] Inputting the infrared image into a YOLO network to obtain a first predicted frame containing a person in the current infrared image;

[0009] When the number of the first prediction frames is greater than or equal to 1, acquiring motion information of the person from the infrared image;

[0010] Inputting the motion information into a motion model to obtain a second predicted frame containing a person in the current infrared image;

[0011] Determining a first position of a person in the infrared image based on the first prediction frame and the second prediction frame;

[0012] Based on a homography matrix, the first position is converted to a second position of the person in the indoor space; the floor in the indoor space is flat; the homography matrix is ​​formed by selecting positions from an elliptical or quasi-elliptical air supply area isotherm map of the floor when the air conditioner is activated and the temperature of the indoor space is stabilized;

[0013] The air conditioner is controlled to supply air to the second position according to a preset air supply strategy.

[0014] Optionally, the ground in the indoor space is flat; before the step of converting the first position to the second position of the person in the indoor space based on the homography matrix, the method further includes:

[0015] When the air conditioner is started, after the temperature of the indoor space is stabilized, obtaining an isothermal map of the elliptical or quasi-elliptical air supply area on the ground;

[0016] Select at least 4 positions from the air supply area isotherm diagram;

[0017] Based on the at least four positions, establishing a coordinate system of the ground and a homography matrix corresponding to a coordinate system of the infrared image;

[0018] The converting the first position to a second position of the person in the indoor space based on the homography matrix includes:

[0019] Based on the homography matrix, the first position is converted into a second position of the character on the ground.

[0020] Optionally, when the number of the first prediction frames is greater than or equal to 1, before the step of acquiring motion information of the person from the infrared image, the method further includes:

[0021] From all first prediction frames corresponding to the current infrared image, the first prediction frame that does not include the human body temperature point is deleted.

[0022] Optionally, inputting the infrared image into a YOLO network to obtain a first predicted frame containing a person in the current infrared image includes:

[0023] When no person appears in a previous infrared image corresponding to the current infrared image, and it is determined based on an inter-frame difference method that the current infrared image contains a person, the infrared image is input into a YOLO network to obtain a first prediction frame containing a person in the current infrared image.

[0024] Optionally, obtaining motion information of a person from the infrared image includes:

[0025] Acquire a first shooting interval duration between at least two frames of the historical infrared images;

[0026] Acquire a first relative displacement of the person in the at least two frames of historical infrared images, and a third position of the person in the last frame of the at least two frames of historical infrared images;

[0027] Based on the homography matrix, converting the first relative displacement into a second relative displacement of the person in the indoor space, and converting the third position into a fourth position of the person in the indoor space;

[0028] Determining a historical speed and a historical acceleration of the person in the indoor space based on the second relative displacement and the first shooting interval;

[0029] Obtaining a second shooting interval time between a last frame of the at least two frames of historical infrared images and the current infrared image;

[0030] Inputting the motion information into a motion model to obtain a second predicted frame containing a person in the current infrared image includes:

[0031] The motion model obtains a second prediction frame containing a person in the current infrared image according to the fourth position, the second shooting interval, the historical speed, and the historical acceleration.

[0032] Optionally, determining a first position of a person in the infrared image based on the first prediction frame and the second prediction frame includes:

[0033] For each of the first prediction frames, determining an intersection-over-union (IoU) ratio between the second prediction frame and the first prediction frame;

[0034] For each of the first prediction frames, when the intersection-over-union ratio is greater than a preset intersection-over-union ratio, increasing the confidence of the first prediction frame according to a preset method;

[0035] The position of the first prediction frame with the highest confidence among all the first prediction frames corresponding to the current infrared image is determined as the first position.

[0036] Optionally, the resolution of a single infrared camera is m×n, where m and n are both natural numbers ranging from 16 to 48.

[0037] In a second aspect, a control device for an air conditioner is provided, comprising:

[0038] An image acquisition module is used to acquire infrared images of an indoor space through a single infrared camera; the infrared images include: historical infrared images and current infrared images;

[0039] A first prediction frame acquisition module is used to input the infrared image into a YOLO network to obtain a first prediction frame containing a person in the current infrared image;

[0040] a motion information acquisition module, configured to acquire motion information of a person from the infrared image when the number of the first prediction frames is greater than or equal to 1;

[0041] A second prediction frame acquisition module is used to input the motion information into a motion model to obtain a second prediction frame containing a person in the current infrared image;

[0042] a first position determining module, configured to determine a first position of a person in the infrared image based on the first prediction frame and the second prediction frame;

[0043] a conversion module configured to convert the first position into a second position of the person in the indoor space based on a homography matrix; the floor in the indoor space being a flat floor; the homography matrix being formed by selecting positions from an elliptical or quasi-elliptical air supply area isotherm map of the floor when the air conditioner is activated and the temperature of the indoor space is stabilized;

[0044] The air supply module is used to control the air conditioner to supply air to the second position according to a preset air supply strategy.

[0045] According to a third aspect, an electronic device is provided, including:

[0046] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the aforementioned air conditioner control methods is implemented.

[0047] In a fourth aspect, a readable storage medium is provided, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute any of the aforementioned air conditioner control methods.

[0048] In a fifth aspect, a computer program product is provided, comprising instructions, which, when executed by a processor in an electronic device, enable the electronic device to execute any one of the aforementioned air conditioner control methods.

[0049] In the embodiments of the present application, firstly, the intelligent control of the air conditioner only requires a single infrared camera in conjunction with the YOLO network, motion model, etc. to realize the positioning of the person, which overcomes the hardware dependence of the existing air conditioning control on multiple sensors (such as depth cameras, lidars), and solves the problems of high cost and complex wiring; secondly, dynamic person tracking is realized through the intervention of the second prediction frame obtained by the motion information of the person and the motion model. Even in the case of fast movement or occlusion, the recognition accuracy is still high, ensuring the real-time and accuracy of the person positioning, reducing the missed detection rate, avoiding the problem of air supply lag, and having strong environmental adaptability, and being able to maintain accuracy under different lighting conditions and background interference. Maintain stable positioning of people; thirdly, the conversion of the first position in the infrared image to the second position in the indoor space is performed through the homography matrix, which has a fast response speed and shortens the time for air supply to the designated area; fourthly, when the ground in the indoor space is flat, since the establishment of the homography matrix is ​​based on the case where the air conditioner is started and the temperature of the indoor space is stabilized, the position is selected from the elliptical or quasi-elliptical air supply area isotherm map of the ground. The selection of the position is in line with the air supply law of air conditioning, and the position has an accurate correspondence between the ground and the plane where the infrared image is located, so the accuracy of the established homography matrix is ​​higher, which makes the accuracy of position conversion higher, and further improves the accuracy of air supply. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0051] Figure 1 This is a flowchart of a method for controlling an air conditioner according to an embodiment of the present invention;

[0052] Figure 2 This is an elliptical or quasi-elliptical air supply area isotherm diagram of an air conditioner provided by an embodiment of the present invention;

[0053] Figure 3 It is a non-elliptical or non-quasi-elliptical air supply area isotherm diagram of an air conditioner;

[0054] Figure 4 is a flowchart of another air conditioner control method provided by an embodiment of the present invention;

[0055] Figure 5 1 is a schematic structural diagram of a control device for an air conditioner provided by an embodiment of the present invention;

[0056] Figure 6This is a flowchart of the steps of another air conditioner control method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0058] This application provides a method for preparing an air conditioner control method, referring to Figure 1 , the method includes the following steps.

[0059] Step 101: Acquire an infrared image of an indoor space through a single infrared camera; the infrared image includes: a historical infrared image and a current infrared image.

[0060] Infrared cameras detect and utilize infrared radiation (typically with wavelengths between 0.76 microns and 1000 microns). In nature, all objects with temperatures above absolute zero (-273.15°C) emit infrared radiation, and the intensity of this radiation is directly related to the object's temperature. Infrared cameras capture this infrared radiation with built-in infrared detectors, convert it into electrical signals, and then use image processing algorithms to generate visible thermal images or videos. Infrared cameras also have temperature measurement capabilities, displaying the temperature distribution of target objects in real time. Certain infrared wavelengths can penetrate plastic, clothing, and other media, making them suitable for covert detection.

[0061] There are no restrictions on the size of indoor spaces. A single infrared camera refers to a single infrared camera, which can be fixed at a specific location indoors. Its field of view is fixed or relatively fixed during capture, allowing it to capture the entire indoor space. A single infrared camera can capture images approximately once per second, and the interval between captures can be slightly extended to reduce power consumption. The specific capture frequency is determined by detection accuracy and is not limited.

[0062] Historical infrared images refer to infrared images taken before the current infrared image. There is no limit on the number of historical infrared images that can be associated with a current infrared image.

[0063] Optionally, the resolution of the infrared camera is m×n, and m and n are both 16 to 48. Specifically, m refers to width and n refers to height. For infrared cameras, the medium resolution is generally around 60×80. In this application, m and n are both natural numbers from 16 to 48, indicating that the infrared camera in this application is a low-resolution infrared camera, which can replace the complex configuration of high-resolution infrared arrays or multimodal sensors in related technologies, thereby further reducing costs. In this application, even low-resolution infrared cameras can achieve high-precision character positioning by combining software such as the YOLO network and motion models. The relative sizes of m and n here are not limited.

[0064] For example, the resolution of the infrared camera may be 24×32. For another example, the resolution of the infrared camera may be 16×16, 16×24, 24×16, 24×48, 32×24, 32×48, etc.

[0065] Step 102: Input the infrared image into a YOLO network to obtain a first prediction frame containing a person in the current infrared image.

[0066] The YOLO network achieves end-to-end object detection and recognition through a single neural network, requiring only a single look at the network. This approach offers real-time performance and high accuracy. Specifically, both the historical infrared image and the current infrared image are fed into the YOLO network, which then outputs the first predicted box containing a person in the current infrared image. If the infrared camera captures infrared data, it also requires conversion to an image format, though this is not a requirement.

[0067] The main process of this step may include: dividing the input infrared image into S×S grids, each grid is responsible for predicting the target whose center point falls within it; predicting B bounding boxes for each grid, each bounding box contains the position (x, y, w, h), confidence (confidence) and category probability (C categories); the category here specifically refers to people; removing redundant bounding boxes through non-maximum suppression (NMS), and outputting the final detection result, which includes: the first predicted box containing the person.

[0068] For the current infrared image, there is no limit on the number of first prediction boxes containing people output by the YOLO network.

[0069] Step 103: When the number of the first prediction frames is greater than or equal to 1, obtain motion information of the person from the infrared image.

[0070] If the number of first prediction frames is greater than or equal to 1, it indicates that the current infrared image may contain a person. To further improve the accuracy of person location, motion information of the person is obtained from the infrared image. This motion information may include speed, acceleration, direction of movement, and motion state. Examples of motion states include static standing, sitting, lying down, and dynamic walking.

[0071] Step 104 : Input the motion information into a motion model to obtain a second predicted frame containing a person in the current infrared image.

[0072] The motion model primarily predicts a character's next location based on their historical positions. A prerequisite for the motion model is the motion hypothesis, which is based on two core assumptions: the continuity of a character's movement and the smoothness of their speed and direction. Motion prediction requires information about the character's movement, including speed, direction, acceleration, and motion state.

[0073] Step 105 : Determine a first position of the person in the infrared image based on the first prediction frame and the second prediction frame.

[0074] This step is to use the second prediction frame output by the motion model to appropriately correct or supplement the first prediction frame, reduce the missed detection rate and false detection rate and improve the detection accuracy, and then accurately determine the position of the person in the current infrared image.

[0075] Step 106: Based on a homography matrix, convert the first position to a second position of the person in the indoor space; the ground in the indoor space is flat; the homography matrix is ​​formed by selecting positions from an elliptical or quasi-elliptical air supply area isotherm map of the ground when the air conditioner is turned on and the temperature of the indoor space is stabilized.

[0076] A homography is a 3×3 reversible matrix used in computer vision to describe the projective transformation relationship between two planes in the same scene. Its core function is to map points on one plane to corresponding points on another plane. In this application, a homography is a reversible matrix that describes the transformation relationship between the plane containing the infrared image of the indoor space captured by the aforementioned single infrared camera and a plane within the indoor space.

[0077] In this application, the homography matrix is ​​combined with algorithms such as RANSAC (Random Sample Consensus Algorithm) to significantly improve robustness. The RANSAC algorithm can effectively eliminate unqualified data and provide faster and more accurate recognition results for data samples containing some erroneous data.

[0078] This step is to accurately and quickly convert the first position in the current infrared image to the second position in the indoor space through the homography matrix.

[0079] The floor in the indoor space is flat. Flat here means that the surface has little undulation, for example, the distance between the highest point and the lowest point of the floor is less than or equal to 10 cm. Specifically, when the air is centrally supplied by air conditioning, if the air supply area is flat, according to fluid mechanics, after the indoor space stabilizes, the temperature distribution range on the floor is a relatively regular ellipse ( Figure 2 As shown in ), if there is an obstacle blocking the area (i.e., a non-flat area), the isothermal map of the action area is quite different from the ellipse (as shown in Figure 3 As shown in Figure 2 ). Therefore, if the ground is flat, the isothermal map of the supply air area temperature distribution on the ground is elliptical or quasi-elliptical. From this elliptical or quasi-elliptical isothermal map of the supply air area, a location can be selected and the homography matrix can be formed based on the selected location. After a certain period of time after the air conditioner is turned on, it can be assumed that the temperature of the indoor space is stable.

[0080] When the floor of the indoor space is flat, the homography matrix is ​​established by selecting a position from the isothermal map of the elliptical or quasi-elliptical air supply area on the ground after the air conditioner is started and the temperature of the indoor space is stabilized. The selection of this position conforms to the air supply law of the air conditioner, and the position has an accurate correspondence between the ground and the plane of the infrared image. Therefore, the accuracy of the established homography matrix is ​​higher, which makes the position conversion more accurate and further improves the accuracy of the air supply.

[0081] Step 107: Control the air conditioner to supply air to the second location according to a preset air supply strategy.

[0082] Preset air supply strategies may include direct air supply, wind-shielded air supply, etc. Direct air supply may control the air conditioner to blow air directly on the user. Wind-shielded air supply may control the air conditioner to blow air away from the user, but still provide the desired temperature.

[0083] For example, here, the air conditioner's air guide and fan can be adjusted, and combined with the heat source movement direction in the infrared image, the air conditioner's air supply area can be gradually approached. This air supply area is determined by the air supply strategy and the second position. For example, by controlling the air guide and wind speed, the air conditioner's air supply area can be quickly moved along the relative position vector to the second position where the person is located, based on an iterative method. Then, fine-tuning is performed according to the air supply strategy until the designated air supply area is reached. For example, based on the second position and the air supply strategy, the air supply position on the ground is determined to have coordinates (3, 6) relative to the second position. By controlling the air guide and wind speed, the air conditioner moves the active area a small step in the direction of (3, 6) to (1, 2). At this point, the relative position becomes (2, 4). The above steps are repeated until the designated position is reached.

[0084] In the embodiments of the present application, firstly, the intelligent control of the air conditioner only requires a single infrared camera in conjunction with the YOLO network, motion model, etc. to realize the positioning of the person, which overcomes the hardware dependence of the existing air conditioning control on multiple sensors (such as depth cameras, lidars), and solves the problems of high cost and complex wiring; secondly, dynamic person tracking is realized through the intervention of the second prediction frame obtained by the motion information of the person and the motion model. Even in the case of fast movement or occlusion, the recognition accuracy is still high, ensuring the real-time and accuracy of the person positioning, reducing the missed detection rate, avoiding the problem of air supply lag, and having strong environmental adaptability, and being able to maintain accuracy under different lighting conditions and background interference. Maintain stable positioning of people; thirdly, the conversion of the first position in the infrared image to the second position in the indoor space is performed through the homography matrix, which has a fast response speed and shortens the time for air supply to the designated area; fourthly, when the ground in the indoor space is flat, since the establishment of the homography matrix is ​​based on the case where the air conditioner is started and the temperature of the indoor space is stabilized, the position is selected from the elliptical or quasi-elliptical air supply area isotherm map of the ground. The selection of the position is in line with the air supply law of air conditioning, and the position has an accurate correspondence between the ground and the plane where the infrared image is located, so the accuracy of the established homography matrix is ​​higher, which makes the accuracy of position conversion higher, and further improves the accuracy of air supply.

[0085] This application provides a method for preparing an air conditioner control method, referring to Figure 4 , the method includes the following steps.

[0086] Step 201 : Acquire an infrared image of an indoor space through a single infrared camera; the infrared image includes: a historical infrared image and a current infrared image.

[0087] Step 201 can refer to the description of the aforementioned step 101, and will not be repeated here to avoid repetition.

[0088] In step 202 , when no person appears in a previous infrared image corresponding to the current infrared image and it is determined based on an inter-frame difference method that the current infrared image contains a person, the infrared image is input into a YOLO network to obtain a first prediction frame containing a person in the current infrared image.

[0089] The previous infrared image corresponding to the current infrared image refers to an infrared image that is captured at a time adjacent to that of the current infrared image and is captured before the current infrared image.

[0090] When there is no person in the previous infrared image corresponding to the current infrared image, the inter-frame difference method can be used to determine whether the current infrared image contains a person. The inter-frame difference method (frame difference method) is a simple and efficient moving target detection algorithm based on the pixel difference of consecutive frame images in a video sequence. Specifically, the difference between the two frames can be obtained by subtracting the two frames, for example, the brightness difference between the two frames can be obtained. Absolute value , to determine whether it is greater than Threshold To analyze the motion characteristics of a video or image sequence and determine whether there is human movement or human intrusion into an indoor space in the image sequence. When the frame difference method is used to determine that the current infrared image contains a person, the above infrared image is input into the YOLO network to obtain the first prediction frame containing the person in the current infrared image. Compared with the YOLO network, the aforementioned frame difference method has a significantly smaller amount of calculation and can also ensure a higher accuracy rate. The present application significantly reduces the amount of calculation and has lower power consumption. It only uses the difference between the two frames of images to further accurately determine whether there is a person intruding. If not, there is no one in this frame of infrared image. The power consumption is low, the required amount of calculation is small, and the implementation speed is fast.

[0091] The steps for obtaining the first prediction frame can refer to the corresponding records above. In order to avoid repetition, they will not be repeated here.

[0092] Step 203 : Deleting the first prediction frame that does not include the human body temperature point from all first prediction frames corresponding to the current infrared image.

[0093] The current infrared image may contain temperature information. The human body temperature is generally within a limited range, for example, around 36°C to 37°C. The human body temperature is usually somewhat different from the ambient temperature of the indoor space. Therefore, the first prediction box is screened based on the temperature distribution in the current infrared image. The first prediction box that does not contain the human body temperature point is likely a misjudgment and is deleted. This can effectively avoid false detections by the YOLO network, not only improving the accuracy of subsequent human position detection, but also requiring less computational effort and faster speed for temperature point detection.

[0094] Step 204 : When the number of the first prediction frames is greater than or equal to 1, obtain motion information of the person from the infrared image.

[0095] For the current infrared image, after the aforementioned step 203 , the number of first prediction frames is the number of first prediction frames obtained for the current infrared image.

[0096] Specifically, step 204 can protect the following steps. Step 2041, obtain the first shooting interval between at least two frames of historical infrared images. Step 2042, obtain the first relative displacement of the person in the at least two frames of historical infrared images, and the third position of the person in the last frame of the at least two frames of historical infrared images. Step 2043, based on the homography matrix, convert the first relative displacement into the second relative displacement of the person in the indoor space, and convert the third position into the fourth position of the person in the indoor space. Step 2044, based on the second relative displacement and the first shooting interval, determine the historical speed and historical acceleration of the person in the indoor space. Step 2045, obtain the second shooting interval between the last frame of the at least two frames of historical infrared images and the current infrared image.

[0097] The at least two historical infrared images herein may be at least two infrared images captured at adjacent times to the current infrared image and captured before the current infrared image. The first shooting interval may refer to the time interval between the capture times of the first infrared image and the last infrared image in the at least two historical infrared images. The at least two historical infrared images may both contain the same person. The third position refers to the position of the person in the last infrared image in the at least two historical infrared images.

[0098] The first relative displacement may refer to: in at least two frames of historical infrared images, the displacement between the position of the person in the first infrared image and the position of the person in the last infrared image.

[0099] In this application, a homography is a reversible matrix that describes the transformation relationship between the plane containing the infrared image of the indoor space captured by the single infrared camera and a plane within the indoor space. Therefore, the homography can be used to convert a first relative displacement obtained in the infrared image into a second relative displacement of the person within the indoor space. The homography can also be used to convert a third positional displacement obtained in the infrared image into a fourth position of the person within the indoor space.

[0100] The second relative displacement = historical speed × first shooting interval length + × historical acceleration × first shooting interval length × first shooting interval length, this calculation method is used to determine the historical acceleration and historical speed here.

[0101] Second shooting interval time It refers to the shooting interval between the shooting time of the last frame of the at least two frames of historical infrared images and the shooting time of the current infrared image.

[0102] Step 205: Input the motion information into a motion model to obtain a second predicted frame containing a person in the current infrared image.

[0103] On the basis of the aforementioned step 204 including step 2041 and step 2045, the step 205 may include: the motion model obtains a second prediction frame containing a person in the current infrared image based on the fourth position, the second shooting interval, the historical speed and the historical acceleration.

[0104] Specifically, the fourth position is , and calculate the historical speed of the character , the historical acceleration is , we can predict the position of the person in the indoor space at the time of shooting the current frame infrared image according to the following formula 1 and formula 2 ( ).

[0105] Formula 1

[0106] Formula 2

[0107] Then, based on the aforementioned homography matrix, the predicted shooting time of the current frame infrared image and the position of the person in the indoor space are converted into the shooting time of the current frame infrared image and the position of the person in the current frame infrared image, and the size of the second prediction box can be predicted.

[0108] The above method of determining the second prediction frame conforms to the relevant laws of kinematics and the continuity, speed and direction smoothness of human movement. The second prediction frame determined in this way is more accurate, which is conducive to improving the accuracy of character position prediction.

[0109] Step 206: Determine a first position of the person in the infrared image based on the first prediction frame and the second prediction frame.

[0110] Step 206 may include the following steps. Step 2061: For each of the first prediction frames, determine the intersection-over-union (IoU) of the second prediction frame with the first prediction frame. Step 2062: For each of the first prediction frames, if the IoU is greater than a preset IoU, increase the confidence of the first prediction frame in a preset manner. Step 2063: Determine the position of the first prediction frame with the highest confidence among all the first prediction frames corresponding to the infrared image as the first position.

[0111] For the current infrared image, after step 203, the number of first prediction frames is the number of first prediction frames obtained for the current infrared image. Using the motion model, one second prediction frame is typically obtained for the current infrared image. Here, for each first prediction frame in the current infrared image, the intersection over union (IOU) ratio (IOU) is calculated with the second prediction frame in the same frame.

[0112] For each first prediction frame of the current infrared image of the same person in the same frame, if the aforementioned IoU is greater than the preset IoU, it indicates that the YOLO network and motion model have both predicted the presence of a person in the first prediction frame and its surroundings, and the confidence level can be appropriately increased. The confidence level of the first prediction frame can be increased according to the preset method shown in the following formula 3.

[0113]

[0114] In the above formula 3, refers to the confidence of the first prediction box after improvement, p is the confidence of the first prediction box before improvement, or in other words, p is the confidence of the first prediction box output by the YOLO network. T is the preset intersection-over-union ratio.

[0115] Then, the position of the first prediction frame with the highest confidence among all the first prediction frames corresponding to the infrared image is determined as the first position of the person in the current infrared image.

[0116] For each first prediction frame of the current infrared image of the same person in the same frame, if the aforementioned intersection-over-union ratio is greater than the preset intersection-over-union ratio, it means that the YOLO network and motion model in the first prediction frame and its surroundings have predicted the existence of a person. The confidence level can be appropriately increased. From all the first prediction frames corresponding to the current infrared image, the position of the first prediction frame with the highest confidence level is selected as the predicted position of the person in the current infrared image, which can reduce the false detection rate or missed detection rate.

[0117] Step 207: Based on a homography matrix, convert the first position to a second position of the person in the indoor space; the ground in the indoor space is flat; the homography matrix is ​​formed by selecting positions from an elliptical or quasi-elliptical air supply area isotherm map of the ground when the air conditioner is turned on and the temperature of the indoor space is stabilized.

[0118] Before step 207, the method may further include: step S1, after the temperature of the indoor space is stabilized, obtaining an elliptical or quasi-elliptical air supply area isotherm map of the ground; step S2, selecting at least 4 positions from the air supply area isotherm map; step S3, based on the at least 4 positions, establishing a coordinate system of the ground and a homography matrix corresponding to the coordinate system of the infrared image.

[0119] Specifically, referring to the above description, when the ground surface is flat, the isothermal map of the air supply area of ​​the ground temperature distribution is elliptical or quasi-elliptical. From this elliptical or quasi-elliptical isothermal map of the air supply area, at least four positions can be randomly selected, or at least four positions can be equidistantly selected, etc., without specific limitation.

[0120] The coordinates of at least four locations selected from the elliptical or quasi-elliptical air supply area isotherm map may be (xj, yj), where j is a natural number ranging from 1 to 4. The homography matrix may be determined using the following formula 4. In the following formula 4, the coordinates in the infrared image coordinate system may be (ui, vi).

[0121]

[0122] In the above formula 4, s is the scale factor. RANSAC loss function , the loss function can be determined by the following formula , loss function The pros and cons of homography matrices.

[0123]

[0124] In the above formulas 4 and 5, h 11 、h 12 、h 21 、h 22 : Controls the rotation, scaling and shearing transformations of an image. For example: h 11 and h 12 Affects the scaling and rotation of the x-coordinate. 21 and h 22 Affects the scaling and rotation of the y coordinate. 13 and h 23 : Controls the translation transformation of the image. For example: h 13 Affects the translation of the x coordinate. h 23 Affects the translation of the y coordinate. h 31 and h 32 : Controls the perspective transformation of the image (i.e. nonlinear transformation). For example: h 31 Affects the perspective transformation of the x′ coordinate. h 32 Affects the perspective transformation of the y coordinate. h 33 : Usually set to 1 for scale normalization.

[0125] When the floor in the indoor space is flat, the homography matrix is ​​established based on at least four positions in the isothermal map of the elliptical or quasi-elliptical air supply area. The selection of the at least four positions conforms to the air supply law of air conditioning, and the at least four positions have an accurate correspondence between the ground and the plane where the infrared image is located. Therefore, the accuracy of the established homography matrix is ​​higher, which makes the position conversion more accurate.

[0126] It should be noted that the construction of the homography matrix can also be performed before step 2043.

[0127] Step 208: Control the air conditioner to supply air to the second location according to a preset air supply strategy.

[0128] The step 208 may refer to the aforementioned step 107 and will not be described again here to avoid repetition.

[0129] This application also provides a control device for an air conditioner, referring to Figure 5 ,include:

[0130] The image acquisition module 301 is used to acquire infrared images of the indoor space through a single infrared camera; the infrared images include: historical infrared images and current infrared images;

[0131] A first prediction frame acquisition module 302 is configured to input the infrared image into a YOLO network to acquire a first prediction frame containing a person in the current infrared image;

[0132] A motion information acquisition module 303 is configured to acquire motion information of a person from the infrared image when the number of the first prediction frames is greater than or equal to 1;

[0133] A second prediction frame acquisition module 304 is configured to input the motion information into a motion model to acquire a second prediction frame containing a person in the current infrared image;

[0134] A first position determining module 305 is configured to determine a first position of a person in the infrared image based on the first prediction frame and the second prediction frame;

[0135] A conversion module 306 is configured to convert the first position into a second position of the person in the indoor space based on a homography matrix; the floor in the indoor space is flat; the homography matrix is ​​formed by selecting positions from an elliptical or quasi-elliptical air supply area isotherm map of the floor when the air conditioner is activated and the temperature of the indoor space is stabilized;

[0136] The air supply module 307 is used to control the air conditioner to supply air to the second location according to a preset air supply strategy.

[0137] Optionally, the device further includes:

[0138] an isotherm acquisition module, configured to acquire an isotherm of an elliptical or quasi-elliptical air supply area on the ground after the temperature of the indoor space stabilizes when the air conditioner is started;

[0139] A position selection module, configured to select at least four positions from the air supply area isotherm map;

[0140] a homography matrix establishing module, configured to establish, based on the at least four positions, a homography matrix corresponding to the coordinate system of the ground and the coordinate system of the infrared image;

[0141] The conversion module 306 includes:

[0142] A first conversion unit is configured to convert the first position into a second position of the person on the ground based on the homography matrix.

[0143] Optionally, the device further includes:

[0144] The deletion module is used to delete the first prediction frame that does not include the human body temperature point from all first prediction frames corresponding to the current infrared image.

[0145] Optionally, the first predicted frame acquisition module 302 includes:

[0146] The first prediction frame acquisition unit is used to input the infrared image into the YOLO network to obtain a first prediction frame containing the person in the current infrared image when no person appears in the previous infrared image corresponding to the current infrared image and it is determined based on the inter-frame difference method that the current infrared image contains the person.

[0147] Optionally, the motion information acquisition module 303 includes:

[0148] A first shooting interval duration acquisition unit, configured to acquire a first shooting interval duration between at least two frames of historical infrared images;

[0149] a displacement and position acquisition unit, configured to acquire a first relative displacement of the person in the at least two frames of historical infrared images, and a third position of the person in the last frame of the at least two frames of historical infrared images;

[0150] a second conversion unit, configured to convert the first relative displacement into a second relative displacement of the person in the indoor space, and convert the third position into a fourth position of the person in the indoor space based on the homography matrix;

[0151] a historical speed obtaining unit, configured to determine a historical speed and a historical acceleration of the person in the indoor space based on the second relative displacement and the first shooting interval;

[0152] A second shooting interval duration acquisition unit is configured to acquire a second shooting interval duration between a last frame of the at least two frames of historical infrared images and the current infrared image;

[0153] The second prediction frame acquisition module 304 includes:

[0154] A second prediction frame acquisition unit is used for the motion model to acquire a second prediction frame containing a person in the current infrared image according to the fourth position, the second shooting interval, the historical speed and the historical acceleration.

[0155] Optionally, the first location determination module 305 includes:

[0156] an IoU determining unit, configured to determine, for each of the first prediction frames, an IoU between the second prediction frame and the first prediction frame;

[0157] a confidence improving unit, configured to, for each of the first prediction frames, improve the confidence of the first prediction frame in a preset manner when the intersection-over-union ratio is greater than a preset intersection-over-union ratio;

[0158] The first position determining unit is configured to determine the position of the first prediction frame having the highest confidence among all the first prediction frames corresponding to the current infrared image as the first position.

[0159] Optionally, the resolution of a single infrared camera is m×n, where m and n are both natural numbers ranging from 16 to 48.

[0160] The following combination Figure 6 , combined with specific embodiments, further explain this application.

[0161] In the first step, the floor of the indoor space is flat. When the air conditioner is started, after the temperature of the indoor space stabilizes, an isothermal map of an elliptical or quasi-elliptical air supply area of ​​the floor is obtained.

[0162] The second step is to select at least 4 positions from the above-mentioned isothermal map of the air supply area. For example, 25 positions are selected at equal intervals from the isothermal map of the air supply area.

[0163] The first and second steps correspond to Figure 6 Filter the heat source center.

[0164] The third step is to establish the coordinate system of the ground and the homography matrix corresponding to the coordinate system of the infrared image based on the aforementioned at least four positions.

[0165] The fourth step is to obtain an infrared image of the indoor space through a single infrared camera; the infrared image includes: a historical infrared image and a current infrared image;

[0166] The fifth step is to input the above infrared image into the YOLO network to obtain the first prediction frame containing the person in the current infrared image (corresponding to Figure 6 Detecting people in .

[0167] In the sixth step, when the number of the first prediction frames is greater than or equal to 1, the motion information of the person is obtained from the infrared image.

[0168] Step 7: Input the above motion information into the motion model to obtain the second prediction frame containing the person in the current infrared image (steps 6 and 7 correspond to Figure 6 Motion model detection in .

[0169] Step 8: Based on the first prediction frame and the second prediction frame, determine the first position of the person in the current infrared image (corresponding to the first prediction frame and the second prediction frame). Figure 6 In the case of a person, the ninth step is performed; in the case of a person, the eleventh step is performed.

[0170] The ninth step is to convert the first position into the second position of the person in the indoor space based on the homography matrix. Figure 6 The relative position vector in the transformation corresponds to ).

[0171] Step 10: Control the air conditioner to supply air to the second position according to the preset air supply strategy (with Figure 6 (corresponding to the movement of the middle air supply area).

[0172] The eleventh step is to determine whether the next infrared image contains a person (with Figure 6 If there is a person, continue with step 5. If there is no person, continue to use the frame difference method to determine whether the next infrared image contains a person.

[0173] It should be noted that, in the embodiments, all calculation formulas in each step, etc., refer to the above description, and in order to avoid repetition, they are not repeated here.

[0174] The present application also provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the aforementioned air conditioner control methods when executing the program.

[0175] The present application also provides a readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any of the aforementioned air conditioner control methods.

[0176] The present application also provides a computer program product, comprising instructions, which, when executed by a processor in an electronic device, enable the electronic device to execute any of the aforementioned air conditioner control methods.

[0177] It should be noted that the control device, electronic device, readable storage medium and computer program product can all refer to the description of the aforementioned control method and have the same or similar beneficial effects. In order to avoid repetition, they will not be described here.

[0178] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0179] The present application also provides an electrical appliance, including any of the aforementioned air conditioner control methods. The type of the electrical appliance is not specifically limited; for example, the electrical appliance may be an air conditioner, a robotic arm, or the like. The preparation method for the electrical appliance and the aforementioned air conditioner control method has the same or similar beneficial effects as any of the aforementioned air conditioner control methods, and to avoid repetition, is not further described here.

[0180] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0181] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for controlling an air conditioner, characterized in that: include: Acquire infrared images of indoor spaces through a single infrared camera; The infrared image includes: a historical infrared image and a current infrared image; Inputting the infrared image into a YOLO network to obtain a first predicted frame containing a person in the current infrared image; When the number of the first prediction frames is greater than or equal to 1, acquiring motion information of the person from the infrared image; Inputting the motion information into a motion model to obtain a second predicted frame containing a person in the current infrared image; Determining a first position of a person in the infrared image based on the first prediction frame and the second prediction frame; Converting the first position to a second position of the person in the indoor space based on a homography matrix; controlling the air conditioner to supply air to the second location according to a preset air supply strategy; The ground in the indoor space is flat; before the step of converting the first position into the second position of the person in the indoor space based on the homography matrix, the method further includes: When the air conditioner is started, after the temperature of the indoor space is stabilized, obtaining an isothermal map of the elliptical or quasi-elliptical air supply area on the ground; Select at least 4 positions from the air supply area isotherm diagram; Based on the at least four positions, establishing a coordinate system of the ground and a homography matrix corresponding to a coordinate system of the infrared image; The converting the first position to a second position of the person in the indoor space based on the homography matrix includes: Based on the homography matrix, the first position is converted into a second position of the character on the ground.

2. The air conditioner control method according to claim 1, characterized in that: Before the step of acquiring motion information of a person from the infrared image when the number of the first prediction frames is greater than or equal to 1, the method further includes: From all first prediction frames corresponding to the current infrared image, the first prediction frame that does not include the human body temperature point is deleted.

3. The air conditioner control method according to claim 1, characterized in that: Inputting the infrared image into a YOLO network to obtain a first predicted frame containing a person in the current infrared image includes: When no person appears in a previous infrared image corresponding to the current infrared image, and it is determined based on an inter-frame difference method that the current infrared image contains a person, the infrared image is input into a YOLO network to obtain a first prediction frame containing a person in the current infrared image.

4. The air conditioner control method according to claim 1, characterized in that: The acquiring the motion information of the person from the infrared image includes: Obtaining a first shooting interval duration between at least two frames of historical infrared images; Acquire a first relative displacement of the person in the at least two frames of historical infrared images, and a third position of the person in the last frame of the at least two frames of historical infrared images; Based on the homography matrix, converting the first relative displacement into a second relative displacement of the person in the indoor space, and converting the third position into a fourth position of the person in the indoor space; Determining a historical speed and a historical acceleration of the person in the indoor space based on the second relative displacement and the first shooting interval; Obtaining a second shooting interval time between a last frame of the at least two frames of historical infrared images and the current infrared image; Inputting the motion information into a motion model to obtain a second predicted frame containing a person in the current infrared image includes: The motion model obtains a second prediction frame containing a person in the current infrared image according to the fourth position, the second shooting interval, the historical speed, and the historical acceleration.

5. The air conditioner control method according to any one of claims 1 to 4, characterized in that: The determining, based on the first prediction frame and the second prediction frame, a first position of the person in the infrared image includes: For each of the first prediction frames, determining an intersection-over-union (IoU) ratio between the second prediction frame and the first prediction frame; For each of the first prediction frames, when the intersection-over-union ratio is greater than a preset intersection-over-union ratio, increasing the confidence of the first prediction frame according to a preset method; The position of the first prediction frame with the highest confidence among all the first prediction frames corresponding to the current infrared image is determined as the first position.

6. The air conditioner control method according to any one of claims 1 to 4, characterized in that: The resolution of a single infrared camera is m×n, where m and n are both natural numbers ranging from 16 to 48.

7. A control device for an air conditioner, characterized in that: include: An image acquisition module, used to acquire infrared images of indoor spaces through a single infrared camera; The infrared image includes: a historical infrared image and a current infrared image; A first prediction frame acquisition module is used to input the infrared image into a YOLO network to obtain a first prediction frame containing a person in the current infrared image; a motion information acquisition module, configured to acquire motion information of a person from the infrared image when the number of the first prediction frames is greater than or equal to 1; A second prediction frame acquisition module is used to input the motion information into a motion model to obtain a second prediction frame containing a person in the current infrared image; a first position determining module, configured to determine a first position of a person in the infrared image based on the first prediction frame and the second prediction frame; A conversion module, configured to convert the first position into a second position of the person in the indoor space based on a homography matrix; an air supply module, configured to control the air conditioner to supply air to the second location according to a preset air supply strategy; The floor in the indoor space is a flat floor; the device further comprises: an isotherm acquisition module, configured to acquire an isotherm of an elliptical or quasi-elliptical air supply area on the ground after the temperature of the indoor space stabilizes when the air conditioner is started; A position selection module, configured to select at least four positions from the air supply area isotherm map; a homography matrix establishing module, configured to establish, based on the at least four positions, a homography matrix corresponding to the coordinate system of the ground and the coordinate system of the infrared image; The conversion module includes: A first conversion unit is configured to convert the first position into a second position of the person on the ground based on the homography matrix.

8. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the air conditioner control method according to any one of claims 1 to 6 when executing the program.

9. A readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the air conditioner control method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises instructions, which, when executed by a processor in an electronic device, enable the electronic device to execute the air conditioner control method according to any one of claims 1 to 6.

Images