Air conditioner control method and device, electronic equipment, storage medium and program product

Abstract

The invention provides an air conditioner control method and device, electronic equipment, a storage medium and a program product, and relates to the field of air conditioner control. The method comprises the following steps: acquiring an infrared image of an indoor space through a single infrared camera; the infrared image comprises a historical infrared image and a current infrared image; inputting the infrared image into a YOLO network, and obtaining a first prediction frame containing a figure in the current infrared image; under the condition that the number of the first prediction frames is greater than or equal to 1, obtaining motion information of a person from the infrared image; inputting the motion information into a motion model, and obtaining a second prediction frame containing a figure in the current infrared image; determining a first position of the person in the infrared image based on the first prediction frame and the second prediction frame; converting the first position into a second position of the person in the indoor space based on a homography matrix; and controlling the air conditioner to supply air to the second position according to a preset air supply strategy. According to the invention, the cost is reduced, and the problem of air supply lag is avoided.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of air conditioner control, and in particular, to a control method and device for an air conditioner, an electronic device, a storage medium, and a program product. Background Art

[0002] With the progress of technology, people's demands for air conditioners have changed, and the pursuit of health and comfort is getting higher and higher. Therefore, the intelligent air supply mode has correspondingly emerged.

[0003] Currently, in an air conditioner, intelligent air supply is usually achieved by relying on multi-sensor fusion or high-cost depth cameras and other hardware.

[0004] However, the existing intelligent air supply mode of air conditioners has a high degree of dependence on hardware, and the cost of the above-mentioned hardware is high, resulting in a high cost of the air conditioner and complex wiring. Summary of the Invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide a control method and device for an air conditioner, an electronic device, a storage medium, and a program product that overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, an embodiment of the present application discloses a control method for an air conditioner, including: Obtaining an infrared image of an indoor space through a single infrared camera; the infrared image includes: a historical infrared image and a current infrared image; Inputting the infrared image into a YOLO network to obtain a first prediction box containing a person in the current infrared image; When the number of the first prediction boxes is greater than or equal to 1, obtaining motion information of the person from the infrared image; Inputting the motion information into a motion model to obtain a second prediction box containing a person in the current infrared image; Based on the first prediction box and the second prediction box, determining a first position of the person in the infrared image; Based on a homography matrix, converting the first position into a second position of the person in the indoor space; the ground in the indoor space is a flat ground; the homography matrix is formed by selecting positions from an isothermal map of an elliptical or quasi-elliptical air supply area on the ground after the temperature in the indoor space is stable when the air conditioner is started; Controlling the air conditioner to supply air to the second position according to a preset air supply strategy.

[0007] Optionally, the ground in the indoor space is a flat ground; before the step of converting the first position into the second position of the person in the indoor space based on the homography matrix, the method further includes: When the air conditioner is started, after the temperature in the indoor space stabilizes, obtain an isothermal map of an elliptical or quasi-elliptical air supply area on the ground; Select at least 4 positions from the isothermal map of the air supply area; Based on the at least 4 positions, establish a homography matrix corresponding to the coordinate system of the ground and the coordinate system of the infrared image; The converting the first position to the second position of the person in the indoor space based on the homography matrix includes: Based on the homography matrix, convert the first position to the second position of the person on the ground.

[0008] Optionally, before the step of obtaining the motion information of the person from the infrared image when the number of the first prediction boxes is greater than or equal to 1, the method further includes: Delete the first prediction boxes that do not contain human body temperature points from all the first prediction boxes corresponding to the current infrared image.

[0009] Optionally, the inputting the infrared image into the YOLO network to obtain the first prediction boxes containing people in the current infrared image includes: When there is no person in the previous infrared image corresponding to the current infrared image and it is determined that there is a person in the current infrared image based on the inter-frame difference method, input the infrared image into the YOLO network to obtain the first prediction boxes containing people in the current infrared image.

[0010] Optionally, the obtaining the motion information of the person from the infrared image includes: Obtain the first shooting interval duration between at least two frames of the historical infrared images; Obtain the first relative displacement of the person in the at least two frames of historical infrared images, and the third position of the person in the last frame of the at least two frames of historical infrared images; Based on the homography matrix, convert the first relative displacement to the second relative displacement of the person in the indoor space, and convert the third position to the fourth position of the person in the indoor space; Based on the second relative displacement and the first shooting interval duration, determine the historical speed and historical acceleration of the person in the indoor space; Obtain the second shooting interval duration between the last frame of the at least two frames of historical infrared images and the current infrared image; The inputting the motion information into the motion model to obtain the second prediction boxes containing people in the current infrared image includes: The motion model obtains a second prediction box containing a person in the current infrared image according to the fourth position, the second shooting interval duration, the historical speed, and the historical acceleration.

[0011] Optionally, determining the first position of the person in the infrared image based on the first prediction box and the second prediction box includes: For each of the first prediction boxes, determining the intersection over union (IoU) between the second prediction box and the first prediction box; For each of the first prediction boxes, when the IoU is greater than a preset IoU, increasing the confidence of the first prediction box in a preset manner; Determining the position where the first prediction box with the highest confidence among all the first prediction boxes corresponding to the current infrared image is located as the first position.

[0012] Optionally, the resolution of a single infrared camera is m×n, where both m and n are natural numbers from 16 to 48.

[0013] In a second aspect, a control device for an air conditioner is provided, including: An image acquisition module, configured to acquire an infrared image of an indoor space through a single infrared camera; the infrared image includes: a historical infrared image and a current infrared image; A first prediction box acquisition module, configured to input the infrared image into a YOLO network to acquire a first prediction box containing a person in the current infrared image; A motion information acquisition module, configured to acquire the motion information of the person from the infrared image when the number of the first prediction boxes is greater than or equal to 1; A second prediction box acquisition module, configured to input the motion information into a motion model to acquire a second prediction box containing a person in the current infrared image; A first position determination module, configured to determine the first position of the person in the infrared image based on the first prediction box and the second prediction box; A conversion module, configured to convert the first position into a second position of the person in the indoor space based on a homography matrix; the ground in the indoor space is a flat ground; the homography matrix is formed by selecting positions from an isothermal map of an elliptical or quasi-elliptical air supply area on the ground after the temperature in the indoor space stabilizes when the air conditioner is started; An air supply module, configured to control the air conditioner to supply air to the second position according to a preset air supply strategy.

[0014] In a third aspect, an electronic device is provided, including: A processor, a memory, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, it implements any of the foregoing air conditioner control methods.

[0015] In a fourth aspect, a readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any of the foregoing air conditioner control methods.

[0016] In a fifth aspect, a computer program product is provided, including instructions that, when executed by a processor in an electronic device, cause the electronic device to execute any of the foregoing air conditioner control methods.

[0017] In the embodiments of the present application, first, for the intelligent control of the air conditioner, only a single infrared camera in cooperation with a YOLO network, a motion model, etc. can be used to achieve human positioning, overcoming the hardware dependence on multiple sensors (such as depth cameras, lidar) in the existing air conditioner control, and solving problems such as high cost and complex wiring; second, through the intervention of the second prediction box obtained from the motion information of the person and the motion model, dynamic human tracking is realized. Even in the case of fast movement or occlusion, the recognition accuracy is still relatively high, ensuring the real-time performance and accuracy of human positioning, reducing the missed detection rate, avoiding the problem of delayed air supply, and having strong environmental adaptability, and being able to maintain stable human positioning under different lighting conditions and background interferences; third, through the homography matrix, the conversion of the first position in the infrared image to the second position in the indoor space is carried out, with a fast response speed, shortening the time for the air to be sent to the designated area; fourth, when the ground in the indoor space is a flat ground, due to the establishment of this homography matrix, which is based on the situation after the air conditioner is started and the temperature in the indoor space is stable, the position is selected from the isothermal diagram of the elliptical or quasi-elliptical air supply area on the ground. The selection of this position conforms to the air supply law of the air conditioner, and there is an accurate corresponding relationship between this position on the ground and the plane where the infrared image is located. Therefore, the accuracy of the established homography matrix is higher, making the accuracy of the position conversion higher, and further improving the accuracy of the air supply. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is a flowchart of the steps of a method for controlling an air conditioner provided by an embodiment of the present invention; Figure 2It is an isothermal diagram of an elliptical or quasi-elliptical air supply area of an air conditioner provided by an embodiment of the present invention; Figure 3 It is an isothermal diagram of a non-elliptical or non-quasi-elliptical air supply area of an air conditioner; Figure 4 It is a step flow chart of another control method of an air conditioner provided by an embodiment of the present invention; Figure 5 It is a schematic structural diagram of a control device of an air conditioner provided by an embodiment of the present invention; Figure 6 It is a step flow chart of still another control method of an air conditioner provided by an embodiment of the present invention. Detailed implementation manners

[0020] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be fully conveyed to those skilled in the art.

[0021] The present application provides a preparation method for a control method of an air conditioner. Referring to Figure 1 , the method includes the following steps.

[0022] Step 101, obtaining an infrared image of the indoor space through a single infrared camera; the infrared image includes: a historical infrared image and a current infrared image.

[0023] An infrared camera can detect and utilize infrared radiation (the wavelength is usually between 0.76 micrometers and 1000 micrometers). In nature, all objects with a temperature higher than absolute zero (-273.15 °C) will emit infrared radiation, and the radiation intensity is directly related to the object temperature. The infrared camera captures this infrared radiation through a built-in infrared detector, converts it into an electrical signal, and then generates a visible thermal image or video through an image processing algorithm. The infrared camera also has a temperature measurement function and can display the temperature distribution of the target object in real time. Infrared rays in some bands can also penetrate media such as plastics and clothes and can also be applicable to concealed detection.

[0024] The area of the indoor space is not limited. A single infrared camera means only one infrared camera, which can be fixed at a certain position in the room, and its field of view range is fixed or relatively fixed during shooting, and the entire indoor space can be photographed. The shooting frequency of a single infrared camera can be about 1 time per second, and the shooting interval can also be slightly extended to reduce power consumption. The specific shooting frequency is determined by detection accuracy, etc., and is not limited thereto.

[0025] A historical infrared image refers to an infrared image whose shooting time is before the shooting time of the current infrared image. For a current infrared image, the number of corresponding historical infrared images is not limited.

[0026] Optionally, the resolution of the infrared camera is m×n, where both m and n are between 16 and 48. Specifically, m refers to the width and n refers to the height. For an infrared camera, the medium resolution is generally around 60×80. In this application, since both m and n are natural numbers between 16 and 48, it means that the infrared camera in this application is a low-resolution infrared camera. It can replace the complex configuration of high-resolution infrared arrays or multi-modal sensors in related technologies, and can further reduce costs. In this application, even a low-resolution infrared camera can achieve a relatively high-precision person localization by combining software such as the YOLO network and motion models. The relative sizes of m and n here are not limited.

[0027] For example, the resolution of the infrared camera can be 24×32. For another example, the resolution of the infrared camera can be 16×16, 16×24, 24×16, 24×48, 32×24, 32×48, etc.

[0028] Step 102: Input the infrared image into the YOLO network to obtain a first prediction box containing a person in the current infrared image.

[0029] The YOLO network refers to a You Only Look Once network, which realizes end-to-end object detection and recognition through a single neural network, and has characteristics such as strong real-time performance and high accuracy. Specifically here, both the above-mentioned historical infrared image and the current infrared image are input into the YOLO network, and the YOLO network outputs a first prediction box containing a person in the current infrared image. In the case where the infrared camera collects infrared data, it is also necessary to first convert the infrared data into an image format, and this is not limited.

[0030] The main process of this step may include: dividing the input infrared image into S×S grids, and each grid is responsible for predicting the object whose center point falls within it; each grid predicts B bounding boxes, and each bounding box contains position (x, y, w, h), confidence, and class probability (C classes); here the class specifically refers to a person; redundant bounding boxes are removed through non-maximum suppression (NMS), and the final detection result is output, and this detection result includes: a first prediction box containing a person.

[0031] For the current infrared image, the number of first prediction boxes containing a person output by the YOLO network is not limited.

[0032] Step 103, when the number of the first prediction boxes is greater than or equal to 1, obtain the motion information of the person from the infrared image.

[0033] When the number of the first prediction boxes is greater than or equal to 1, it indicates that there may be a person in the current infrared image. To further improve the accuracy of person positioning, obtain the motion information of the person from the infrared image. The motion information of the person may include: speed, acceleration, motion direction, motion state, etc. The motion state of the person, for example, static standing, sitting, lying, and dynamic walking, etc.

[0034] Step 104, input the motion information into the motion model to obtain a second prediction box containing a person in the current infrared image.

[0035] The motion model mainly predicts the next position of a person based on the positions of historical persons. A prerequisite for the motion model is the motion hypothesis. The motion hypothesis is based on two core hypotheses: the continuity of human motion and the smoothness of speed and direction. To perform motion prediction, it is necessary to obtain the motion information of the person, mainly including speed, direction, acceleration, and motion state, etc.

[0036] Step 105, based on the first prediction box and the second prediction box, determine the first position of the person in the infrared image.

[0037] This step is to appropriately correct or supplement the first prediction box through the second prediction box output by the motion model, reduce the missed detection rate and false detection rate, and improve the detection accuracy, so as to accurately determine the position of the person in the current infrared image.

[0038] Step 106, based on the homography matrix, convert the first position to the second position of the person in the indoor space; the ground in the indoor space is a flat ground; the homography matrix is formed by selecting positions from an isothermal diagram of an elliptical or quasi-elliptical air supply area on the ground after the temperature in the indoor space is stable when the air conditioner is started.

[0039] The homography matrix is a 3×3 invertible matrix used in computer vision to describe the projective transformation relationship between two planes in the same scene. Its core function is to map points on one plane to corresponding points on another plane. In this application, the homography matrix is an invertible matrix describing the conversion relationship between the plane where the infrared image of the indoor space obtained by the aforementioned single infrared camera is located and a plane in the indoor space.

[0040] In this application, the homography matrix combined with algorithms such as RANSAC (Random Sample Consensus algorithm) can significantly improve the robustness. The RANSAC algorithm can effectively eliminate unqualified data and can give a more accurate identification result faster for data samples with some incorrect data.

[0041] This step is to accurately and quickly transform the first position in the current infrared image to the second position in the indoor space through a homography matrix.

[0042] The ground in the indoor space is a flat ground. Here, "flat" means that the degree of surface undulation is small. For example, the distance between the highest point and the lowest point on the ground is less than or equal to 10 cm. Specifically, when the air conditioner supplies air centrally, if the air supply area is a flat ground, according to fluid mechanics, it can be deduced that after the indoor space stabilizes, the temperature distribution range on the ground is a relatively regular ellipse ( Figure 2 as shown). If there are obstacles blocking (i.e., non-flat areas), the isotherm map of the action area is quite different from the ellipse (such as Figure 3 shown). Therefore, when the ground is a flat ground, the isotherm map of the air supply area on the ground is an ellipse or a quasi-ellipse. From this ellipse or quasi-ellipse isotherm map of the air supply area on the ground, a position can be selected, and based on the selected position, this homography matrix is formed. After the air conditioner is turned on, after a certain period of time, it can be considered that the temperature in the indoor space stabilizes after the air conditioner is turned on.

[0043] When the ground in the indoor space is a flat ground, since the establishment of this homography matrix is based on the situation after the air conditioner is started and after the temperature in the indoor space stabilizes, a position is selected from the ellipse or quasi-ellipse isotherm map of the air supply area on the ground. The selection of this position conforms to the air supply law of the air conditioner, and this position has an accurate corresponding relationship between the ground and the plane where the infrared image is located. Therefore, the accuracy of the established homography matrix is higher, making the accuracy of position transformation higher, and further improving the accuracy of air supply.

[0044] Step 107, control the air conditioner to supply air to the second position according to a preset air supply strategy.

[0045] The preset air supply strategy can include: direct blowing air supply, sheltered air supply, etc. Direct blowing air supply can mean controlling the air conditioner to blow the air on people. Sheltered air supply can mean controlling the air conditioner to blow the air to a position outside the person, but it can bring the appropriate temperature required by the person.

[0046] For example, here, the air-conditioning air deflector and the fan can be adjusted, and combined with the moving direction of the heat source on the infrared image, gradually approach the specified air supply area, where there is an air supply strategy and a second position determination. For example, by controlling the air deflector and the wind speed, the air supply area of the air conditioner is quickly moved to the second position where the person is located along the relative position vector based on an iterative manner, and then fine-tuned according to the air supply strategy until the specified air supply area is reached. For example, according to the second position and the air supply strategy, the determined air supply position on the ground has coordinates relative to the second position of (3, 6). The air conditioner controls the air deflector and the wind speed to move the action area a small step in the direction of (3, 6) to reach (1, 2). At this time, the relative position becomes (2, 4), and the above steps are continued to be repeated until the specified position is reached.

[0047] In the embodiments of the present application, firstly, the intelligent control of the air conditioner only requires a single infrared camera to cooperate with the YOLO network, motion model, etc. to achieve human positioning, overcoming the hardware dependence on multiple sensors (such as depth cameras, lidar) in the existing air conditioner control, and solving problems such as high cost and complex wiring; secondly, through the intervention of the second prediction box obtained from the motion information of the person and the motion model, dynamic human tracking is realized. Even in the case of fast movement or occlusion, the recognition accuracy is still relatively high, ensuring the real-time performance and accuracy of human positioning, reducing the missed detection rate, avoiding the problem of air supply lag, and having strong environmental adaptability, and being able to maintain stable human positioning under different lighting conditions and background interferences; thirdly, through the homography matrix, the conversion of the first position in the infrared image to the second position in the indoor space is carried out, with a fast response speed and shortening the time for air supply to the specified area; fourthly, when the ground in the indoor space is a flat ground, due to the establishment of this homography matrix, which is based on the situation after the air conditioner is started and the temperature in the indoor space is stable, the position is selected from the isothermal diagram of the elliptical or quasi-elliptical air supply area on the ground, and this position selection conforms to the air supply law of the air conditioner, and this position has an accurate corresponding relationship between the ground and the plane where the infrared image is located, so the accuracy of the established homography matrix is higher, making the accuracy of position conversion higher, and further improving the accuracy of air supply.

[0048] The present application provides a preparation method for a control method of an air conditioner, referring to Figure 4 , and the method includes the following steps.

[0049] Step 201, obtain an infrared image of the indoor space through a single infrared camera; the infrared image includes: a historical infrared image and a current infrared image.

[0050] Step 201 can refer to the description of the foregoing step 101. To avoid repetition, it will not be elaborated here.

[0051] Step 202, when there is no person in the previous infrared image corresponding to the current infrared image and it is determined that the current infrared image contains a person based on the frame difference method, input the infrared image into the YOLO network to obtain a first prediction box of the person contained in the current infrared image.

[0052] The previous infrared image corresponding to the current infrared image refers to an infrared image that is adjacent to the shooting time of the current infrared image and is taken before the current infrared image.

[0053] When there is no person in the previous infrared image corresponding to the current infrared image, the frame difference method can be used to determine whether the current infrared image contains a person. The frame difference method is a simple and efficient moving object detection algorithm based on the pixel differences of consecutive frame images in a video sequence. Specifically, by subtracting two frames, the difference between the two frames can be obtained. For example, the brightness difference between the two frames can be obtained. Absolute value , and judge whether it is greater than Threshold value to analyze the motion characteristics of the video or image sequence and determine whether there is human motion or human intrusion into the indoor space in the image sequence. When it is determined that the current infrared image contains a person through the frame difference method, then input the above infrared image into the YOLO network to obtain a first prediction box of the person contained in the current infrared image. Compared with the YOLO network, the computational complexity of the aforementioned frame difference method is significantly smaller, and a relatively high accuracy can also be guaranteed. This application significantly reduces the computational complexity and has lower power consumption. Only through the difference between two frames of images, it can be further accurately judged whether there is a person intrusion. If not, there is also no person in this frame of infrared image, with low power consumption, small required computational complexity, and fast implementation speed.

[0054] The steps to obtain the first prediction box can refer to the corresponding records mentioned above. To avoid repetition, it will not be elaborated here.

[0055] Step 203, delete the first prediction boxes that do not contain human temperature points from all the first prediction boxes corresponding to the current infrared image.

[0056] The current infrared image may contain temperature information. The human body temperature is generally within a limited range. For example, around 36°C to 37°C. The human body temperature usually has a certain difference from the environmental temperature of the indoor space. Therefore, according to the temperature distribution in the current infrared image, the first prediction boxes are screened. The first prediction boxes that do not contain human temperature points are likely to be misjudged and are deleted. This can effectively avoid the misdetection of the YOLO network, not only improving the detection accuracy of the subsequent human body position, but also the computational complexity required for the temperature point detection here is small and the speed is fast.

[0057] Step 204, when the number of the first prediction boxes is greater than or equal to 1, obtain the motion information of the person from the infrared image.

[0058] For the current infrared image, after the aforementioned step 203, the number of the first prediction boxes is the number of the first prediction boxes obtained for the current infrared image.

[0059] Specifically, step 204 may include the following steps. Step 2041: Obtain the first shooting interval duration between at least two frames of historical infrared images. Step 2042: Obtain the first relative displacement of the person in the at least two frames of historical infrared images, and the third position of the person in the last frame of the at least two frames of historical infrared images. Step 2043: Based on the homography matrix, convert the first relative displacement into the second relative displacement of the person in the indoor space, and convert the third position into the fourth position of the person in the indoor space. Step 2044: Determine the historical speed and historical acceleration of the person in the indoor space based on the second relative displacement and the first shooting interval duration. Step 2045: Obtain the second shooting interval duration between the last frame of the at least two frames of historical infrared images and the current infrared image.

[0060] The at least two frames of historical infrared images here may be at least two frames of infrared images in the historical infrared images whose shooting moments are adjacent to the shooting moment of the current infrared image and are shot before the current infrared image. The first shooting interval duration may refer to the time interval between the shooting moment of the first frame of infrared image and the shooting moment of the last frame of infrared image among the at least two frames of historical infrared images. The same person may be included in the aforementioned at least two frames of historical infrared images. The third position refers to the position where the person is located in the last frame of the at least two frames of historical infrared images.

[0061] The first relative displacement may refer to the displacement between the position of the person in the first frame of infrared image and the position of the person in the last frame of infrared image among the at least two frames of historical infrared images.

[0062] In this application, since the homography matrix is an invertible matrix describing the conversion relationship between the plane where the infrared image of the indoor space obtained by the aforementioned single infrared camera is located and a plane in the indoor space. Therefore, the first relative displacement obtained from the aforementioned infrared image can be converted into the second relative displacement of the person in the indoor space through the homography matrix. The third position displacement obtained from the aforementioned infrared image can be converted into the fourth position of the person in the indoor space through the homography matrix.

[0063] The second relative displacement can be obtained by Second relative displacement = historical speed × first shooting interval duration + The historical acceleration, the first shooting interval duration, and the first shooting interval duration are used in such a calculation method to determine the historical acceleration and historical speed here.

[0064] The second shooting interval duration It refers to the shooting interval duration between the shooting moment of the last historical infrared image among the at least two historical infrared images and the shooting moment of the current infrared image.

[0065] Step 205: Input the motion information into the motion model to obtain a second prediction box containing a person in the current infrared image.

[0066] Based on the premise that the aforementioned step 204 includes steps 2041 to 2045, this step 205 may include: The motion model obtains the second prediction box containing a person in the current infrared image according to the fourth position, the second shooting interval duration, the historical speed, and the historical acceleration.

[0067] Specifically, the aforementioned fourth position is , and the historical speed of this person is calculated , and the historical acceleration is , and the position of the person in the indoor space at the shooting moment of the current frame infrared image can be predicted according to the following formula 1 and formula 2 ( ).

[0068] Formula 1 Formula 2 Then, according to the aforementioned homography matrix, the position of the person in the indoor space at the shooting moment of the predicted current frame infrared image is converted into the position of the person in the current frame infrared image at the shooting moment of the current frame infrared image, and the size of the second prediction box can be predicted.

[0069] The above method for determining the second prediction box conforms to the relevant laws of kinematics and the relevant laws such as the continuity of human motion and the smoothness of speed and direction. The second prediction box determined thereby has higher accuracy, which is conducive to improving the accuracy of predicting the position of a person.

[0070] Step 206: Based on the first prediction box and the second prediction box, determine the first position of the person in the infrared image.

[0071] Step 206 may include the following steps. Step 2061: For each of the first prediction boxes, determine the intersection over union (IoU) between the second prediction box and the first prediction box. Step 2062: For each of the first prediction boxes, when the IoU is greater than a preset IoU, increase the confidence of the first prediction box in a preset manner. Step 2063: Determine the position where the first prediction box with the highest confidence among all the first prediction boxes corresponding to the infrared image is located as the first position.

[0072] For the current infrared image, after the aforementioned step 203, the number of first prediction boxes is the number of first prediction boxes obtained for the current infrared image. Through the motion model, usually one second prediction box is obtained for the current infrared image. Here, for each first prediction box of the same frame of the current infrared image, calculate the intersection over union (IoU) with the second prediction box of the same frame of the current infrared image.

[0073] For each first prediction box of the same person in the same frame of the current infrared image, when the aforementioned IoU is greater than the preset IoU, it indicates that both the YOLO network and the motion model predict the presence of a person in and around the first prediction box, and the confidence can be appropriately increased. The confidence of the first prediction box can be increased in the preset manner shown in the following formula 3.

[0074]

[0075] In the above formula 3, refers to the confidence of the first prediction box after increase, p is the confidence of the first prediction box before increase, or rather, p is the confidence of the first prediction box output by the YOLO network. T is the preset IoU.

[0076] Then, determine the position where the first prediction box with the highest confidence among all the first prediction boxes corresponding to the infrared image is located as the first position of the person in the current infrared image.

[0077] For each first prediction box of the same person in the same frame of the current infrared image, when the aforementioned IoU is greater than the preset IoU, it indicates that both the YOLO network and the motion model predict the presence of a person in and around the first prediction box, and the confidence can be appropriately increased. Select the position where the first prediction box with the highest confidence is located from all the first prediction boxes corresponding to the current infrared image as the position of the predicted person in the current infrared image, which can reduce the false detection rate or the missed detection rate.

[0078] Step 207: Based on the homography matrix, convert the first position to the second position of the person within the indoor space; the ground in the indoor space is a flat ground; the homography matrix is formed by selecting positions from the isothermal diagram of the elliptical or quasi-elliptical air supply area on the ground after the temperature in the indoor space stabilizes when the air conditioner is started.

[0079] Before this step 207, the method may further include: Step S1: After the temperature in the indoor space stabilizes, obtain the isothermal diagram of the elliptical or quasi-elliptical air supply area on the ground; Step S2: Select at least 4 positions from the isothermal diagram of the air supply area; Step S3: Based on the at least 4 positions, establish a homography matrix corresponding to the coordinate system of the ground and the coordinate system of the infrared image.

[0080] Specifically, referring to the foregoing description, when the ground is a flat ground, the isothermal diagram of the air supply area with the temperature distribution on the ground is elliptical or quasi-elliptical. From this elliptical or quasi-elliptical isothermal diagram of the air supply area, at least 4 positions can be randomly selected, or at least 4 positions can be selected at equal intervals, etc., and no specific limitation is made on this.

[0081] From this elliptical or quasi-elliptical isothermal diagram of the air supply area, the coordinates of at least 4 selected positions can be (xj, yj), and the value range of j is a natural number from 1 to 4. The homography matrix can be determined by the following formula 4. In the following formula 4, the coordinates in the infrared image coordinate system can be (ui, vi).

[0082]

[0083] Among them, in the above formula 4, s is the scale factor. The RANSAC loss function , the loss function can be determined by the following formula , the loss function The advantages and disadvantages of the homography matrix.

[0084]

[0085] In the above formula 4 and formula 5, h 11 、h 12 、h 21 、h 22 : Control the rotation, scaling, and shear transformation of the image. For example: h 11 and h 12 Affect the scaling and rotation of the x coordinate. h 21 and h 22 Affect the scaling and rotation of the y coordinate. h 13 and h 23 : Control the translation transformation of the image. For example: h 13Translation that affects the x - coordinate. h 23 Translation that affects the y - coordinate. h 31 and h 32 : Controls the perspective transformation (i.e., non - linear transformation) of the image. For example: h 31 Perspective transformation that affects the x' - coordinate. h 32 Perspective transformation that affects the y - coordinate. h 33 : Usually set to 1 for scale normalization.

[0086] When the ground in the indoor space is a flat ground, due to the establishment of this homography matrix, which is determined based on at least 4 positions in the isothermal diagram of the elliptical or quasi - elliptical air - supply area, the selection of these at least 4 positions conforms to the air - conditioning air - supply law, and there is an accurate corresponding relationship between these at least four positions on the ground and the plane where the infrared image is located, the accuracy of the established homography matrix is higher, making the accuracy of position conversion higher.

[0087] It should be noted that the construction of the homography matrix can also be carried out before step 2043.

[0088] Step 208, control the air conditioner to supply air to the second position according to a preset air - supply strategy.

[0089] This step 208 can refer to the aforementioned step 107. To avoid repetition, it will not be elaborated here.

[0090] This application also provides a control device for an air conditioner. Referring to Figure 5 , including: An image acquisition module 301, used to acquire an infrared image of the indoor space through a single infrared camera; the infrared image includes: a historical infrared image and a current infrared image; A first prediction box acquisition module 302, used to input the infrared image into the YOLO network to obtain a first prediction box containing a person in the current infrared image; A motion information acquisition module 303, used to acquire the motion information of a person from the infrared image when the number of the first prediction boxes is greater than or equal to 1; A second prediction box acquisition module 304, used to input the motion information into a motion model to obtain a second prediction box containing a person in the current infrared image; A first position determination module 305, used to determine the first position of a person in the infrared image based on the first prediction box and the second prediction box; A conversion module 306 is configured to convert the first position into a second position of the person in the indoor space based on a homography matrix; the ground in the indoor space is a flat ground; the homography matrix is formed by selecting positions from an isothermal diagram of an elliptical or quasi-elliptical air supply area on the ground after the temperature in the indoor space stabilizes when the air conditioner is started. An air supply module 307 is configured to control the air conditioner to supply air to the second position according to a preset air supply strategy.

[0091] Optionally, the device further includes: An isothermal diagram acquisition module is configured to acquire an isothermal diagram of an elliptical or quasi-elliptical air supply area on the ground after the temperature in the indoor space stabilizes when the air conditioner is started. A position selection module is configured to select at least 4 positions from the isothermal diagram of the air supply area. A homography matrix establishment module is configured to establish a homography matrix corresponding to the coordinate system of the ground and the coordinate system of the infrared image based on the at least 4 positions. The conversion module 306 includes: A first conversion unit is configured to convert the first position into a second position of the person on the ground based on the homography matrix.

[0092] Optionally, the device further includes: A deletion module is configured to delete the first prediction boxes that do not contain human body temperature points from all the first prediction boxes corresponding to the current infrared image.

[0093] Optionally, the first prediction box acquisition module 302 includes: A first prediction box acquisition unit is configured to input the infrared image into a YOLO network to acquire a first prediction box containing a person in the current infrared image when there is no person in the previous infrared image corresponding to the current infrared image and it is determined that there is a person in the current infrared image based on the inter-frame difference method.

[0094] Optionally, the motion information acquisition module 303 includes: A first shooting interval duration acquisition unit is configured to acquire a first shooting interval duration between at least two historical infrared images. A displacement and position acquisition unit is configured to acquire a first relative displacement of the person in the at least two historical infrared images and a third position of the person in the last historical infrared image of the at least two historical infrared images. A second conversion unit, configured to convert the first relative displacement into a second relative displacement of the person within the indoor space based on the homography matrix, and convert the third position into a fourth position of the person within the indoor space; A historical speed acquisition unit, configured to determine a historical speed and a historical acceleration of the person within the indoor space based on the second relative displacement and the first shooting interval duration; A second shooting interval duration acquisition unit, configured to acquire a second shooting interval duration between the last historical infrared image and the current infrared image among the at least two historical infrared images; The second prediction box acquisition module 304 includes: A second prediction box acquisition unit, configured to obtain, according to the motion model, a second prediction box containing a person in the current infrared image based on the fourth position, the second shooting interval duration, the historical speed, and the historical acceleration.

[0095] Optionally, the first position determination module 305 includes: An intersection-over-union determination unit, configured to determine an intersection-over-union of the second prediction box and the first prediction box for each of the first prediction boxes; A confidence improvement unit, configured to, for each of the first prediction boxes, improve the confidence of the first prediction box in a preset manner when the intersection-over-union is greater than a preset intersection-over-union; A first position determination unit, configured to determine, as the first position, a position where the first prediction box with the highest confidence among all the first prediction boxes corresponding to the current infrared image is located.

[0096] Optionally, the resolution of a single infrared camera is m×n, where both m and n are natural numbers from 16 to 48.

[0097] Next, in conjunction with Figure 6 , specific embodiments are used to further explain and illustrate the present application.

[0098] First step, the ground in the indoor space is a flat ground. When the air conditioner is started and the temperature in the indoor space is stable, an isothermal diagram of an elliptical or quasi-elliptical air supply area on the ground is acquired.

[0099] Second step, at least 4 positions are selected from the above isothermal diagram of the air supply area. For example, 25 positions are selected at equal intervals from the isothermal diagram of the air supply area.

[0100] The first step and the second step correspond to Figure 6 the screening of the heat source center in

[0101] In the third step, based on the foregoing at least 4 positions, establish a homography matrix corresponding to the coordinate system where the ground is located and the coordinate system of the infrared image.

[0102] In the fourth step, obtain an infrared image of the indoor space through a single infrared camera; the infrared image includes: a historical infrared image and a current infrared image; In the fifth step, input the above infrared image into the YOLO network to obtain a first prediction box containing a person in the current infrared image (corresponding to Figure 6 person detection in

[0103] In the sixth step, when the number of the first prediction boxes is greater than or equal to 1, obtain the motion information of the person from the above infrared image.

[0104] In the seventh step, input the above motion information into the motion model to obtain a second prediction box containing a person in the current infrared image (the sixth and seventh steps correspond to Figure 6 motion model detection in

[0105] In the eighth step, based on the first prediction box and the second prediction box, determine the first position of the person in the current infrared image (corresponding to Figure 6 improving the regional confidence in

[0106] and whether there is a person corresponding). When there is a person, perform the subsequent ninth step, and when there is no person, perform the subsequent eleventh step. Figure 6 In the ninth step, based on the homography matrix, convert the first position to the second position of the person in the indoor space (corresponding to

[0107] the conversion of the relative position vector in Figure 6 In the tenth step, control the air conditioner to supply air to the second position according to a preset air supply strategy (corresponding to

[0108] the movement of the air supply area in Figure 6 In the eleventh step, based on the frame difference method, determine whether there is a person in the next infrared image (corresponding to

[0109] the frame difference method in

[0110] It should be noted that in the embodiments, all the operation formulas and the like in each step are referred to the foregoing records. For the sake of avoiding repetition, they are not elaborated herein.

[0111] The present application also provides a readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any one of the foregoing air conditioner control methods.

[0112] The present application also provides a computer program product, including instructions. When the instructions are executed by a processor in an electronic device, the electronic device executes any one of the foregoing air conditioner control methods.

[0113] It should be noted that the control device, the electronic device, the readable storage medium, and the computer program product can all refer to the description of the foregoing control method and have the same or similar beneficial effects. To avoid repetition, details are not described herein again.

[0114] It should be noted that for the method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequences, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0115] The present application also provides an electrical appliance, which includes any one of the foregoing air conditioner control methods. The type of the electrical appliance is not specifically limited. For example, the electrical appliance can be an air conditioner, a robotic arm, etc. The preparation method of the electrical appliance and the foregoing air conditioner control method has the same or similar beneficial effects. To avoid repetition, details are not described herein again.

[0116] After considering the specification and practicing the application disclosed herein, those skilled in the art will readily think of other embodiments of the present application. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include well-known common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0117] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A control method for an air conditioner, characterized in that, Including: Obtaining an infrared image of the indoor space through a single infrared camera; The infrared image includes: a historical infrared image and a current infrared image; Inputting the infrared image into the YOLO network to obtain a first prediction box containing a person in the current infrared image; In the case where the number of the first prediction boxes is greater than or equal to 1, obtaining the movement information of the person from the infrared image; Inputting the movement information into a movement model to obtain a second prediction box containing a person in the current infrared image; Based on the first prediction box and the second prediction box, determining a first position of the person in the infrared image; Based on a homography matrix, converting the first position into a second position of the person in the indoor space; the ground in the indoor space is a flat ground; the homography matrix is formed by selecting positions from an isothermal map of an elliptical or quasi-elliptical air supply area on the ground after the temperature in the indoor space stabilizes when the air conditioner is started; Controlling the air conditioner to supply air to the second position according to a preset air supply strategy.

2. The control method of the air conditioner according to claim 1, wherein, Before the step of converting the first position into the second position of the person in the indoor space based on the homography matrix, the method further includes: When the air conditioner is started, after the temperature in the indoor space stabilizes, obtaining an isothermal map of an elliptical or quasi-elliptical air supply area on the ground; Selecting at least 4 positions from the isothermal map of the air supply area; Based on the at least 4 positions, establishing a homography matrix corresponding to the coordinate system of the ground and the coordinate system of the infrared image; The converting the first position into the second position of the person in the indoor space based on the homography matrix includes: Based on the homography matrix, converting the first position into the second position of the person on the ground.

3. The control method of the air conditioner according to claim 1, characterized in that, Before the step of obtaining the movement information of the person from the infrared image in the case where the number of the first prediction boxes is greater than or equal to 1, the method further includes: Deleting the first prediction boxes that do not contain human body temperature points from all the first prediction boxes corresponding to the current infrared image.

4. The control method of the air conditioner according to claim 1, characterized in that, The inputting the infrared image into the YOLO network to obtain a first prediction box containing a person in the current infrared image includes: In the case where there is no person in the previous infrared image corresponding to the current infrared image and it is determined that there is a person in the current infrared image based on the frame difference method, inputting the infrared image into the YOLO network to obtain a first prediction box containing a person in the current infrared image.

5. The control method of the air conditioner according to claim 1, characterized in that, The obtaining the movement information of the person from the infrared image includes: Obtaining a first shooting interval duration between at least two frames of historical infrared images; Obtaining a first relative displacement of the person in the at least two frames of historical infrared images and a third position of the person in the last frame of the at least two frames of historical infrared images; Based on the homography matrix, converting the first relative displacement into a second relative displacement of the person in the indoor space and converting the third position into a fourth position of the person in the indoor space; Determine the historical speed and historical acceleration of the person in the indoor space based on the second relative displacement and the first shooting interval duration; Obtain the second shooting interval duration between the last historical infrared image among the at least two historical infrared images and the current infrared image; The step of inputting the motion information into a motion model to obtain a second prediction box containing a person in the current infrared image includes: The motion model obtains the second prediction box containing a person in the current infrared image according to the fourth position, the second shooting interval duration, the historical speed, and the historical acceleration.

6. The control method of the air conditioner according to any one of claims 1 to 5, characterized in that The step of determining a first position of the person in the infrared image based on the first prediction box and the second prediction box includes: For each of the first prediction boxes, determine the intersection-over-union ratio of the second prediction box and the first prediction box; For each of the first prediction boxes, when the intersection-over-union ratio is greater than a preset intersection-over-union ratio, increase the confidence level of the first prediction box in a preset manner; Determine the position where the first prediction box with the highest confidence level among all the first prediction boxes corresponding to the current infrared image is located as the first position.

7. The control method of the air conditioner according to any one of claims 1 to 5, characterized in that The resolution of a single infrared camera is m×n, where both m and n are natural numbers from 16 to 48.

8. A control device for an air conditioner, characterized in that, It includes: An image acquisition module for acquiring an infrared image of an indoor space through a single infrared camera; The infrared image includes: a historical infrared image and a current infrared image; A first prediction box acquisition module for inputting the infrared image into a YOLO network to obtain a first prediction box containing a person in the current infrared image; A motion information acquisition module for, when the number of the first prediction boxes is greater than or equal to 1, acquiring the motion information of the person from the infrared image; A second prediction box acquisition module for inputting the motion information into a motion model to obtain a second prediction box containing a person in the current infrared image; A first position determination module for determining a first position of the person in the infrared image based on the first prediction box and the second prediction box; A conversion module for converting the first position into a second position of the person in the indoor space based on a homography matrix; the ground in the indoor space is a flat ground; the homography matrix is formed by selecting positions from an isothermal map of an elliptical or quasi-elliptical air supply area on the ground after the temperature in the indoor space is stabilized when the air conditioner is started; An air supply module for controlling the air conditioner to supply air to the second position according to a preset air supply strategy.

9. An electronic device, characterized in that, It includes: A processor, a memory, and a computer program stored on the memory and executable on the processor, wherein the processor implements the air conditioner control method according to any one of claims 1 to 7 when executing the program.

10. A readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of an electronic device, the electronic device can execute the air conditioner control method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, Comprising instructions which, when executed by a processor in an electronic device, cause the electronic device to execute the control method of the air conditioner according to any one of claims 1 to 7.

Images