Air conditioner, control method and device thereof, storage medium and computer program product

Abstract

The application provides an air conditioner and a control method, device, storage medium and computer program product thereof, and belongs to energy-saving refrigeration and air conditioning equipment. The method comprises the following steps: collecting infrared images of N preset directions in a room through an infrared camera; wherein N is an even number; splicing the infrared images of the N directions to obtain a spliced infrared image of the room; constructing a space temperature field of the room based on the obtained spliced infrared image of the room; and adjusting the operation of the air conditioner according to the constructed space temperature field of the room. The scheme provided by the application can solve the problem of local overcooling or overheating in the air conditioner control scheme of single-point temperature sensing by constructing a space temperature field, and can also reduce energy consumption loss by adjusting the operation of the air conditioner according to the constructed space temperature field, thereby achieving the energy-saving effect.

Description

Technical Field

[0001] This invention relates to the field of control, and more particularly to an air conditioner and its control method, device, storage medium and computer program product, belonging to energy-saving refrigeration and air conditioning equipment. Background Technology

[0002] Current air conditioning control systems primarily rely on the return air temperature or a single temperature setting on the remote control to sense the ambient temperature and adjust accordingly. However, this single-point temperature sensing-based control method has inherent and significant limitations, failing to meet the demands of modern smart homes for personalized comfort and refined energy conservation. Specifically, indoor spaces exhibit localized high-temperature and low-temperature zones due to complex factors such as human activity and solar radiation, resulting in a significant temperature unevenness. The single-point temperature sensors in these technologies can only acquire the temperature near their installation location, failing to comprehensively perceive the temperature distribution across the entire space. This leads to localized overheating or undercooling of the air conditioning while other areas remain unsuitable, resulting in poor overall comfort and high energy consumption. Summary of the Invention

[0003] The technical problem to be solved by this invention is that the air conditioning control scheme based on single-point temperature sensing in the related technology has the problem of local overcooling or overheating. Therefore, this invention provides an air conditioner and its control method, device, storage medium and computer program product, which belong to energy-saving refrigeration and air conditioning equipment.

[0004] The present invention provides a method for controlling an air conditioner, comprising: acquiring infrared images of N preset locations within a room using an infrared camera, wherein N is an even number; stitching the infrared images of the N locations together to obtain a stitched infrared image of the room; constructing a spatial temperature field of the room based on the obtained stitched infrared image; and adjusting the operation of the air conditioner according to the constructed spatial temperature field of the room.

[0005] Optionally, the infrared images from the N directions are stitched together to obtain a stitched infrared image of the room. This includes: calculating the angle of the infrared camera's field of view occupied by each pixel in the infrared images captured by the infrared camera; calculating the number of overlapping pixels between any two adjacent infrared images in the N directions based on the calculated angle of the infrared camera's field of view occupied by each pixel and the rotation angle of the infrared camera; cropping the overlapping portion of the N directions of infrared images based on the calculated number of overlapping pixels between any two adjacent images; and stitching the N directions of infrared images together to obtain a stitched infrared image of the room.

[0006] Optionally, based on the calculated angle of the infrared camera's field of view occupied by each pixel and the rotation angle of the infrared camera, the number of overlapping pixels between any two adjacent infrared images in the N azimuth infrared images is calculated, including: the number of overlapping pixels between any two adjacent infrared images is equal to the difference between the number of pixels in the image width or height minus the ratio of the rotation angle of the infrared camera to the angle of the infrared camera's field of view occupied by each pixel.

[0007] Optionally, constructing the spatial temperature field of the room based on the obtained stitched infrared image of the room includes: binarizing the stitched infrared image to obtain a binarized stitched infrared image, and determining the horizontal central axis in the binarized stitched infrared image; determining the intersection point of the horizontal central axis in the binarized stitched infrared image with the connected regions on the left and right sides of the stitched infrared image that satisfy preset conditions; and determining the window area, left and right wall areas, front wall area, and ground area in the stitched infrared image according to the determined horizontal central axis and the intersection point, thereby obtaining the spatial temperature field of the room.

[0008] Optionally, the connected regions on the left and right sides of the stitched infrared image that meet preset conditions are the connected regions where the left and right sides of the stitched infrared image have temperature boundaries with other regions, the number of pixels is greater than a preset ratio of the total number of pixels in the stitched infrared image, and the area located above the horizontal central axis accounts for more than half of the total area of ​​the region.

[0009] Optionally, determining the horizontal central axis in the binarized stitched infrared image includes: determining the initial horizontal central axis position of the stitched infrared image based on the image height of the stitched infrared image; counting the number of pixels with a value of 0 in each row of pixels within a range of M pixels above and below the initial horizontal central axis position, and sorting the pixels in each row according to the number of pixels with a value of 0; selecting the row of pixels with the largest number of pixels with a value of 0 and closest to the initial central axis as the final horizontal central axis position.

[0010] Optionally, it also includes: after obtaining the spatial temperature field of the room, assigning the same color to different areas, and adjusting the color depth according to the temperature.

[0011] Optionally, it further includes: identifying target objects in the collected infrared images from the N directions, and converting the pixel coordinates of the target objects into world coordinates using a pre-set homography matrix; performing coordinate fusion on the world coordinates of the target objects in the identified infrared images from the N directions to obtain the world coordinates of the target objects in the spatial temperature field of the room.

[0012] Optionally, the target object includes: a human body and / or a heat source; coordinate fusion of the world coordinates of the target object in the infrared images of the N directions includes: for the human body, performing coordinate fusion of the world coordinates of the target object in the infrared images of the N directions by clustering; and / or, for the heat source, performing coordinate fusion of the world coordinates of the target object in the infrared images of the N directions by a weighted average algorithm.

[0013] Another aspect of the present invention provides a control device for an air conditioner, comprising: a data acquisition unit for acquiring infrared images of N preset locations within a room via an infrared camera, wherein N is an even number; a stitching unit for stitching the infrared images of the N locations together to obtain a stitched infrared image of the room; a construction unit for constructing a spatial temperature field of the room based on the obtained stitched infrared image; and an adjustment unit for adjusting the operation of the air conditioner according to the constructed spatial temperature field of the room.

[0014] Optionally, the stitching unit stitches together the infrared images from the N directions to obtain a stitched infrared image of the room, including: calculating the angle of the infrared camera's field of view occupied by each pixel in the infrared images captured by the infrared camera; calculating the number of overlapping pixels between any two adjacent infrared images in the N directions based on the calculated angle of the infrared camera's field of view occupied by each pixel and the rotation angle of the infrared camera; cropping the overlapping portion of the N directions of infrared images based on the calculated number of overlapping pixels between any two adjacent images; and stitching the N directions of infrared images together to obtain a stitched infrared image of the room.

[0015] Optionally, the stitching unit calculates the number of overlapping pixels between any two adjacent infrared images in the N azimuth infrared images based on the calculated angle of the infrared camera's field of view occupied by each pixel and the rotation angle of the infrared camera. This includes: the number of overlapping pixels between any two adjacent infrared images is equal to the difference between the number of pixels in the image width or height and the ratio of the rotation angle of the infrared camera to the angle of the infrared camera's field of view occupied by each pixel.

[0016] Optionally, the construction unit constructs the spatial temperature field of the room based on the obtained stitched infrared image of the room, including: binarizing the stitched infrared image to obtain a binarized stitched infrared image, and determining the horizontal central axis in the binarized stitched infrared image; determining the intersection point of the horizontal central axis in the binarized stitched infrared image with the connected regions on the left and right sides of the stitched infrared image that meet preset conditions; and determining the window area, left and right wall areas, front wall area, and ground area in the stitched infrared image according to the determined horizontal central axis and the intersection point, thereby obtaining the spatial temperature field of the room.

[0017] Optionally, the connected regions on the left and right sides of the stitched infrared image that meet preset conditions are the connected regions where the left and right sides of the stitched infrared image have temperature boundaries with other regions, the number of pixels is greater than a preset ratio of the total number of pixels in the stitched infrared image, and the area located above the horizontal central axis accounts for more than half of the total area of ​​the region.

[0018] Optionally, the construction unit determines the horizontal central axis in the binarized stitched infrared image by: determining the initial horizontal central axis position of the stitched infrared image based on the image height of the stitched infrared image; counting the number of pixels with a value of 0 in each row of pixels within a range of M pixels above and below the initial horizontal central axis position, and sorting the pixels in each row of pixels according to the number of pixels with a value of 0; and selecting the row of pixels with the largest number of pixels with a value of 0 and closest to the initial central axis as the final horizontal central axis position.

[0019] Optionally, the construction unit, which constructs the spatial temperature field of the room based on the obtained stitched infrared image of the room, further includes: after obtaining the spatial temperature field of the room, assigning the same color to different areas, and adjusting the color depth according to the temperature.

[0020] Optionally, it further includes: an identification unit, used to identify target objects in the acquired infrared images from the N directions, and convert the pixel coordinates of the target objects into world coordinates using a pre-set homography matrix; and a fusion unit, used to perform coordinate fusion on the identified target objects in the infrared images from the N directions to obtain the world coordinates of the target objects in the spatial temperature field of the room.

[0021] Optionally, the target object includes: a human body and / or a heat source; the fusion unit performs coordinate fusion on the world coordinates of the target object in the infrared images from the N directions, including: for the human body, performing coordinate fusion on the world coordinates of the target object in the infrared images from the N directions by clustering; and / or, for the heat source, performing coordinate fusion on the world coordinates of the target object in the infrared images from the N directions by a weighted average algorithm.

[0022] In another aspect, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0023] In another aspect, the present invention provides an air conditioner, including a processor, a memory, and a computer program stored in the memory that can run on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0024] In another aspect, the present invention provides an air conditioner including any of the control devices described above.

[0025] In another aspect, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0026] According to the technical solution of the present invention, infrared sensors are controlled to collect infrared images from different directions and stitch them together to obtain a stitched infrared image. Based on the obtained stitched infrared image, a spatial temperature field is constructed. The air conditioner is then adjusted according to the constructed spatial temperature field. This effectively expands the field of view of the infrared camera and constructs a more accurate spatial temperature field. Compared with the original infrared image, this method is more scientific and reasonable, which is conducive to obtaining the ambient temperature of the space where the human body is located, thereby realizing personalized air conditioning environment adjustment based on different human bodies.

[0027] According to the technical solution of the present invention, based on the stitched infrared images of infrared images from different directions of the room, the room area is divided after the images are binarized, thereby constructing a spatial temperature field. The spatial temperature field is constructed in a low-resource-consuming, fast and automatic manner so that the ambient temperature of the space in which the human body is located can be accurately identified and the thermal comfort needs of users in different scenarios can be met.

[0028] According to the technical solution of the present invention, the operation of the air conditioner is adjusted based on the constructed spatial temperature field. The temperature of the human body in the space is identified by the divided temperature field and used as an indicator to measure human comfort. While improving the user's comfort experience, it also reduces energy loss and has an energy-saving effect.

[0029] The technical solution of this invention has a wide range of applications and is not limited to any specific location or environment. This invention can effectively expand the field of view of infrared sensors and construct an accurate spatial temperature field, making it more scientific and reasonable than the original infrared image identification. Attached Figure Description

[0030] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0031] Figure 1 This is a schematic diagram of an embodiment of the air conditioner control method provided by the present invention;

[0032] Figure 2 The flowchart illustrates the steps of stitching together infrared images from the N directions to obtain a stitched infrared image of the room.

[0033] Figure 3 The process of infrared image stitching is shown;

[0034] Figure 4 The original stitched infrared image obtained after stitching is shown;

[0035] Figure 5 The image shows the stitched image after binarization;

[0036] Figure 6 This shows the effect after the region was divided;

[0037] Figure 7 A schematic diagram illustrating the steps for constructing the spatial temperature field of a room based on the obtained stitched infrared image of the room is shown.

[0038] Figure 8 The diagram shows the temperature distribution in the window area, the left and right wall areas, the front wall area, and the floor area.

[0039] Figure 9 The temperature distribution is shown in a colored diagram.

[0040] Figure 10 A visualization of the process of constructing a space temperature field is shown;

[0041] Figure 11 This is a schematic diagram of another embodiment of the air conditioner control method provided by the present invention;

[0042] Figure 12 This is a schematic diagram of a specific embodiment of the air conditioner control method provided by the present invention;

[0043] Figure 13The system framework diagram of the present invention is shown;

[0044] Figure 14 This is a structural block diagram of an embodiment of the air conditioner control device provided by the present invention;

[0045] Figure 15 This is a structural block diagram of another embodiment of the air conditioner control device provided by the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0047] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0048] This invention provides a method for controlling an air conditioner.

[0049] Figure 1 This is a schematic diagram of an embodiment of the air conditioner control method provided by the present invention.

[0050] like Figure 1 As shown, according to an embodiment of the present invention, the air conditioner control method includes at least steps S110, S140, S150 and S160.

[0051] Step S110: Use an infrared camera to collect infrared images from N preset locations within the room.

[0052] Specifically, to cover the entire area of ​​the room, infrared cameras capture infrared images from N directions. The infrared camera includes an infrared sensor array (each pixel corresponds to one infrared sensor), has a small field of view, and experiences minimal distortion when capturing images from different directions. In one specific embodiment, the infrared camera can be mounted on a rotating device, which controls the camera to capture images from different directions. Preferably, N is an even number. More preferably, N=4, meaning infrared images from four preset directions within the room. For example, the rotating device controls the infrared camera to capture infrared images from the top left, top right, bottom left, and bottom right directions, respectively, as the base infrared images.

[0053] Step S140: The infrared images from the N directions are stitched together to obtain a stitched infrared image of the room.

[0054] Figure 2 This diagram illustrates the steps of stitching together infrared images from the N directions to obtain a stitched infrared image of the room. Figure 2 The step S140 shown may specifically include steps S141, S142 and S143.

[0055] Step S141: Calculate the angle of the infrared camera's field of view occupied by each pixel in the infrared image captured by the infrared camera.

[0056] The field of view (FOV) of an infrared camera refers to the range of spatial angles that the infrared camera can observe at a specific distance. The FOV of an infrared camera includes angles in both the horizontal and vertical directions. The horizontal FOV is the range of angles that the infrared camera can observe in the horizontal direction; the vertical FOV is the range of angles that the infrared camera can observe in the vertical direction.

[0057] The angle of the infrared camera's field of view occupied by each pixel includes the angle of the infrared camera's field of view in the horizontal direction and the angle of the infrared camera's field of view occupied by each pixel in the vertical direction. The angle of the infrared camera's field of view occupied by each pixel in the horizontal direction is equal to the ratio of the horizontal field of view of the infrared camera to the width (number of pixels) of the captured image. The angle of the infrared camera's field of view occupied by each pixel in the vertical direction is equal to the ratio of the vertical field of view of the infrared camera to the height (number of pixels) of the captured image. For example, if the image resolution is h*w (where h represents height and w represents width), the horizontal field of view is h_nominal, and the vertical field of view is v_nominal, then the angle of the infrared camera's field of view occupied by each pixel in the horizontal direction (i.e., the instantaneous field of view: the angle subtended by a single pixel in space) can be calculated as: h_nominal / w. Similarly, the angle of the infrared camera's field of view occupied by each pixel in the vertical direction (instantaneous field of view) is: v_nominal / h.

[0058] Step S142: Calculate the number of overlapping pixels between each pair of adjacent infrared images in the N azimuth infrared images based on the angle of the infrared camera's field of view occupied by each pixel and the rotation angle of the infrared camera.

[0059] Specifically, ignoring lens distortion and perspective distortion, the number of overlapping pixels between any two adjacent infrared images is equal to the number of pixels in the image width or height (pixel height or pixel width) minus the difference between the rotation angle of the infrared camera and the angle of the infrared camera's field of view occupied by each pixel. The rotation angle of the infrared camera is the angular difference between any two adjacent infrared images in the N directions acquired by the infrared camera, that is, the rotation angle from the center position of one image to the center position of the other. The rotation angle includes horizontal and vertical rotation angles. For example, the rotation angle between two adjacent infrared images vertically is the vertical rotation angle, and the rotation angle between two adjacent infrared images horizontally is the horizontal rotation angle.

[0060] Taking the horizontal direction as an example: the image width (number of pixels) is w, and the angle of the infrared camera's field of view occupied by each pixel in the horizontal direction is k = h_nominal / w. If the rotation angle (e.g., the horizontal rotation angle of the camera from the center position of the "left image" to the center position of the "right image") is θ, which is equivalent to moving s = θ / k pixels, then the number of overlapping pixels is: P_overlap = w - s = w - θ / k. When performing image stitching, it is necessary to ensure that there is an overlapping part; otherwise, effective stitching cannot be completed. Therefore, the number of overlapping pixels cannot be 0, that is, P_overlap must be greater than 0. Thus, the number of overlapping pixels is: P_overlap = max(0, w - θ / k).

[0061] Step S143: Based on the calculated number of overlapping pixels between each pair of adjacent images, the overlapping portions of the infrared images in the N directions are cropped, and the infrared images in the N directions are stitched together to obtain the stitched infrared image of the room.

[0062] That is, the overlapping portion of one image with the other is cropped from two adjacent images, while the overlapping portion of the other image is retained. Preferably, when acquiring infrared images from four preset locations within the room, one infrared image is used as the reference image. The overlapping portions of the other infrared images are cropped based on the calculated number of overlapping pixels between each pair of adjacent images, and the four infrared images are then stitched together to obtain a stitched infrared image. For example, using the infrared image from the top left as the reference, the overlapping portions of the other three infrared images are cropped and stitched together to obtain a stitched infrared image. Optionally, after obtaining the stitched infrared image, it can be mirrored to suit user viewing habits, i.e., the image can be flipped along the vertical or horizontal axis. The stitching process can be referenced... Figure 3 . Figure 3 The process of stitching infrared images is shown. Figure 4 The original stitched infrared image obtained after stitching is shown.

[0063] By fixing the shooting angle for each movement, the overlapping portion of each image remains consistent across different scenes and locations when stitching them together. Therefore, by determining the infrared camera's movement angle and overlapping portion only once during the experiment, the stitching method remains the same regardless of the infrared camera's position.

[0064] Step S150: Construct the spatial temperature field of the room based on the obtained stitched infrared image of the room.

[0065] In actual infrared thermal imaging images, edge areas (e.g., windows, walls, etc.) have relatively clear boundaries with other areas. For example... Figure 4As shown, the left side of the image is the window area, which has a relatively clear boundary with other areas. The grayscale values ​​are continuous, and the entire area has a high brightness value in the image, showing obvious regional characteristics.

[0066] Figure 5 The stitched image after binarization is shown. Using the high brightness and continuity of the window region as feature values, the binarized result is as follows. Figure 5 As shown, after binarization, the window area is marked as a connected highlighted area with a label of 1 (1 represents white, 0 represents black), which is clearly separated from the background.

[0067] To define the temperature field, the following prior knowledge is introduced, taking into account real-life scenarios:

[0068] ① In real life, when an air conditioner is installed directly in front of you, there is a significant temperature difference between the left and right wall areas. If there is a window area in the opposite direction, the temperature in the window area will definitely be the highest among the three areas (the left and right wall areas, and the opposite wall area). (In summer, when the air conditioner is cooling, the temperature at the window is higher than the left and right walls, the opposite wall, and the indoor space due to solar radiation).

[0069] ② From the perspective of an infrared sensor installed on an air conditioner (e.g., at the bottom of the air conditioner), the ground and walls are always divided into upper and lower parts. The upper part is the wall, and the lower part is the ground.

[0070] ③ The wall areas on the left and right sides of the stitched infrared image will always pass through the horizontal central axis and extend to the lower left and lower right corners (the lower left and lower right vertices of the stitched infrared image). The line where the wall directly opposite the infrared camera intersects with the ground is the horizontal central axis.

[0071] Based on the three prior knowledge points mentioned above, firstly, the horizontal central axis can be divided into three equal parts, resulting in two trisection points. Using these two trisection points, the area separating the front, left, and right walls, as well as the ground area located below the horizontal central axis, can be drawn without any constraints. Figure 6 The effect after region division is shown. The horizontal and vertical axes are in pixels, and the grayscale value represents temperature.

[0072] Although the temperature distribution of the space was drawn based on prior knowledge, it is clear that it differs significantly from the actual spatial structure. The window area is not necessarily located at one-third of the horizontal central axis, and there may not be a wall or window on the right side. Therefore, the temperature distribution needs to be divided according to the actual spatial structure.

[0073] Figure 7This diagram illustrates the steps involved in constructing the spatial temperature field of a room based on a stitched infrared image of the room. Figure 7 As shown, step S150 may include steps S151, S152 and S153.

[0074] Step S151: Binarize the stitched infrared image to obtain a binarized stitched infrared image, and determine the horizontal central axis in the binarized stitched infrared image.

[0075] Specifically, the grayscale values ​​of all pixels in the stitched infrared image are forcibly replaced with two specific values, such as 0 and 255. Black represents the background or non-target, and white represents the target or foreground. Figure 5 The image shows the binarized stitched infrared image.

[0076] The line where the wall opposite the infrared camera (air conditioner) intersects with the ground is the horizontal centerline. Specifically, the initial horizontal centerline position of the stitched infrared image is determined based on the image height. Within a range of M pixels above and below the initial horizontal centerline position, the number of pixels with a value of 0 in each row is counted, and the rows are sorted according to the number of pixels with a value of 0. The row with the largest number of pixels with a value of 0 and closest to the initial centerline is selected as the final horizontal centerline position.

[0077] For example, the steps to identify the lateral midline in a binarized stitched infrared image:

[0078] 1. Obtain the height h (h pixels) of the stitched image.

[0079] 2. The initial position of the centerline is h / / 2 (h / / 2 means h / 2 rounded down).

[0080] 3. Move up and down by h / / 4 pixels (h / / 4 means h / 4 rounded down) near the central axis.

[0081] 4. Within the range of (max(0, h / 2 - h / / 4), min(h, h / / 2 + h / / 4)), count the number of 0 pixels in each row.

[0082] 5. Sort by the number of pixels with a value of 0.

[0083] 6. Select the position with the largest number of pixels with a pixel value of 0 that is closest to the initial centerline as the final centerline position.

[0084] That is, based on the image height h of the stitched infrared image, the initial horizontal centerline position (h / / 2) of the stitched infrared image is determined; the image is moved up and down by h / / 4 pixels near the initial horizontal centerline, and within the range of height (max(0,h / 2 - h / / 4), min(h, h / / 2 + h / / 4)), the number of pixels with a value of 0 in each row is counted, and the pixels are sorted according to the number of pixels with a value of 0; the row of pixels with the largest number of pixels with a value of 0 and closest to the initial centerline is selected as the final horizontal centerline position.

[0085] Step S152: Determine the intersection point of the horizontal central axis and the connected regions on the left and right sides that meet the preset conditions in the binarized stitched infrared image.

[0086] The connected regions (referred to as large connected regions) on the left and right sides of the stitched infrared image that meet preset conditions are specifically defined as connected regions where the left and right sides of the stitched infrared image have temperature boundaries with other regions, the number of pixels in these regions is greater than a preset proportion of the total number of pixels in the stitched infrared image, and the area located above the horizontal central axis accounts for more than half of the total area of ​​these regions. In other words, in the stitched infrared image, there are connected regions on the left and right sides with obvious temperature boundaries with other regions, the number of pixels in these connected regions is greater than a preset proportion of the total number of pixels in the stitched infrared image, and the area located above the horizontal central axis in these connected regions accounts for more than half of the total area of ​​these regions. For example, the number of pixels in these connected regions accounts for more than 5% of the total number of pixels in the stitched infrared image, and the area located above the horizontal central axis accounts for more than half of the total area of ​​these regions. Among the intersection points of the horizontal central axis and the connected regions on the left and right sides that meet the preset conditions, the intersection point on the left is designated as the left point, and the intersection point on the right is designated as the right point.

[0087] Step S153: Based on the determined horizontal central axis and the intersection point, determine the window area, left and right wall areas, front wall area and ground area in the stitched infrared image to obtain the spatial temperature field of the room.

[0088] Specifically, the positions of the window area, left and right wall areas, front wall area, and ground area in the stitched infrared image are determined based on the intersection point and the horizontal central axis determined in the binarized stitched infrared image.

[0089] Figure 8 This image shows the temperature distribution in the window area, the left and right wall areas, the front wall area, and the floor area. Both the horizontal and vertical axes are in pixels. The final image is shown below. Figure 8 As shown. From Figure 8As can be seen, the binarization result effectively distinguishes the window's location and identifies the area on the right where no other temperature boundaries exist. Furthermore, it effectively integrates prior knowledge to separate the ground area where people are active from the wall area. To further visualize the temperature distribution results, different areas can be assigned the same color, with the color depth adjusted according to the temperature. The final coloring effect is as follows: Figure 9 As shown. Figure 9 The image shows a colorized representation of the temperature distribution. The horizontal and vertical axes are in pixels, and the color depth represents the temperature magnitude, in degrees Celsius (°C).

[0090] Furthermore, not all infrared images show that the highlighted areas (window areas) are located at the left and right edges. Figure 10 A visualization of the process of constructing a space temperature field is shown. For example... Figure 10 As shown in the last two images, the highlighted area is in the middle. Therefore, infrared images with a large window area (the proportion of window area pixels in the total number of pixels in the infrared image is greater than the preset proportion) are processed.

[0091] like Figure 10 As shown in the last two images, after binarization, the intersection of the horizontal line's central axis and the highly connected region may be located at the lower edge of the highlighted area (window area) (see...). Figure 10 In the grayscale image on the left side of the third row, the horizontal central axis is below the large highlighted area representing the window (the intersection point is also below). The vertical central axis extending from this intersection point divides the highlighted area. Therefore, after obtaining the intersection point of the horizontal central axis and the highlighted area (window area), it is further determined whether the vertical line extending upwards from this intersection point still contains a highlighted area label (the label corresponding to the highlighted area after binarization is 1). If a highlighted area label still exists, it is necessary to determine whether the intersection point needs to be moved left or right to satisfy the physically meaningful window position. This can be analyzed based on a one-dimensional projection of the temperature gradient.

[0092] 1. Calculate the temperature gradient map G along the horizontal direction (X-axis) of the panoramic image excluding the highlighted areas on the horizontal central axis. The gradient is calculated as |T(x+1) - T(x)|;

[0093] 2. Sum the temperature gradient graph G along the vertical direction (Y-axis).

[0094] Because the temperature changes drastically at the junction of the wall where the window is located and the adjacent wall, the sum of the temperature gradients in this column will be significantly higher than that in the left and right areas.

[0095] 3. Find the peak value from step 2, which is to estimate the approximate location of the intersection line.

[0096] 4. Determine whether the intersection line is to the left or right of the current intersection point. If it is to the right, move the intersection point one pixel to the right until there are no highlighted area labels on the vertical line. Conversely, move it to the right.

[0097] Using the above method, even if the highlighted area is in the middle, the floor, wall and window areas can be separated well.

[0098] Step S160: Adjust the operation of the air conditioner according to the constructed spatial temperature field of the room.

[0099] Specifically, based on the constructed spatial temperature field, the high-temperature zone (e.g., a zone with a temperature higher than a first preset temperature threshold), the low-temperature zone (e.g., a zone with a temperature lower than a second preset temperature threshold), and the overall temperature gradient change of the room are determined, as well as whether the temperature at the location of the human body is within a preset comfortable temperature range. Based on the determined high-temperature zone of the room, the overall temperature gradient change, and whether the temperature at the location of the human body is within the preset comfortable temperature range, the operation of the air conditioner is adjusted, for example, by adjusting the airflow direction, airflow speed, and / or air conditioner power.

[0100] For example, a spatial temperature field can be created by assigning the same color to different areas and adjusting the color depth according to temperature. Based on the color, one can determine the temperature gradient changes in high-temperature and low-temperature areas, as well as the overall temperature gradient, and the temperature comfort level at a human location. If the temperature at a human location is uncomfortable, appropriate adjustments can be made based on the human's location coordinates.

[0101] For example, including the following situations:

[0102] 1. Determine if there are any abnormal temperature points, such as electrical malfunctions, and provide feedback to the user via an app or other means.

[0103] 2. If the overall temperature is too low, increase the temperature setting and run the air conditioner at a lower power. If the overall temperature is too high, decrease the temperature setting and increase the fan speed.

[0104] 3. When the temperature at the location of the human body is within the comfortable temperature range, no adjustment is required; when the temperature at the location of the human body is not within the comfortable temperature range, the wind speed is dynamically adjusted according to the spatial temperature gradient distribution at the location of the human body to bring the temperature at the location of the human body within the comfortable temperature range. For example, when the temperature at the location of the human body is below the lower limit of the comfortable temperature range, the wind speed is reduced and / or the wind is directed away from the human body until the temperature at the location of the human body is within the comfortable temperature range. When the temperature at the location of the human body is above the upper limit of the comfortable temperature range, the wind speed is increased and / or the wind is directed towards the human body until the temperature at the location of the human body is within the comfortable temperature range.

[0105] Figure 11This is a schematic diagram of another embodiment of the air conditioner control method provided by the present invention. Figure 11 As shown, the air conditioner control method further includes steps S120 and S130.

[0106] Step S120: Identify the target object in the collected infrared images from the N directions, and convert the pixel coordinates of the target object into world coordinates using a pre-set homography matrix.

[0107] In one specific implementation, a target detection algorithm identifies target objects (such as human bodies, heat sources, etc.) in the acquired infrared image. The algorithm outputs the target object's category, bounding box coordinates [x_center, y_center, width, height], and confidence score. x_center is the x-coordinate of the target bounding box center point, y_center is the y-coordinate of the target bounding box center point, width is the width of the bounding box, height is the height of the bounding box, and confidence score is the detection confidence score. When the confidence score is greater than a preset probability threshold (confidence > threshold), the pixel coordinates of the corresponding object are mapped to world coordinates using the corresponding homography matrix H. The threshold represents the probability threshold for identifying the object as a target.

[0108] Specifically, the pixel positions of the acquired infrared images from the N directions are converted into spatial positions on the ground plane according to a pre-set homography matrix. Different homography matrices correspond to different infrared images from different directions.

[0109] To cover the entire room, this invention collects infrared images from N directions. Because the angles from which the infrared images are collected differ, the pixel position of a point varies in different infrared images, requiring different transformation formulas to map it to the same world point.

[0110] The method for setting the homography matrix corresponding to infrared images from different orientations is the same. It is important to note that a unified reference point must be used: all coordinates should have the same origin (0,0) at the same corner of the room, all coordinate units should be the same (e.g., meters), and the X and Y axes should have the same defined direction.

[0111] How to set up a homography matrix:

[0112] Step 1: Select four non-collinear points in the room whose corresponding real-world coordinates (xj, yj) are known.

[0113] Step 2: Acquire infrared images containing the four selected points using an infrared camera, and mark the pixel coordinates (ui, vi) of each point in the acquired infrared images.

[0114] Step 3: Establish the mapping relationship between the pixel coordinates (ui,vi) of the infrared image and the real-world coordinates (xj,yj).

[0115] Specifically, the mapping relationship between the pixel coordinates (ui, vi) of the infrared image and the real-world coordinates (xj, yj) is as follows:

[0116]

[0117] Where s is the scale factor, It is a homography matrix.

[0118] The scale factor *s* is used to convert the "pixel units" in an image to "physical units" (such as meters and centimeters) in the real world. The homography matrix itself is a projective geometric transformation, losing depth information, and therefore cannot directly provide the absolute size in the real world. The size can be determined using a known reference object. For example, in image measurement: using image processing algorithms (such as edge detection), find two feature points of the reference object in the image and calculate the pixel distance *Dpixel* between them. To obtain the true size: the true physical distance *Dreal* of the reference object, the scale factor is equal to *Dreal* / *Dpixel* (unit: meters / pixel). For example, if a 60cm * 60cm floor tile is laid on a floor, and in an infrared image, the diagonal length of one tile is measured to be 1000 pixels, and the actual diagonal length of the tile is approximately 0.848, then the scale factor is: *S* = 0.848 / 1000.

[0119] Step four: Substitute the pixel coordinates and world coordinates of the selected four points into the mapping relationship to obtain the corresponding homogeneous equation, and solve the homogeneous equation to obtain the homography matrix.

[0120] Specifically, by substituting the pixel coordinates and world coordinates of the selected four points into the above mapping relationship, the corresponding homogeneous equation can be obtained. Solving the homogeneous equation yields h. ij， The homography matrix H can then be obtained.

[0121] Step S130: Perform coordinate fusion on the world coordinates of the target object in the infrared images of the N directions to obtain the world coordinates of the target object in the spatial temperature field of the room.

[0122] The target object can specifically include: a human body and / or a heat source (an object that generates heat). For the human body, a clustering algorithm is used to fuse the world coordinates of the target object in the infrared images from the N directions to obtain the human body's position. For example, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm can be used to fuse the world coordinates of the target object in the infrared images from the N directions. This is because the world coordinates of the same person should be similar in the N directions, while the world coordinates of people in different locations are not similar. Based on this, assuming there are two people and both are captured in images from each direction, using a clustering algorithm to cluster the 2N world coordinates can yield two labels, thus obtaining the coordinates of the two people.

[0123] Clustering using the DBSCAN clustering algorithm can specifically include the following steps:

[0124] 1. Determine the model parameters:

[0125] esp (radius): Set according to the room size and accuracy requirements. For example, if two people cannot stand too close, it can be set to 0.5m. The specific value depends on the resolution.

[0126] Min_samples (minimum number of samples): set to 1, because even if only one camera sees a person, they can be identified as a single individual.

[0127] 2. Input the model parameters into the DBSCAN algorithm to obtain the cluster label for each coordinate point.

[0128] 3. Traverse the cluster labels, calculate the average of all points belonging to the same cluster, and obtain the coordinates of each individual.

[0129] For the heat source, the location of the heat source is obtained by averaging the world coordinates of the target object in the infrared images from the N directions. That is, the world coordinates in the infrared images from each direction can be directly averaged.

[0130] Because there may be multiple people involved, the coordinates calculated from N angles are mixed together. Using clustering methods will better distinguish the number of people and their corresponding coordinates. There may only be one heat source, so the coordinates calculated from N angles can be directly weighted and averaged. The corresponding world coordinates are calculated based on the homography matrix. If the heat source is captured from all N directions, N world coordinates are obtained. These N world coordinates are then averaged. The specific world coordinate units are (X, Y), so the average is calculated separately for X and Y.

[0131] To clearly illustrate the technical solution of the present invention, the execution flow of the air conditioner control method provided by the present invention will be described below with reference to a specific embodiment.

[0132] Figure 12 This is a schematic diagram of a specific embodiment of the air conditioner control method provided by the present invention.

[0133] Step 1: Infrared camera scanning. The infrared sensor is swung in four directions using a rotating device to acquire thermal images from each location.

[0134] Step 2: Geometric transformation calculation. Obtain the bounding box coordinates and confidence of the human body, heat source, etc. in each infrared image. When confidence > threshold, map the corresponding object pixels to world coordinates using the corresponding homography matrix H.

[0135] Step 3: Coordinate Fusion. For the human body, clustering is used for coordinate fusion; for the heat source, weighted averaging is used for coordinate fusion. This yields a temperature field with coordinates.

[0136] Step 5: Temperature Field Analysis. Analyze the temperature gradient changes in high-temperature regions, low-temperature regions, and the overall region to determine spatial temperature non-uniformity. Analyze the temperature at the human body's location coordinates to determine the direction of human comfort adjustment.

[0137] Step 6: Decision Generation. Based on the analysis results from Step 5, determine the adjustments to wind direction, wind speed, power, etc.

[0138] Step 7: Coordinated Adjustment of Air Conditioning Parameters. Based on the decision, determine and adjust the angle of the air deflector, wind speed, and air volume under the temperature gradient of the entire space.

[0139] Step 8: Every N time intervals, return to Step 1 and adjust again.

[0140] Figure 13 A system framework diagram of the present invention is shown. (See diagram below.) Figure 13 As shown, the system framework of this invention includes a sensing unit, a decision-making unit, and an execution unit.

[0141] Sensing unit: Infrared camera, used to acquire spatial temperature field distribution.

[0142] Decision-making unit: AI chip / server / cloud. Used for stitching together multiple infrared images extracted by the sensing unit, constructing temperature distribution, calculating the geometric transformation between infrared pixel positions and the spatial position of the floor plane, predicting using AI algorithms, and analyzing spatial temperature, thereby generating dynamic adjustment strategies based on the current spatial temperature field.

[0143] Execution unit: The air conditioning main control board receives the adjustment strategy from the decision-making unit and drives the actuators. Examples include a stepper motor for controlling the direction of the air sweeping blades and a fan for controlling the airflow adjustment.

[0144] The present invention also provides a control device for an air conditioner.

[0145] Figure 14 This is a structural block diagram of an embodiment of the air conditioner control device provided by the present invention. Figure 13 As shown, the control device 100 of the air conditioner includes: a data acquisition unit 110, a splicing unit 140, a construction unit 150, and an adjustment unit 160.

[0146] The acquisition unit 110 is used to acquire infrared images of N preset locations in the room using an infrared camera; where N is an even number.

[0147] Specifically, to cover the entire area of ​​the room, infrared cameras capture infrared images from N directions. The infrared camera includes an infrared sensor array (each pixel corresponds to one infrared sensor), has a small field of view, and experiences minimal distortion when capturing images from different directions. In one specific embodiment, the infrared camera can be mounted on a rotating device, which controls the camera to capture images from different directions. Preferably, N is an even number. More preferably, N=4, meaning infrared images from four preset directions within the room. For example, the rotating device controls the infrared camera to capture infrared images from the top left, top right, bottom left, and bottom right directions, respectively, as the base infrared images.

[0148] The stitching unit 120 is used to stitch together the infrared images from the N directions to obtain a stitched infrared image of the room.

[0149] In one specific embodiment, the step of stitching together the infrared images from the N directions to obtain a stitched infrared image of the room may specifically include the following steps:

[0150] (1) Calculate the angle of the infrared camera's field of view occupied by each pixel in the infrared image captured by the infrared camera.

[0151] The field of view (FOV) of an infrared camera refers to the range of spatial angles that the infrared camera can observe at a specific distance. The FOV of an infrared camera includes angles in both the horizontal and vertical directions. The horizontal FOV is the range of angles that the infrared camera can observe in the horizontal direction; the vertical FOV is the range of angles that the infrared camera can observe in the vertical direction.

[0152] The angle of the infrared camera's field of view occupied by each pixel includes the angle of the infrared camera's field of view in the horizontal direction and the angle of the infrared camera's field of view occupied by each pixel in the vertical direction. The angle of the infrared camera's field of view occupied by each pixel in the horizontal direction is equal to the ratio of the horizontal field of view of the infrared camera to the width (number of pixels) of the captured image. The angle of the infrared camera's field of view occupied by each pixel in the vertical direction is equal to the ratio of the vertical field of view of the infrared camera to the height (number of pixels) of the captured image. For example, if the image resolution is h*w (where h represents height and w represents width), the horizontal field of view is h_nominal, and the vertical field of view is v_nominal, then the angle of the infrared camera's field of view occupied by each pixel in the horizontal direction (i.e., the instantaneous field of view: the angle subtended by a single pixel in space) can be calculated as: h_nominal / w. Similarly, the angle of the infrared camera's field of view occupied by each pixel in the vertical direction (instantaneous field of view) is: v_nominal / h.

[0153] (2) Calculate the number of overlapping pixels between each pair of adjacent infrared images in the N directions based on the angle of the infrared camera's field of view occupied by each pixel and the rotation angle of the infrared camera.

[0154] Specifically, ignoring lens distortion and perspective distortion, the number of overlapping pixels between any two adjacent infrared images is equal to the number of pixels in the image width or height (pixel height or pixel width) minus the difference between the ratio of the rotation angle of the infrared camera and the angle of the infrared camera's field of view occupied by each pixel.

[0155] The rotation angle of the infrared camera is the angle difference between two adjacent infrared images in the N directions captured by the infrared camera. That is, the rotation angle from the center position of one of the two adjacent infrared images to the center position of the other image. The rotation angle includes the horizontal rotation angle and the vertical rotation angle. For example, the rotation angle between two adjacent infrared images is the vertical rotation angle, and the rotation angle between two adjacent infrared images is the horizontal rotation angle.

[0156] Taking the horizontal direction as an example: the image width (number of pixels) is w, and the angle of the infrared camera's field of view occupied by each pixel in the horizontal direction is k = h_nominal / w. If the rotation angle (e.g., the horizontal rotation angle of the camera from the center position of the "left image" to the center position of the "right image") is θ, which is equivalent to moving s = θ / k pixels, then the number of overlapping pixels is: P_overlap = w - s = w - θ / k. When performing image stitching, it is necessary to ensure that there is an overlapping part; otherwise, effective stitching cannot be completed. Therefore, the number of overlapping pixels cannot be 0, that is, P_overlap must be greater than 0. Thus, the number of overlapping pixels is: P_overlap = max(0, w - θ / k).

[0157] (3) Based on the number of overlapping pixels between each pair of adjacent images, the overlapping parts of the infrared images in the N directions are cropped, and the infrared images in the N directions are stitched together to obtain the stitched infrared image of the room.

[0158] That is, the overlapping portion of one image with the other is cropped from two adjacent images, while the overlapping portion of the other image is retained. Preferably, when acquiring infrared images from four preset locations within the room, one infrared image is used as the reference image. The overlapping portions of the other infrared images are cropped based on the calculated number of overlapping pixels between each pair of adjacent images, and the four infrared images are then stitched together to obtain a stitched infrared image. For example, using the infrared image from the top left as the reference, the overlapping portions of the other three infrared images are cropped and stitched together to obtain a stitched infrared image. Optionally, after obtaining the stitched infrared image, it can be mirrored to suit user viewing habits, i.e., the image can be flipped along the vertical or horizontal axis. The stitching process can be referenced... Figure 3 . Figure 3 The process of stitching infrared images is shown. Figure 4 The original stitched infrared image obtained after stitching is shown.

[0159] By fixing the shooting angle for each movement, the overlapping portion of each image remains consistent across different scenes and locations when stitching them together. Therefore, by determining the infrared camera's movement angle and overlapping portion only once during the experiment, the stitching method remains the same regardless of the infrared camera's position.

[0160] The construction unit 150 is used to construct the spatial temperature field of the room based on the obtained stitched infrared image of the room.

[0161] In actual infrared thermal imaging images, edge areas (e.g., windows, walls, etc.) have relatively clear boundaries with other areas. For example... Figure 4As shown, the left side of the image is the window area, which has a relatively clear boundary with other areas. The grayscale values ​​are continuous, and the entire area has a high brightness value in the image, showing obvious regional characteristics.

[0162] Using the high brightness and continuity of the window area as feature values, the binarized result is as follows: Figure 5 As shown, after binarization, the window area is marked as a connected highlighted area with a label of 1 (1 represents white, 0 represents black), which is clearly separated from the background.

[0163] To define the temperature field, the following prior knowledge is introduced, taking into account real-life scenarios:

[0164] ① In real life, when an air conditioner is installed directly in front of you, there is a significant temperature difference between the left and right wall areas. If there is a window area in the opposite direction, the temperature in the window area will definitely be the highest among the three areas (the left and right wall areas, and the opposite wall area). (In summer, when the air conditioner is cooling, the temperature at the window is higher than the left and right walls, the opposite wall, and the indoor space due to solar radiation).

[0165] ② From the perspective of an infrared sensor installed on an air conditioner (e.g., at the bottom of the air conditioner), the ground and walls are always divided into upper and lower parts. The upper part is the wall, and the lower part is the ground.

[0166] ③ The wall areas on the left and right sides of the stitched infrared image always pass through the horizontal central axis and extend to the lower left and lower right corners (the lower left and lower right vertices of the stitched infrared image). The line where the wall directly opposite the infrared sensor intersects with the ground is the horizontal central axis.

[0167] Based on the three prior knowledge points mentioned above, firstly, the horizontal central axis can be divided into three equal parts, resulting in two trisection points. Using these two trisection points, the area separating the front, left, and right walls, as well as the ground area located below the central axis, can be drawn without any constraints. Figure 6 The effect after dividing the displayable region is shown.

[0168] Although the temperature distribution of the space was drawn based on prior knowledge, it is clear that it differs significantly from the actual spatial structure. The window area is not necessarily located at one-third of the horizontal central axis, and there may not be a wall or window on the right side. Therefore, the temperature distribution needs to be divided according to the actual spatial structure.

[0169] In one specific implementation, the step of constructing the spatial temperature field of the room based on the obtained stitched infrared image of the room may specifically include the following steps:

[0170] (1) Binarize the stitched infrared image to obtain the binarized stitched infrared image, and determine the horizontal central axis in the binarized stitched infrared image.

[0171] Specifically, the grayscale values ​​of all pixels in the stitched infrared image are forcibly replaced with two specific values, such as 0 and 255. Black represents the background or non-target, and white represents the target or foreground. Figure 5 The image shows the binarized stitched infrared image.

[0172] The line where the wall opposite the infrared camera (air conditioner) intersects with the ground is the horizontal centerline. Specifically, the initial horizontal centerline position of the stitched infrared image is determined based on the image height. Within a range of M pixels above and below the initial horizontal centerline position, the number of pixels with a value of 0 in each row is counted, and the rows are sorted according to the number of pixels with a value of 0. The row with the largest number of pixels with a value of 0 and closest to the initial centerline is selected as the final horizontal centerline position.

[0173] For example, the steps to identify the lateral midline in a binarized stitched infrared image:

[0174] 7. Obtain the height h (h pixels) of the stitched image.

[0175] 8. The initial position of the centerline is h / / 2 (h / / 2 means h / 2 rounded down).

[0176] 9. Move up or down by h / / 4 pixels near the center line (h / / 4 means h / 4 rounded down).

[0177] 10. Within the range of (max(0, h / 2 - h / / 4), min(h, h / / 2 + h / / 4)), count the number of 0 pixels in each row.

[0178] 11. Sort by the number of pixels with a value of 0.

[0179] 12. Select the position with the largest number of pixels with a pixel value of 0 that is closest to the initial centerline as the final centerline position.

[0180] That is, based on the image height h of the stitched infrared image, the initial horizontal centerline position (h / / 2) of the stitched infrared image is determined; the image is moved up and down by h / / 4 pixels near the initial horizontal centerline, and within the range of height (max(0,h / 2 - h / / 4), min(h, h / / 2 + h / / 4)), the number of pixels with a value of 0 in each row is counted, and the pixels are sorted according to the number of pixels with a value of 0; the row of pixels with the largest number of pixels with a value of 0 and closest to the initial centerline is selected as the final horizontal centerline position.

[0181] (2) Determine the intersection point of the horizontal central axis and the connected regions on the left and right sides that meet the preset conditions in the binarized stitched infrared image;

[0182] The connected regions on the left and right sides of the stitched infrared image that meet preset conditions are specifically connected regions where the left and right sides of the stitched infrared image have a temperature boundary with other areas, the number of pixels in the connected region is greater than a preset proportion of the total number of pixels in the stitched infrared image, and the area located above the horizontal central axis accounts for more than half of the total area of ​​the region. In other words, in the stitched infrared image, there are connected regions on the left and right sides with a clear temperature boundary with other areas, the number of pixels in the connected region is greater than a preset proportion of the total number of pixels in the stitched infrared image, and the area located above the horizontal central axis in the connected region accounts for more than half of the total area of ​​the region. For example, the number of pixels in the connected region accounts for more than 5% of the total number of pixels in the stitched infrared image, and the area located above the horizontal central axis accounts for more than half of the total area of ​​the region. Among the intersection points of the horizontal central axis and the connected regions on the left and right sides that meet the preset conditions, the intersection point on the left is defined as the left point, and the intersection point on the right is defined as the right point.

[0183] (3) Based on the determined horizontal central axis and the intersection point, determine the window area, left and right wall areas, front wall area and ground area in the stitched infrared image to obtain the spatial temperature field of the room.

[0184] Specifically, the positions of the window area, left and right wall areas, front wall area, and ground area in the stitched infrared image are determined based on the intersection point and the horizontal central axis determined in the binarized stitched infrared image.

[0185] Figure 8 This diagram illustrates the temperature distribution in the window area, the left and right wall areas, the front wall area, and the floor area. The final effect is shown below. Figure 8 As shown. From Figure 8As can be seen, the binarization result effectively distinguishes the window's location and identifies the area on the right where no other temperature boundaries exist. Furthermore, it effectively integrates prior knowledge to separate the ground area where people are active from the wall area. To further visualize the temperature distribution results, different areas can be assigned the same color, with the color depth adjusted according to the temperature. The final coloring effect is as follows: Figure 9 As shown. Figure 9 The image shows a colorized representation of the temperature distribution. The horizontal and vertical axes are in pixels, and the color depth represents the temperature magnitude.

[0186] Furthermore, not all infrared images show that the highlighted areas (window areas) are located at the left and right edges. Figure 10 A visualization of the process of constructing a space temperature field is shown. For example... Figure 10 As shown in the last two images, the highlighted area is located in the middle. To further evaluate the effectiveness of the proposed solution, infrared images with a large window area (the proportion of window area pixels in the total number of pixels in the infrared image is greater than the preset proportion) were processed.

[0187] like Figure 10 As shown in the last two images, after binarization, the intersection of the horizontal line's central axis and the highly connected region may be located at the lower edge of the highlighted area (window area) (see...). Figure 10 In the grayscale image on the left side of the third row, the horizontal central axis is below the large highlighted area representing the window (the intersection point is also below). The vertical central axis extending from this intersection point divides the highlighted area. Therefore, after obtaining the intersection point of the horizontal central axis and the highlighted area (window area), it is further determined whether the vertical line extending upwards from this intersection point still contains a highlighted area label (the label corresponding to the highlighted area after binarization is 1). If a highlighted area label still exists, it is necessary to determine whether the intersection point needs to be moved left or right to satisfy the physically meaningful window position. This can be analyzed based on a one-dimensional projection of the temperature gradient.

[0188] 5. Calculate the temperature gradient map G along the horizontal direction (X-axis) for the panoramic view excluding the highlighted areas on the horizontal central axis. The gradient is calculated as |T(x+1) - T(x)|;

[0189] 6. Sum the temperature gradient graph G along the vertical direction (Y-axis).

[0190] Because the temperature changes drastically at the junction of the wall where the window is located and the adjacent wall, the sum of the temperature gradients in this column will be significantly higher than that in the left and right areas.

[0191] 7. Find the peak value from step 2, which is to estimate the approximate location of the intersection line.

[0192] 8. Determine whether the intersection line is to the left or right of the current intersection point. If it is to the right, move the intersection point one pixel to the right until there are no highlighted area labels on the vertical line. Conversely, move it to the right.

[0193] Using the above method, even if the highlighted area is in the middle, the floor, wall and window areas can be separated well.

[0194] The adjustment unit 160 is used to adjust the operation of the air conditioner according to the spatial temperature field of the room.

[0195] Specifically, based on the constructed spatial temperature field, the high-temperature zone (e.g., a zone with a temperature higher than a first preset temperature threshold), the low-temperature zone (e.g., a zone with a temperature lower than a second preset temperature threshold), and the overall temperature gradient change of the room are determined, as well as whether the temperature at the location of the human body is within a preset comfortable temperature range. Based on the determined high-temperature zone of the room, the overall temperature gradient change, and whether the temperature at the location of the human body is within the preset comfortable temperature range, the operation of the air conditioner is adjusted, for example, by adjusting the airflow direction, airflow speed, and / or air conditioner power.

[0196] For example, a spatial temperature field can be created by assigning the same color to different areas and adjusting the color depth according to temperature. Based on the color, one can determine the temperature gradient changes in high-temperature and low-temperature areas, as well as the overall temperature gradient, and the temperature comfort level at a human location. If the temperature at a human location is uncomfortable, appropriate adjustments can be made based on the human's location coordinates.

[0197] For example, including the following situations:

[0198] 1. Determine if there are any abnormal temperature points, such as electrical malfunctions, and provide feedback to the user via an app or other means.

[0199] 2. If the overall temperature is too low, increase the temperature setting and run the air conditioner at a lower power. If the overall temperature is too high, decrease the temperature setting and increase the fan speed.

[0200] 3. When the temperature of the person's location is within the comfortable temperature range, no adjustment is needed; when the temperature of the person's location is not within the comfortable temperature range, the wind speed is dynamically adjusted according to the spatial temperature gradient distribution of the person's location to bring the temperature of the person's location within the comfortable temperature range. For example, when the temperature of the person's location is below the lower limit of the comfortable temperature range, the wind speed is reduced and / or the wind is directed away from the person; when the temperature of the person's location is above the upper limit of the comfortable temperature range, the wind speed is increased and / or the wind is directed towards the person.

[0201] Figure 15 This is a structural block diagram of another embodiment of the air conditioner control device provided by the present invention. Figure 15As shown, the air conditioner control device 100 further includes an identification unit 120 and a fusion unit 130.

[0202] The identification unit 130 is used to identify target objects in the acquired infrared images from the N directions, and to convert the pixel coordinates of the target objects into world coordinates using a pre-set homography matrix.

[0203] In one specific implementation, a target detection algorithm identifies target objects (such as human bodies, heat sources, etc.) in the acquired infrared image. The algorithm outputs the target object's category, bounding box coordinates [x_center, y_center, width, height], and confidence score. x_center is the x-coordinate of the target bounding box center point, y_center is the y-coordinate of the target bounding box center point, width is the width of the bounding box, height is the height of the bounding box, and confidence score is the detection confidence score. When the confidence score is greater than a preset probability threshold (confidence > threshold), the pixel coordinates of the corresponding object are mapped to world coordinates using the corresponding homography matrix H. The threshold represents the probability threshold for identifying the object as a target.

[0204] Specifically, the pixel positions of the acquired infrared images from the N directions are converted into spatial positions on the ground plane according to a pre-set homography matrix. Different homography matrices correspond to different infrared images from different directions.

[0205] To cover the entire room, this invention collects infrared images from N directions. Because the angles from which the infrared images are collected differ, the pixel position of a point varies in different infrared images, requiring different transformation formulas to map it to the same world point.

[0206] The method for setting the homography matrix corresponding to infrared images from different orientations is the same. It is important to note that a unified reference point must be used: all coordinates should have the same origin (0,0) at the same corner of the room, all coordinate units should be the same (e.g., meters), and the X and Y axes should have the same defined direction.

[0207] How to set up a homography matrix:

[0208] Step 1: Select four non-collinear points in the room whose corresponding real-world coordinates (xj, yj) are known.

[0209] Step 2: Acquire infrared images containing the four selected points using an infrared camera, and mark the pixel coordinates (ui, vi) of each point in the acquired infrared images.

[0210] Step 3: Establish the mapping relationship between the pixel coordinates (ui,vi) of the infrared image and the real-world coordinates (xj,yj).

[0211] Specifically, the mapping relationship between the pixel coordinates (ui, vi) of the infrared image and the real-world coordinates (xj, yj) is as follows:

[0212]

[0213] Where s is the scale factor, It is a homography matrix.

[0214] The scale factor *s* is used to convert the "pixel units" in an image to "physical units" (such as meters and centimeters) in the real world. The homography matrix itself is a projective geometric transformation, losing depth information, and therefore cannot directly provide the absolute size in the real world. The size can be determined using a known reference object. For example, in image measurement: using image processing algorithms (such as edge detection), find two feature points of the reference object in the image and calculate the pixel distance *Dpixel* between them. To obtain the true size: the true physical distance *Dreal* of the reference object, the scale factor is equal to *Dreal* / *Dpixel* (unit: meters / pixel). For example, if a 60cm * 60cm floor tile is laid on a floor, and in an infrared image, the diagonal length of one tile is measured to be 1000 pixels, and the actual diagonal length of the tile is approximately 0.848, then the scale factor is: *S* = 0.848 / 1000.

[0215] Step four: Substitute the pixel coordinates and world coordinates of the selected four points into the mapping relationship to obtain the corresponding homogeneous equation, and solve the homogeneous equation to obtain the homography matrix.

[0216] Specifically, by substituting the pixel coordinates and world coordinates of the selected four points into the above mapping relationship, the corresponding homogeneous equation can be obtained. Solving the homogeneous equation yields h. ij， The homography matrix H can then be obtained.

[0217] The fusion unit 140 is used to perform coordinate fusion on the world coordinates of the target object in the infrared images of the N directions to obtain the world coordinates of the target object in the spatial temperature field of the room.

[0218] The target object may specifically include: a human body and / or a heat source (an object that generates heat); the fusion unit performs coordinate fusion on the world coordinates of the target object in the infrared images from the N directions, including: for the human body, performing coordinate fusion on the world coordinates of the target object in the infrared images from the N directions using a clustering algorithm to obtain the human body's position; for example, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm can be used to perform coordinate fusion on the world coordinates in the infrared images from the N directions. This is because the world coordinates of the same person should be similar in the N directions, while the world coordinates of people in different locations are not similar. Based on this, assuming there are two people and both people are captured in images from each direction, using a clustering algorithm to cluster the 2N world coordinates can yield two labels, i.e., the coordinates of the two people.

[0219] Clustering using the DBSCAN clustering algorithm can specifically include the following steps:

[0220] 1. Determine the model parameters:

[0221] esp (radius): Set according to the room size and accuracy requirements. For example, if two people cannot stand too close, it can be set to 0.5m. The specific value depends on the resolution.

[0222] Min_samples (minimum number of samples): set to 1, because even if only one camera sees a person, they can be identified as a single individual.

[0223] 2. Input the model parameters into the DBSCAN algorithm to obtain the cluster label for each coordinate point.

[0224] 3. Traverse the cluster labels, calculate the average of all points belonging to the same cluster, and obtain the coordinates of each individual.

[0225] For the heat source, the location of the heat source is obtained by averaging the world coordinates of the target object in the infrared images from the N directions. That is, the world coordinates in the infrared images from each direction can be directly averaged.

[0226] Because there may be multiple people involved, the coordinates calculated from N angles are mixed together. Using clustering methods will better distinguish the number of people and their corresponding coordinates. There may only be one heat source, so the coordinates calculated from N angles can be directly weighted and averaged. The corresponding world coordinates are calculated based on the homography matrix. If the heat source is captured from all N directions, N world coordinates are obtained. These N world coordinates are then averaged. The specific world coordinate units are (X, Y), so the average is calculated separately for X and Y.

[0227] The present invention also provides a storage medium corresponding to the control method of the air conditioner, wherein a computer program is stored thereon, and when the program is executed by a processor, the steps of any of the aforementioned methods are implemented.

[0228] The present invention also provides an air conditioner corresponding to the control method of the air conditioner, comprising a processor, a memory, and a computer program stored in the memory that can run on the processor, wherein the processor executes the program to implement the steps of any of the aforementioned methods.

[0229] The present invention also provides an air conditioner corresponding to the control device of the air conditioner, including any of the control devices described above.

[0230] The present invention also provides a computer program product corresponding to the control method of the air conditioner, including a computer program that, when executed by a processor, implements the steps of any of the aforementioned methods.

[0231] In related technologies, single-point temperature sensing control methods suffer from problems such as localized overcooling and overheating, resulting in poor user comfort and high energy consumption. The technical solution of this invention addresses the problem of localized overcooling or overheating in single-point temperature sensing air conditioning control schemes by constructing a spatial temperature field. Based on this constructed spatial temperature field, the operation of the air conditioner is adjusted, which can improve user comfort while reducing energy loss, thus achieving energy-saving effects.

[0232] In related technologies, the field of view of infrared sensors is too small, resulting in a limited thermal imaging field of view that cannot cover the entire room. The technical solution of this invention controls an infrared sensor to acquire infrared images from different directions, and stitches them together based on the rotation angle of the infrared camera and the angle of the infrared camera's field of view occupied by each pixel to obtain a stitched infrared image. This stitched infrared image is then used to construct a spatial temperature field, effectively expanding the field of view of the infrared camera and constructing a more accurate spatial temperature field.

[0233] In related technologies, thermal imaging images have low resolution, resulting in blurred boundaries between various structures in space and an inability to accurately distinguish different spatial structural regions. The technical solution of this invention, by binarizing the image and dividing the room region based on the horizontal central axis of the image, can accurately distinguish different spatial structural regions.

[0234] In related technologies, air conditioning control strategies, such as fixed-sweep and manual louver control, lack precise understanding of the location and temperature characteristics of heat / cold sources in the space. This results in low air conditioning efficiency, easily causing discomfort to users due to drafts, and high energy consumption. This invention, by fusing the coordinates of a target object in infrared images from various angles, can accurately locate the target object (human body and / or heat source) within the spatial temperature field. Based on changes in the spatial temperature field, the air conditioning is adjusted, improving user comfort while reducing energy loss and achieving energy-saving effects.

[0235] Accordingly, the solution provided by the present invention acquires thermal imaging images of users based on infrared visual detection devices without infringing on user privacy, and collects infrared thermal imaging images from different directions; by removing overlapping parts and stitching four images together using a method based on the movement of the infrared camera and the field of view, a thermal imaging image with an increased field of view is obtained, which can effectively expand the field of view of the infrared camera and construct a more accurate spatial temperature field.

[0236] The functions described herein can be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions can be stored as one or more instructions or codes on or transmitted via a computer-readable medium. Other examples and embodiments are within the scope and spirit of this invention and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Furthermore, the functional units can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit.

[0237] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0238] The units described as separate components may or may not be physically separate. Similarly, the components of the control device may or may not be physical units; they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0239] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0240] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for controlling an air conditioner, characterized in that, include: Infrared images are captured from N preset locations within the room using infrared cameras; where N is an even number. The infrared images from the N directions are stitched together to obtain the stitched infrared image of the room. The spatial temperature field of the room is constructed based on the obtained stitched infrared image of the room. The operation of the air conditioner is adjusted according to the spatial temperature field of the room.

2. The method according to claim 1, characterized in that, The infrared images from the N directions are stitched together to obtain a stitched infrared image of the room, including: Calculate the angle of the infrared camera's field of view occupied by each pixel in the infrared image captured by the infrared camera; Based on the angle of the infrared camera's field of view occupied by each pixel and the rotation angle of the infrared camera, calculate the number of overlapping pixels between each pair of adjacent infrared images in the N azimuth infrared images. The overlapping portions of the N infrared images are cropped based on the number of overlapping pixels between each pair of adjacent images, and the N infrared images are then stitched together to obtain the stitched infrared image of the room.

3. The method according to claim 2, characterized in that, Based on the calculated angle of the infrared camera's field of view occupied by each pixel and the rotation angle of the infrared camera, the number of overlapping pixels between any two adjacent infrared images in the N azimuths is calculated, including: The number of overlapping pixels between any two adjacent infrared images is equal to the difference between the number of pixels in the image width or height and the ratio of the rotation angle of the infrared camera to the angle of the infrared camera's field of view occupied by each pixel.

4. The method according to any one of claims 1-3, characterized in that, Based on the obtained stitched infrared image of the room, the spatial temperature field of the room is constructed, including: The stitched infrared image is binarized to obtain a binarized stitched infrared image, and the horizontal central axis in the binarized stitched infrared image is determined. Determine the intersection point between the horizontal central axis of the binarized stitched infrared image and the connected regions on the left and right sides of the stitched infrared image that meet preset conditions; Based on the determined horizontal central axis and the intersection point, the window area, left and right wall areas, front wall area and ground area in the stitched infrared image are determined to obtain the spatial temperature field of the room.

5. The method according to claim 4, characterized in that, The connected regions on the left and right sides of the stitched infrared image that meet the preset conditions are the connected regions where the left and right sides of the stitched infrared image have temperature boundaries with other regions, the number of pixels is greater than a preset ratio of the total number of pixels in the stitched infrared image, and the area located above the horizontal central axis accounts for more than half of the total area of ​​the region.

6. The method according to claim 4 or 5, characterized in that, Determining the horizontal central axis in the binarized stitched infrared image includes: The initial horizontal centerline position of the stitched infrared image is determined based on the image height of the stitched infrared image. Within a range of M pixels above and below the initial horizontal centerline position, count the number of pixels with a value of 0 in each row, and sort the pixels in each row according to the number of pixels with a value of 0. The row of pixels with the largest number of pixels having a value of 0 and closest to the initial center line is selected as the final horizontal center line position.

7. The method according to claim 4 or 5, characterized in that, Also includes: After obtaining the spatial temperature field of the room, different areas are assigned the same color, and the color depth is adjusted according to the temperature.

8. The method according to any one of claims 1-3, characterized in that, Also includes: Identify target objects in the collected infrared images from the N directions, and convert the pixel coordinates of the target objects into world coordinates using a pre-set homography matrix; The world coordinates of the target object in the infrared images from the N directions are fused to obtain the world coordinates of the target object in the spatial temperature field of the room.

9. The method according to any one of claims 1-3, characterized in that, The target object includes: a human body and / or a heat source; The world coordinates of the target object in the identified N infrared images are fused, including: For the human body, the world coordinates of the target object in the collected infrared images from the N directions are fused by clustering; And / or, For the heat source, the world coordinates of the target object in the collected infrared images from the N directions are fused using a weighted average algorithm.

10. A control device for an air conditioner, characterized in that, include: The acquisition unit is used to acquire infrared images of N preset locations within the room using an infrared camera; where N is an even number. The stitching unit is used to stitch together the infrared images from the N directions to obtain a stitched infrared image of the room. The construction unit is used to construct the spatial temperature field of the room based on the obtained stitched infrared image of the room. The regulating unit is used to regulate the operation of the air conditioner according to the spatial temperature field of the room.

11. A storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-9.

12. An air conditioner, characterized in that, It includes a processor, a memory, and a computer program stored in the memory that can run on the processor, wherein the processor executes the program to implement the steps of any of the methods of claims 1-9, or includes the control device as described in claim 10.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-9.

Images