An air conditioner indoor unit fan blade control method and device, edge device, and storage medium

Abstract

This invention provides a method, device, edge device, and storage medium for controlling the fan blades of an air conditioner indoor unit, relating to the field of air conditioning technology. It applies computer vision algorithms to the control of the fan blades of an air conditioner indoor unit. By acquiring an image of the indoor environment where the air conditioner is located, a pre-trained target detection model is used to detect people in the indoor environment image, obtaining target detection boxes representing the positions of people in the indoor environment image. The rotation direction of the fan blades of the air conditioner indoor unit is adjusted according to the target detection boxes, thereby automatically adjusting the airflow direction of the air conditioner according to the actual position of the people in the room, so that people can feel a noticeable breeze and improve the user experience.

Description

Technical Field

[0001] This invention relates to the field of air conditioning technology, and more specifically, to a method, device, peripheral equipment, and storage medium for controlling the fan blades of an indoor air conditioning unit. Background Technology

[0002] Currently, the direction of fan blade rotation in air conditioner indoor units is mainly divided into left-right swing, up-down swing, and fixed swing. However, in reality, people indoors cannot always be in the direction of the airflow, resulting in a weak airflow and a generally poor user experience. Summary of the Invention

[0003] The problem solved by this invention is how to automatically adjust the airflow direction of the air conditioner according to the actual position of the people in the room, so that people can feel a noticeable breeze.

[0004] To address the above problems, this invention provides a method for controlling the fan blades of an air conditioner indoor unit, applied to an edge device. The edge device is communicatively connected to an air conditioner. The method for controlling the fan blades of the air conditioner indoor unit includes:

[0005] Acquire an image of the indoor environment where the air conditioner is located, the image of the indoor environment including people in the room;

[0006] The indoor people in the indoor environment image are detected by using a pre-trained target detection model, and target detection boxes representing the positions of the indoor people in the indoor environment image are obtained.

[0007] Adjust the rotation direction of the fan blades of the indoor air conditioner according to the target detection box.

[0008] Compared with the prior art, the air conditioner indoor unit fan blade control method of the present invention has the following advantages: by acquiring an image of the indoor environment where the air conditioner is located, using a pre-trained target detection model to detect indoor people in the indoor environment image, a target detection box representing the position of indoor people in the indoor environment image is obtained, and the rotation direction of the air conditioner indoor unit fan blades is adjusted according to the target detection box, so that the air blowing direction of the air conditioner can be automatically adjusted according to the actual position of the indoor people, so that people can feel a clear wind and improve the user experience.

[0009] Optionally, the target detection model includes a backbone network, a neck network, and a head network;

[0010] The step of using a pre-trained target detection model to detect indoor occupants in the indoor environment image and obtaining target detection boxes representing the positions of indoor occupants in the indoor environment image includes:

[0011] Based on the indoor environment image, the input image is obtained;

[0012] The input image is input into the backbone network, and the backbone network is used to extract features from the input image, outputting two feature maps of different scales: a first feature map and a second feature map.

[0013] The first feature map and the second feature map are input into the neck network, and the neck network performs feature enhancement extraction on the first feature map and the second feature map based on an attention mechanism to obtain a first enhanced feature map and a second enhanced feature map.

[0014] The first and second enhanced feature maps are input into the head network for image regression and classification to obtain two target prediction boxes of different scales.

[0015] The two target prediction boxes at different scales are post-processed to obtain the target detection boxes.

[0016] Optionally, the backbone network includes CBR units, max pooling layers, and feature extraction units of three different scales;

[0017] The step of extracting features from the input image using the backbone network and outputting two feature maps of different scales, namely a first feature map and a second feature map, includes:

[0018] The input image is processed by convolution, batch normalization, and ReLU activation function using the CBR unit to obtain a first input feature map, the size of which is 1 / 2 of the input image.

[0019] The first input feature map is subjected to max pooling using the max pooling layer to obtain a second input feature map, the size of which is 1 / 2 of the first input feature map;

[0020] The first feature extraction unit is used to perform first-level feature extraction on the second input feature map to obtain a third input feature map, the size of which is 1 / 2 of the second input feature map;

[0021] The second feature extraction unit is used to perform second-level feature extraction on the third input feature map to obtain the first feature map, the size of which is 1 / 2 of the third input feature map;

[0022] The third feature extraction unit is used to perform third-level feature extraction on the first feature map to obtain the second feature map, the size of which is 1 / 2 of the size of the first feature map.

[0023] Optionally, the first feature extraction unit includes three ShuffleNetV2 basic units and one basic unit with spatial downsampling; the second feature extraction unit includes seven ShuffleNetV2 basic units and one basic unit with spatial downsampling; and the third feature extraction unit includes three ShuffleNetV2 basic units, one basic unit with spatial downsampling, and one spatial pyramid pooling layer.

[0024] Optionally, the neck network includes three SimAM attention units, a CBR unit, a depthwise separable convolutional layer, an upsampling layer, and a stitching layer;

[0025] The step of the neck network performing feature enhancement extraction on the first feature map and the second feature map based on an attention mechanism to obtain the first enhanced feature map and the second enhanced feature map includes:

[0026] The second feature map is processed using the SimAM attention mechanism by the first SimAM attention unit to obtain a first output feature map. The size of the first output feature map is the same as that of the second feature map. Figure 1 To;

[0027] The first output feature map is processed by convolution, batch normalization, and ReLU activation function using the CBR unit to obtain the first enhanced feature map. The size of the first enhanced feature map is the same as that of the first output feature map. Figure 1 To;

[0028] The first enhanced feature map is convolved using the depthwise separable convolutional layer to obtain a second output feature map, the size of which is the same as that of the first enhanced feature map. Figure 1 To;

[0029] The second output feature map is upsampled using the upsampling layer to obtain a third output feature map, the size of which is twice that of the second output feature map;

[0030] The third output feature map is processed using the SimAM attention mechanism by a second SimAM attention unit to obtain a fourth output feature map. The size of the fourth output feature map is the same as that of the third output feature map. Figure 1 To;

[0031] The fourth output feature map and the first feature map are concatenated using the concatenation layer to obtain the second enhanced feature map. The size of the second enhanced feature map is the same as that of the first feature map. Figure 1 To.

[0032] Optionally, the step of adjusting the rotation direction of the fan blades of the indoor air conditioner based on the target detection box includes:

[0033] Calculate the center coordinates of the target detection box based on the vertex coordinates of the target detection box;

[0034] Obtain a pre-established reference image, wherein the size of the reference image is consistent with that of the indoor environment image and is divided into m*n regions, each region has a set control signal, m represents the number of rows of the region, n represents the number of columns of the region, and m and n are both integers greater than or equal to 1;

[0035] Based on the reference map, the target area where the center coordinates are located is determined from m*n regions;

[0036] The control signal corresponding to the target area is sent to the air conditioner so that the air conditioner adjusts the rotation direction of the fan blades of its indoor unit according to the control signal.

[0037] Optionally, there are multiple indoor occupants and multiple target detection boxes, with one target detection box corresponding to one indoor occupant;

[0038] The step of adjusting the rotation direction of the fan blades of the indoor air conditioner based on the target detection box includes:

[0039] Calculate the center coordinates of each target detection box based on the vertex coordinates of each target detection box;

[0040] Obtain a pre-established reference image, wherein the reference image is the same size as the indoor environment image and is divided into m*n regions, each region has a set control signal, m represents the number of rows of the region, n represents the number of columns of the region, and m and n are both integers greater than or equal to 1;

[0041] Based on the reference map, the target area where each of the center coordinates is located is determined from m*n regions;

[0042] If all the center coordinates are located in the same target area, then the control signal corresponding to the target area is sent to the air conditioner so that the air conditioner adjusts the rotation direction of the fan blades of its indoor unit according to the control signal.

[0043] If there are multiple target areas where all center coordinates are located, then the direction of rotation of the air conditioner's indoor unit's fan blades remains unchanged.

[0044] The present invention also provides an air conditioner indoor unit fan blade control device, applied to an edge device, wherein the edge device is communicatively connected to an air conditioner, and the air conditioner indoor unit fan blade control device includes:

[0045] The image acquisition module is used to acquire an image of the indoor environment where the air conditioner is located, the image of the indoor environment including people in the room;

[0046] The target detection module is used to detect indoor people in the indoor environment image using a pre-trained target detection model, and obtain target detection boxes that represent the positions of indoor people in the indoor environment image.

[0047] The rotation direction adjustment module is used to adjust the rotation direction of the fan blades of the indoor air conditioner according to the target detection box.

[0048] The present invention also provides an edge device, including a processor and a memory, the memory being used to store a program, and the processor being used to implement the above-described air conditioner indoor unit fan blade control method when executing the program.

[0049] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for controlling the fan blades of an indoor air conditioner. Attached Figure Description

[0050] Figure 1 This is an application scenario diagram of the air conditioner indoor unit fan blade control method provided by the present invention.

[0051] Figure 2 Flowchart of the air conditioner indoor unit fan blade control method provided by the present invention Figure 1 .

[0052] Figure 3 This is a block diagram illustrating the target detection model provided by the present invention.

[0053] Figure 4 for Figure 3 The diagram shows the box layouts for Unit 1 and Unit 2 in the target detection model.

[0054] Figure 5 for Figure 2 The flowchart of step S102 in the air conditioner indoor unit fan blade control method is shown.

[0055] Figure 6 for Figure 2 A flowchart illustrating step S103 of the air conditioner indoor unit fan blade control method shown.

[0056] Figure 7 A block diagram for reference provided for this invention.

[0057] Figure 8 Flowchart of the air conditioner indoor unit fan blade control method provided by the present invention Figure 2 .

[0058] Figure 9for Figure 2 Another flowchart of step S103 in the air conditioner indoor unit fan blade control method shown.

[0059] Figure 10 A block diagram of the air conditioner indoor unit fan blade control device provided by the present invention.

[0060] Figure 11 A block diagram illustrating the edge device provided by the present invention.

[0061] Explanation of reference numerals in the attached figures:

[0062] 10-Edge device; 11-Processor; 12-Memory; 13-Bus; 20-Air conditioner; 101-Image acquisition module; 102-Target detection module; 103-Rotation direction adjustment module. Detailed Implementation

[0063] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0064] As smart home systems become increasingly integrated into every household, the intelligence of furniture is becoming more and more important. In recent years, products such as smart robots, smart voice control, and smart robot vacuums have been continuously iterated, with smart air conditioners mainly focusing on functions such as voice control. The intelligence of air conditioners should also be reflected in the control of temperature, airflow direction, etc., that is, without human active control, the air conditioner can automatically control airflow direction, temperature, etc., according to the actual environment.

[0065] For example, the direction of the fan blades in the indoor unit of an air conditioner is mainly divided into left-right swing, up-down swing, and fixed swing. However, in real life, people indoors cannot always be in the direction of the airflow, resulting in the air conditioner not providing a strong breeze and a generally poor user experience.

[0066] Therefore, it is important for those skilled in the art to know how to automatically adjust the airflow direction of the air conditioner according to the actual location of the people in the room.

[0067] Based on this, the embodiments of the present invention apply computer vision algorithms to the fan blade control of the indoor unit of the air conditioner. By acquiring an image of the indoor environment where the air conditioner is located, a pre-trained target detection model is used to detect people in the indoor environment image to obtain a target detection box representing the position of people in the indoor environment image. The fan blade rotation direction of the indoor unit of the air conditioner is adjusted according to the target detection box, so that the air blowing direction of the air conditioner can be automatically adjusted according to the actual position of people in the room, so that people can feel a clear breeze and improve the user experience.

[0068] The following is a detailed description with reference to the accompanying drawings.

[0069] The air conditioner indoor unit fan blade control method provided in this embodiment of the invention is applied to edge device 10. Please refer to... Figure 1 The edge device 10 communicates with the air conditioner 20 via a network, which can be a wired or wireless network. The edge device 10 is used to analyze indoor environmental images using computer vision algorithms to automatically adjust the airflow direction of the air conditioner 20.

[0070] Edge device 10 refers to a device with certain computing and processing capabilities, capable of processing and analyzing image and video data locally without relying on cloud servers for large-scale data transmission and processing. Edge device 10 can quickly respond to and process real-time computer vision tasks, such as face recognition, object detection and tracking, and scene analysis, providing more convenient and real-time visual intelligence applications.

[0071] In this embodiment, the edge device 10 can be an embedded device or a portable device, such as a desktop computer, laptop computer, smartphone, tablet computer, smart camera, smartwatch, smart glasses, etc. The edge device 10 can also be a chip system, which can be composed of chips or include chips and other discrete components; this embodiment does not impose any limitations on this.

[0072] Please refer to Figure 2 , Figure 2 A flowchart illustrating the air conditioner indoor unit fan blade control method provided by the present invention is shown. This data transmission method may include the following steps:

[0073] S101, acquire an image of the indoor environment where the air conditioner is located, including the indoor occupants.

[0074] In this embodiment, indoor environmental images can be captured by a camera module. The field of view of the camera module is consistent with the air blowing area of ​​the air conditioner 20. For example, if the air conditioner 20 is located in the living room, then the field of view of the camera module should also be the living room.

[0075] The camera module can be a standalone device, such as a camera or webcam, and communicate with the edge device 10 to capture images of the indoor environment where the air conditioner 20 is located and transmit them to the edge device 10 for further processing. Alternatively, the camera module can be integrated into the edge device 10, such as a smartphone camera, to capture images of the indoor environment where the air conditioner 20 is located and transmit them to the processor of the edge device 10 for further processing.

[0076] S102, use a pre-trained target detection model to detect indoor people in the indoor environment image, and obtain target detection boxes representing the location of indoor people in the indoor environment image.

[0077] In this embodiment, the object detection model can be a commonly used object detection model, such as the YOLO series models, such as YOLOv3, YOLOv3-tiny, YOLOv4, YOLOv4-tiny, etc., or it can be a convolutional neural network, or it can be a self-designed object detection model. This embodiment does not impose any restrictions on this. The following embodiment uses a self-designed improved YOLOv4-tiny network as an example for description.

[0078] Please refer to Figure 3 The target detection model can include a backbone network, a neck network, and a head network. Compared with the existing YOLOv4-tiny, the target detection model provided in this invention is an improved lightweight target detection model based on YOLOv4-tiny. The improvements of the model mainly include the following three aspects:

[0079] 1. In the backbone network, the lightweight network ShuffleNetv2 is used to replace the original backbone network. Depth-separable convolutional fusion is used to reduce the number of parameters and computation. The network structure is also modified to better fit the network's detection layer.

[0080] 2. In the backbone network, features with a larger receptive field are obtained by adding a spatial pyramid pooling (SPP) structure.

[0081] 3. The SimAM parameterless attention mechanism was added to the neck network to improve the network's feature extraction capability.

[0082] exist Figure 3 Based on the target detection model shown, please refer to Figure 5 Step S102, which uses a pre-trained target detection model to detect indoor people in an indoor environment image and obtains a target detection box representing the location of indoor people in the indoor environment image, may include sub-steps S1021 to S1025.

[0083] S1021, Based on the indoor environment image, obtain the input image.

[0084] In this embodiment, after the edge device 10 receives the indoor environment image captured by the camera module, it can first perform a series of processing on the indoor environment image, such as image size adjustment and image enhancement operations, to facilitate subsequent processing by the target detection model.

[0085] For example, if the indoor environment image is 640*640*3 (i.e., the size is 640*640 and the number of channels is 3), adjusting the size of the indoor environment image will give the input image 416*416*3 (i.e., the size is 416*416 and the number of channels is 3).

[0086] S1022, the input image is input into the backbone network, and the backbone network is used to extract features from the input image, outputting a first feature map and a second feature map at two different scales.

[0087] In this embodiment, the backbone network is mainly composed of some ShuffleNetv2 basic units and basic units with spatial downsampling. An SPP spatial pyramid pooling structure is also added at the tail to extract features from the input image and output two feature maps of different scales: a first feature map and a second feature map.

[0088] like Figure 3 As shown, the backbone network includes CBR units, max pooling layers, and three feature extraction units of different scales, namely, Figure 3 The CBR, Maxpool, Stage 1, Stage 2, and Stage 3 are shown.

[0089] CBR stands for Conv+BN+ReLU, which consists of a Convolutional Layer (Conv), a Batch Normalization Layer (BN), and a ReLU activation function. In CBR, Conv extracts input features through filters and maps them to the output feature space. BN normalizes the output features to accelerate the training process and reduce the model's sensitivity to hyperparameters. The ReLU activation function is applied to the feature map to enhance the model's nonlinear characteristics.

[0090] The first feature extraction unit (Stage 1) consists of 3 Unit 1 and Unit 2 units, the second feature extraction unit (Stage 2) consists of 7 Unit 1 and Unit 2 units, and the third feature extraction unit (Stage 3) consists of 3 Unit 1, Unit 2, and SPP units. Unit 1 represents the basic unit of ShuffleNetV2, Unit 2 represents the basic unit with spatial downsampling, and SPP represents the spatial pyramid pooling layer.

[0091] Please refer to Figure 4 Unit 1 consists of a Convolutional Batch Renderer (CBR), a Depthwise Convolutional Layer (DwConv), and a Concat layer. It concatenates the input with the input processed by a CBR, DwConv, and a CBR to obtain the output. Unit 2 consists of a CBR, DwCBR, DwConv, Batch Normalization (BN), and Concat. It concatenates the input processed by DwConv, BN, and CBR with the input processed by a CBR, DwCBR, and a CBR to obtain the output.

[0092] Therefore, the process of extracting features from the input image using the backbone network in step S1022 and outputting two feature maps of different scales, namely the first feature map and the second feature map, may include sub-steps S10221 to S10225.

[0093] S10221, The input image is processed by convolution, batch normalization and ReLU activation function using CBR unit to obtain the first input feature map. The size of the first input feature map is 1 / 2 of the input image.

[0094] In this embodiment, the input image is 416*416*3, that is, the size is 416*416 and the number of channels is 3. The first input feature map is 208*208*24, that is, the size is 208*208 and the number of channels is 24.

[0095] S10222, Max pooling is performed on the first input feature map using a max pooling layer to obtain a second input feature map, the size of which is 1 / 2 of the size of the first input feature map.

[0096] In this embodiment, the second input feature map is 104*104*24, that is, the size is 104*104 and the number of channels is 24.

[0097] S10223, the first feature extraction unit is used to perform first-level feature extraction on the second input feature map to obtain the third input feature map, the size of which is 1 / 2 of the second input feature map.

[0098] In this embodiment, the third input feature map is 52*52*116, that is, the size is 52*52 and the number of channels is 116.

[0099] S10224, the second feature extraction unit is used to perform second-level feature extraction on the third input feature map to obtain the first feature map, the size of which is 1 / 2 of the third input feature map.

[0100] In this embodiment, the first feature map is 26*26*232, that is, the size is 26*26 and the number of channels is 232.

[0101] S10225, the third feature extraction unit is used to perform third-level feature extraction on the first feature map to obtain the second feature map, the size of which is 1 / 2 of the size of the first feature map.

[0102] In this embodiment, the second feature map is 13*13*464, that is, the size is 13*13 and the number of channels is 464.

[0103] S1023, the first feature map and the second feature map are input into the neck network. The neck network performs feature enhancement extraction on the first feature map and the second feature map based on the attention mechanism to obtain the first enhanced feature map and the second enhanced feature map.

[0104] In this embodiment, compared with the neck structure of YOLOv4-tiny, the improved neck network incorporates three SimAM attention mechanisms after the Stage2 and SPP structures, as well as during the upsampling of the 13×13 feature map, to enhance and extract information from the feature output of the backbone network.

[0105] like Figure 3 As shown, the neck network includes three SimAM attention units, a CBR unit, a depthwise separable convolutional layer, an upsampling layer, and a stitching layer, i.e. Figure 3 The three are shown: SimAM, CBR, Upsample, DwConv, and Concat.

[0106] Therefore, the process of the neck network performing feature enhancement extraction on the first feature map and the second feature map based on the attention mechanism in step S1023 to obtain the first enhanced feature map and the second enhanced feature map may include sub-steps S10231 to S10236.

[0107] S10231, the second feature map is processed using the SimAM attention mechanism by the first SimAM attention unit to obtain the first output feature map. The size of the first output feature map is the same as that of the second feature map. Figure 1 To.

[0108] In this embodiment, the second feature map is 13*13*464, that is, the size is 13*13 and the number of channels is 464. The first output feature map is 13*13*464, that is, the size is 13*13 and the number of channels is 464.

[0109] S10232, the first output feature map is processed by convolution, batch normalization, and ReLU activation function using the CBR unit to obtain the first enhanced feature map. The size of the first enhanced feature map is the same as that of the first output feature map. Figure 1 To.

[0110] In this embodiment, the first enhanced feature map is 13*13*256, that is, the size is 13*13 and the number of channels is 256.

[0111] S10233, a depthwise separable convolutional layer is used to convolve the first enhanced feature map to obtain a second output feature map. The size of the second output feature map is the same as that of the first enhanced feature map. Figure 1 To.

[0112] In this embodiment, the second output feature map is 13*13*128, that is, the size is 13*13 and the number of channels is 128.

[0113] S10234, the second output feature map is upsampled using an upsampling layer to obtain a third output feature map, the size of which is twice that of the second output feature map.

[0114] In this embodiment, the third output feature map is 26*26*128, that is, the size is 26*26 and the number of channels is 128.

[0115] S10235, the third output feature map is processed using the SimAM attention mechanism by the second SimAM attention unit to obtain the fourth output feature map. The size of the fourth output feature map is the same as that of the third output feature map. Figure 1 To.

[0116] In this embodiment, the fourth output feature map is 26*26*128, that is, the size is 26*26 and the number of channels is 128.

[0117] S10236, the fourth output feature map and the first feature map are concatenated using a concatenation layer to obtain a second enhanced feature map. The size of the second enhanced feature map is the same as that of the first feature map. Figure 1 To.

[0118] In this embodiment, the first feature map is 26*26*232, that is, the size is 26*26 and the number of channels is 232. The second enhanced feature map is 26*26*360, that is, the size is 26*26 and the number of channels is 360.

[0119] S1024, the first and second enhanced feature maps are input into the head network for image regression and classification to obtain two target prediction boxes of different scales.

[0120] like Figure 3 As shown, the first enhanced feature map (13*13*256) and the second enhanced feature map (26*26*360) are input into the head network for image regression and classification, resulting in two target prediction boxes of different scales: target detection boxes of 26*26*3*(1+4+NumCls) and 13*13*3*(1+4+NumCls). Here, 26*26 and 13*13 represent the size of the target detection boxes, 3 represents the number of channels, and NumCls represents the number of categories.

[0121] S1025, post-process the two target prediction boxes at different scales to obtain the target detection boxes.

[0122] In this embodiment, after obtaining two target prediction boxes of different scales through the above steps, the target prediction boxes can be filtered using the matrix nonmaximum suppression (Matrix NMS) algorithm, NMS post-processing strategy, etc., to obtain the final target detection box.

[0123] The backbone network structure of the improved lightweight target detection model based on YOLOv4-tiny provided in this invention is summarized in Table 1 below:

[0124] Table 1 Improved YOLOv4-tiny backbone network structure

[0125]

[0126] Meanwhile, the performance of the improved YOLOv4-tiny network structure provided by this invention was verified on the VOC2007+12 public dataset, and the results are shown in Table 2 below:

[0127] Table 2 compares the improved YOLOv4-tiny with other algorithms.

[0128]

[0129] Among them, YOLOv4-tiny-S 416 This refers to the improved YOLOv4-tiny network provided by the present invention. As can be clearly seen from Table 2, the improved YOLOv4-tiny network provided by the present invention has achieved certain improvements in the number of parameters, computational load, and accuracy compared with YOLOv4-tiny.

[0130] S103, adjust the rotation direction of the air conditioner indoor unit's fan blades according to the target detection box.

[0131] As can be seen from the foregoing, after inputting the indoor environment image into the target detection model to obtain the target detection box representing the position of the indoor people in the indoor environment image, the center position of the indoor people can be calculated based on the coordinates of the target detection box, and the rotation direction of the fan blades of the indoor unit of the air conditioner 20 can be adjusted according to the center position of the indoor people.

[0132] In one possible scenario, when there is only one person indoors, the target detection box will also be only one. Please refer to [the relevant documentation / reference]. Figure 6 The process of adjusting the rotation direction of the air conditioner's indoor unit's fan blades according to the target detection box in step S103 may include sub-steps S1031 to S1034.

[0133] S1031, Calculate the center coordinates of the target detection box based on the vertex coordinates of the target detection box.

[0134] In this embodiment, since the target detection box represents the position of people in the indoor environment image, the center coordinates can be calculated based on the coordinates of the four vertices of the target detection box, and the center coordinates are the center position of the people in the indoor environment.

[0135] S1032, Obtain a pre-established reference image, wherein the size of the reference image is consistent with the indoor environment image and is divided into m*n regions. Each region has a set control signal, where m represents the number of rows in the region and n represents the number of columns in the region. Both m and n are integers greater than or equal to 1.

[0136] In this embodiment, a reference map can be pre-established based on the indoor environment image. The reference map is the same size as the indoor environment image and is divided into m*n regions, and each region has a set control signal.

[0137] For example, please refer to Figure 7 The reference diagram is 640*640 pixels in size and is divided into 9 areas. These 9 areas are represented by numbers 0 to 8. Each area has a set control signal. For example, the control signal for area 0 is up + left, which means that the fan blades are pointing upwards and to the left, i.e., blowing air to the upper left.

[0138] S1033, Based on the reference map, determine the target region where the center coordinates are located from m*n regions.

[0139] In this embodiment, since the target detection box is marked in the indoor environment image, representing the position of people in the indoor environment image, and the indoor environment image is the same size as the reference image, the target area with the center coordinates can be determined from the m*n regions of the reference image.

[0140] S1034, send the control signal corresponding to the target area to the air conditioner so that the air conditioner can adjust the rotation direction of the fan blades of its indoor unit according to the control signal.

[0141] In this embodiment, after identifying the target area where the center coordinates are located in the reference diagram, it indicates that the people in the room are currently in the target area. It is necessary to adjust the direction of the fan blades of the indoor unit to blow air towards the target area, so that the people in the room can feel a noticeable breeze.

[0142] Since each area in the reference diagram has a set control signal, the control signal corresponding to the target area is sent to the main control MCU of the air conditioner 20. The main control MCU can then adjust the rotation direction of the indoor unit's fan blades according to the control signal to blow air towards the target area.

[0143] That is, such as Figure 8As shown, the camera module acquires indoor environmental images, and the edge device 10 uses a pre-trained target detection model to detect indoor people in the indoor environmental images, obtaining target detection boxes that represent the positions of indoor people in the indoor environmental images. Then, the center position of the indoor people is calculated based on the target detection boxes, and the corresponding control signal is sent to the main control MCU of the air conditioner 20. The main control MCU of the air conditioner 20 then controls the rotation direction of the fan blades of the indoor unit to blow air towards the area where the indoor people are located, so that the indoor people feel a significant breeze.

[0144] In another possible scenario, when there are multiple people indoors, there will also be multiple target detection boxes. Please refer to [the relevant documentation / reference]. Figure 9 The process of adjusting the rotation direction of the air conditioner's indoor unit's fan blades according to the target detection box in step S103 may include sub-steps S103a to S103e.

[0145] S103a, calculate the center coordinates of each target detection box based on the vertex coordinates of each target detection box.

[0146] S103b, Obtain a pre-established reference image, wherein the size of the reference image is consistent with the indoor environment image and is divided into m*n regions. Each region has a set control signal, where m represents the number of rows in the region and n represents the number of columns in the region. Both m and n are integers greater than or equal to 1.

[0147] S103c, based on the reference map, determines the target region where each center coordinate is located from m*n regions.

[0148] S103d, if all center coordinates are located in the same target area, then the control signal corresponding to the target area is sent to the air conditioner so that the air conditioner adjusts the rotation direction of the fan blades of its indoor unit according to the control signal.

[0149] S103e, if there are multiple target areas where all center coordinates are located, then keep the direction of the air conditioner indoor unit's fan blades unchanged.

[0150] Combined with Figure 7 Assuming there are two people indoors, and the calculations above determine that they are in zone 0 and zone 4 respectively, since zone 0 corresponds to air blowing to the upper left and zone 4 corresponds to air blowing to the center, it is impossible to blow air to each person indoors in this case. Therefore, the direction of the fan blades of the indoor unit of air conditioner 20 remains unchanged.

[0151] In order to perform the corresponding steps in the above embodiments and various possible implementations, an implementation method of an air conditioner indoor unit fan blade control device is given below.

[0152] Please refer to Figure 10This is a functional module diagram of the air conditioner indoor unit fan blade control device 100 provided by the present invention. It should be noted that the basic principle and technical effects of the air conditioner indoor unit fan blade control device 100 described in this embodiment are the same as those in the aforementioned method embodiments. For the sake of brevity, parts not mentioned in this embodiment can be referred to the corresponding content of the aforementioned method embodiments. This air conditioner indoor unit fan blade control device 100 is applied to the edge device 10, and the following is a detailed description in conjunction with... Figure 10 The air conditioner indoor unit fan blade control device 100 is described below. The air conditioner indoor unit fan blade control device 100 includes: an image acquisition module 101, a target detection module 102, and a rotation direction adjustment module 103.

[0153] The image acquisition module 101 is used to acquire images of the indoor environment where the air conditioner is located, including images of people inside the room.

[0154] The target detection module 102 is used to detect indoor people in an indoor environment image using a pre-trained target detection model, and obtain target detection boxes that represent the location of indoor people in the indoor environment image.

[0155] The rotation direction adjustment module 103 is used to adjust the rotation direction of the fan blades of the indoor air conditioner according to the target detection box.

[0156] Optionally, the object detection model includes a backbone network, a neck network, and a head network; the object detection module 102 is specifically used for:

[0157] The input image is obtained based on the indoor environment image;

[0158] The input image is fed into the backbone network, which extracts features from the input image and outputs two feature maps of different scales: the first feature map and the second feature map.

[0159] The first feature map and the second feature map are input into the neck network. The neck network performs feature enhancement extraction on the first feature map and the second feature map based on the attention mechanism to obtain the first enhanced feature map and the second enhanced feature map.

[0160] The first and second enhanced feature maps are input into the head network for image regression and classification, resulting in two target prediction boxes at different scales.

[0161] Optionally, the backbone network includes CBR units, max pooling layers, and three feature extraction units at different scales; the object detection module 102 performs feature extraction on the input image using the backbone network, outputting a first feature map and a second feature map at two different scales, including:

[0162] The input image is processed by convolution, batch normalization, and ReLU activation function using CBR units to obtain the first input feature map, the size of which is 1 / 2 of the input image.

[0163] The first input feature map is max-pooled using a max-pooling layer to obtain the second input feature map, the size of which is 1 / 2 of the first input feature map.

[0164] The first feature extraction unit is used to perform first-level feature extraction on the second input feature map to obtain the third input feature map, the size of which is 1 / 2 of the second input feature map;

[0165] The second feature extraction unit is used to perform second-level feature extraction on the third input feature map to obtain the first feature map, the size of which is 1 / 2 of the third input feature map;

[0166] The third feature extraction unit is used to perform third-level feature extraction on the first feature map to obtain the second feature map, the size of which is 1 / 2 of the size of the first feature map.

[0167] Optionally, the first feature extraction unit includes three ShuffleNetV2 basic units and one basic unit with spatial downsampling; the second feature extraction unit includes seven ShuffleNetV2 basic units and one basic unit with spatial downsampling; and the third feature extraction unit includes three ShuffleNetV2 basic units, one basic unit with spatial downsampling, and one spatial pyramid pooling layer.

[0168] Optionally, the neck network includes three SimAM attention units, a CBR unit, a depthwise separable convolutional layer, an upsampling layer, and a concatenation layer; the object detection module 102 executes the neck network to perform feature enhancement extraction on the first and second feature maps based on the attention mechanism, obtaining the first and second enhanced feature maps in the following ways:

[0169] The first SimAM attention unit is used to process the second feature map using the SimAM attention mechanism, resulting in a first output feature map. The size of the first output feature map is the same as that of the second feature map. Figure 1 To;

[0170] The first output feature map is processed by convolution, batch normalization, and ReLU activation function using a CBR unit to obtain the first enhanced feature map. The size of the first enhanced feature map is the same as that of the first output feature map. Figure 1 To;

[0171] The first enhanced feature map is convolved using a depthwise separable convolutional layer to obtain a second output feature map. The size of the second output feature map is the same as that of the first enhanced feature map. Figure 1 To;

[0172] The second output feature map is upsampled using an upsampling layer to obtain a third output feature map, the size of which is twice that of the second output feature map.

[0173] The third output feature map is processed using the SimAM attention mechanism by the second SimAM attention unit to obtain the fourth output feature map. The size of the fourth output feature map is the same as that of the third output feature map. Figure 1 To;

[0174] The fourth output feature map and the first feature map are concatenated using a concatenation layer to obtain a second enhanced feature map. The size of the second enhanced feature map is the same as that of the first feature map. Figure 1 To.

[0175] Optionally, the rotation direction adjustment module 103 is specifically used for:

[0176] Calculate the center coordinates of the target detection box based on the vertex coordinates;

[0177] Obtain a pre-established reference image, which is the same size as the indoor environment image and is divided into m*n regions. Each region has a set control signal, where m represents the number of rows in the region and n represents the number of columns in the region. Both m and n are integers greater than or equal to 1.

[0178] Based on the reference map, the target area with the center coordinates is determined from m*n regions;

[0179] The control signal corresponding to the target area is sent to the air conditioner so that the air conditioner can adjust the rotation direction of the fan blades of its indoor unit according to the control signal.

[0180] Optionally, there are multiple people indoors and multiple target detection boxes, with one target detection box corresponding to one person indoors; the rotation direction adjustment module 103 is also specifically used for:

[0181] Calculate the center coordinates of each object detection box based on the vertex coordinates of each object detection box;

[0182] Obtain a pre-established reference image, which is the same size as the indoor environment image and is divided into m*n regions. Each region has a set control signal, where m represents the number of rows in the region and n represents the number of columns in the region. Both m and n are integers greater than or equal to 1.

[0183] Based on the reference map, the target area where each center coordinate is located is determined from m*n regions;

[0184] If all the center coordinates are located in the same target area, the control signal corresponding to the target area will be sent to the air conditioner so that the air conditioner can adjust the direction of the fan blades of its indoor unit according to the control signal.

[0185] If there are multiple target areas where all center coordinates are located, then keep the direction of the air conditioner's indoor unit's fan blades unchanged.

[0186] Please refer to Figure 11 , Figure 11 This is a block diagram of an edge device 10 provided by the present invention. The edge device 10 includes a processor 11, a memory 12, and a bus 13. The processor 11 is connected to the memory 12 via the bus 13.

[0187] The memory 12 is used to store programs. After receiving an execution instruction, the processor 11 executes the programs to implement the air conditioner indoor unit fan blade control method disclosed in the above embodiments.

[0188] The memory 12 may include high-speed random access memory (RAM) or non-volatile memory (NVM).

[0189] Processor 11 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of processor 11 or through software instructions. Processor 11 can be a general-purpose processor, including a Central Processing Unit (CPU), a Microcontroller Unit (MCU), a Complex Programmable Logic Device (CPLD), a Field Programmable Gate Array (FPGA), embedded ARM chips, etc.

[0190] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by processor 11, implements the air conditioner indoor unit fan blade control method disclosed in the above embodiments.

[0191] In summary, the present invention provides an air conditioner indoor unit fan blade control method, device, edge device, and storage medium that applies computer vision algorithms to air conditioner indoor unit fan blade control. By acquiring an image of the indoor environment where the air conditioner is located, a pre-trained target detection model is used to detect people in the indoor environment image to obtain target detection boxes representing the positions of people in the indoor environment image. The rotation direction of the air conditioner indoor unit fan blades is adjusted according to the target detection boxes, thereby automatically adjusting the airflow direction of the air conditioner according to the actual position of people in the room, so that people can feel a noticeable breeze and improve the user experience.

[0192] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A method for controlling the fan blades of an indoor air conditioning unit, characterized in that, Applied to an edge device (10), the edge device (10) is communicatively connected to an air conditioner (20), and the air conditioner indoor unit fan blade control method includes: Acquire an image of the indoor environment where the air conditioner (20) is located, the image of the indoor environment including people in the room; The indoor environment image is used to detect indoor people using a pre-trained target detection model, and target detection boxes representing the positions of indoor people in the indoor environment image are obtained. According to the target detection box, adjust the rotation direction of the fan blades of the indoor unit of the air conditioner (20); The target detection model includes a backbone network, a neck network, and a head network; The step of using a pre-trained target detection model to detect indoor occupants in the indoor environment image and obtaining target detection boxes representing the positions of indoor occupants in the indoor environment image includes: Based on the indoor environment image, the input image is obtained; The input image is input into the backbone network, and the backbone network is used to extract features from the input image, outputting two feature maps of different scales: a first feature map and a second feature map. The first feature map and the second feature map are input into the neck network, and the neck network performs feature enhancement extraction on the first feature map and the second feature map based on an attention mechanism to obtain a first enhanced feature map and a second enhanced feature map. The first and second enhanced feature maps are input into the head network for image regression and classification to obtain two target prediction boxes of different scales. The two target prediction boxes at different scales are post-processed to obtain the target detection boxes; The neck network includes three SimAM attention units, a CBR unit, a depthwise separable convolutional layer, an upsampling layer, and a splicing layer. The step of the neck network performing feature enhancement extraction on the first feature map and the second feature map based on an attention mechanism to obtain the first enhanced feature map and the second enhanced feature map includes: The second feature map is processed by the SimAM attention mechanism using the first SimAM attention unit to obtain the first output feature map, the size of which is the same as that of the second feature map. The first output feature map is processed by convolution, batch normalization and ReLU activation function using the CBR unit to obtain the first enhanced feature map, and the size of the first enhanced feature map is the same as that of the first output feature map. The first enhanced feature map is convolved using the depthwise separable convolutional layer to obtain a second output feature map, the size of which is the same as that of the first enhanced feature map. The second output feature map is upsampled using the upsampling layer to obtain a third output feature map, the size of which is twice that of the second output feature map; The third output feature map is processed by the SimAM attention mechanism using the second SimAM attention unit to obtain a fourth output feature map, the size of which is the same as that of the third output feature map. The fourth output feature map and the first feature map are spliced ​​together using the splicing layer to obtain the second enhanced feature map, the size of which is the same as that of the first feature map. 2.The method of claim 1, wherein, The backbone network includes CBR units, max pooling layers, and feature extraction units of three different scales. The step of extracting features from the input image using the backbone network and outputting two feature maps of different scales, namely a first feature map and a second feature map, includes: The input image is processed by convolution, batch normalization, and ReLU activation function using the CBR unit to obtain a first input feature map, the size of which is 1 / 2 of the input image. The first input feature map is subjected to max pooling using the max pooling layer to obtain a second input feature map, the size of which is 1 / 2 of the first input feature map; The first feature extraction unit is used to perform first-level feature extraction on the second input feature map to obtain a third input feature map, the size of which is 1 / 2 of the second input feature map; The second feature extraction unit is used to perform second-level feature extraction on the third input feature map to obtain the first feature map, the size of which is 1 / 2 of the third input feature map; The third feature extraction unit is used to perform third-level feature extraction on the first feature map to obtain the second feature map, the size of which is 1 / 2 of the size of the first feature map. 3.The method of claim 2, wherein, The first feature extraction unit includes three ShuffleNetV2 basic units and one basic unit with spatial downsampling; the second feature extraction unit includes seven ShuffleNetV2 basic units and one basic unit with spatial downsampling; the third feature extraction unit includes three ShuffleNetV2 basic units, one basic unit with spatial downsampling, and one spatial pyramid pooling layer. 4.The method of claim 1, wherein, The step of adjusting the rotation direction of the fan blades of the indoor unit of the air conditioner (20) according to the target detection box includes: Calculate the center coordinates of the target detection box based on the vertex coordinates of the target detection box; Obtain a pre-established reference image, wherein the size of the reference image is consistent with that of the indoor environment image and is divided into m*n regions, each region has a set control signal, m represents the number of rows of the region, n represents the number of columns of the region, and m and n are both integers greater than or equal to 1; Based on the reference map, the target area where the center coordinates are located is determined from m*n regions; The control signal corresponding to the target area is sent to the air conditioner (20) so that the air conditioner (20) adjusts the rotation direction of the fan blades of its indoor unit according to the control signal. 5.The method of claim 1, wherein, There are multiple indoor occupants and multiple target detection boxes, with one target detection box corresponding to one indoor occupant; The step of adjusting the rotation direction of the fan blades of the indoor unit of the air conditioner (20) according to the target detection box includes: Calculate the center coordinates of each target detection box based on the vertex coordinates of each target detection box; Obtain a pre-established reference image, wherein the reference image is the same size as the indoor environment image and is divided into m*n regions, each region has a set control signal, m represents the number of rows of the region, n represents the number of columns of the region, and m and n are both integers greater than or equal to 1; Based on the reference map, the target area where each of the center coordinates is located is determined from m*n regions; If all the center coordinates are located in the same target area, then the control signal corresponding to the target area is sent to the air conditioner (20) so that the air conditioner (20) adjusts the rotation direction of the fan blades of its indoor unit according to the control signal; If there are multiple target areas where all center coordinates are located, then the direction of the fan blade rotation of the indoor unit of the air conditioner (20) remains unchanged.

6. An air conditioner indoor unit fan blade control device, characterized by, Applied to an edge device (10), the edge device (10) is communicatively connected to an air conditioner (20), and the air conditioner indoor unit fan blade control device (100) includes: The image acquisition module (101) is used to acquire an image of the indoor environment where the air conditioner (20) is located, the image of the indoor environment including people in the room; The target detection module (102) is used to detect indoor people in the indoor environment image using a pre-trained target detection model, and obtain a target detection box representing the position of indoor people in the indoor environment image; The rotation direction adjustment module (103) is used to adjust the rotation direction of the fan blades of the indoor unit of the air conditioner (20) according to the target detection box; The target detection model includes a backbone network, a neck network, and a head network. The target detection module (102) is specifically used for: obtaining an input image based on the indoor environment image; inputting the input image into the backbone network, using the backbone network to extract features from the input image, and outputting two feature maps of different scales: a first feature map and a second feature map; inputting the first feature map and the second feature map into the neck network, where the neck network performs feature enhancement extraction based on an attention mechanism, obtaining a first enhanced feature map and a second enhanced feature map; inputting the first enhanced feature map and the second enhanced feature map into the head network for image regression and classification, obtaining two target prediction boxes of different scales; and post-processing the two target prediction boxes of different scales to obtain the target detection box. The neck network includes three SimAM attention units, a CBR unit, a depthwise separable convolutional layer, an upsampling layer, and a concatenation layer. The target detection module (102) executes the neck network to perform feature enhancement extraction on the first feature map and the second feature map based on the attention mechanism to obtain a first enhanced feature map and a second enhanced feature map. This includes: using the first SimAM attention unit to process the second feature map using the SimAM attention mechanism to obtain a first output feature map, the size of which is the same as that of the second feature map; and using the CBR unit to process the first output feature map using convolution, batch normalization, and ReLU activation function to obtain the first enhanced feature map, the size of which is the same as that of the first output feature map. The first enhanced feature map is convolved using the depthwise separable convolutional layer to obtain a second output feature map, the size of which is the same as that of the first enhanced feature map. The second output feature map is then upsampled using the upsampling layer to obtain a third output feature map, the size of which is twice that of the second output feature map. The third output feature map is then processed using a second SimAM attention unit to obtain a fourth output feature map, the size of which is the same as that of the third output feature map. Finally, the fourth output feature map and the first feature map are concatenated using the splicing layer to obtain a second enhanced feature map, the size of which is the same as that of the first feature map.

7. An edge device, characterized by It includes a processor (11) and a memory (12), the memory (12) being used to store a program, and the processor (11) being used to implement the air conditioner indoor unit fan blade control method according to any one of claims 1-5 when executing the program.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the processor (11), implements the air conditioning indoor unit fan blade control method as described in any one of claims 1-5.

Images