Image denoising method and device, air conditioner and storage medium

Abstract

The application relates to an image denoising method and device, an air conditioner and a storage medium. The method comprises the following steps: inputting a noisy image sample into a generator, extracting human behavior features in the noisy image sample through a behavior feature representation network, and obtaining a denoised image output by the generator; inputting samples of the denoised image and preset training samples into an identifier, wherein the training samples comprise noisy image samples and non-noisy image samples; calculating an error between an identification result and a true label of the noisy image sample through the identifier, optimizing network parameters of a generative adversarial network by using a back propagation algorithm, obtaining a trained generator, and taking the trained generator as a denoising model; and inputting a target noisy image into the denoising model to obtain a target denoised image. The application improves the accuracy of personnel detection results.

Description

Technical Field

[0001] This application relates to the field of neural networks, and more particularly to an image denoising method, apparatus, air conditioner, and storage medium. Background Technology

[0002] Human-detecting radar (millimeter-wave, ultra-wideband, and other Doppler radars) determines the presence of people in a scene by detecting periodic bodily movements (chest vibrations caused by breathing and heart rate). Specifically, when detecting vital signs, human breathing causes the chest cavity to rise and fall, which affects the radar echo pulses. By continuously receiving periodic echo pulses, the human body can be detected. However, the echo signal is often affected by adverse external factors, leading to inaccurate detection results.

[0003] There is currently no good solution to the problem that radar detection is easily affected by external interference, leading to inaccurate detection results. Summary of the Invention

[0004] This application provides an image denoising method, apparatus, air conditioner, and storage medium to solve the problem of inaccurate detection results.

[0005] In a first aspect, this application provides an image denoising method, the method comprising: inputting a noisy image sample into a generator, extracting human behavior features from the noisy image sample through a behavioral feature representation network, and obtaining a denoised image output by the generator; inputting a sample of the denoised image and a preset training sample into a recognizer, wherein the training sample includes noisy image samples and uncluttered image samples; calculating the error between the recognition result and the true label of the noisy image sample through the recognizer, optimizing the network parameters of the generative adversarial network using a backpropagation algorithm, obtaining a trained generator, and using the trained generator as a denoising model; and inputting a target noisy image into the denoising model to obtain a target denoised image.

[0006] Optionally, inputting the target noisy image into the denoising model to obtain the target denoised image includes: inputting the target noisy image into the denoising model; during downsampling, reducing the size of the feature map by adjusting the convolution stride to obtain a first feature map; during deep feature processing, fusing the first feature map and the second feature map after two convolutions through dense connection units; during upsampling, filling the feature map of the fused feature map with a preset resolution using 0 elements, and then performing convolution to obtain the target denoised image.

[0007] Optionally, the step of reducing the size of the feature map by adjusting the convolution stride during downsampling to obtain the first feature map includes: during downsampling, using a 7×7 convolutional layer to extract contextual information within the receptive field of the target noisy image; sequentially inputting the input of the 7×7 convolutional layer into two 4×4 convolutional layers to obtain the first feature map, wherein the size of the feature map output by the 4×4 convolutional layer is reduced by half compared to the size of the feature map output by the 7×7 convolutional layer.

[0008] Optionally, in the deep feature processing, fusing the first feature map and the second feature map after two convolutions using densely connected units includes: in the deep feature processing, convolving the first feature map through a 3×3 convolutional layer and a 1×1 convolutional layer to obtain the second feature map, wherein the 3×3 convolutional layer performs 1-pixel edge padding, the 1×1 convolutional layer has a stride of 1 and no edge padding, and the 1×1 convolutional layer is used to perform nonlinear transformation on the features output by the 3×3 convolutional layer; and fusing the first feature map and the second feature map after two convolutions using a channel concatenation method.

[0009] Optionally, four transition layers are provided between each of the densely connected units, each of the transition layers being a 1×1 convolutional layer with a convolution stride of 1 and no edge padding.

[0010] Optionally, during the upsampling process, filling the fused feature map with zero elements at a preset resolution and then performing convolution to obtain the target denoised image includes: in the transposed convolution, filling the fused feature map with zero elements at a preset resolution; performing convolution operations on the filled fused feature map using the first two 4×4 convolutional layers to obtain the feature map of the original size, wherein the stride of the first two 4×4 convolutional layers is 2 and an edge padding of 1 pixel is set; performing convolution operations on the feature map of the original size using the last 7×7 convolutional layer, wherein the number of output channels of the last convolutional layer is 3; and normalizing the output of the last convolutional layer using the hyperbolic tangent function to obtain the target denoised image.

[0011] Optionally, after obtaining the target denoised image, the method further includes: if it is continuously determined that there are people or no people in the target denoised image, the count is incremented by one for each determination; if the number of consecutive counts does not reach the target preset number, the preset time is extended until the number of consecutive counts is greater than the target preset number, and then the air conditioner is turned on or off according to the air conditioner's on / off status, wherein the preset time can be manually adjusted.

[0012] Optionally, if the number of consecutive counts does not reach the target preset number, the preset duration is extended until the number of consecutive counts exceeds the target preset number. Then, the air conditioner is turned on or off based on its status: if someone is present, the preset duration is extended if the number of consecutive counts does not reach the first preset number; if the number of consecutive counts reaches the first preset number, the air conditioner is turned on if it is not turned on.

[0013] Optionally, if the number of consecutive counts does not reach the target preset number, the preset duration is extended until the number of consecutive counts exceeds the target preset number. Then, the air conditioner is turned on or off based on its status: if no one is present, the preset duration is extended if the number of consecutive counts does not reach the second preset number; if the number of consecutive counts reaches the second preset number, the air conditioner is turned off if it is turned on.

[0014] Secondly, an image denoising device is provided, comprising: an extraction module for inputting noisy image samples into a generator, extracting human behavioral features from the noisy image samples through a behavioral feature representation network, and obtaining a denoised image output by the generator; an input module for inputting samples of the denoised image and preset training samples into a recognizer, wherein the training samples include noisy image samples and noisy image samples; a generation module for calculating the error between the recognition result and the true label of the noisy image sample through the recognizer, optimizing the network parameters of the generative adversarial network using a backpropagation algorithm, obtaining a trained generator, and using the trained generator as a denoising model; and an obtaining module for inputting a target noisy image into the denoising model to obtain a target denoised image.

[0015] Thirdly, this application provides an air conditioner, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus.

[0016] Fourthly, this application also provides a computer storage medium storing computer-executable instructions for performing the image denoising method described in any of the preceding claims of this application.

[0017] Compared with the prior art, the above-mentioned technical solution provided in this application has the following advantages: This application trains the generator through a generative adversarial network and incorporates the extraction of human behavioral features during the training process. The trained generator can perform denoising while retaining the original motion information of the human body. Even if the image detected by radar is affected by external interference, it can still be denoised by the trained denoising model, thereby improving the accuracy of personnel detection results. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0021] Figure 1 A flowchart of an image denoising method provided in this application embodiment;

[0022] Figure 2 A schematic diagram of the overall structure of the GAN model provided in the embodiments of this application;

[0023] Figure 3 A schematic diagram of deep learning for the model provided in the embodiments of this application;

[0024] Figure 4 This is a schematic diagram of radar echo signal denoising provided in an embodiment of this application;

[0025] Figure 5 A schematic diagram of a densely connected unit provided in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of the pooling operation based on zero-element filling provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the transposed convolution operation provided in the embodiments of this application;

[0028] Figure 8 This is a schematic diagram of the automatic control process for air conditioning provided in an embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the overall process of image denoising provided in an embodiment of this application;

[0030] Figure 10 This is a schematic diagram of the structure of an image denoising device provided in an embodiment of this application;

[0031] Figure 11 This is a schematic diagram of the structure of an air conditioner provided in an embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0034] This application provides an image denoising method, such as Figure 1 As shown, the method includes:

[0035] Step 101: Input the noisy image sample into the generator, extract the human behavior features in the noisy image sample through the behavior feature representation network, and obtain the denoised image output by the generator.

[0036] The overall structure of the GAN model is as follows: Figure 2 As shown, the training process of GAN is a process in which G (generator) gradually fits the denoised mapping, and G (generator) is the most important part of GAN. During training, G attempts to generate realistic denoised images that D (decision processor) cannot recognize. D optimizes its own parameters based on the error of the recognition result, and the recognition result is fed back to G for training. G and D continuously train in a game of mutual competition, gradually improving their performance. The behavioral feature representation network H is responsible for extracting human behavioral features from radar images. It uses the error of the denoising result on human behavioral features to supervise the training of G, so that the output of G not only achieves pixel-level similarity with the corresponding noiseless image, but also retains the original motion information.

[0037] The dataset used to train the denoising model typically needs to contain both noise-free and noisy samples. The following is a basic data processing flow for preparing the dataset.

[0038] Data Acquisition: Based on relevant databases and radar modules, simulations and human detection are performed to construct simulated and measured radar noise image datasets. Human echo signals can be acquired using radar equipment in interference-free environments. These signals should be raw, noise-free signals, and relevant environmental and radar parameters need to be recorded during acquisition.

[0039] Adding noise: Based on the collected noise-free signal, noise is introduced one by one and the corresponding noisy signals are collected. Interference in the real environment can be simulated by placing noise sources (such as cats, dogs, moving curtains, fans, wind disturbances, etc.) within the radar's radiation range. Ensure that each noise source is introduced independently to avoid mutual interference.

[0040] Data labeling: The collected noisy signals are labeled and paired with corresponding noise-free signals. Labeling can be done using file names, data tables, or metadata to ensure the correspondence between noise-free and noisy data.

[0041] Standardization: Standardizing a signal makes its mean 0 and its variance 1. The standardization formula is:

[0042] I_normalized=(I-mean(I)) / std(I)

[0043] Where I_normalized is the standardized signal data, I is the value of the original data, mean(I) is the mean of the original data, and std(I) is the standard deviation of the original data.

[0044] The noisy image sample with added noise is input into the generator. During the denoising process, the generator extracts human behavior features from the noisy image sample through its internal behavioral feature representation network. The generator then suppresses the noise components in the noisy image sample to obtain the denoised image output by the generator.

[0045] Step 102: Input the denoised image samples and the preset training samples into the recognizer.

[0046] The training samples include noisy image samples and noisy image samples.

[0047] The training samples include noisy image samples and noiseless image samples. The noisy image samples introduce noise into a noiseless signal, while the noiseless image samples contain no noise. The server inputs both the denoised image samples and the preset training samples into the recognizer to train it. Both the training samples and the denoised image samples include labels that describe whether the image is noisy or noiseless.

[0048] Step 103: Calculate the error between the recognition result and the real label of the noisy image sample using the recognizer, optimize the network parameters of the generative adversarial network using the backpropagation algorithm, obtain the trained generator, and use the trained generator as the denoising model.

[0049] Specifically, the denoising model trains the generator from three levels: pixel distribution, semantic discrimination, and behavioral features.

[0050] If the recognition result output by the recognizer is noisy, then based on the error between the noisy result and the true label of the noisy image sample, the backpropagation algorithm is used to optimize the network parameters of the generative adversarial network, resulting in a trained generator. Simultaneously, the recognizer continuously trains to improve recognition accuracy. However, in the GAN structure, the recognizer and the behavioral feature representation network are only responsible for supervising the generator's training. Once the model training converges, the generator is extracted separately as a denoising model, while the recognizer and behavioral feature representation network no longer play a role. The trained generator can learn the mapping relationship between noisy and noiseless images, and thus serves as the denoising model.

[0051] Figure 3 This is a schematic diagram of the deep learning model.

[0052] Step 104: Input the noisy target image into the denoising model to obtain the denoised target image.

[0053] The server inputs the target noisy image into the denoising model and obtains the target denoised image output by the denoising model.

[0054] This application trains a generator using a generative adversarial network and incorporates the extraction of human behavioral features during the training process. The trained generator can denoise while retaining the original motion information of the human body. Even if the image detected by radar is affected by external interference, it can still be denoised by the trained denoising model, thereby improving the accuracy of personnel detection results.

[0055] Figure 4 The diagram illustrates the denoising of radar echo signals. It can be seen that after denoising by the CNN denoising model, most of the noise signal has been removed.

[0056] As an optional implementation, inputting the target noisy image into a denoising model to obtain the target denoised image includes: inputting the target noisy image into the denoising model; during downsampling, reducing the size of the feature map by adjusting the convolution stride to obtain a first feature map; during deep feature processing, fusing the first feature map and the second feature map after two convolutions through dense connection units; during upsampling, filling the feature map of the fused feature map with a preset resolution using 0 elements, and then performing convolution to obtain the target denoised image.

[0057] The generative network is built using a CNN and mainly consists of three parts: a downsampling part, a deep feature processing part, and an upsampling part. The generative network uses the CNN to capture information from the local receptive fields of the input image and reconstructs the micro-Doppler signal based on these local features. Simultaneously, leveraging the weight-sharing characteristic of CNNs, the convolutional layers capturing local features can form effective denoising maps in different regions. Through learning by the convolutional layers, the denoising maps can match the global noise distribution, enabling the CNN to remove noise of similar intensity.

[0058] Specifically, when a noisy target image is input into a denoising model, the downsampling part is located in a shallow layer of the CNN during the downsampling process, which is prone to losing some details. Therefore, the size of the feature map can be reduced by adjusting the convolution stride to reduce the loss of details during the downsampling process and obtain the first feature map.

[0059] The core structure of the deep feature processing section is the DenseBlock (DB). Both the input and output of a DB are feature maps, and the two feature maps are connected by two convolutional layers. The DB fuses the first feature map with the second feature map (after two convolutions) to obtain a fused feature map, which serves as the output of the DB layer. This cross-layer connection design makes the information feedforward in the entire network denser, improving the utilization of deep features.

[0060] Common upsampling methods include un-pooling, bilinear interpolation, and transposed convolution. In un-pooling and transposed convolution, this application pads the low-resolution feature map in the fused feature map with zero elements before convolution to achieve upsampling. This operation is equivalent to rearranging the input feature map and the convolutional layer, padding with zero elements, and then performing matrix multiplication to achieve the inverse operation of convolution, which can improve the generalization ability of the model.

[0061] Common CNN architectures compress images to extremely low-resolution feature maps. However, radar image denoising should rely on the signal within the noisy image to suppress noise. While downsampling can highly abstract semantic features, it destroys neighborhood information and loses details in the radar image. Therefore, RnGAN makes a trade-off between downsampling and radar image denoising. It reduces the size of the feature map by adjusting the convolution stride, while focusing on processing deep features. It fuses the input and output feature maps of densely connected units, improving the utilization of deep features. It suppresses noise by identifying human motion information in the signal. Finally, it pads the low-resolution feature map with zeros before convolution, achieving upsampling and improving the model's generalization ability.

[0062] The following sections will provide detailed descriptions of the downsampling, deep feature processing, and upsampling parts.

[0063] As an optional implementation, during the downsampling process, the size of the feature map is reduced by adjusting the convolution stride to obtain the first feature map, which includes: during the downsampling process, a 7×7 convolutional layer is used to extract contextual information within the receptive field of the target noisy image; the input of the 7×7 convolutional layer is sequentially input into two 4×4 convolutional layers to obtain the first feature map, wherein the size of the feature map output by the 4×4 convolutional layer is reduced by half compared to the size of the feature map output by the 7×7 convolutional layer.

[0064] The downsampling part is located in the shallow layers of the CNN, mainly learning general local texture features, which primarily reflect information such as signal strength and boundaries. Under low signal-to-noise ratio conditions, the noise intensity is close to the human body echo signal intensity, and motion signals in local areas may be completely covered by noise. In order to fully respond to motion signals, it is necessary to appropriately expand the receptive field and use larger convolutional layers to fully extract the contextual information within the receptive field. Therefore, the first convolutional layer uses a 7×7 convolutional layer.

[0065] The second and third layers of the downsampling part are both designed as 4×4 convolutional layers. The feature map size output by the 4×4 convolution is reduced by half compared to the feature map size output by the 7×7 convolutional layer. The two 4×4 convolutional layers output a feature map of size 30×30. By adjusting the convolution stride, the model reduces the size of the feature map, thereby reducing the loss of detail during the downsampling process.

[0066] Each convolutional layer is followed by Batch Normalization (BN) and an activation function. BN normalizes the features within a batch by subtracting their mean and then dividing by their variance. As the number of network layers increases, the distribution of features gradually shifts. Periodically applying BN in the network can normalize the features to an approximate standard normal distribution, ensuring that the gradient remains within a reasonable range during backpropagation, effectively avoiding gradient vanishing and gradient exploding. This paper selects the ReLU function as the activation function in CNNs to perform non-linear activation on the features.

[0067] As an optional implementation, in the deep feature processing, fusing the first feature map and the second feature map after two convolutions through densely connected units includes: in the deep feature processing, the first feature map is convolved through a 3×3 convolutional layer and a 1×1 convolutional layer to obtain the second feature map, wherein the 3×3 convolutional layer performs 1-pixel edge padding, the 1×1 convolutional layer has a stride of 1 and no edge padding, and the 1×1 convolutional layer is used to perform nonlinear transformation on the features output by the 3×3 convolutional layer; the first feature map and the second feature map after two convolutions are fused using a channel concatenation method.

[0068] After downsampling, a shallow feature map is obtained. To extract and process deep behavioral features from the shallow feature map, this application designs a deep feature processing section in the center of the CNN to extract and process deep semantic features while suppressing noise components in the image. The core structure of the deep feature processing section is a densely connected unit (DB), as shown in Figure 5.

[0069] Figure 5 This diagram illustrates a densely connected unit (DCU), where both input and output are feature maps. Two feature maps are connected sequentially by a 3×3 convolutional layer and a 1×1 convolutional layer. The first rectangle in the diagram represents a 3×3 convolutional layer with a stride of 1. Odd-sized convolutional layers have a definite center point and better symmetry, making them more suitable for processing deep features. Furthermore, in the deeper parts of the network, smaller convolutions outperform larger ones because while larger convolutions widen the receptive field, they introduce more parameters, significantly increasing model complexity and computation. In contrast, using more small convolutional layers can achieve the same receptive field with fewer parameters. A 3×3 convolution is the smallest odd-sized convolutional layer with the largest receptive field; therefore, the DB structure employs multiple 3×3 convolutions to achieve better fitting performance. The 3×3 convolutional layers in DB implement 1-pixel edge padding to ensure that the size of the convolutional feature maps remains unchanged, thus preserving the neighborhood structure of the features and retaining more detailed information. Figure 5 The second rectangle in the diagram represents a 1×1 convolutional layer with a stride of 1 and no edge padding. The purpose of the 1×1 convolutional layer is to perform a non-linear transformation on the features output by the 3×3 convolutional layer, thereby improving the network's resistance to overfitting.

[0070] In this process, the output of each convolutional layer is normalized by Batch Normalization (BN) and then activated by a ReLU layer.

[0071] The feature map size remains unchanged after two convolution operations. Then, the first feature map, which is the input, and the second feature map, which is the result of two convolutions, are fused together as the output of this layer's DB. This cross-layer connection design makes the information feedforward in the entire network denser, improves the utilization of deep features, and realizes the reuse of multi-scale receptive field features.

[0072] Furthermore, in DB training, gradients can be backpropagated across different areas, making the training of the entire model more efficient.

[0073] Regarding feature fusion methods, common fusion strategies include pointwise addition, pointwise multiplication, and channel concatenation. This application selects channel concatenation as the feature fusion method to retain deep features to the maximum extent. As can be seen from the DB operation, as the number of layers increases, more and more features are concatenated along the channels. To reduce the computational load of the model and further reduce the risk of overfitting, four transition layers are set between densely connected units. These transition layers are 1×1 convolutional layers with a stride of 1 and no edge padding. Their function is to compress the number of channels in the feature map. For example, the deep feature processing part outputs a (batch×256×30×30) dimensional feature tensor, which contains feature maps with 256 channels.

[0074] As an optional implementation, during the upsampling process, filling the fused feature map with zero elements at a preset resolution and then performing convolution to obtain the target denoised image includes: in the transposed convolution, filling the fused feature map with zero elements at a preset resolution; performing convolution operations on the filled fused feature map using the first two 4×4 convolutional layers to obtain the feature map of the original size, wherein the stride of the first two 4×4 convolutional layers is 2 and an edge padding of 1 pixel is set; performing convolution operations on the feature map of the original size using the last 7×7 convolutional layer, wherein the number of output channels of the last convolutional layer is 3; and normalizing the output of the last convolutional layer using the hyperbolic tangent function to obtain the target denoised image.

[0075] Commonly used upsampling methods include: un-pooling, bilinear interpolation, or transposed convolution.

[0076] Bilinear interpolation is often used for super-resolution restoration of visual images. The principle is to perform two bilinear interpolations in two directions in the matrix, determine the weights by the distance between the interpolation point and the four neighboring elements, and calculate the weighted average of the four neighboring elements as the element value of the interpolation point.

[0077] Upper pooling often occurs in pairs with pooling. It restores the position of elements in a high-resolution feature map and fills other elements with zero elements or its own elements. Upper pooling based on zero-element filling is as follows: Figure 6 As shown.

[0078] Transposed convolution is the inverse operation of convolution. It achieves upsampling by padding the low-resolution feature map with zeros and then convolving it with the fused feature map. This operation is equivalent to rearranging the input feature map and the convolutional layer, padding with zeros, and then performing matrix multiplication, thus achieving the inverse operation of convolution. The operation of transposed convolution is as follows: Figure 7 As shown.

[0079] The first two layers of the upsampling part are 4×4 transposed convolutional layers with a stride of 2 and 1-pixel edge padding. When the stride of the transposed convolution cannot be divided evenly by the convolutional layer size, a large number of convolutional overlap regions will appear, resulting in a grid-like pixel value distribution in the output image, known as the "chessboard effect". The model in this application sets the convolutional layer size in the transposed convolution to 4×4 and the convolutional stride to 2 to avoid overlapping regions in the transposed convolution. After two layers of transposed convolution, the feature map of the original size is obtained. The last layer of the network is set to a 7×7 convolutional layer, which performs convolution processing again within a large receptive field. The number of output channels of the last convolutional layer is set to 3 to achieve channel compression of high-dimensional features. Then, the output is normalized by a hyperbolic tangent function to obtain the final output of the generator network, i.e., the denoised image.

[0080] The trained denoising model is applied to the actual radar echo signal enhancement task, and the denoising model is encapsulated into the radar human sensing module.

[0081] Furthermore, currently, air conditioners require users to manually decide whether to turn them on and off. For example, if a user stays indoors for an extended period, the air conditioner needs to be turned on; if the user only enters the room briefly, it doesn't need to be turned on. Similarly, if a user leaves the room for an extended period, they need to manually turn off the air conditioner; if the user only leaves briefly, it doesn't need to be turned off. Currently, users need to manually turn the air conditioner on or off for both periods of time, resulting in a low level of intelligence in air conditioner control.

[0082] This application proposes a solution to this problem. Specifically, after obtaining the target denoised image, if it is determined that there are people or no people in the target denoised image, the count is incremented by one for each determination. If the number of consecutive counts does not reach the target preset number, the preset time is extended until the number of consecutive counts exceeds the target preset number. Then, the air conditioner is turned on or off according to the air conditioner's on / off status. The preset time can be manually adjusted.

[0083] Figure 8 This is a schematic diagram of the automatic control process for air conditioning.

[0084] If someone is detected consecutively, the timer count M increments by one, where M is a positive integer or zero. Each consecutive count triggers a custom delay module, which is used for a delay period that the user can adjust to their preset duration.

[0085] If the number of consecutive counts M does not reach the first preset number, it indicates that the number of counts is too low and the waiting time has not yet been reached. The person may be staying in the room for a short time. In this case, the preset time should be extended. If the number of consecutive counts M reaches the first preset number, it indicates that the waiting time is long enough and the person should be staying in the room for a long time. If the air conditioner is not turned on, then the air conditioner should be turned on.

[0086] like Figure 8 In the process, when M is 0, 1, or 2, a 5-minute delay is required. Therefore, it is necessary to wait until M is 3, which is 15 minutes later, before it is determined that the person has been present for an extended period of time. At this point, the air conditioning needs to be turned on. If the air conditioning is already on, there is no need to adjust it. If the air conditioning is not on, then turn it on.

[0087] If no one is detected consecutively, the timer count N is incremented by one, where N is a positive integer or zero. If the consecutive count N does not reach the second preset count, it indicates that the number of checks is too low and the required waiting time has not yet been reached. The person may not have returned to the room for a short time or may have had an emergency. In this case, the preset time is extended. If the consecutive count N reaches the second preset count, it indicates that the waiting time is long enough and the person is unlikely to return to the room for an extended period. If the air conditioner is on, it is turned off.

[0088] like Figure 8 In the calculation, when N is 0, 1, 2, or 3, a 5-minute delay is required. Therefore, it is necessary to wait until N is 5, which is 20 minutes, before determining that the person will not return to the room for an extended period. At this point, the air conditioner needs to be turned off. If the air conditioner is already off, no adjustments are needed; otherwise, it should be turned off.

[0089] This application allows for automatic control of the air conditioner's on / off state regardless of whether the room is occupied or unoccupied. Users can also customize the preset duration of the delay module to meet their individual needs.

[0090] The overall flowchart of this application can be found in [reference needed]. Figure 9 This includes the following steps:

[0091] Step 901: Power on the radar module while the air conditioner is in standby mode.

[0092] Step 902: The radar module acquires echo signals and generates radar images.

[0093] Step 903: Prepare a sample dataset based on radar images, including noisy images and noisy images.

[0094] Step 904: Train the CNN network using the sample dataset to obtain the denoising model.

[0095] Step 905: After denoising the image using a denoising model, the air conditioner is automatically controlled to open and close based on the number of people in the room.

[0096] Based on the same technical concept, this application provides an image denoising apparatus, such as... Figure 10 As shown, the device includes:

[0097] The extraction module 1001 is used to input noisy image samples into the generator, extract human behavior features from the noisy image samples through the behavior feature representation network, and obtain the denoised image output by the generator.

[0098] The input module 1002 is used to input the denoised image samples and the preset training samples into the recognizer, wherein the training samples include noisy image samples and noisy image samples.

[0099] The generation module 1003 is used to calculate the error between the recognition result and the real label of the noisy image sample through the recognizer, optimize the network parameters of the generative adversarial network using the backpropagation algorithm, obtain the trained generator, and use the trained generator as the denoising model.

[0100] Module 1004 is used to input the target noisy image into the denoising model to obtain the target denoised image.

[0101] Optionally, module 1004 is used for:

[0102] Input the noisy target image into the denoising model;

[0103] During the downsampling process, the size of the feature map is reduced by adjusting the convolution stride to obtain the first feature map;

[0104] In the deep feature processing, the first feature map and the second feature map after two convolutions are fused through densely connected units;

[0105] During the upsampling process, zero elements are used to fill the feature map at a preset resolution in the fused feature map, and then convolution is performed to obtain the target denoised image.

[0106] Optionally, module 1004 is used for:

[0107] During the downsampling process, a 7×7 convolutional layer is used to extract contextual information within the receptive field of the noisy target image;

[0108] The input of a 7×7 convolutional layer is sequentially fed into two 4×4 convolutional layers to obtain the first feature map. The feature map size output by the 4×4 convolutional layer is reduced by half compared to the feature map size output by the 7×7 convolutional layer.

[0109] Optionally, module 1004 is used for:

[0110] In the deep feature processing, the first feature map is convolved through a 3×3 convolutional layer and a 1×1 convolutional layer to obtain the second feature map. The 3×3 convolutional layer performs 1-pixel edge padding, the 1×1 convolutional layer has a stride of 1 and no edge padding, and the 1×1 convolutional layer is used to perform non-linear transformation on the features output by the 3×3 convolutional layer.

[0111] The first feature map and the second feature map after two convolutions are fused using a channel concatenation method.

[0112] Optionally, four transition layers are provided between each densely connected unit. Each transition layer is a 1×1 convolutional layer with a convolution stride of 1 and no edge padding.

[0113] Optionally, module 1004 is used for:

[0114] In the transposed convolution, zero elements are used to fill the feature map of the fused feature map at a preset resolution;

[0115] The first two 4×4 convolutional layers are used to perform convolution operations on the padded fused feature map to obtain the feature map of the original size. The stride of the first two 4×4 convolutional layers is 2, and 1 pixel of edge padding is set.

[0116] A 7×7 convolutional layer is used as the last convolutional layer to perform convolution operations on the feature map of the original size. The output channel number of the last convolutional layer is 3.

[0117] The output of the last convolutional layer is normalized using the hyperbolic tangent function to obtain the target denoised image.

[0118] Optionally, the device is also used for:

[0119] If the presence or absence of people is continuously determined in the target denoised image, the count is incremented by one for each determination.

[0120] If the number of consecutive counts does not reach the target preset number, the preset time will be extended until the number of consecutive counts exceeds the target preset number. Then, the air conditioner will be turned on or off depending on whether the air conditioner is on or off. The preset time can be manually adjusted.

[0121] Optionally, the device is also used for:

[0122] If someone is present, extend the preset duration if the consecutive count does not reach the first preset number;

[0123] If the number of consecutive counts reaches the first preset number, and the air conditioner is not turned on, then the air conditioner will be turned on.

[0124] Optionally, the device is also used for:

[0125] If no one is present, extend the preset duration if the consecutive count does not reach the second preset number;

[0126] If the number of consecutive counts reaches the second preset number, and the air conditioner is on, then turn it off.

[0127] like Figure 11 As shown in the figure, this application provides an air conditioner, including a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104, wherein the processor 1101, the communication interface 1102 and the memory 1103 communicate with each other through the communication bus 1104.

[0128] Memory 1103 is used to store computer programs.

[0129] In one embodiment of this application, the processor 1101, when executing the program stored in the memory 1103, implements the image denoising method provided in any of the foregoing method embodiments.

[0130] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the image denoising method provided in any of the foregoing method embodiments.

[0131] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0133] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0134] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An image denoising method, applied to a radar human detection module in an air conditioner for personnel detection, characterized in that, The method includes: Noisy image samples are input into the generator, and human behavior features are extracted from the noisy image samples through the behavior feature representation network in the generator to obtain the denoised image output by the generator. The human behavior features are used to supervise the training of the generator by using the error of the denoising result on the human behavior features, so that the output of the generator is pixel-level similar to the corresponding noiseless image, while retaining the original motion information. The denoised image samples and preset training samples are input into the recognizer, wherein the training samples include noisy image samples and noisy image samples; The error between the recognition result and the true label of the noisy image sample is calculated by the recognizer. The network parameters of the generative adversarial network are optimized using the backpropagation algorithm to obtain a trained generator. The trained generator is then used as a denoising model. The denoising model includes a downsampling network, a deep feature processing network, and an upsampling network. The downsampling network consists of three convolutional layers connected sequentially: 7×7, 4×4, and 4×4. The feature map size output by the 4×4 convolution is reduced by half compared to the feature map size output by the 7×7 convolutional layer. This reduction in feature map size by adjusting the convolution stride minimizes detail loss during downsampling. The deep feature processing network... The layer feature processing network includes densely connected units, each consisting of a 3×3 convolutional layer and a 1×1 convolutional layer connected sequentially. The 3×3 convolutional layer is used to perform 1-pixel edge padding to maintain the size of the convolutional feature map. The 1×1 convolutional layer is used to perform non-linear transformation on the features output by the 3×3 convolutional layer. The upsampling network consists of three layers connected sequentially: a 4×4 transposed convolutional layer, a 4×4 transposed convolutional layer, and a 7×7 convolutional layer. The first two 4×4 transposed convolutional layers are used to perform convolution operations on the padded fused feature map to obtain the original size feature map. The 7×7 convolutional layer is used to perform convolution operations on the original size feature map. The trained denoising model was applied to the actual radar echo signal enhancement task, and the denoising model was encapsulated into the radar human sensing module of the air conditioner. The target noisy image is input into the denoising model to obtain the target denoised image.

2. The method according to claim 1, characterized in that, Inputting the target noisy image into the denoising model to obtain the target denoised image includes: Input the target noisy image into the denoising model; During the downsampling process, the size of the feature map is reduced by adjusting the convolution stride to obtain the first feature map; In the deep feature processing, the first feature map and the second feature map after two convolutions are fused through dense connection units; During the upsampling process, zero elements are used to fill the feature map at a preset resolution in the fused feature map, and then convolution is performed to obtain the target denoised image.

3. The method according to claim 2, characterized in that, The process of reducing the size of the feature map by adjusting the convolution stride during downsampling to obtain the first feature map includes: During the downsampling process, a 7×7 convolutional layer is used to extract the contextual information within the receptive field of the target noisy image; The input of the 7×7 convolutional layer is sequentially fed into two 4×4 convolutional layers to obtain the first feature map, wherein the size of the feature map output by the 4×4 convolutional layer is reduced by half compared to the size of the feature map output by the 7×7 convolutional layer.

4. The method according to claim 2, characterized in that, In the deep feature processing, the fusion of the first feature map and the second feature map after two convolutions is performed using densely connected units, including: In the deep feature processing, the first feature map is convolved through a 3×3 convolutional layer and a 1×1 convolutional layer to obtain the second feature map. The 3×3 convolutional layer performs 1-pixel edge padding, and the 1×1 convolutional layer has a stride of 1 and no edge padding. The 1×1 convolutional layer is used to perform non-linear transformation on the features output by the 3×3 convolutional layer. The first feature map and the second feature map after two convolutions are fused using a channel concatenation method.

5. The method according to claim 4, characterized in that, Four transition layers are provided between each of the densely connected units. Each transition layer is a 1×1 convolutional layer with a convolution stride of 1 and no edge padding.

6. The method according to claim 2, characterized in that, During the upsampling process, zero elements are used to fill the feature map at a preset resolution in the fused feature map, and then convolution is performed to obtain the target denoised image, including: In the transposed convolution, zero elements are used to fill the feature map of the fused feature map at a preset resolution; The first two 4×4 convolutional layers are used to perform convolution operations on the filled fused feature map to obtain the feature map of the original size. The stride of the first two 4×4 convolutional layers is 2, and an edge padding of 1 pixel is set. A 7×7 convolutional layer is used as the last convolutional layer to perform a convolution operation on the feature map of the original size, wherein the number of output channels of the last convolutional layer is 3; The output of the last convolutional layer is normalized using the hyperbolic tangent function to obtain the target denoised image.

7. The method according to claim 2, characterized in that, After obtaining the target denoised image, the method further includes: If the presence or absence of people is continuously determined in the target denoised image, the count is incremented by one for each determination. If the number of consecutive counts does not reach the target preset number, the preset time is extended until the number of consecutive counts exceeds the target preset number. Then, the air conditioner is turned on or off based on the air conditioner's on / off status. The preset time can be manually adjusted.

8. The method according to claim 7, characterized in that, If the consecutive count does not reach the target preset count, the preset time is extended until the consecutive count exceeds the target preset count. Then, the air conditioner is turned on or off based on its on / off status, including: If someone is present, extend the preset duration if the consecutive count does not reach the first preset number; If the number of consecutive counts reaches the first preset number, and the air conditioner is not turned on, then the air conditioner will be turned on.

9. The method according to claim 7, characterized in that, If the consecutive count does not reach the target preset count, the preset time is extended until the consecutive count exceeds the target preset count. Then, the air conditioner is turned on or off based on its on / off status, including: If no one is present, extend the preset duration if the consecutive count does not reach the second preset number; If the number of consecutive counts reaches the second preset number, and the air conditioner is on, then turn it off.

10. An image denoising apparatus for implementing the image denoising method as described in any one of claims 1 to 9, characterized in that, The device includes: The extraction module is used to input noisy image samples into the generator, extract human behavior features from the noisy image samples through the behavior feature representation network, and obtain the denoised image output by the generator. The input module is used to input the denoised image samples and preset training samples into the recognizer, wherein the training samples include noisy image samples and noisy image samples; The generation module is used to calculate the error between the recognition result and the real label of the noisy image sample through the recognizer, optimize the network parameters of the generative adversarial network using the backpropagation algorithm, obtain the trained generator, and use the trained generator as a denoising model. The module is used to input the target noisy image into the denoising model to obtain the target denoised image.

11. An air conditioner, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

Images