A window state detection method, system, terminal and storage medium

Abstract

The application provides a window state detection method, system, terminal and storage medium, relates to the technical field of window state detection, and includes obtaining a system trigger signal; starting a preset dual-mode camera to collect a full-image visible light image and a full-image infrared image based on the system trigger signal; analyzing the full-image visible light image and the full-image infrared image to determine a final infrared image; obtaining a position prompt coordinate of a preset four-corner point; inputting the position prompt coordinate, the full-image visible light image and the final infrared image into a preset convolutional neural network to generate a position prompt feature and a global fusion feature; analyzing the position prompt feature and the global fusion feature to determine a region enhancement feature; and inputting the region enhancement feature into a preset state classification network to generate a current window state. The application has the effect of ensuring high accuracy of the window state detection method.

Description

Technical Field

[0001] This application relates to the technical field of window status detection, and in particular to a window status detection method, system, terminal and storage medium. Background Technology

[0002] Window status detection refers to the process of using sensors and visual recognition technology to detect and judge the physical state of windows, with the aim of improving the accuracy of window status detection.

[0003] In related technologies, window status detection methods typically use infrared sensors and cameras to collect image information, then preprocess the image information to select effective window areas, extract features through convolutional neural networks, and then determine the opening and closing status of the window through a set feature threshold, finally outputting the specific opening and closing status of the window.

[0004] Regarding the aforementioned technologies, infrared sensors and cameras are used to collect image information. However, when using infrared sensors, if the window surface has a coating, it will change the infrared emissivity of the window, resulting in the captured infrared image lacking normal details and temperature contrast, leading to inaccurate window status detection. There is still room for improvement. Summary of the Invention

[0005] To ensure the high accuracy of the window status detection method, this application provides a window status detection method, system, terminal, and storage medium.

[0006] Firstly, this application provides a window status detection method, which adopts the following technical solution:

[0007] A window status detection method, comprising:

[0008] Obtain system trigger signals;

[0009] The system triggers a pre-set dual-mode camera to acquire full-view visible light and full-view infrared images.

[0010] The visible light image and the infrared image of the entire image are analyzed to determine the final infrared image;

[0011] Get the coordinates of the preset four corner points;

[0012] The location cue coordinates, the full-image visible light image, and the final infrared image are input into a pre-defined convolutional neural network to generate location cue features and global fusion features;

[0013] The location cue features and global fusion features are analyzed to determine the region enhancement features;

[0014] The region enhancement features are input into a preset state classification network to generate the current window state.

[0015] By adopting the above technical solution, a dual-modal camera is activated to acquire full-view visible light and full-view infrared images. After analyzing the full-view visible light and full-view infrared images, the final infrared image is determined. Then, the location cue coordinates, the full-view visible light image, and the final infrared image are input into a convolutional neural network to generate location cue features and global fusion features. After analyzing the location cue features and global fusion features, the region enhancement features are determined. The region enhancement features are input into a state classification network to generate the current window state, thereby offsetting the influence of the coating on the window surface on the full-view infrared image, ensuring the high accuracy of the window state detection method.

[0016] Optionally, the step of analyzing the full-image visible light image and the full-image infrared image to determine the final infrared image includes:

[0017] Obtain the visible light pixel values ​​and pixel count values ​​in the entire visible light image;

[0018] The visible light pixel values ​​are weighted and summed according to the preset channel brightness weight parameters to generate grayscale pixel values;

[0019] Analyze the grayscale pixel values ​​and the number of pixels to generate a pixel grayscale image;

[0020] Analyze the pixel grayscale image and the full-image infrared image to determine the average window brightness and adjust the infrared image;

[0021] The adjusted infrared images were analyzed to determine the maximum temperature difference and crack detection results;

[0022] The crack detection result is determined to be either a preset result indicating the presence of a crack or a preset result indicating the absence of a crack.

[0023] If the result shows cracks, a damage warning message will be generated;

[0024] If the result is without cracks, the average window brightness, maximum temperature difference, pixel grayscale image, and adjusted infrared image are analyzed to determine the final infrared image.

[0025] By adopting the above technical solution, the grayscale pixel value is determined by weighted summation of visible light pixel values ​​based on channel brightness weight parameters. After analyzing the grayscale pixel value and the number of pixels, a pixel grayscale image is generated. Then, the average window brightness and the adjusted infrared image are determined by analyzing the pixel grayscale image and the full-image infrared image. After analyzing the adjusted infrared image, the maximum temperature difference and crack detection result are determined. When a crack is determined, a damage warning message is directly generated. When no crack is determined, the final infrared image is determined by analyzing the average window brightness, the maximum temperature difference, the pixel grayscale image, and the adjusted infrared image. This offsets the impact of local temperature differences caused by window cracks on the full-image infrared image, improves the safety of window use, and ensures the high accuracy of the window condition detection method.

[0026] Optionally, the steps of analyzing the pixel grayscale image and the full-image infrared image to determine the average window brightness and adjust the infrared image include:

[0027] Obtain the window pixel value and window pixel count value of a preset window region in the pixel grayscale image;

[0028] Calculate the mean of the window's pixel values ​​to generate the mean window brightness;

[0029] Calculate the Laplacian operator for the full-image infrared image to generate infrared image flatness;

[0030] Calculate the variance of the infrared image flatness to generate the average flatness;

[0031] Determine whether the average flatness is greater than the preset standard flatness;

[0032] If it is greater than that, then the full-image infrared image is defined as the adjusted infrared image;

[0033] If the value is not greater than the specified value, the window pixel value, the average window brightness, the number of window pixels, the pixel grayscale image, and the full-image infrared image are analyzed to determine the infrared image to be adjusted.

[0034] By adopting the above technical solution, after calculating the mean value of window pixel values, the mean value of window brightness is generated. After calculating the Laplacian operator of the full-image infrared image, the infrared image flatness is generated. Then, the variance of the infrared image flatness is calculated to generate the average flatness. If the average flatness is greater than the standard flatness, the full-image infrared image is directly defined as the adjusted infrared image. If it is not greater, the adjusted infrared image is determined by analyzing the window pixel values, the mean value of window brightness, the number of window pixels, the pixel grayscale image, and the full-image infrared image. Thus, the average flatness is used to determine whether the infrared image is affected by the coating. In the case of coating influence, the full-image infrared image is corrected to offset the influence of changes in infrared emissivity or reflectivity caused by the coating.

[0035] Optionally, the window pixel values, average window brightness, number of window pixels, pixel grayscale image, and full-image infrared image are analyzed to determine the steps for adjusting the infrared image, including:

[0036] The window pixel values ​​and preset standard brightness values ​​are sorted to determine the number of brightness pixels;

[0037] Calculate the quotient of the number of brightness pixels to the number of window pixels to generate an adjustment brightness index;

[0038] The brightness index, preset standard intensity factor, and preset intensity adjustment range are analyzed to determine the intensity adjustment factor.

[0039] Obtain the background pixel value of a preset background area in a pixel grayscale image;

[0040] Calculate the mean value of the background pixels to generate the mean background brightness.

[0041] The average window brightness, average background brightness, and preset offset adjustment range are analyzed to determine the offset adjustment factor.

[0042] The intensity adjustment factor, the full-image infrared image, and the offset adjustment factor were analyzed to determine the adjustment of the infrared image.

[0043] By adopting the above technical solution, the number of brightness pixels is determined after sorting the window pixel values ​​and standard brightness values. The quotient of the number of brightness pixels and the number of window pixels is calculated to determine the brightness adjustment index. Then, the intensity adjustment factor is determined after analyzing the brightness adjustment index, standard intensity factor, and intensity adjustment range. The mean value of background pixel values ​​is calculated to generate the mean value of background brightness. Then, the offset adjustment factor is determined after analyzing the mean value of window brightness, mean value of background brightness, and offset adjustment range. Finally, the infrared image is adjusted after analyzing the intensity adjustment factor, the full-image infrared image, and the offset adjustment factor. This adjusts the contrast and offset of the full-image infrared image, reduces the systematic deviation between the temperature of the full-image infrared image and the actual situation, and thus improves the accuracy of window status detection.

[0044] Optionally, the steps of analyzing the adjusted infrared images to determine the maximum temperature difference and crack detection results include:

[0045] Obtain the infrared temperature value of the adjusted infrared image and the neighborhood temperature value of the preset infrared image neighborhood;

[0046] Sort the neighborhood temperature values ​​to determine the median temperature value;

[0047] Calculate the absolute value of the difference between the infrared temperature value and the median temperature value to generate the infrared temperature difference value;

[0048] The infrared temperature differences are sorted to determine the maximum temperature difference.

[0049] Determine whether the maximum temperature difference is greater than the preset standard temperature difference.

[0050] If the result is greater than the preset value, the result with cracks will be defined as the crack detection result.

[0051] If the result is not greater than the preset no-crack result, then the crack detection result will be defined as the crack detection result.

[0052] By adopting the above technical solution, the median temperature value is determined after sorting the neighborhood temperature values. The absolute value of the difference between the infrared temperature value and the median temperature value is calculated to generate the infrared temperature difference value. The maximum temperature difference value is then determined after sorting the infrared temperature difference values. When the maximum temperature difference value is greater than the standard temperature difference value, the result with cracks is directly defined as the crack detection result; if it is not greater, the result without cracks is directly defined as the crack detection result. Thus, the presence or absence of cracks is determined based on the temperature difference between each pixel and its neighborhood, thereby improving the safety of window use and the accuracy of infrared images, and ensuring the high accuracy of the window status detection method.

[0053] Optionally, the steps of analyzing the average window brightness, maximum temperature difference, pixel grayscale image, and adjusted infrared image to determine the final infrared image include:

[0054] Calculate the gradient magnitude of the pixel grayscale image to generate a gradient magnitude image;

[0055] Obtain the gradient pixel values ​​of the gradient magnitude image;

[0056] Calculate the mean of the gradient pixel values ​​to generate the gradient average;

[0057] Determine whether the average gradient value is less than the preset standard gradient mean;

[0058] If it is less than, then the adjusted infrared image will be defined as the final infrared image;

[0059] If it is not less than, then the average window brightness, adjusted infrared image, and maximum temperature difference are analyzed to determine the final infrared image.

[0060] By adopting the above technical solution, the gradient magnitude image is generated after calculating the gradient magnitude of the pixel grayscale image, and the gradient average value is generated after calculating the mean of the gradient pixel values. When the gradient average value is determined to be less than the standard gradient average value, the adjusted infrared image is directly defined as the final infrared image; if it is not less than the standard gradient average value, the final infrared image is determined after analyzing the window brightness average value, the adjusted infrared image, and the maximum temperature difference value. Thus, the complexity of the adjusted infrared image is judged based on the gradient average value, thereby determining whether there are decorations on the window, so as to determine the final infrared image.

[0061] Optionally, the steps of analyzing the average window brightness, adjusted infrared image, and maximum temperature difference to determine the final infrared image include:

[0062] Determine whether the maximum temperature difference is less than the preset reference temperature difference;

[0063] If it is less than, then the adjusted infrared image will be defined as the final infrared image;

[0064] If it is not less than, then the reference brightness value is determined according to the correspondence between the average window brightness and the preset brightness threshold.

[0065] Determine whether the average brightness of the window is greater than the reference brightness value;

[0066] If it is greater than the preset reflection correction factor, the product of the adjusted infrared image and the preset reflection correction factor is calculated to generate the final infrared image.

[0067] If it is not greater than, then calculate the product of the adjusted infrared image and the preset light absorption correction factor to generate the final infrared image.

[0068] By adopting the above technical solution, when the maximum temperature difference is less than the reference temperature difference, the adjusted infrared image is directly defined as the final infrared image; if it is not less than the reference temperature difference, the reference brightness value is determined according to the correspondence between the average window brightness and the brightness threshold. When the average window brightness is greater than the reference brightness value, the final infrared image is generated by calculating the product of the adjusted infrared image and the reflection correction factor; if it is not greater than the reference brightness, the final infrared image is generated by calculating the product of the adjusted infrared image and the light absorption correction factor. Thus, the window surface is determined to be a decoration or the window's own pattern based on the maximum temperature difference. Then, the adjusted infrared image is adjusted by the brightness of the full visible light image to facilitate the subsequent generation of global fusion features.

[0069] Secondly, this application provides a window status detection system, which adopts the following technical solution:

[0070] A window status detection system, comprising:

[0071] The acquisition module is used to acquire system trigger signals and location prompt coordinates;

[0072] A memory for storing a program for a window state detection method as described in any of the preceding claims;

[0073] The processor and the program in the memory can be loaded and executed by the processor to implement a window state detection method as described in any of the above.

[0074] By adopting the above technical solution, a window state detection method program stored in memory is loaded and executed by a processor. The acquisition module is controlled to acquire a series of data related to the window state detection method, thereby activating a dual-modal camera to acquire full-view visible light and full-view infrared images. After analyzing the full-view visible light and full-view infrared images, the final infrared image is determined. The position cue coordinates, the full-view visible light image, and the final infrared image are then input into a convolutional neural network to generate position cue features and global fusion features. After analyzing the position cue features and global fusion features, region enhancement features are determined. The region enhancement features are then input into a state classification network to generate the current window state, thereby offsetting the influence of the coating on the window surface on the full-view infrared image, ensuring the high accuracy of the window state detection method.

[0075] Thirdly, this application provides a smart terminal, which adopts the following technical solution:

[0076] A smart terminal includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any of the preceding claims for window state detection.

[0077] By adopting the above technical solution, a window state detection method program stored in memory is loaded and executed by a processor. The acquisition module is controlled to acquire a series of data related to the window state detection method, thereby activating a dual-modal camera to acquire full-view visible light and full-view infrared images. After analyzing the full-view visible light and full-view infrared images, the final infrared image is determined. The position cue coordinates, the full-view visible light image, and the final infrared image are then input into a convolutional neural network to generate position cue features and global fusion features. After analyzing the position cue features and global fusion features, region enhancement features are determined. The region enhancement features are then input into a state classification network to generate the current window state, thereby offsetting the influence of the coating on the window surface on the full-view infrared image, ensuring the high accuracy of the window state detection method.

[0078] Fourthly, this application provides a computer storage medium capable of storing corresponding programs, which facilitates the implementation of a method for ensuring high accuracy in window state detection, and adopts the following technical solution:

[0079] A computer-readable storage medium storing a computer program that can be loaded by a processor and executed by any of the above-described window state detection methods.

[0080] By adopting the above technical solution, a window state detection method program stored in memory is loaded and executed by a processor. The acquisition module is controlled to acquire a series of data related to the window state detection method, thereby activating a dual-modal camera to acquire full-view visible light and full-view infrared images. After analyzing the full-view visible light and full-view infrared images, the final infrared image is determined. The position cue coordinates, the full-view visible light image, and the final infrared image are then input into a convolutional neural network to generate position cue features and global fusion features. After analyzing the position cue features and global fusion features, region enhancement features are determined. The region enhancement features are then input into a state classification network to generate the current window state, thereby offsetting the influence of the coating on the window surface on the full-view infrared image, ensuring the high accuracy of the window state detection method.

[0081] In summary, this application includes at least one of the following beneficial technical effects:

[0082] 1. By activating a dual-modal camera to acquire full-view visible light and full-view infrared images, the final infrared image is determined after analyzing the full-view visible light and full-view infrared images. The location cue coordinates, the full-view visible light image, and the final infrared image are then input into a convolutional neural network to generate location cue features and global fusion features. The location cue features and global fusion features are then analyzed to determine the region enhancement features. The region enhancement features are then input into a state classification network to generate the current window state, thereby offsetting the influence of the coating on the full-view infrared image caused by the window surface, ensuring the high accuracy of the window state detection method.

[0083] 2. By weighting and summing visible light pixel values ​​according to channel brightness weight parameters, grayscale pixel values ​​are determined. After analyzing the grayscale pixel values ​​and pixel count, a pixel grayscale image is generated. Then, by analyzing the pixel grayscale image and the full-image infrared image, the average window brightness and the adjusted infrared image are determined. After analyzing the adjusted infrared image, the maximum temperature difference and crack detection results are determined. When a crack is detected, a damage warning message is directly generated. When no crack is detected, the final infrared image is determined by analyzing the average window brightness, maximum temperature difference, pixel grayscale image, and adjusted infrared image. This offsets the impact of local temperature differences caused by window cracks on the full-image infrared image, improves the safety of window use, and ensures the high accuracy of the window condition detection method.

[0084] 3. After sorting the window pixel values ​​and standard brightness values, the number of brightness pixels is determined. The quotient of the number of brightness pixels and the number of window pixels is calculated to determine the brightness adjustment index. Then, the intensity adjustment factor is determined after analyzing the brightness adjustment index, standard intensity factor, and intensity adjustment range. The mean value of background pixel values ​​is calculated to generate the mean value of background brightness. Then, the offset adjustment factor is determined after analyzing the mean value of window brightness, mean value of background brightness, and offset adjustment range. Then, the adjustment infrared image is determined after analyzing the intensity adjustment factor, the full-image infrared image, and the offset adjustment factor. This adjusts the contrast and offset of the full-image infrared image, reduces the systematic deviation between the temperature of the full-image infrared image and the actual situation, and thus improves the accuracy of the window state detection method. Attached Figure Description

[0085] Figure 1 This is a flowchart of a window state detection method in an embodiment of this application.

[0086] Figure 2 This is a flowchart illustrating the steps in this application embodiment to analyze a full-image visible light image and a full-image infrared image to determine the final infrared image.

[0087] Figure 3 This is a flowchart illustrating the steps in this application embodiment to analyze pixel grayscale images and full-image infrared images to determine the average window brightness and adjust the infrared image.

[0088] Figure 4 This is a flowchart illustrating the steps for analyzing window pixel values, average window brightness, number of window pixels, pixel grayscale images, and full-image infrared images in this application embodiment to determine the steps for adjusting the infrared image.

[0089] Figure 5 This is a flowchart of the steps in this application embodiment to analyze the adjusted infrared image to determine the maximum temperature difference and the crack detection result.

[0090] Figure 6 This is a flowchart of the steps in this application embodiment to analyze the average window brightness, maximum temperature difference, pixel grayscale image, and adjusted infrared image to determine the final infrared image.

[0091] Figure 7 This is a flowchart of the steps in this application embodiment to analyze the average window brightness, adjust the infrared image, and the maximum temperature difference to determine the final infrared image. Detailed Implementation

[0092] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figures 1 to 7The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0093] This application discloses a window state detection method, specifically a dual-modal camera and a processing terminal. The processing terminal is communicatively connected to the dual-modal camera to achieve data interaction and control. After receiving a system trigger signal, the processing terminal starts the dual-modal camera to acquire a full-view visible light image and a full-view infrared image. After analyzing the full-view visible light image and the full-view infrared image, the final infrared image is determined. The location cues, the full-view visible light image, and the final infrared image are then input into a convolutional neural network to generate location cues and global fusion features. After analyzing the location cues and global fusion features, region enhancement features are determined. The region enhancement features are then input into a state classification network to generate the current window state, thereby offsetting the influence of the coating on the full-view infrared image on the window surface, ensuring the high accuracy of the window state detection method.

[0094] Reference Figure 1 This application discloses a window state detection method, including the following steps:

[0095] Step S100: Obtain the system trigger signal.

[0096] Among them, the system trigger signal refers to the system's trigger signal, which is triggered by the operator to activate the system's start switch, thereby sending the level signal representing the system trigger signal to the processing terminal to obtain the start conditions for the dual-mode camera.

[0097] Step S101: Based on the system trigger signal, start the preset dual-mode camera to acquire the full-view visible light image and the full-view infrared image.

[0098] After receiving the system trigger signal, the processing terminal responds to the system trigger signal by starting the dual-mode camera to acquire the full-view visible light image and the full-view infrared image, so as to determine the final infrared image later.

[0099] A dual-modal camera is a camera capable of acquiring both RGB and infrared images. It integrates an RGB sensor and an infrared thermal imaging sensor and is pre-installed by the operator. By deploying a dual-modal camera, images needed for window status detection are acquired to subsequently determine the final infrared image.

[0100] A full-view visible light image is an image that displays details such as the color and texture of an object, and is captured by a dual-modal camera. By acquiring a full-view visible light image, the full-view infrared image is corrected based on the brightness and texture information reflected in the image, in order to determine the final infrared image.

[0101] A full-view infrared image is an image that shows the heat distribution of an object and is captured by a dual-modal camera. By acquiring full-view infrared images, it is possible to determine whether a window has cracks based on local temperature differences, thereby improving the safety of window use.

[0102] Step S102: Analyze the full-image visible light image and the full-image infrared image to determine the final infrared image.

[0103] The final infrared image refers to the image after correction of the full-image infrared image based on whether the window has a coating or decoration. It is obtained by the processing terminal through analysis of the full-image visible light image and the full-image infrared image. Specific methods are described in [reference needed]. Figure 2 The steps involved are as follows. By determining the final infrared image, the phenomenon of an overly smooth infrared image due to the influence of the coating, as well as local temperature differences caused by obstruction by decorative elements, can be eliminated, thereby ensuring the high accuracy of the window condition detection method.

[0104] Step S103: Obtain the position coordinates of the preset four corner points.

[0105] The four corner points refer to the four corner points of the target window, which are marked by the operator on the full visible light image. By determining the positions of the four corner points, the area of ​​the target window and the background area can be identified, which facilitates subsequent adjustments to the infrared image.

[0106] Location cue coordinates refer to the coordinates of the four corner points of the target window, obtained by the processing terminal after converting the positions of the four corner points into coordinates. By determining the location cue coordinates, the position of the target window is made more accurate, providing data support for the subsequent generation of location cue features.

[0107] Step S104: Input the location cue coordinates, the full-image visible light image, and the final infrared image into a preset convolutional neural network to generate location cue features and global fusion features.

[0108] Among them, location cue features refer to the features obtained by enhancing the location cue coordinates through a convolutional neural network structure. The processing terminal inputs the location cue coordinates into the convolutional neural network. Through the enhancement of the convolutional structure in the convolutional neural network, enhanced location features are obtained to facilitate subsequent determination of region enhancement features.

[0109] Global fusion features refer to more comprehensive features obtained by extracting information from the full visible light image and the final infrared image. These features are generated by the processing terminal by inputting the full visible light image and the final infrared image into a convolutional neural network. By determining global fusion features, the texture, shape, color, and temperature distribution of the window can be fused to obtain complete feature information related to the window, facilitating the subsequent determination of the current window state.

[0110] A convolutional neural network (CNN) is a deep learning model used to process window-related data. It consists of convolutional layers, pooling layers, and fully connected layers. The convolutional layers initially extract local features from the full-image visible light data and the final infrared data. The pooling layers reduce the dimensionality of the features, thereby reducing computational cost. Then, the full-image visible light data branch is input into the main encoder to extract global visible light features, and the final infrared data branch is input into a separate encoder to extract global infrared features. These two types of features are fused using learnable gating parameters to generate a global fused feature. Finally, the current window state is output through a fully connected layer.

[0111] Step S105: Analyze the location cue features and global fusion features to determine the region enhancement features.

[0112] Among them, region enhancement features refer to spatially dimensional features resulting from the interaction of location features and global fusion features. These features are obtained by the processing terminal through a spatial attention mechanism that interacts with the location cue features and the global fusion features. By determining region enhancement features, the convolutional network model can focus on features related to the window position within the global fusion features, making the detection of window position and state more accurate, thereby ensuring the high accuracy of the window state detection method.

[0113] Step S106: Input the region enhancement features into a preset state classification network to generate the current window state.

[0114] The current window state refers to the window's open / closed state, including open, closed, or half-open, which is obtained by the processing terminal after inputting the region enhancement features into the state classification network. Through classification by the state classification network, the current open / closed posture of the window can be determined, thereby improving the accuracy of the window state detection method.

[0115] State classification networks are networks that classify the opening and closing states of windows based on region enhancement features. They use fully connected layers to map the output region enhancement features to the classification decision boundary of the window state category, and finally output the current window state, thus ensuring the high accuracy of the window state detection method.

[0116] Reference Figure 2 The steps for analyzing the full-image visible light image and the full-image infrared image to determine the final infrared image include:

[0117] Step S200: Obtain the visible light pixel value and pixel count value in the full visible light image.

[0118] The visible light pixel value refers to the color quantization value of the visible light image, which is read by the processing terminal from the full visible light image information. By determining the visible light pixel value, the three-channel values ​​of RGB can be converted into single-channel values ​​of the grayscale image, providing data support for subsequent adjustments to the infrared image.

[0119] The pixel count value refers to the spatial resolution of the entire visible light image, which is obtained by the processing terminal from the visible light image information. By determining the pixel count value, the pixel count value of the pixel grayscale image is determined, and then the pixel grayscale image is generated to facilitate subsequent adjustments to the infrared image.

[0120] Step S201: The visible light pixel values ​​are weighted and summed according to the preset channel brightness weight parameters to generate grayscale pixel values.

[0121] Here, grayscale pixel value refers to the luminance quantization value of a pixel grayscale image, which is obtained by the processing terminal through a weighted sum of visible light pixel values ​​based on channel luminance weight parameters. By determining the grayscale pixel value, the pixel value of the pixel grayscale image can be determined, thereby compressing three-dimensional color information into one-dimensional luminance information, which facilitates subsequent determination and adjustment of the infrared image.

[0122] Channel brightness weighting parameters refer to the weighting coefficients that map the three color channels of the entire visible light image to the brightness of a single channel. For example, the weight of the red channel is 0.299, the weight of the green channel is 0.587, and the weight of the blue channel is 0.114, which are preset by the operator. By determining the channel brightness weighting parameters, grayscale distortion caused by average weighting can be avoided, thereby preserving the brightness information and details of the entire visible light image to the greatest extent and providing data support for determining grayscale pixel values.

[0123] Step S202: Analyze the grayscale pixel values ​​and the number of pixels to generate a pixel grayscale image.

[0124] A pixel grayscale image is an image that displays the brightness of an object. It is obtained by the processing terminal determining a two-dimensional pixel matrix based on the number of pixels, and then filling the two-dimensional matrix with each grayscale pixel value. By determining the pixel grayscale image, the three-channel data of the entire visible light image can be compressed into single-channel data, effectively avoiding interference from color information on the target object and significantly reducing the computational load of subsequent processing.

[0125] Step S203: Analyze the pixel grayscale image and the full-image infrared image to determine the average window brightness and adjust the infrared image.

[0126] The average window brightness refers to the average brightness value of the window area, which is obtained by the processing terminal after analyzing the pixel grayscale image. For specific methods, please refer to... Figure 3The steps are as follows. By determining the average brightness of the window area, the average brightness of the window area can be determined. This allows for the adjustment of the offset of the entire infrared image based on the brightness difference between the window area and the background area, providing data support for subsequent determination of the offset adjustment factor.

[0127] Adjusting the infrared image refers to correcting the entire infrared image based on whether the window surface has a coating. This is done by the processing terminal analyzing the pixel grayscale image and the entire infrared image. For specific methods, please refer to [link / reference needed]. Figure 3 The steps are as follows. By adjusting the infrared image, the phenomenon of an overly smooth full-image infrared image caused by the coating effect can be effectively eliminated, thereby obtaining more accurate temperature information and ensuring the high accuracy of the window status detection method.

[0128] Step S204: Analyze the adjusted infrared image to determine the maximum temperature difference and crack detection results.

[0129] The maximum temperature difference refers to the maximum value of all infrared temperature differences, which is obtained by the processing terminal after analyzing the adjusted infrared image. For specific methods, please refer to [reference needed]. Figure 5 The steps are as follows. By determining the maximum temperature difference, it can be determined whether there is a significant difference between its temperature and the normal temperature of its neighborhood, thereby determining whether the window has cracks and thus improving the safety of window use.

[0130] Crack detection results refer to the results of detecting whether a window has cracks, including results showing cracks and results showing no cracks. This is obtained by the processing terminal after analyzing the adjusted infrared image. For specific methods, please refer to [link / reference needed]. Figure 5 The steps involved are as follows. By determining the crack detection results, it can be found that there are cracks on the window surface, thus providing timely warnings in the event of cracks and improving the safety of window use.

[0131] Step S205: Determine whether the crack detection result is a preset result of having cracks or a preset result of not having cracks.

[0132] The method determines whether there is a crack on the window surface by judging whether the crack detection result is a cracked result or not. In the case of a crack, an early warning message is issued in time, and in the case of no crack, the infrared image is adjusted and corrected to improve the accuracy of the window condition detection method.

[0133] A "cracked" result refers to the presence of cracks on the window surface, which is stored by the operator at the processing terminal. Determining the presence of cracks confirms the existence of cracks on the window surface, allowing for timely issuance of early warning information to improve window safety.

[0134] A "crack-free" result refers to the absence of cracks on the window surface, and this result is stored by the operator at the processing terminal. By confirming a crack-free result, it can be determined that the window surface is free of cracks. Therefore, even in the absence of cracks, the infrared image can be further adjusted based on the presence of decorative elements on the window surface, thus ensuring the high accuracy of the window condition detection method.

[0135] Step S2051: If the result is a crack, a damage warning message is generated.

[0136] If the result indicates a crack, it means that there is a crack on the window surface. In this case, a damage warning message is generated directly to improve the safety of window use.

[0137] Damage alerts indicate the presence of cracks on the window surface and are generated by the processing terminal after confirming the presence of cracks. Generating damage alerts allows for the timely detection of damaged windows, thereby reducing the risk of window breakage.

[0138] Step S2052: If the result is without cracks, analyze the average window brightness, maximum temperature difference, pixel grayscale image and adjusted infrared image to determine the final infrared image.

[0139] If the result is "no cracks," it means there are no cracks on the window surface. In this case, the processing terminal analyzes the average window brightness, maximum temperature difference, pixel grayscale image, and adjusted infrared image to determine the final infrared image. The specific method is described in [reference needed]. Figure 6 The steps are as follows. By determining the final infrared image, local temperature differences caused by decorative obstructions on the window surface can be eliminated, thus ensuring the high accuracy of the window condition detection method.

[0140] Reference Figure 3 The steps for analyzing pixel grayscale images and full-image infrared images to determine the average window brightness and adjust the infrared image include:

[0141] Step S300: Obtain the window pixel value and window pixel count value of the preset window area in the pixel grayscale image.

[0142] The window area refers to the region in the pixel grayscale image where the window is located, which is predetermined by the operator. By determining the window area, the position of the target window in the pixel grayscale image can be determined, thereby determining the pixel value and number of pixels in the window area, which is helpful for subsequent adjustments to the infrared image.

[0143] Window pixel values ​​refer to the pixel values ​​within a window area, which are read by the processing terminal from the pixel grayscale image information. By determining the window pixel values, the brightness within the window area can be determined, thereby determining the average brightness of the window area, providing data support for subsequent determination of the offset adjustment factor.

[0144] The window pixel count value refers to the number of pixels within a window area, which is read by the processing terminal from the pixel grayscale image information. By determining the window pixel count, the number of pixels within the window area can be determined, thereby determining the proportion of bright pixels within the window area, providing data support for subsequent determination and adjustment of the brightness index.

[0145] Step S301: Calculate the mean of the window pixel values ​​to generate the mean window brightness.

[0146] After determining the window pixel values, the processing terminal calculates the average value of the window pixel values ​​to generate the average window brightness. Then, by comparing the difference between the average brightness of the window and the average brightness of the background, the offset of the whole infrared image is adjusted.

[0147] Step S302: Calculate the Laplacian operator for the full-image infrared image to generate infrared image flatness.

[0148] Infrared image smoothness refers to a value reflecting the temperature detail changes in the entire infrared image, obtained by the processing terminal after calculating the Laplacian operator of the entire infrared image. By determining the infrared image smoothness, the quantified value of the temperature change in the entire infrared image can be determined, thus providing data support for subsequently determining the average smoothness.

[0149] Step S303: Calculate the variance of the infrared image flatness to generate the average flatness.

[0150] The average flatness refers to the value that measures the dispersion of temperature changes in the entire infrared image. It is obtained by calculating the variance of the infrared image flatness from the processing terminal. By determining the average flatness, it can be determined that the smaller the average flatness, the fewer detailed changes are detected. At this time, the temperature distribution of the entire infrared image exhibits an abnormally smooth phenomenon, that is, the entire infrared image is affected by the coating on the window surface, so as to facilitate subsequent determination and adjustment of the infrared image.

[0151] Step S304: Determine whether the average flatness is greater than the preset standard flatness.

[0152] The standard flatness refers to a standard threshold for measuring the dispersion of temperature changes, which is set in advance by the operator. By determining whether the average flatness is greater than the standard flatness, it can be determined whether the window is affected by the surface coating. In the case of coating influence, the full-image infrared image can be corrected to ensure the high accuracy of the window condition detection method.

[0153] Step S3041: If it is greater than, then define the full-image infrared image as the adjusted infrared image.

[0154] If the average flatness is greater than the standard flatness, it means that the temperature detail changes detected by the Laplacian operator are very diverse, which indicates that there is an uneven temperature gradient in the full-image infrared image. This shows that the full-image infrared image is not affected by the coating. In this case, the full-image infrared image is directly defined as the adjusted infrared image to facilitate the subsequent determination of the final infrared image.

[0155] Step S3042: If it is not greater than, then analyze the window pixel value, the average window brightness, the number of window pixels, the pixel grayscale image, and the full-image infrared image to determine the infrared image to be adjusted.

[0156] If the average flatness is not greater than the standard flatness, it indicates that the temperature change detected by the Laplacian operator is very small, suggesting that the temperature distribution of the full-image infrared image is abnormally smooth. This indicates that the full-image infrared image is affected by the surface coating. In this case, the processing terminal analyzes the window pixel values, average window brightness, number of window pixels, pixel grayscale image, and full-image infrared image to determine the adjustment of the infrared image. The specific method is as follows: Figure 4 These steps ensure the high accuracy of the window status detection method.

[0157] Reference Figure 4 The steps for adjusting the infrared image are determined by analyzing the window pixel values, average window brightness, number of window pixels, pixel grayscale image, and full-image infrared image:

[0158] Step S400: Sort the window pixel values ​​and preset standard brightness values ​​to determine the number of brightness pixels.

[0159] The number of luminance pixels refers to the number of window pixel values ​​that are greater than the standard luminance value. It is obtained by the processing terminal sorting and counting the window pixel values ​​and the standard luminance value. By determining the number of luminance pixels, the proportion of bright pixels within the window area can be determined, thereby determining the degree of influence of the coating on the brightness of the entire visible light image, providing data support for subsequent determination and adjustment of the luminance index.

[0160] The standard brightness value is a threshold for measuring the normal brightness of the entire visible light image, and it is set in advance by the operator. By determining the standard brightness value, the proportion of bright pixels in the entire visible light image can be determined, which facilitates subsequent determination and adjustment of the brightness index.

[0161] Step S401: Calculate the quotient of the number of luminance pixels and the number of window pixels to generate an adjusted luminance index.

[0162] The brightness adjustment index measures the overall brightness intensity of the visible light image and is calculated by the processing terminal by dividing the number of brightness pixels by the number of window pixels. A higher brightness adjustment index indicates a brighter overall visible light image and a greater impact on the infrared image, providing data support for determining the intensity adjustment factor.

[0163] Step S402: Analyze the adjusted brightness index, the preset standard intensity factor, and the preset intensity adjustment range to determine the intensity adjustment factor.

[0164] The intensity adjustment factor refers to the factor that adjusts the intensity of the entire infrared image. It is obtained by the processing terminal after analyzing the adjustment brightness index, standard intensity factor, and intensity adjustment range, and can be expressed as follows: ,in Indicates the intensity adjustment factor. Indicates the standard strength factor. Indicates the intensity adjustment range. This indicates the adjustment of the brightness index. A higher brightness index indicates more severe reflection from the window surface, which has a greater impact on the overall infrared image. Therefore, a larger intensity adjustment factor is needed to provide data support for subsequent adjustments to the infrared image.

[0165] The standard intensity factor is a value used to measure the standard intensity of a full-image infrared image, which is set in advance by the operator. By determining the standard intensity factor, the intensity of the full-image infrared image can be enhanced when there is severe glare, while maintaining the intensity of the full-image infrared image unchanged when there is not severe glare, thus facilitating subsequent adjustments to the infrared image.

[0166] The intensity adjustment range refers to the maximum impact of reflectivity on the intensity correction of the entire infrared image, and is set in advance by the operator. By determining the intensity adjustment range, the maximum range that the entire infrared image can be adjusted when reflectivity is severe can be controlled, thus determining the final correction range.

[0167] Step S403: Obtain the background pixel value of the preset background area in the pixel grayscale image.

[0168] The background area refers to the region outside the window area in the pixel grayscale image, which is predetermined by the operator. By determining the background area, the pixel values ​​within it can be determined, thereby establishing the average brightness of the background area, which is then used to determine the offset adjustment factor.

[0169] Background pixel values ​​refer to the quantized brightness values ​​of the background region in a pixel grayscale image, which are read by the processing terminal from the pixel grayscale image information. By determining the background pixel values, the brightness of the background region can be determined, thereby identifying the brightness difference between the window area and the background area, providing data support for subsequently determining the average background brightness.

[0170] Step S404: Calculate the mean value of the background pixels to generate the mean background brightness.

[0171] Among them, the average background brightness value refers to the average brightness value of the background area, which is obtained by the processing terminal after calculating the average value of the background pixels. This determines the brightness difference between the window area and the background area, and then determines the overall offset of the infrared signal, so as to facilitate the subsequent determination of the offset adjustment factor.

[0172] Step S405: Analyze the average window brightness, the average background brightness, and the preset offset adjustment range to determine the offset adjustment factor.

[0173] The offset adjustment factor refers to the factor that adjusts the offset of the entire infrared image. It is obtained by the processing terminal after analyzing the average window brightness, average background brightness, and offset adjustment magnitude, and can be expressed as: ,in Indicates the offset adjustment factor. Indicates the offset adjustment range. This represents the average brightness of the window. This represents the average background brightness. The greater the difference between the average window brightness and the average background brightness, the greater the brightness difference between the window area and the background area. This results in a greater overall offset of the infrared signal, and thus a larger offset adjustment factor to counteract the effects of the infrared signal offset.

[0174] The offset adjustment range refers to the maximum impact of brightness differences on the offset correction of the entire infrared image, and is set in advance by the operator. By determining the offset adjustment range, the maximum impact on the offset correction of the entire infrared image when brightness differences are large can be controlled, which facilitates subsequent determination of the infrared image adjustment.

[0175] Step S406: Analyze the intensity adjustment factor, the full-image infrared image, and the offset adjustment factor to determine the adjustment of the infrared image.

[0176] After determining the offset adjustment factor, the processing terminal analyzes the intensity adjustment factor, the full-image infrared image, and the offset adjustment factor to obtain the adjusted infrared image, which can be represented as follows: ,in This indicates adjustment of the infrared image. Indicates the intensity adjustment factor. Represents a full-image infrared image. This indicates the offset adjustment factor. The larger the intensity adjustment factor and offset adjustment factor, the greater the influence of the window surface coating on the full-image infrared image. In this case, the adjustment range of the full-image infrared image is larger, thereby offsetting the temperature offset caused by the coating.

[0177] Reference Figure 5 The steps for analyzing adjusted infrared images to determine the maximum temperature difference and crack detection results include:

[0178] Step S500: Obtain the infrared temperature value of the adjusted infrared image and the neighborhood temperature value of the preset infrared image neighborhood.

[0179] The infrared temperature value refers to the temperature value corresponding to each pixel in the adjusted infrared image, which is read by the processing terminal from the adjusted infrared image information. By determining the infrared temperature value, the temperature value corresponding to each pixel in the adjusted infrared image can be determined, thereby determining whether the temperature on the window surface is uniformly distributed, and further determining whether there are cracks on the window surface, providing data support for subsequent determination of infrared temperature difference.

[0180] Infrared image neighborhood refers to the neighborhood corresponding to each pixel in an infrared image, which is pre-set by the operator. By determining the infrared image neighborhood, the difference between the temperature value of each pixel and the temperature value of its neighbors can be captured, thus reflecting this local anomaly and facilitating subsequent determination of crack detection results.

[0181] Neighborhood temperature value refers to the temperature value corresponding to each pixel in the neighborhood of an infrared image, which is read by the processing terminal from the adjusted infrared image information. By determining the neighborhood temperature value, the median temperature value within the neighborhood can be determined, thereby selecting a value that can represent the average temperature of the neighborhood, so as to facilitate the subsequent determination of the infrared temperature difference.

[0182] Step S501: Sort the neighborhood temperature values ​​to determine the median temperature value.

[0183] The median temperature value refers to the median of the neighborhood temperature values, which is obtained by sorting the neighborhood temperature values ​​and taking the median after processing. By determining the median temperature value, a value that can represent the average temperature of the neighborhood can be selected, thus avoiding the influence of extreme temperature values ​​when taking the average.

[0184] Step S502: Calculate the absolute value of the difference between the infrared temperature value and the median temperature value to generate the infrared temperature difference value.

[0185] The infrared temperature difference refers to the difference between the temperature of a pixel and the average temperature of its neighborhood. It is obtained by the processing terminal calculating the absolute value of the difference between the infrared temperature value and the median temperature value. By determining the infrared temperature difference, the degree of deviation between the pixel's temperature and the temperature of its neighborhood can be determined. Based on the degree of deviation, it can be determined whether there are cracks on the window surface, so as to facilitate the subsequent determination of crack detection results.

[0186] Step S503: Sort the infrared temperature differences to determine the maximum temperature difference.

[0187] In this process, after determining the infrared temperature difference values, the processing terminal sorts the infrared temperature differences and takes the maximum value as the maximum temperature difference. By determining the maximum temperature difference, the maximum temperature difference between a pixel and its neighborhood in the adjusted infrared image can be determined, thereby determining whether there are cracks on the window surface.

[0188] Step S504: Determine whether the maximum temperature difference is greater than the preset standard temperature difference.

[0189] The standard temperature difference refers to a standard threshold for measuring the temperature difference between a pixel and its neighborhood, which is set in advance by the operator. By determining whether the maximum temperature difference is greater than the standard temperature difference, it can be determined whether there is an anomaly in the local temperature of the adjusted infrared image, thereby determining whether there are cracks on the window surface.

[0190] Step S5041: If it is greater than the preset crack result, then define the crack detection result as the crack detection result.

[0191] If the maximum temperature difference is greater than the standard temperature difference, it means that the local temperature difference in the adjusted infrared image is greater than the standard threshold. In this case, the result with cracks is directly defined as the crack detection result to improve the safety of window use.

[0192] Step S5042: If it is not greater than, then the preset no-crack result is defined as the crack detection result.

[0193] If the maximum temperature difference is not greater than the standard temperature difference, it means that the local temperature difference in the adjusted infrared image is not significant. In this case, the result without cracks is directly defined as the crack detection result, so as to facilitate the subsequent determination of the final infrared image.

[0194] Reference Figure 6 The steps to determine the final infrared image include analyzing the average window brightness, maximum temperature difference, pixel grayscale image, and adjusted infrared image.

[0195] Step S600: Calculate the gradient magnitude of the pixel grayscale image to generate a gradient magnitude image.

[0196] The gradient magnitude image is an image that displays the intensity of brightness changes at each pixel in the grayscale image. It is obtained by the processing terminal calculating the gradient magnitude of the pixel grayscale image. By determining the gradient magnitude image, the intensity of brightness changes at each location in the entire visible light image can be determined, thereby determining whether the texture of the window surface is complex, which is helpful for subsequently determining the final infrared image.

[0197] Step S601: Obtain the gradient pixel values ​​of the gradient magnitude image.

[0198] The gradient pixel value refers to the pixel value in the gradient magnitude image, which is read by the processing terminal from the gradient magnitude image information. By obtaining the gradient pixel value, the intensity of brightness change at each location in the entire visible light image can be determined, providing data support for subsequent calculation of the gradient average value.

[0199] Step S602: Calculate the mean of the gradient pixel values ​​to generate the gradient average value.

[0200] The gradient average value refers to the average value of the gradient pixels, which is calculated by the processing terminal. A higher gradient average value indicates a more complex texture on the window surface, suggesting the presence of decorations on the window surface, which helps in determining the final infrared image.

[0201] Step S603: Determine whether the average gradient value is less than the preset standard average gradient value.

[0202] The standard gradient mean is a standard threshold for measuring the complexity of the window surface texture, which is set in advance by the operator. By determining whether the average gradient is less than the standard gradient mean, the complexity of the window surface texture can be determined, thereby identifying whether there are decorative elements obstructing the window surface, thus ensuring the high accuracy of the window condition detection method.

[0203] Step S6031: If it is less than, then the adjusted infrared image is defined as the final infrared image.

[0204] If the average gradient is less than the average standard gradient, it means that the texture complexity of the window is within the normal range and there are no decorative obstructions on the window surface. The adjusted infrared image is then directly defined as the final infrared image to ensure the high accuracy of the window state detection method.

[0205] Step S6032: If it is not less than, then analyze the average window brightness, the adjusted infrared image, and the maximum temperature difference to determine the final infrared image.

[0206] If the average gradient is not less than the standard average gradient, it indicates that the texture complexity of the window is outside the normal range. In this case, there may be decorative obstructions on the window surface. Therefore, the processing terminal analyzes the average window brightness, adjusted infrared image, and maximum temperature difference to determine the final infrared image. The specific method is described in [reference needed]. Figure 7 This process eliminates the inaccuracy in adjusting infrared images caused by obstructions from decorations.

[0207] Reference Figure 7 The steps for determining the final infrared image by analyzing the average window brightness, adjusting the infrared image, and the maximum temperature difference include:

[0208] Step S700: Determine whether the maximum temperature difference is less than the preset reference temperature difference.

[0209] The reference temperature difference is a threshold value set in advance by the operator to measure whether there are decorative obstructions on the window surface. By determining whether the maximum temperature difference is less than the reference temperature difference, it can be determined whether there are pixels outside the normal temperature difference range, and thus whether there are decorative obstructions on the window surface.

[0210] Step S7001: If it is less than, then the adjusted infrared image is defined as the final infrared image.

[0211] If the maximum temperature difference is less than the reference temperature difference, it means that the temperature difference of the adjusted infrared image is within the temperature variation range of the normal window texture. This indicates that the normal temperature difference is generated by the texture of the window itself. In this case, the adjusted infrared image is directly defined as the final infrared image to facilitate the subsequent determination of global fusion features.

[0212] Step S7002: If it is not less than, then determine the reference brightness value according to the correspondence between the average window brightness and the preset brightness threshold.

[0213] If the maximum temperature difference is not less than the reference temperature difference, it means that the temperature difference of the adjusted infrared image is not within the temperature change range of the normal window texture. This indicates that there is a decorative object blocking the window surface. At this time, the processing terminal determines the reference brightness value based on the correspondence between the average window brightness and the brightness threshold, thereby determining whether the decorative object on the window surface is a light-absorbing material or a reflective material, so as to facilitate the subsequent determination of the final infrared image.

[0214] The brightness threshold correspondence refers to the correspondence between the average brightness of a window and the standard brightness threshold. For example, when the average brightness of a window is very low, the decoration is made of light-absorbing material, and the corresponding reference brightness value is 0.1. The operator will form a mapping table by matching the average brightness of the window with the reference brightness value one by one.

[0215] The baseline brightness threshold refers to the brightness threshold at which the decorative object is a light-absorbing or reflective material in the full visible light image. It is obtained by the processing terminal by looking up the corresponding brightness threshold in a mapping table based on the average brightness of the window. By determining the baseline brightness threshold, it is possible to determine whether the decorative object is a light-absorbing or reflective material, thereby determining the final infrared image and ensuring high accuracy in window status detection.

[0216] Step S70021: Determine whether the average brightness of the window is greater than the reference brightness value.

[0217] In this process, by determining whether the average brightness of the window is greater than the reference brightness value, it is possible to determine whether the window surface decoration is a light-absorbing or light-reflecting material, and then determine the influence of the decoration material on adjusting the infrared image, so as to facilitate the subsequent determination of the final infrared image.

[0218] Step S700211: If it is greater than, calculate the product of the adjusted infrared image and the preset reflection correction factor to generate the final infrared image.

[0219] If the average brightness of the window is greater than the reference brightness value, it means that the window area is very bright in the visible light image of the whole picture. In this case, it means that the decorative material is a reflective material. The processing terminal calculates the product of the adjusted infrared image and the reflectivity correction factor to generate the final infrared image. In this way, the adjusted infrared image is corrected according to the reflectivity of the decorative material, thereby ensuring the high accuracy of the window status detection method.

[0220] The reflection correction factor refers to the degree of correction applied to the infrared image based on the reflectivity of the decoration, and is preset by the operator. By determining the reflection correction factor, the temperature rise in the adjusted infrared image caused by the reflection of the decoration can be eliminated, thereby determining the final infrared image.

[0221] Step S700212: If it is not greater than, calculate the product of the adjusted infrared image and the preset light absorption correction factor to generate the final infrared image.

[0222] If the average brightness of the window is not greater than the reference brightness value, it means that the window area in the visible light image is very dark. In this case, it means that the decorative material is a light-absorbing material. The processing terminal calculates the product of the adjusted infrared image and the light absorption correction factor to generate the final infrared image. Thus, the adjusted infrared image is corrected according to the light absorption degree of the decorative material, thereby ensuring the high accuracy of the window status detection method.

[0223] The light absorption correction factor refers to the degree of correction applied to the infrared image based on the light absorption of the decoration, and is preset by the operator. By determining the light absorption correction factor, the phenomenon of temperature reduction in the adjusted infrared image caused by light absorption by the decoration can be eliminated, thereby determining the final infrared image.

[0224] Based on the same inventive concept, embodiments of this application provide a window status detection system, including:

[0225] The acquisition module is used to acquire system trigger signals, location prompt coordinates, visible light pixel values, pixel count values, window pixel values, background pixel values, infrared temperature values, neighborhood temperature values, and gradient pixel values.

[0226] Memory for storing a program for a window state detection method;

[0227] The processor can load and execute programs in memory and implement a window state detection method.

[0228] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0229] This application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed as a window state detection method.

[0230] Computer storage media include, for example, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code.

[0231] Based on the same inventive concept, embodiments of this application provide a smart terminal, including a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as a window state detection method.

[0232] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0233] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A method for detecting the state of a window, characterized in that, include: Obtain system trigger signals; The system triggers a pre-set dual-mode camera to acquire full-view visible light and full-view infrared images. The visible light image and the infrared image of the entire image are analyzed to determine the final infrared image; Get the coordinates of the preset four corner points; The location cue coordinates, the full-image visible light image, and the final infrared image are input into a pre-defined convolutional neural network to generate location cue features and global fusion features; The location cue features and global fusion features are analyzed to determine the region enhancement features; The region enhancement features are input into a preset state classification network to generate the current window state; The steps for analyzing the full-image visible light image and the full-image infrared image to determine the final infrared image include: Obtain the visible light pixel values ​​and pixel count values ​​in the entire visible light image; The visible light pixel values ​​are weighted and summed according to the preset channel brightness weight parameters to generate grayscale pixel values; Analyze the grayscale pixel values ​​and the number of pixels to generate a pixel grayscale image; Analyze the pixel grayscale image and the full-image infrared image to determine the average window brightness and adjust the infrared image; The adjusted infrared images were analyzed to determine the maximum temperature difference and crack detection results; The crack detection result is determined to be either a preset result indicating the presence of a crack or a preset result indicating the absence of a crack. If the result shows cracks, a damage warning message will be generated; If the result is without cracks, the average window brightness, maximum temperature difference, pixel grayscale image, and adjusted infrared image are analyzed to determine the final infrared image. The steps for analyzing pixel grayscale images and full-image infrared images to determine the average window brightness and adjust the infrared image include: Obtain the window pixel value and window pixel count value of a preset window region in the pixel grayscale image; Calculate the mean of the window's pixel values ​​to generate the mean window brightness; Calculate the Laplacian operator for the full-image infrared image to generate infrared image flatness; Calculate the variance of the infrared image flatness to generate the average flatness; Determine whether the average flatness is greater than the preset standard flatness; If it is greater than that, then the full-image infrared image is defined as the adjusted infrared image; If it is not greater than, then analyze the window pixel value, window average brightness value, window pixel count value, pixel grayscale image and full-image infrared image to determine the infrared image to be adjusted; The steps for adjusting the infrared image are determined by analyzing window pixel values, average window brightness, number of window pixels, pixel grayscale images, and the full-image infrared image: The window pixel values ​​and preset standard brightness values ​​are sorted to determine the number of brightness pixels; Calculate the quotient of the number of brightness pixels to the number of window pixels to generate an adjustment brightness index; The brightness index, preset standard intensity factor, and preset intensity adjustment range are analyzed to determine the intensity adjustment factor. Obtain the background pixel value of a preset background area in a pixel grayscale image; Calculate the mean value of the background pixels to generate the mean background brightness. The average window brightness, average background brightness, and preset offset adjustment range are analyzed to determine the offset adjustment factor. The intensity adjustment factor, the full-image infrared image, and the offset adjustment factor were analyzed to determine the adjustment of the infrared image.

2. The window status detection method according to claim 1, characterized in that, The steps for analyzing adjusted infrared images to determine the maximum temperature difference and crack detection results include: Obtain the infrared temperature value of the adjusted infrared image and the neighborhood temperature value of the preset infrared image neighborhood; Sort the neighborhood temperature values ​​to determine the median temperature value; Calculate the absolute value of the difference between the infrared temperature value and the median temperature value to generate the infrared temperature difference value; The infrared temperature differences are sorted to determine the maximum temperature difference. Determine whether the maximum temperature difference is greater than the preset standard temperature difference. If the result is greater than the preset value, the result with cracks will be defined as the crack detection result. If the result is not greater than the preset no-crack result, then the crack detection result will be defined as the crack detection result.

3. The window status detection method according to claim 1, characterized in that, The steps to determine the final infrared image by analyzing the average window brightness, maximum temperature difference, pixel grayscale image, and adjusted infrared image include: Calculate the gradient magnitude of the pixel grayscale image to generate a gradient magnitude image; Obtain the gradient pixel values ​​of the gradient magnitude image; Calculate the mean of the gradient pixel values ​​to generate the gradient average; Determine whether the average gradient value is less than the preset standard gradient mean; If it is less than, then the adjusted infrared image will be defined as the final infrared image; If it is not less than, then the average window brightness, adjusted infrared image, and maximum temperature difference are analyzed to determine the final infrared image.

4. The window status detection method according to claim 3, characterized in that, The steps to determine the final infrared image by analyzing the average window brightness, adjusting the infrared image, and the maximum temperature difference include: Determine whether the maximum temperature difference is less than the preset reference temperature difference; If it is less than, then the adjusted infrared image will be defined as the final infrared image; If it is not less than, then the reference brightness value is determined according to the correspondence between the average window brightness and the preset brightness threshold. Determine whether the average brightness of the window is greater than the reference brightness value; If it is greater than the preset reflection correction factor, the product of the adjusted infrared image and the preset reflection correction factor is calculated to generate the final infrared image. If it is not greater than, then calculate the product of the adjusted infrared image and the preset light absorption correction factor to generate the final infrared image.

5. A window status detection system, characterized in that, include: The acquisition module is used to acquire system trigger signals and location prompt coordinates; A memory for storing a program for a window state detection method as described in any one of claims 1 to 4; The processor and the program in the memory can be loaded and executed by the processor to implement the window state detection method as described in any one of claims 1 to 4.

6. A smart terminal, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1 to 4.

Images