A solar heat collecting device operation state recognition method based on a structure perception segmentation network

Abstract

This invention provides a method for recognizing the operating status of solar thermal collectors based on a structure-aware segmentation network, belonging to the field of solar thermal equipment detection technology based on computer vision. First, it acquires equipment images and video data under different operating conditions through multi-source visual acquisition methods, and constructs a multidimensional dataset containing structural semantic labels, status labels, and temporal labels. Then, it builds a structure-aware semantic segmentation model based on an improved DeepLab network, and achieves high-precision segmentation of key structural regions of the thermal collector by introducing structural prior coding and structural constraint attention mechanisms. On this basis, it combines a temporal consistency discrimination module to model structural changes in continuous video frames, achieving dynamic recognition of states such as occlusion, offset, damage, and abnormal installation angles. Finally, it fuses the segmentation results and status discrimination results to output the equipment operating status and anomaly type, realizing automated detection and analysis.

Description

Technical Field

[0001] This invention belongs to the field of solar energy operation equipment detection technology based on computer vision, and particularly relates to a method for identifying the operating status of solar thermal collectors based on structure-aware segmentation networks. Background Technology

[0002] With the rapid development of renewable energy technologies, solar thermal utilization technology has been widely applied in industrial heating, district heating, and solar thermal power generation. As a core component of such systems, the operating status of solar thermal collectors directly affects the overall energy efficiency and safety of the system. In practical engineering applications, collectors typically consist of various structural units such as collector tubes, reflectors, supports, and connecting components. These structures are susceptible to environmental factors (such as wind, sand, rain, snow, and temperature variations) and differences in construction quality during long-term operation, leading to problems such as shading, misalignment, structural damage, or abnormal installation angles. Currently, the detection of the operating status of solar thermal collectors mainly relies on manual inspection or traditional image processing methods. Manual inspection suffers from low efficiency, strong subjectivity, and difficulty in achieving continuous monitoring, making it difficult to meet the intelligent operation and maintenance needs of large-scale solar power plants. On the other hand, traditional visual inspection technologies based on edge detection and threshold segmentation have poor adaptability to complex environments. In situations with drastic changes in lighting, complex background interference, or slender and highly similar equipment structures, false positives and false negatives are prone to occur.

[0003] In recent years, with the development of deep learning technology, semantic segmentation methods have been widely used in the field of industrial visual inspection. Convolutional neural network models, such as DeepLab, can achieve pixel-level target recognition, improving detection accuracy to a certain extent. However, most existing methods focus on learning texture and appearance features, lacking the ability to model equipment structural information and failing to fully utilize the inherent geometric and topological relationships between the components of solar thermal collectors. For example, collector tubes are usually arranged in parallel, and there is a fixed connection between the reflector and the support; these structural priors are not effectively utilized in traditional deep learning models. Furthermore, most existing methods are based on single-frame image analysis, lacking the utilization of temporal information and making it difficult to dynamically model changes in equipment state. In real-world scenarios, problems such as short-term occlusion and slight equipment displacement often require the combination of continuous frame information for judgment; relying solely on single-frame results can easily lead to misjudgments. Therefore, how to integrate structural and temporal information on the basis of semantic segmentation to achieve accurate identification of the operating state of solar thermal collectors has become an urgent technical problem to be solved. Summary of the Invention

[0004] To address the above problems, this invention proposes a method for identifying the operating status of solar thermal collectors based on a structure-aware segmentation network, comprising the following steps: S1: Collect image sequence data during the operation of the solar thermal collector and preprocess it to obtain normalized images; obtain the structural topology adjacency matrix by constructing the structural topology relationship of the equipment and calculating the distance between the structural centers. S2, based on the constructed structure-aware segmentation network, normalized image input encoder is used for feature extraction to obtain image features, structure topological adjacency matrix is ​​input into structure prior encoding module to generate structure features, the two features are fused to obtain structure-enhanced features, and input into decoder for pixel-by-pixel classification, outputting structure-enhanced semantic segmentation results; S3, based on the constructed state discrimination model, converts the semantic segmentation results into a deterministic category label map, and aggregates the categories to obtain category regions. It calculates the spatial center position of the category region and concatenates it with the area feature to obtain the structural feature vector. The vector is then input into the temporal modeling network module to obtain temporal features. The structural change between adjacent frames is introduced and combined with the temporal features to construct a comprehensive feature vector. Finally, the vector is input into the state classification network to output the device operation state discrimination result.

[0005] Preferably, after outputting the device's operating status, the result is corrected based on structural constraint rules, specifically as follows: By comparing the changes in area, position, and relative spatial relationships of structural regions in consecutive frames, the following discrimination criteria are constructed: When the area of ​​a certain structural region decreases by more than a certain amount compared to the previous frame, it is determined to be an occlusion state; When the change in the center position of the structure exceeds a preset threshold, it is determined to be a structural offset state. When a structural region becomes discontinuous or disappears, it is determined to be in a state of structural damage. When the relative positional relationship between multiple structures changes, it is determined to be an abnormal installation angle state.

[0006] Preferably, the process also includes the following: Combining the obtained state discrimination result and the semantic segmentation result obtained in S2 as input, the device operating state is fused and expressed and the result is output, including: classifying each pixel position to obtain a structural segmentation mask, then making a category decision to obtain the final state label; then fusing the segmentation mask and the final state label to obtain a single frame output result; finally, performing temporal state statistical analysis, counting the number of device state transitions, and generating a device operating state report.

[0007] Preferably, the structural topological adjacency matrix is ​​obtained in the following way: First, pixel-level semantic segmentation and annotation are performed on the image to divide the device into four types of structural regions. The area includes the heat collection tube, the reflector surface, the support frame, and the connecting components. Extract the connected regions for each category and calculate the geometric center position for each structure instance. The geometric center is obtained by averaging the coordinates of all pixels within the region. Subsequently, a structural topology diagram is constructed. , The set of nodes; each node corresponds to a specific physical structural component, and in the image, it is an independent structural unit corresponding to a spatially connected pixel region, and the set of edges. Generated by distance constraints, the device topology is constructed based on the spatial distance relationships between structures, defining any two nodes. and , and Let the coordinates of the geometric centers of the two structural instances be given, and calculate the Euclidean distance between the two structural centers. ; when Less than the preset threshold When two structures are considered to be connected, an edge connection is established in the topological graph and set to 1; otherwise, it is set to 0. This process ultimately yields the structural topological adjacency matrix. .

[0008] Preferably, the encoder in the structure-aware segmentation network employs an improved DeepLab backbone network for normalized images. Perform layer-by-layer convolution processing to extract deep semantic features; First, multi-scale contextual information is obtained through dilated convolutional structures to obtain the initial feature representation. A convolutional thinning module is connected in series after the output of the DeepLab backbone network to refine the output. Further feature transformations are performed to obtain the final encoded image features. ; The convolutional thinning module consists of three concatenated convolutional layers, each with a kernel size of [size missing]. The convolution stride is 1, and each convolutional layer is followed by a non-linear mapping using the ReLU activation function. The encoder introduces a voided convolution structure after the second and third convolution layers, with different void ratios set for each layer. This allows the convolution kernel to expand the receptive field without increasing the number of parameters, thus sensing the slender structure of the heat collection tube and the wide area of ​​the reflective surface.

[0009] Preferably, the specific data processing procedure of the structure prior coding module is as follows: structural topology matrix Feature encoding is performed using three fully connected layers, with ReLU activation applied after each fully connected layer to output the final structural feature vector. During the processing, learnable parameters are used to map topological relationships, enabling the connections between structures to be expressed in vector form. The structural features are calculated as follows: ,in For structural encoding parameters, It is a nonlinear mapping function; Subsequently, through the mapping function Structural features Mapping to image features The same feature space.

[0010] Preferably, the mapping function The feature transformation module, composed of learnable parameters, is implemented through a fully connected layer and tensor reconstruction operations: First, the structural feature vectors are transformed... A fully connected mapping layer is input, and its channel dimensions are linearly transformed to obtain an intermediate feature vector. Subsequently, this intermediate feature vector is reconstructed into a shape similar to the image features through a dimension expansion operation. The feature representation is consistent across the channel dimension and replicated and expanded in the spatial dimension to make its size consistent with... The spatial resolution is consistent.

[0011] Preferably, the specific data processing procedure in the time-series modeling stage of the state discrimination model is as follows: First, the semantic segmentation result sequence for each frame. At each pixel position The image is discretized, and a maximum probability selection operation is performed to obtain a discrete label map. : ; Indicates pixel position Category tags, The category with the highest probability is selected as the final category label; Secondly, for each category Extract its corresponding spatial region, and then select all regions that satisfy the condition that the category label equals the category. The pixels are represented as a set: ; That is to say, the first Frame Category The corresponding complete spatial region; Finally, for each category of area The spatial center position is calculated by averaging the coordinates of all pixels within the region. The center position coordinates are obtained by averaging the x-coordinate and y-coordinate respectively. ; Simultaneously, the number of pixels in the category region is used as the area feature of the structural region for statistical analysis to obtain the region area. ; After calculating all categories, the center position and area features of each category are concatenated in category order to construct the structural feature vector of the current frame. .

[0012] Preferably, the temporal modeling network module employs a three-layer cascaded long short-term memory network; firstly, the structural feature vectors are... The information is fed into the first LSTM layer as input. This layer filters and updates the historical information through input gates, forget gates, and output gates, and outputs the first layer hidden state. Its input dimension is The hidden layer dimension is set to ; then, Input the second LSTM layer to obtain the second hidden state, with the hidden layer dimension set to 1. This is used to further extract time-dependent features, and the output is... Finally, Input a third LSTM layer, with hidden layer dimensions set to... Output the final time series features .

[0013] Preferably, for each pixel position Category determination: ; in, Represents a structural segmentation mask. Represents pixels The structural category label; The category with the highest probability is selected as the final category label; Indicates category; The current frame state is determined using the maximum probability principle: ; in, This represents the final state label for frame t. Indicates belonging to a state The probability of; Segmentation mask With final state label The components are then fused to create a single-frame output: .

[0014] Compared with the prior art, the present invention has the following innovative features: (1) Introducing a structure-aware mechanism, the geometric topological relationship of the solar thermal collector is embedded into the semantic segmentation network as prior information, realizing the transformation from "pure data-driven" to "structure-guided modeling", effectively improving the segmentation accuracy of slender structures and complex connection areas; (2) Design a structural constraint attention mechanism to strengthen structural consistency constraints during feature reconstruction and improve the model's ability to express structural integrity; (3) A state discrimination method based on temporal consistency is proposed, which introduces video sequence information into the device state recognition process, thereby extending from single-frame static analysis to multi-frame dynamic analysis; (4) Achieve joint modeling of segmentation and state recognition, so that the model can not only identify "what kind of structure" but also determine "what state it is in", thereby enhancing the practicality and engineering adaptability of the method.

[0015] The beneficial effects of this invention include: (1) By fusing structural information and depth features, the segmentation accuracy and robustness under complex lighting and occlusion conditions are improved, and the false detection rate and false detection rate are reduced. (2) By introducing a time series analysis mechanism, it is possible to effectively identify short-term changes and gradual anomalies, thereby improving the accuracy and stability of state identification; (3) This method realizes automated visual inspection of solar thermal collectors, reduces reliance on manual inspection, and significantly improves inspection efficiency; (4) It can provide reliable data support for equipment construction quality assessment and operation and maintenance, which helps to improve the overall operation safety and energy efficiency of the system and has good engineering application value and promotion prospects. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the following description is only one embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the overall technical route of the present invention.

[0018] Figure 2 This is a structural diagram of the structure-aware semantic segmentation network of the present invention.

[0019] Figure 3 This is a structural diagram of the time-series modeling network module of the present invention.

[0020] Figure 4 This is a comparison chart showing the improvement in segmentation accuracy brought about by structure perception in the embodiments.

[0021] Figure 5 This is a diagram showing the segmentation performance changes of key structures in consecutive frames in the embodiment.

[0022] Figure 6 The diagram shows the continuous frame segmentation performance and state probability of the key structure in the embodiment. Detailed Implementation

[0023] To achieve real-time and accurate identification and intelligent decision-making regarding steel plate surface defects, this invention proposes a method for identifying the operating status of solar thermal collectors based on a structure-aware segmentation network. The overall process is as follows: Figure 1 As shown: S1, Multi-source Scene Data Acquisition and Annotation Construction. First, multi-source visual data acquisition is conducted during the actual operation of the solar thermal collector. This includes high-resolution image sequence data of the solar thermal utilization equipment. and operating conditions Subsequently, the collected data was finely labeled to obtain normalized images. The structural topological adjacency matrix is ​​obtained by calculating the distance between the structural centers. Based on this, the device status change information is further labeled on the time series data to form a dataset for subsequent model training.

[0024] S2, Construction of the Structure-Aware Semantic Segmentation Network. In the model design phase, based on the DeepLab semantic segmentation network framework, a structure-aware mechanism is introduced to construct the Structure-Aware Segmentation Network (SAS-Net). Specifically, using... and As input, in the encoder section, the normalized image is... Multi-scale image features are extracted, and dilated convolution is introduced to enhance the model's ability to perceive targets of different sizes, outputting multi-scale feature maps. In the structural prior encoding module, based on the structural topological adjacency matrix... This maps structural features to a feature map space, and then compares them with the encoder's output. Alignment and fusion yield structural enhancement features .

[0025] During the decoding phase, structurally enhanced features are applied through a structurally constrained attention mechanism. Weighted reconstruction is performed to enable the model to incorporate device structural consistency during pixel-level classification, resulting in structure-enhanced, high-precision semantic segmentation results. .

[0026] S3, a state discrimination model based on temporal consistency is constructed to obtain high-precision semantic segmentation results. Then, a temporal consistency discrimination model is constructed to dynamically analyze the device's operating status. First, the high-precision semantic segmentation results are converted into deterministic category label maps. And collect the categories to obtain Secondly, its spatial center position is calculated and concatenated with the area feature to obtain the structural feature vector. Then, a temporal feature sequence is constructed. Next, an LSTM module is introduced to construct state discrimination feature vectors. Finally, the integrated feature vectors will be... An input state classification network is used to determine the operating status of the device. Combined with pre-defined structural constraint rules, it corrects for abnormal states and ultimately outputs the device operating status determination result. .

[0027] S4, Operational Status Identification and Result Output. This is combined with the obtained status determination results. The high-precision semantic segmentation result obtained in S2 As input, the device operating status is fused and expressed, and the result is output. This includes using the same discretization method as in S3 to classify each pixel location to obtain a structural segmentation mask. Secondly, a category decision is made to obtain the final state label. Next, the segmentation mask and the final state label are fused to obtain the single-frame output result. Finally, a time-series status statistical analysis is performed to count the number of device state transitions and generate a device operating status report R.

[0028] The implementation process of the present invention will be further described below with reference to specific embodiments.

[0029] S1. Multi-source scene data acquisition and annotation construction S1-1 Multi-source visual data acquisition. Fixed monitoring cameras are deployed at the target site, and mobile inspection and acquisition terminals are configured to simultaneously acquire data from different perspectives on the same device. The resulting set of image data is defined as... The image is an RGB image. It is a visual image of the solar thermal collector during operation, and video sequence data is constructed based on consecutive frame images.

[0030] During image acquisition, scene conditions are strictly labeled, and a set of condition categories is defined. The conditions are categorized into five types: sunny, cloudy, direct sunlight, backlighting, and occluded scenes. Each frame is uniquely assigned to a specific condition category for subsequent data analysis and to improve the model's generalization capabilities.

[0031] Simultaneously, the operating status of the equipment is recorded synchronously, and status categories are defined. The states are: normal state, obstructed state, structural offset state, structural damage state, and abnormal installation angle state.

[0032] The operating condition category and the status category are combined to obtain the status label. Each frame of the image is assigned a corresponding state label, thus completing the initial association between the raw data and state information. The final result is a raw dataset with condition labels and state categories. .

[0033] S1-2 Data Preprocessing and Standardization. Based on the collected raw dataset... After image normalization, a standardized image set is obtained. Specifically: All images are resampled to a fixed resolution of 512*512 to ensure consistency of subsequent network inputs. The images are then normalized using the following standardization formula: .in, This represents the three-channel mean vector. This is the standard deviation vector for the three channels. This is the normalized image.

[0034] S1-3 Structural semantic annotation and topological relation construction. Based on the obtained normalized image. As input, semantic annotation is performed, and the output is a semantic segmentation label graph and structural topology relationship data.

[0035] First, pixel-level semantic segmentation and annotation are performed on the image to divide the device into four types of structural regions. The areas include the heat collection pipe area, the reflector surface area, the support area, and the connecting component area. A pixel-level label mapping function is constructed. , This represents the set of image pixel coordinates, assigning a unique category label to each pixel. The label also includes the background region, thus obtaining the semantic segmentation label set. .

[0036] Based on structural semantic annotation, the topological relationships of device structures are constructed. Connected regions for each category are extracted, and the geometric center of each structural instance is calculated. The geometric center is obtained by averaging the coordinates of all pixels within the region and is used to describe the spatial location of the structure in the image.

[0037] Subsequently, a structural topology diagram is constructed. , This is a set of nodes. Each node corresponds to a specific physical structural component, and in the image, it is an independent structural unit corresponding to a spatially connected pixel region. The set of edges... Generated through distance constraints. The device topology is constructed based on the spatial distance relationships between structures. Any two nodes are defined. and , and Let represent the coordinates of the geometric centers of the two structural instances. Calculate the Euclidean distance between their two structural centers. .

[0038] When distance Less than the preset threshold If a connection is found between two structures, an edge connection is established in the topological graph and set to 1; otherwise, it is set to 0. The final structure topological adjacency matrix is ​​obtained. , used to describe the spatial relationships between the various components of a device.

[0039] S1-4 Temporal State Labeling and Multidimensional Dataset Construction. This step is based on normalized images. Data and its semantic segmentation label set and structural topological adjacency matrix and status labels The training dataset is then constructed.

[0040] First, the data is organized chronologically by sorting consecutively acquired image frames according to their time index, forming an ordered sequence that enables the normalized images to be processed. Data and its semantic segmentation label set Structural topological adjacency matrix and status labels Strictly aligned in the time dimension.

[0041] Secondly, consistency checks are performed on adjacent frames in the time dimension. For any adjacent frame, the semantic segmentation results are compared. and The structural regions are analyzed to determine whether significant structural changes have occurred. Specifically, based on the semantic segmentation results, the first... Frame and the The area and geometric center location of each type of structural region in the frame, where the first... Class structure in the first The area in the frame is denoted as In the The area in the frame is denoted as The corresponding structural center positions are denoted as follows: and .

[0042] Based on this, structural changes are quantitatively described by calculating the degree of area and spatial position change of the same type of structure between adjacent frames. Specifically, the degree of area change is determined by comparing... and The magnitude of the numerical difference is used to measure the change in spatial position, while the degree of change in spatial position is measured by comparing the Euclidean distance between the two central positions.

[0043] When the area change of any type of structure exceeds a preset ratio threshold Or its center position offset exceeds a preset distance threshold. If the frame structure changes significantly, it is determined that the frame structure has changed; otherwise, the structure is considered stable. This represents the sensitivity threshold to area changes, a criterion used to control changes in structural dimensions. This represents the threshold for spatial position change, used as a criterion for controlling structural positional deviation.

[0044] When the semantic segmentation result remains stable, its corresponding state label Maintain consistency; preserve the original state labels when significant changes occur in the structural region. Specifically, retain the state labels assigned to each frame during the data acquisition and manual annotation stages. and No modification is made, meaning the labels are not overwritten based on adjacent frame results, thus avoiding incorrect label corrections due to structural abrupt changes; conversely, when the segmentation result is determined to have changed significantly, the label is modified accordingly. The status label of the frame is covered by the first The frame's status label ensures label stability.

[0045] Finally, the aligned image data, semantic labels, structural topological relationships, and state labels are uniformly encapsulated to construct a multidimensional dataset. First of all, with Input data into the model, To output data to the model, a training dataset is constructed for the S2 structure-aware semantic segmentation network. Secondly, with Input data into the model, Output data to the model to obtain the training dataset for the S3 state discrimination model. .

[0046] S2. Construction of Structure-Aware Semantic Segmentation Network This step is based on the training dataset obtained in S1. Based on the DeepLab network, improvements were made by introducing prior structural information to construct the structure-aware semantic segmentation network SAS-Net. The network structure is as follows: Figure 2 As shown, the model possesses structural understanding capabilities while performing pixel-level classification.

[0047] S2-1 Network Overall Structure Design. The SAS-Net consists of three parts: an encoder, a structure prior encoding module, and a decoder. First, the multidimensional dataset obtained in S1... Normalized image in The input encoder is used for feature extraction to obtain image features. At the same time, the structural topological adjacency matrix Input the structure prior encoding module to generate structural features The two features are then fused at the feature level and input into the decoder for pixel-by-pixel classification, ultimately outputting a structure-enhanced, high-precision semantic segmentation result. .

[0048] S2-2 encoder multi-scale feature extraction. Specifically, it extracts features from normalized images. The input backbone network is used to extract multi-scale features. The encoder employs an improved DeepLab backbone network for... Perform layer-by-layer convolution processing to extract deep semantic features and obtain feature maps. .

[0049] Firstly, the DeepLab backbone network is used to process the input image. Feature extraction is performed, and multi-scale contextual information is obtained through dilated convolutional structures to obtain initial feature representations. Building upon this, to further enhance feature representation capabilities, a convolutional thinning module is cascaded after the DeepLab backbone network output. Further feature transformations are performed to obtain the final encoded image features. Specifically, the convolutional thinning module consists of three concatenated convolutional layers, each with a kernel size of [size missing]. The convolution stride is 1, and each convolutional layer is followed by a ReLU activation function for non-linear mapping, refining and enhancing the features layer by layer to obtain the final output image features of the encoder. .

[0050] To enhance the ability to perceive targets at different scales, a dilated convolutional structure is introduced after the second and third convolutional layers in the encoder. Each layer has a different dilation rate, allowing the convolutional kernel to expand the receptive field without increasing the number of parameters. This enables the model to perceive the slender structure of the heat collection pipe and the wide area of ​​the reflective surface without increasing the number of parameters. The specific implementation is as follows: . Indicates the features of the previous layer. This represents the void ratio of the layer. These are the output features of the convolution.

[0051] Multi-scale feature representations are obtained through multiple sets of parallel convolution operations with different void ratios, enabling the model to simultaneously perceive the slender structure of the heat collector tube and the large-scale structure of the reflector surface. The final output of this step is an image feature fused with multi-scale information. .

[0052] S2-3 Structural Prior Encoding Module. To enable the model to have structure-aware capabilities, the structural topological relationships obtained in step S1 are encoded. Introduced into the network, a structure prior coding module is set up, specifically: structural topology matrix Feature encoding is performed using three fully connected layers, with ReLU activation applied after each fully connected layer. The final structural feature vector is output. This process maps topological relationships using learnable parameters, enabling the connections between structures to be expressed in vector form. The structural features are calculated as follows: ,in For structural encoding parameters, It is a non-linear mapping function.

[0053] Subsequently, through the mapping function Structural features Mapping to image features Same feature space, with encoder output Alignment ensures that structural information spatially corresponds one-to-one with image features. The mapping function... This is a feature transformation module composed of learnable parameters, specifically implemented through fully connected layers and tensor reconstruction operations. First, the structural feature vectors... A fully connected mapping layer is input, and its channel dimension is linearly transformed to obtain an intermediate feature vector. Subsequently, this vector is reconstructed using a dimension expansion operation to resemble the image features. The feature representation is consistent across the channel dimension and replicated and expanded in the spatial dimension to make its size consistent with... The spatial resolution is consistent, thus achieving the mapping from structural feature vectors to spatial feature maps. The mapping parameters in this process are automatically optimized through network training. The fusion method is element-wise addition: .in, These are structural enhancement features used for subsequent decoding.

[0054] The S2-4 decoder and structurally constrained attention decoding mechanism. During the decoding phase, structural enhancement features are incorporated. The input decoder performs feature recovery and classification, while introducing a structurally constrained attention mechanism to weight and adjust the features.

[0055] First, the decoder consists of three upsampling convolutional layers, each of which uses bilinear interpolation to gradually restore the feature map size to the original image size. The ReLU activation function is used after each convolutional layer.

[0056] A structurally constrained attention mechanism is introduced during the upsampling process, based on features. Attention weights are calculated to measure the importance of each spatial location. The calculation method for attention weights is as follows: .in This is the attention parameter matrix.

[0057] Subsequently, the attention weights are element-wise multiplied with the features to obtain a weighted feature representation. Finally, the feature resolution is gradually restored through upsampling, and a high-precision semantic segmentation result with the same size as the input image is output. .in Each element in the array represents the probability that a pixel belongs to the k-th class, and includes the heat collection tube, reflective surface, bracket, connecting parts, and background.

[0058] S2-5 Model Training and Output. During the model training phase, the model uses a pixel-level cross-entropy loss function to evaluate the predicted segmentation results. With semantic segmentation label set Compare them.

[0059] ; in, Represents the set of network parameters. A collection of image pixels. For pixel location indexing. Through backpropagation algorithm, based on the loss function... Optimize network parameters Optimize the training. After training, the structure-aware semantic segmentation model is obtained. The output is a structure-enhanced high-precision semantic segmentation result. This step enables joint modeling of structural information and image features, allowing the model to accurately segment the heat collection tubes, reflector surfaces, supports, and connecting components while maintaining structural consistency and spatial integrity.

[0060] S3. Construction of a state discrimination model based on temporal consistency This step takes the obtained structure-enhanced semantic segmentation result sequence as input and constructs a three-stage model of structural feature extraction, temporal modeling, and state classification to achieve the transformation from single-frame segmentation results to dynamic state recognition. Specifically: S3-1 Structural feature extraction based on segmentation results. Because... The output is in probabilistic form; to perform structural analysis, it needs to be converted into a definitive category label map. First, for each frame's semantic segmentation result sequence... At each pixel position The image is discretized, and a maximum probability selection operation is performed to obtain a discrete label map. : ; Indicates pixel position Category tags, The category with the highest probability is selected as the final category label.

[0061] Secondly, for each category Extract its corresponding spatial region. Specifically, extract all regions that satisfy the condition that the category label equals the category. The pixels are represented as a set. This set That is to say, the first Frame Category The corresponding complete spatial region. In this way, pixel-level segmentation results are converted into structural-level region representations.

[0062] Finally, after obtaining the spatial regions for each category, their geometric properties are calculated to construct a structural feature description. For each category of region... Calculate its spatial center position. This center position is determined by averaging the coordinates of all pixels within the region, by averaging the x-coordinate and y-coordinate respectively. , .in, This indicates the number of pixels within the region. It also provides the coordinates of the center position. .

[0063] Simultaneously, the number of pixels in this category region is used as the area feature of the structural region for statistical analysis to obtain the region area. .

[0064] After calculating all categories, the center position and area features of each category are concatenated in category order to construct the structural feature vector of the current frame. This feature vector describes the spatial distribution of the key structures of the device at the current moment.

[0065] S3-2 Temporal Modeling Network Module Construction. To model the changes in device state over time, the structural features of consecutive frames are arranged in chronological order. For the current moment... The construction length is The feature sequence is obtained within a time window: .in This indicates the length of the time window. This sequence reflects the changes in the device structure over a continuous period of time and serves as the input for subsequent time series modeling.

[0066] To extract the dynamic features of the structure over time, this invention designs a temporal modeling network module based on a Long Short-Term Memory (LSTM) network. This network module uses a three-layer cascaded LSTM network for modeling, such as... Figure 3 As shown.

[0067] First, the structural feature vector The information is fed into the first LSTM layer as input. This layer filters and updates the historical information through input gates, forget gates, and output gates, and outputs the first layer hidden state. Its input dimension is The hidden layer dimension is set to Subsequently, Input the second LSTM layer to obtain the second hidden state, with the hidden layer dimension set to 1. This is used to further extract time-dependent features, and the output is... Finally, Input a third LSTM layer, with hidden layer dimensions set to... Output the final time series features .

[0068] Within each LSTM layer, information flow is controlled through input gates, forget gates, and output gates, enabling the model to remember long-term structural changes. The final result... This indicates the temporal characteristics after incorporating historical information.

[0069] S3-3 Structural Change Features and State Discrimination Model Construction. To enhance the detection capability of sudden anomalies, structural changes between adjacent frames are further introduced on the basis of temporal features. For the current frame... Perform difference calculation with the structural features of the previous frame: This difference reflects the degree of structural change over a short period of time.

[0070] Subsequently, the time series characteristics With changing characteristics By splicing the data, we obtain the comprehensive discriminative features: This feature contains both long-term trends and short-term change information.

[0071] Comprehensive features An input state classification network is used to determine the operating status of a device. This classification network consists of two fully connected layers: the first layer maps the input features to a high-dimensional feature space with dimensions of [dimension number missing]. The system first enhances nonlinear expressiveness through the ReLU activation function; the second fully connected layer maps features to the state class space and obtains the probability distribution of each state through the Softmax function. Finally, the state prediction result is obtained. .

[0072] S3-4 State Correction Based on Structural Constraints. After obtaining the preliminary classification results, the results are further corrected by incorporating structural constraint rules.

[0073] Specifically, by comparing the changes in area, position, and relative spatial relationships of structural regions in consecutive frames, discrimination criteria are constructed: (1) When the area of ​​a certain structural region Compared to the previous frame When the value is significantly reduced (set according to the actual scenario), it is determined to be an occlusion state; (2) When the change in the position of the center of the structure exceeds the preset threshold, it is determined to be an offset state; (3) When a structural region becomes discontinuous or disappears, it is determined to be in a damaged state; (4) When the relative positional relationship between multiple structures changes, it is determined to be an abnormal installation angle. The above-mentioned state categories are then grouped into a set. It includes normal state, obstruction state, structural offset state, component damage state, and abnormal installation angle state.

[0074] The classification results are corrected using the above rules to improve the stability and reliability of the identification. The final output is the discrimination result after correction based on the device's operating status. This result integrates semantic segmentation information, temporal features, and structural change information to achieve dynamic identification of the device's operating status.

[0075] S4. Operation Status Identification and Result Output Module S4-1 Structure segmentation mask generation. For each frame of image, high-precision semantic segmentation results with structure enhancement output from S2 are generated. This generates a corresponding structural segmentation mask. Specifically, using the same discretization method as in S3, a segmentation mask is generated for each pixel position. Category determination: ; in, Represents a structural segmentation mask. Represents pixels The structural category label is used to convert the probabilistic segmentation results into an intuitive structural region distribution map for subsequent visualization and analysis.

[0076] S4-2 Single-frame final state label determination. After obtaining the discrimination result output in S3. Then, a category decision is made to obtain the final state label. Specifically, the current frame state is determined using the maximum probability principle: ; in, This represents the final state label for frame t. Indicates belonging to a state The probability of.

[0077] To achieve a unified representation of structural and state information, the segmentation mask will be used. With final state label The components are then fused to create a single-frame output: .in, This represents the comprehensive recognition result for frame t. Wherein, the segmentation mask... The status label represents the pixel-level classification result of each structural region in the input image, which is used to characterize the location distribution and spatial range of each physical structure in the device; This indicates the overall operational status category of the device in the current frame. This is achieved by introducing a segmentation mask. It can achieve precise location of abnormal areas, and expand from whether it is abnormal to where it is abnormal and what kind of structure is abnormal.

[0078] Therefore, the comprehensive identification result The physical meaning is: in the first The frame image simultaneously presents the spatial distribution of the various structural components of the device and the corresponding overall operating status judgment results, among which, Spatial information used to describe the structure of the equipment Semantic information used to describe the operating status of equipment, together with semantic information, constitute an interpretable recognition result for engineering applications.

[0079] In practical applications, status labels can be used. The image is overlaid onto the segmented image and labeled with color codes, thereby enabling synchronous visualization of the device structure and operating status.

[0080] S4-3 Timing Status Statistical Analysis. To obtain the changes in the operating status of the device over a period of time, statistical analysis is performed on the status results of consecutive frames. Let the time interval be... within, share If the frame is defined as the frequency of state occurrence, then: ; in, Representing state The proportion of occurrence, This is an indicator function; it takes a value of 1 when the condition is true and 0 otherwise. This statistic reflects the distribution of various states of the equipment within a given time period.

[0081] Furthermore, to characterize the state change process, state transition counting is introduced: ; This statistic is used to describe the device status. Towards The number of transitions reflects the dynamic characteristics of state changes. Based on the above statistical analysis results, an equipment operating status report is generated.

[0082] The report includes: the occurrence rate of each status category. State transition situation and single-frame output results Finally, the above information is organized into a structured output. .

[0083] Experimental Analysis: (one) Figure 4 Experimental results show that SAS-Net with added structure priors provides clearer segmentation results in image details and edge regions, and significantly improves the IoU metric compared to baseline methods. SAS-Net is particularly accurate in areas such as slender pipes, complex supports, and connecting components, demonstrating that structure-aware constraints can effectively improve the segmentation accuracy and visualization of key functional regions.

[0084] (two) Figure 5 This study demonstrates the segmentation performance variations of key structures across consecutive frames. Each line represents a different device structure, including the heat collector, reflector, support frame, and connector. The lines show the IoU (Intersection over Union) value of each structure as the frame number changes, while the filled error bands represent the fluctuation range of IoU across consecutive frames, providing a clear picture of the model's segmentation stability and robustness. The results show that the heat collector and support frame have generally high IoU values ​​with minimal fluctuation, indicating that the model can stably recognize these elongated or regular structures. The reflector has a slightly lower IoU in some frames, mainly due to the susceptibility to light reflection and occlusion interference in real-world scenes, but overall it remains at a high level. The connector's segmentation accuracy fluctuates slightly, but the average value remains within an acceptable range.

[0085] (three) Figure 6 This study demonstrates the joint distribution of segmentation performance and operational status recognition probability for key structures across consecutive frames. The size of the bubbles corresponds to the segmentation IoU value for each frame, visually reflecting segmentation accuracy, while the bubble color maps to the operational status recognition probability, with a gradient from blue to red representing states from low confidence to high confidence. Experimental results show that the bubbles for the heat collector tube and support are generally large and reddish, indicating stable segmentation accuracy and high operational status recognition confidence across consecutive frames. The bubble size and color for connectors exhibit moderate fluctuations, suggesting that while the model experiences some fluctuations in segmentation and operational status recognition for small or complex connectors, the overall results are acceptable.

[0086] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0087] While the above description illustrates specific embodiments of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for identifying the operating status of a solar thermal collector based on a structure-aware segmentation network, characterized in that, The process includes the following: S1: Collect image sequence data during the operation of the solar thermal collector and preprocess it to obtain normalized images; obtain the structural topology adjacency matrix by constructing the structural topology relationship of the equipment and calculating the distance between the structural centers. S2, based on the constructed structure-aware segmentation network, normalized image input encoder is used for feature extraction to obtain image features, structure topological adjacency matrix is ​​input into structure prior encoding module to generate structure features, the two features are fused to obtain structure-enhanced features, and input into decoder for pixel-by-pixel classification, outputting structure-enhanced semantic segmentation results; S3, based on the constructed state discrimination model, converts the semantic segmentation results into a deterministic category label map, and aggregates the categories to obtain category regions. It calculates the spatial center position of the category region and concatenates it with the area feature to obtain the structural feature vector. The vector is then input into the temporal modeling network module to obtain temporal features. The structural change between adjacent frames is introduced and combined with the temporal features to construct a comprehensive feature vector. Finally, the vector is input into the state classification network to output the device operation state discrimination result.

2. The method for identifying the operating status of a solar thermal collector based on a structure-aware segmentation network as described in claim 1, characterized in that: After outputting the device's operating status, the results are corrected based on structural constraint rules, specifically as follows: By comparing the changes in area, position, and relative spatial relationships of structural regions in consecutive frames, the following discrimination criteria are constructed: When the area of ​​a certain structural region decreases by more than a certain amount compared to the previous frame, it is determined to be an occlusion state; When the change in the center position of the structure exceeds a preset threshold, it is determined to be a structural offset state. When a structural region becomes discontinuous or disappears, it is determined to be in a state of structural damage. When the relative positional relationship between multiple structures changes, it is determined to be an abnormal installation angle state.

3. The method for identifying the operating status of a solar thermal collector based on a structure-aware segmentation network as described in claim 1, characterized in that: It also includes the following processes: Combining the obtained state discrimination result and the semantic segmentation result obtained in S2 as input, the device operating state is fused and expressed and the result is output, including: classifying each pixel position to obtain a structural segmentation mask, then making a category decision to obtain the final state label; then fusing the segmentation mask and the final state label to obtain a single frame output result; finally, performing temporal state statistical analysis, counting the number of device state transitions, and generating a device operating state report.

4. The method for identifying the operating status of a solar thermal collector based on a structure-aware segmentation network as described in claim 1, characterized in that: The structural topological adjacency matrix is ​​obtained in the following way: First, pixel-level semantic segmentation and annotation are performed on the image to divide the device into four types of structural regions. The area includes the heat collection tube, the reflector surface, the support frame, and the connecting components. Extract the connected regions for each category and calculate the geometric center position for each structure instance. The geometric center is obtained by averaging the coordinates of all pixels within the region. Subsequently, a structural topology diagram is constructed. , The set of nodes; each node corresponds to a specific physical structural component, and in the image, it is an independent structural unit corresponding to a spatially connected pixel region, and the set of edges. Generated by distance constraints, the device topology is constructed based on the spatial distance relationships between structures, defining any two nodes. and , and Let the coordinates of the geometric centers of the two structural instances be given, and calculate the Euclidean distance between the two structural centers. ; when Less than the preset threshold When two structures are considered to be connected, an edge connection is established in the topological graph and set to 1; otherwise, it is set to 0. This process ultimately yields the structural topological adjacency matrix. .

5. The method for identifying the operating status of a solar thermal collector based on a structure-aware segmentation network as described in claim 1, characterized in that: The encoder in the structure-aware segmentation network employs an improved DeepLab backbone network for normalized images. Perform layer-by-layer convolution processing to extract deep semantic features; First, multi-scale contextual information is obtained through dilated convolutional structures to obtain the initial feature representation. A convolutional thinning module is connected in series after the output of the DeepLab backbone network to refine the output. Further feature transformations are performed to obtain the final encoded image features. ; The convolutional thinning module consists of three concatenated convolutional layers, each with a kernel size of [size missing]. The convolution stride is 1, and each convolutional layer is followed by a non-linear mapping using the ReLU activation function. The encoder introduces a voided convolution structure after the second and third convolution layers, with different void ratios set for each layer. This allows the convolution kernel to expand the receptive field without increasing the number of parameters, thus sensing the slender structure of the heat collection tube and the wide area of ​​the reflective surface.

6. The method for identifying the operating status of a solar thermal collector based on a structure-aware segmentation network as described in claim 1, characterized in that: The specific data processing procedure of the structure prior coding module is as follows: structural topology matrix Feature encoding is performed using three fully connected layers, with ReLU activation applied after each fully connected layer to output the final structural feature vector. During the processing, learnable parameters are used to map topological relationships, enabling the connections between structures to be expressed in vector form. The structural features are calculated as follows: ,in For structural encoding parameters, It is a nonlinear mapping function; Subsequently, through the mapping function Structural features Mapping to image features The same feature space.

7. The method for identifying the operating status of a solar thermal collector based on a structure-aware segmentation network as described in claim 6, characterized in that: The mapping function The feature transformation module, composed of learnable parameters, is implemented through a fully connected layer and tensor reconstruction operations: First, the structural feature vectors are transformed... A fully connected mapping layer is input, and its channel dimensions are linearly transformed to obtain an intermediate feature vector. Subsequently, this intermediate feature vector is reconstructed into a shape similar to the image features through a dimension expansion operation. The feature representation is consistent across the channel dimension and replicated and expanded in the spatial dimension to make its size consistent with... The spatial resolution is consistent.

8. The method for identifying the operating status of a solar thermal collector based on a structure-aware segmentation network as described in claim 1, characterized in that: The specific data processing procedure for the temporal modeling stage of the state discrimination model is as follows: First, the semantic segmentation result sequence for each frame. At each pixel position The image is discretized, and a maximum probability selection operation is performed to obtain a discrete label map. : ； Indicates pixel position Category tags, The category with the highest probability is selected as the final category label; Secondly, for each category Extract its corresponding spatial region, and then select all regions whose category label equals the category. The pixels are represented as a set: ； That is to say, the first Frame Category The corresponding complete spatial region; Finally, for each category of area The spatial center position is calculated by averaging the coordinates of all pixels within the region. The coordinates of the center position are obtained by averaging the x-coordinate and y-coordinate respectively. ; Simultaneously, the number of pixels in the category region is used as the area feature of the structural region for statistical analysis to obtain the region area. ; After calculating all categories, the center position and area features of each category are concatenated in category order to construct the structural feature vector of the current frame. .

9. The method for identifying the operating status of a solar thermal collector based on a structure-aware segmentation network as described in claim 1, characterized in that: The temporal modeling network module employs a three-layer cascaded long short-term memory network; firstly, the structural feature vectors are... The information is fed into the first LSTM layer as input. This layer filters and updates the historical information through input gates, forget gates, and output gates, and outputs the first layer hidden state. Its input dimension is The hidden layer dimension is set to ; then, Input the second LSTM layer to obtain the second hidden state, with the hidden layer dimension set to 1. This is used to further extract time-dependent features, and the output is... Finally, Input a third LSTM layer, with hidden layer dimensions set to... Output the final time series features .

10. The method for identifying the operating status of a solar thermal collector based on a structure-aware segmentation network as described in claim 3, characterized in that: For each pixel position Category determination: ； in, Represents a structural segmentation mask. Represents pixels The structural category label; The category with the highest probability is selected as the final category label; Indicates category; The current frame state is determined using the maximum probability principle: ； in, This represents the final state label for frame t. Indicates belonging to a state The probability of; Segmentation mask With final state label The components are then fused to create a single-frame output: .

Images