Infrared Radiation De-icing Method and System Based on Blade Tracking

Abstract

The present application discloses an infrared radiation deicing method and system based on blade tracking, which obtains the video stream of the blade area through a high-speed camera, and uses image processing technology based on deep learning to extract the blade state characteristics of each image frame in the video stream of the blade area, and further performs time-series dynamic propagation of the blade state characteristics to mine the change pattern of the blade state over time, capture the motion trajectory and rotation characteristics of the blade, and based on this, realize the prediction of the position of the blade at the next moment. Then, based on the blade position data at the next moment, adjust the direction of infrared radiation. In this way, real-time tracking of high-speed rotating blades and real-time dynamic adjustment of the infrared radiation direction can be achieved, which not only improves the deicing efficiency, reduces unnecessary energy consumption, but also ensures the stability and reliability of the wind energy conversion system.

Description

Technical Field

[0001] This application relates to the field of intelligent de-icing, and more specifically, to an infrared radiation de-icing method and system based on blade tracking. Background Art

[0002] In the field of wind power generation technology, the de-icing problem of blades has always been a technical challenge that needs to be solved urgently. Especially for wind turbines operating in cold climate conditions, ice formation on the blades not only leads to a significant decrease in the efficiency of the wind turbines, but may also cause safety problems, such as unbalanced loads, increased mechanical stress, and potential hazards caused by ice shedding. To this end, engineers have adopted a variety of de-icing methods, including mechanical de-icing, hot air flow de-icing, electric heating de-icing, etc. However, these traditional methods often have problems such as slow response speed, high energy consumption, and potential damage to the blade material and structural strength.

[0003] With the development of technology, infrared radiation de-icing has gradually attracted attention as a new and efficient method. Infrared radiation de-icing refers to using the energy of infrared rays to directly act on the ice layer, so that the ice layer absorbs heat to achieve melting. This method has the advantages of fast response, concentrated energy, non-contact operation, etc., and is very suitable for application on large rotating structures such as wind turbine blades. However, traditional infrared de-icing technologies also face some problems. For example, it is impossible to accurately track the high-speed rotating blades and adjust the direction of infrared radiation in real time to ensure the effective transmission of energy. Due to the large size and high rotation speed of the wind turbine blades, it is difficult for a fixed-direction infrared radiation source to fully cover each blade, which may lead to poor de-icing effect or energy waste.

[0004] Therefore, an infrared radiation de-icing method based on blade tracking is expected. Summary of the Invention

[0005] To solve the above technical problems, this application is proposed. The embodiments of this application provide an infrared radiation de-icing method and system based on blade tracking. It obtains the video stream of the blade area through a high-speed camera, extracts the blade state features of each image frame in the video stream of the blade area by using image processing technology based on deep learning, further mines the change pattern of the blade state over time through temporal dynamic propagation of the blade state features, captures the movement trajectory and rotation characteristics of the blade, and predicts the position of the blade at the next moment based on this. Then, based on the blade position data at the next moment, the direction of infrared radiation is adjusted. In this way, real-time tracking of high-speed rotating blades and real-time dynamic adjustment of the direction of infrared radiation can be achieved, which not only improves the de-icing efficiency, reduces unnecessary energy consumption, but also ensures the stability and reliability of the wind energy conversion system.

[0006] According to one aspect of the present application, there is provided an infrared radiation de-icing method based on blade tracking, which includes:

[0007] Obtaining a video stream of the blade area collected by a high-speed camera;

[0008] Extracting image frames from the video stream of the blade area frame by frame to obtain a time queue of blade area image frames;

[0009] Preprocessing the time queue of the blade area image frames to obtain a time queue of preprocessed blade area image frames;

[0010] Extracting blade state feature to obtain a time queue of blade state semantic coding features from the time queue of the preprocessed blade area image frames;

[0011] Performing multi-gated temporal dynamic propagation guided by the blade state on the time queue of the blade state semantic coding features to obtain blade state temporal tracking semantic coding features;

[0012] Adjusting the direction of infrared radiation based on the blade state temporal tracking semantic coding features.

[0013] According to another aspect of the present application, there is provided an infrared radiation de-icing system based on blade tracking, which includes:

[0014] A video stream acquisition module for obtaining a video stream of the blade area collected by a high-speed camera;

[0015] An image frame extraction module for extracting image frames from the video stream of the blade area frame by frame to obtain a time queue of blade area image frames;

[0016] An image preprocessing module for preprocessing the time queue of the blade area image frames to obtain a time queue of preprocessed blade area image frames;

[0017] A blade state feature extraction module for extracting blade state features from the time queue of the preprocessed blade area image frames to obtain a time queue of blade state semantic coding features;

[0018] A multi-gated temporal dynamic propagation module for performing multi-gated temporal dynamic propagation guided by the blade state on the time queue of the blade state semantic coding features to obtain blade state temporal tracking semantic coding features;

[0019] An infrared radiation direction adjustment module for adjusting the direction of infrared radiation based on the blade state temporal tracking semantic coding features.

[0020] Compared with the prior art, an infrared radiation de-icing method and system based on blade tracking provided by the present application obtains a video stream of the blade area through a high-speed camera, extracts the blade state features of each image frame in the video stream of the blade area by using image processing technology based on deep learning, further mines the change pattern of the blade state over time by performing temporal dynamic propagation on the blade state features, captures the movement trajectory and rotation features of the blade, and predicts the position of the blade at the next moment based on this. Then, based on the blade position data at the next moment, the direction of infrared radiation is adjusted. In this way, real-time tracking of the high-speed rotating blade and real-time dynamic adjustment of the infrared radiation direction can be achieved, which not only improves the de-icing efficiency, reduces unnecessary energy consumption, but also ensures the stability and reliability of the wind energy conversion system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of the present application will become more obvious. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used to explain the present application together with the embodiments of the present application, and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 It is a flowchart of the infrared radiation de-icing method based on blade tracking according to an embodiment of the present application;

[0023] Figure 2 It is a schematic diagram of data flow of the infrared radiation de-icing method based on blade tracking according to an embodiment of the present application;

[0024] Figure 3 It is a flowchart of sub-step S5 of the infrared radiation de-icing method based on blade tracking according to an embodiment of the present application;

[0025] Figure 4 It is a block diagram of the infrared radiation de-icing system based on blade tracking according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] Next, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0027] As shown in this application and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0028] Although this application makes various references to certain modules in the system according to the embodiments of this application, however, any number of different modules can be used and run on the user terminal and / or server. The modules are merely illustrative, and different aspects of the system and method can use different modules.

[0029] Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the previous or following operations are not necessarily executed precisely in sequence. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or several steps can be removed from these processes.

[0030] Next, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments of this application. It should be understood that this application is not limited by the exemplary embodiments described herein.

[0031] In the technical solution of this application, an infrared radiation de-icing method based on blade tracking is proposed. Figure 1 FIG. is a flowchart of an infrared radiation de-icing method based on blade tracking according to an embodiment of this application. Figure 2 FIG. is a schematic diagram of data flow of an infrared radiation de-icing method based on blade tracking according to an embodiment of this application. As Figure 1 and Figure 2 shown, the infrared radiation de-icing method based on blade tracking according to the embodiment of this application includes the steps of: S1, obtaining a video stream of the blade area collected by a high-speed camera; S2, extracting image frames of the video stream of the blade area frame by frame to obtain a time queue of blade area image frames; S3, preprocessing the time queue of the blade area image frames to obtain a time queue of preprocessed blade area image frames; S4, extracting blade state feature extraction from the time queue of the preprocessed blade area image frames to obtain a time queue of blade state semantic coding features; S5, performing multi-gated temporal dynamic propagation based on the blade state guidance on the time queue of the blade state semantic coding features to obtain blade state temporal tracking semantic coding features; S6, adjusting the direction of infrared radiation based on the blade state temporal tracking semantic coding features.

[0032] Specifically, in S1, a video stream of the blade area collected by a high-speed camera is obtained. The high-speed camera can capture images at a frame rate much higher than that of conventional imaging devices, thereby ensuring high-precision dynamic monitoring of rapidly rotating wind turbine blades. The speed of wind turbine blades during operation is extremely fast. Especially in large wind power generation units, their rotational speed can reach dozens or even hundreds of revolutions per minute. In this case, it is difficult for a camera with an ordinary speed to capture clear and stable blade images, while a high-speed camera can record videos at a higher frame rate (the number of images captured per second) than ordinary imaging devices, continuously capturing multiple image frames within a very short time interval, so as to provide sufficient frames to ensure that the position and state of the blade at each moment can be accurately recorded. The continuous video stream captured by the high-speed camera not only contains the spatial information of the blade, but also contains information in the time dimension, which is crucial for understanding the state changes of the blade at different time periods. In an embodiment of the present application, a Phantom V2511 high-speed camera is used to collect the video stream of the blade area. The Phantom V2511 can capture at a speed of up to 36,700 frames per second (fps) in black and white mode, or reach 24,000 fps in color mode. Such a high frame rate is very useful for capturing rapidly changing phenomena such as high-speed rotating wind turbine blades.

[0033] It is worth noting that when installing and positioning the high-speed camera, it should be placed at a position that can best cover the rotation path of the blade, usually on the wind turbine tower or a stable structure around it. The installation position needs to ensure that there are no obstacles in the camera's field of view and minimize external light interference as much as possible. For large wind turbines, multiple cameras may be required to work together to comprehensively monitor the conditions of all blades. In addition, considering various harsh weather conditions (such as strong winds, rain, snow, extremely low temperatures, etc.) in the outdoor environment, the design and protection measures of the high-speed camera also need to be considered specifically. For example, means such as using a waterproof and dustproof housing and heating the glass lens to prevent frosting are adopted to ensure normal operation even in extreme environments.

[0034] Synchronization and calibration are another key step. It is very important to ensure the clock synchronization between the high-speed camera and other system components (such as the image processing unit, control system, etc.), so as to ensure the accuracy of the timestamps of the collected data. During the initial deployment or regular maintenance, the camera must be calibrated, including focal length adjustment, angle correction in the horizontal and vertical directions, etc., to ensure that the captured images can truly reflect the actual movement of the blade. At the same time, the system should include a self-diagnosis function, which can detect the working state of the camera (such as whether there is a fault, whether it deviates from the preset parameters, etc.), and trigger an alarm to notify relevant personnel for inspection or repair when necessary. Regularly execute a health check program to evaluate the camera performance and update the firmware or hardware components as needed to maintain the best operating state of the system.

[0035] Specifically, in S2, the video stream of the blade area is extracted frame by frame to obtain a time queue of blade area image frames. It should be understood that when the high-speed camera captures the video stream of the blade area, these video streams are actually composed of a series of consecutive image frames. However, due to the high-speed rotation characteristics of the wind turbine blade, there may be significant position differences between adjacent frames. If the frame extraction interval is too large, important transitions in the blade state at certain moments may be ignored, thus affecting the accuracy of subsequent analysis. Therefore, in the technical solution of this application, the video stream of the blade area is extracted frame by frame to obtain a time queue of blade area image frames. By extracting frames one by one, independent image frames can be separated from the video stream and arranged in chronological order to form a time queue. The time queue of the blade area image frames reflects the motion trajectory and state change pattern of the blade, where each frame of the image represents the position and state of the blade at a specific time point. In this way, the system can more accurately understand the state of the blade changing over time, thereby improving the accuracy of position prediction.

[0036] Specifically, in S3, preprocess the time queue of the blade area image frames to obtain the time queue of the preprocessed blade area image frames. In a specific example of the present application, the process of preprocessing the time queue of the blade area image frames includes: performing denoising processing on each blade area image frame in the time queue of the blade area image frames to obtain the time queue of the denoised blade area image frames; and performing contrast enhancement on each denoised blade area image frame in the time queue of the denoised blade area image frames to obtain the time queue of the preprocessed blade area image frames. Here, considering that during the operation of the high-speed camera, environmental light changes, complex and variable outdoor weather conditions, and inevitable motion blur during the shooting process may occur, the captured images often contain a large amount of noise. And the noise may seriously affect the accuracy of the wind turbine blade state recognition. For example, in ice detection, the noise may be misidentified as ice layer, resulting in incorrect judgment; while in blade position prediction, the noise may cause the model to learn incorrect patterns, affecting the prediction accuracy. Therefore, perform denoising processing on each blade area image frame in the time queue of the blade area image frames to reduce the interference of the noise on the image and obtain the time queue of the denoised blade area image frames. In addition, considering that the wind turbine blades are usually in a vast natural environment with different lighting conditions, especially in low-light or high-contrast situations, the contrast of the original image may be low, resulting in unclear blade contours and making it difficult to distinguish the blade itself from its background. Therefore, in the technical solution of the present application, further perform contrast enhancement on each denoised blade area image frame in the time queue of the denoised blade area image frames. Through contrast enhancement, the boundary between the blade and the surrounding environment can be made more distinct, facilitating the subsequent processing steps to identify and track the position of the blade, so as to obtain the time queue of the preprocessed blade area image frames

[0037] Specifically, in S4, leaf state feature extraction is performed on the time queue of the preprocessed leaf region image frames to obtain a time queue of leaf state semantic encoding features. In the technical solution of this application, a leaf state feature extractor based on the Mobile-Former model is used to perform leaf state feature extraction on the time queue of the preprocessed leaf region image frames to obtain a time queue of leaf state semantic encoding feature vectors as the time queue of the leaf state semantic encoding features. By using a leaf state feature extractor based on the Mobile-Former model to perform leaf state feature extraction on the time queue of the preprocessed leaf region image frames, deep semantic features of the leaf state are extracted to obtain a time queue of leaf state semantic encoding feature vectors as the time queue of the leaf state semantic encoding features. Since the fan blade is in a constantly changing environment, its surface may be affected by various factors, such as wind speed, temperature, humidity, etc., resulting in different physical phenomena, such as icing. Traditional feature extraction methods often have difficulty capturing these complex patterns. The Mobile-Former model enhances its consistent performance under different conditions by introducing advanced training strategies and technical means, such as self-supervised learning and transfer learning. By using a leaf state feature extractor based on the Mobile-Former model to perform leaf state feature extraction on the time queue of the preprocessed leaf region image frames, the most representative leaf features can be mined from a large amount of image data, including but not limited to the shape, edge contour of the leaf, and any abnormal changes (such as icing), to more accurately identify the state of the leaf, thereby providing a scientific basis for infrared radiation de-icing.

[0038] In one example, while extracting single-frame image features, Mobile-Forme utilizes temporal information to establish connections between adjacent frames and within the entire sequence. This helps capture the leaf movement trajectory and its dynamic characteristics, thereby providing richer context information for subsequent prediction. The feature maps obtained through the above process contain a large amount of information about the leaf state, but they are still in a relatively raw form. To better serve downstream tasks (such as position prediction), these features need to be further mapped to a higher-level space - namely, semantic encoding. In this process, Mobile-Former adopts an attention mechanism to highlight the parts that are most important for describing the leaf state, while suppressing irrelevant or redundant information, to obtain the leaf state semantic encoding feature vector.

[0039] As the features of each frame of the image are gradually extracted, all the generated semantic encoding feature vectors of the blade states are sorted according to the corresponding timestamps, forming a continuous time series. This time series not only records the specific state of the blade at each moment but also implies any changing trends over time, serving as an important basis for subsequent analysis and decision-making.

[0040] It should be noted that the working environment of the wind turbine is constantly changing. Therefore, the feature extraction model used also needs to have a certain degree of flexibility and self-evolution ability. Mobile-Former supports online learning and incremental training, which means it can continuously adjust its own parameters as new data arrives and maintain the best working state.

[0041] Specifically, in S5, multi-gated temporal dynamic propagation guided by the blade state is performed on the time queue of the semantic encoding features of the blade state to obtain the semantic encoding features of blade state temporal tracking. It should be understood that during the high-speed rotation of the wind turbine blade, the movement trajectory and rotation speed of the blade may be affected by various factors, such as wind direction, wind speed, blade shape, and mass distribution of surface ice on the blade. Using the Mobile-Former model to perform static feature extraction on the blade state can describe the blade motion state at a certain moment, but it cannot reveal the internal connection between the blade states at each moment. To accurately predict the future position of the blade, it is necessary to accurately model the movement trajectory of the blade. Therefore, further, multi-gated temporal dynamic propagation guided by the blade state is performed on the time queue of the semantic encoding features of the blade state to capture the trend of the blade state evolving over time and obtain the semantic encoding features of blade state temporal tracking. Specifically, each gating unit can be regarded as a small memory module responsible for recording and updating the state changes of the wind turbine blade within a specific time period. Through a more complex feature fusion and propagation mechanism, the system can better understand the behavior patterns of the blade at different moments, thus providing support for more accurate state assessment. In a specific example of this application, as Figure 3 shown, S5 includes: S51, mapping the time queue of the semantic encoding feature vectors of the blade state to the Poincaré space to obtain the time queue of the semantic encoding feature vectors of blade state exotic modulation; S52, calculating the propagation representation of the time queue of the semantic encoding feature vectors of blade state exotic modulation, and based on the propagation representation, performing significant propagation aggregation on the time queue of the semantic encoding feature vectors of the blade state to obtain the semantic encoding vector of blade state temporal tracking as the semantic encoding features of blade state temporal tracking.

[0042] Specifically, in S51, the time queue of the blade state semantic encoding feature vectors is mapped to the Poincaré space to obtain the time queue of the blade state semantic exotic modulation encoding feature vectors. It should be understood that during the de-icing process, the state changes of the wind turbine blades are highly non-linear, and there are complex hierarchical relationships between the blade states at each moment. In the Poincaré space, this hierarchical structure can be represented more intuitively. By mapping the blade state semantic encoding feature vectors to the Poincaré space, the interaction and dependence relationships between different-level features can be better captured. Compared with the fixed linear distance metric in the Euclidean space, in the Poincaré space, the distance between two points changes dynamically according to the geometric relationship between the two points. This non-linear distance metric is particularly suitable for dealing with data with complex internal structures. For wind turbine blades, their state changes are often not simple linear processes, but the result of the intertwined influence of multiple factors. The distance metric in the Poincaré space can better reflect the essence of these complex changes, making the model more accurate when evaluating the similarity and difference between different states. In addition, due to the boundary effect of the Poincaré space, when two states are very similar, the distance in the Poincaré space will be very close, which helps to improve the sensitivity of the model to subtle changes, thereby enhancing the ability to identify blade state changes. More specifically, the time queue of the blade state semantic encoding feature vectors is mapped to the Poincaré space by the following formula to obtain the time queue of the blade state semantic exotic modulation encoding feature vectors; where the formula is:

[0043] V = {v1, v2,..., v i ,..., v n}

[0044] h i = W1v i W2

[0045] H = {h1, h2,..., h i ,..., h n}

[0046] where V is the time queue of the blade state semantic encoding feature vectors, v1, v2, v i and v n are the 1st, 2nd, i-th, and n-th blade state semantic encoding feature vectors in the time queue of the blade state semantic encoding feature vectors respectively, W1 and W2 are the first weight matrix and the second weight matrix respectively, h1, h2, h i and h nThey are the 1st, 2nd, ith, and nth blade state semantic alien modulation encoded feature vectors in the time queue of the blade state semantic alien modulation encoded feature vectors respectively, and H is the time queue of the blade state semantic alien modulation encoded feature vectors.

[0047] Specifically, in S52, the propagation representation of the time queue of the blade state semantic alien modulation encoded feature vectors is calculated, and based on the propagation representation, significant propagation aggregation is performed on the time queue of the blade state semantic encoded feature vectors to obtain the blade state time series tracking semantic encoded vector as the blade state time series tracking semantic encoded feature. In the embodiments of the present application, first, forward LSTM sequence encoding is performed on the time queue of the blade state semantic alien modulation encoded feature vectors to obtain the blade state semantic feature alien propagation representation vector as the propagation representation. It should be understood that for blade states, some changes may occur instantaneously, such as the rapid adjustment of the blade angle due to sudden changes in wind speed; while other changes may involve a longer time span, such as the accumulation of ice caused by gradually decreasing temperature. Forward LSTM sequence encoding can freely adjust the memory content at different time scales through its internal gating structure (input gate, forget gate, and output gate), so as to avoid overfitting when processing long time series while maintaining the memory of key information. More specifically, the following sequence encoding formula is used to perform forward LSTM sequence encoding on the time queue of the blade state semantic alien modulation encoded feature vectors to obtain the blade state semantic feature alien propagation representation vector; where the sequence encoding formula is:

[0048] h n+1 = LSTM(H)

[0049] where LSTM is the forward LSTM encoding, and h n+1 is the blade state semantic feature alien propagation representation vector.

[0050] Next, based on the propagation representation, calculate the weight values of each blade state semantic exotic modulation coding feature vector in the time queue of the blade state semantic exotic modulation coding feature vectors to obtain the time queue of the blade state semantic feature transfer modulation weights. Specifically, first, calculate the semantic association degree between each blade state semantic exotic modulation coding feature vector in the time queue of the blade state semantic exotic modulation coding feature vectors and the blade state semantic feature exotic propagation representation vector to obtain the time queue of the blade state semantic association degree; here, the semantic association degree is used to represent the semantic internal association degree between the message propagation characteristics in the Poincaré space and each blade state semantic exotic modulation coding feature vector, so as to facilitate the subsequent introduction of the attention mechanism to enhance the model's attention to the blade state features. Next, calculate the semantic space jump value of each blade state semantic exotic modulation coding feature vector in the time queue of the blade state semantic exotic modulation coding feature vectors relative to the blade state semantic feature exotic propagation representation vector to obtain the time queue of the blade state semantic space jump values. This index measures the change trend and rate of feature expression over time through the degree of spatial span difference between the message propagation characteristics in the Poincaré space and each blade state semantic exotic modulation coding feature vector. The calculation of the semantic space jump value can not only reveal the dynamic characteristics of the blade state features, but also discover periodic patterns or abnormal events in the data. Furthermore, perform multiple gated feature modulations on the time queue of the blade state semantic association degree and the time queue of the blade state semantic space jump values to obtain the time queue of the blade state semantic feature transfer modulation weights. It should be understood that the state change of the wind turbine blade is a complex non-linear process, and the importance of features at different time points for understanding the overall behavior is not the same. For example, during the de-icing process, the ice layer on the blade surface may suddenly fall off at certain specific moments, thereby changing the motion characteristics of the blade. In order to accurately capture such sudden state changes, through a multiple gated mechanism combined with the semantic association degree and the semantic space jump value, adaptively and selectively strengthen or weaken the transmission of specific information according to the relationship between features, and assign a transfer modulation weight to the features at each time point to dynamically adjust the direction and intensity of the information flow. This weight reflects the importance of the feature at that moment for subsequent prediction, so that the model can focus more on the key information that truly affects the position state of the blade. More specifically, calculate the weight values of each blade state semantic exotic modulation coding feature vector in the time queue of the blade state semantic exotic modulation coding feature vectors with the following weight calculation formula to obtain the time queue of the blade state semantic feature transfer modulation weights; where the weight calculation formula is:

[0051]

[0052] α = {α1, α2,..., αi ,..., α n}

[0053] β i = ||h n+1 - h i ||

[0054] β = {β1, β2,..., β i ,..., β n}

[0055]

[0056] where ‖·‖ is the norm of the calculation vector, arccosh(·) is the inverse hyperbolic cosine function, α1, α2, α i and α n are respectively the 1st, 2nd, ith, and nth blade state semantic association degrees in the time queue of the blade state semantic association degree, α is the time queue of the blade state semantic association degree, β1, β2, β i and β n are respectively the 1st, 2nd, ith, and nth blade state semantic space jump values in the time queue of the blade state semantic space jump value, β is the time queue of the blade state semantic space jump value, W α and W β are respectively the semantic association weight matrix and the semantic space weight matrix, is matrix multiplication, b is the bias vector, sigmoid(·) is the sigmoid function, γ t is the time queue of the blade state semantic feature transfer modulation weight.

[0057] Furthermore, based on the time queue of the blade state semantic feature transfer modulation weight, calculate the position-wise weighted sum of the time queue of the blade state semantic foreign modulation coding feature vectors to obtain the blade state semantic foreign feature significant propagation representation vector. By means of weighted summation, the importance of the features and the changes over time are comprehensively considered, and a blade state semantic foreign feature significant propagation representation vector that comprehensively reflects the characteristics of the blade state change features over time is obtained.

[0058] Finally, fuse the significant propagation representation vector of the blade state semantic foreign region features and the foreign region propagation representation vector of the blade state semantic features to obtain the blade state time series tracking semantic coding vector. In the technical solution of this application, perform a residual connection on the significant propagation representation vector of the blade state semantic foreign region features and the foreign region propagation representation vector of the blade state semantic features to obtain a significant propagation compensation representation vector of the blade state semantic foreign region features; it should be understood that deep neural networks are prone to problems of gradient vanishing or explosion during the training process, especially when dealing with long time series data, and this phenomenon is more obvious. When the number of network layers increases, the gradients propagated backward may gradually approach zero, resulting in the model being difficult to effectively update parameters and affecting the learning effect. By introducing a residual connection, the gradient can be directly passed from the previous layer to the next layer, avoiding excessive attenuation of the information flow in the network. In this technical solution, the residual connection ensures that even when dealing with complex blade state changes, the model can maintain good training performance, thereby improving the stability and reliability of the system. Further, perform an Euclidean space mapping on the significant propagation compensation representation vector of the blade state semantic foreign region features to obtain the blade state time series tracking semantic coding vector. This process converts the significant propagation compensation representation vector of the blade state semantic foreign region features after complex processing back to the traditional Euclidean space, facilitating the use of subsequent standard machine learning algorithms. Through appropriate mapping functions and parameter adjustments, an effective projection from the high-dimensional feature space to the low-dimensional space can be achieved, providing convenience for feature dimensionality reduction and visualization. More specifically, fuse the significant propagation representation vector of the blade state semantic foreign region features and the foreign region propagation representation vector of the blade state semantic features with the following fusion formula to obtain the blade state time series tracking semantic coding vector; where the fusion formula is:

[0059]

[0060] v n+1 =f euclidean (h n+1 )=W3h n+1 ′W4

[0061] Among them, γ i is the modulation weight of the blade state semantic feature transfer corresponding to h i , n is the number of vectors in the time queue of the blade state semantic foreign modulation coding feature vector, h n+1 ' is the significant propagation compensation representation vector of the blade state semantic foreign region features, W3 and W4 are the third weight matrix and the fourth weight matrix respectively, f euclidean (·) is the Euclidean space mapping operation, and v n+1 is the blade state time series tracking semantic coding vector.

[0062] Specifically, in S6, based on the semantic encoding features traced by the blade state time series, the direction of infrared radiation is adjusted. That is, in the technical solution of this application, first, based on the semantic encoding vector traced by the blade state time series, the blade position data at the next moment is determined. Specifically, the semantic encoding vector traced by the blade state time series is input into a blade position predictor based on a decoder to obtain the blade position data at the next moment. That is, the specific physical quantity, namely the blade position data at the next moment, is recovered from the high-level abstract features (the semantic encoding vector traced by the blade state time series). To achieve this goal, the decoder usually adopts a recurrent neural network (RNN) or its variants (such as LSTM, GRU) because these models are good at capturing long-term dependencies in sequence data and can effectively handle non-linear transformations. Specifically, the decoder parses the received semantic encoding traced by the blade state time series frame by frame through an internal gating mechanism, gradually building a prediction of the future position of the blade. The decoding result of each frame depends not only on the input features at the current moment but also on the prediction results at the previous moments, thus ensuring the coherence and stability of the prediction process. Further, based on the blade position data at the next moment, the direction of infrared radiation is adjusted. Here, the method of blade tracking can perform directional heating according to actual needs. Since it can track the blade position in real time and flexibly adjust the radiation direction, the infrared radiation de-icing method based on blade tracking can better adapt to different icing conditions. For areas with severe local icing, the system can increase the radiation intensity when the blade approaches, concentrating efforts to quickly melt the ice layer; while for slightly iced parts, a lower radiation intensity can be used to avoid material damage caused by overheating. In addition, this dynamic adjustment can effectively prevent the problem of uneven de-icing caused by different blade rotation speeds, ensuring that each part can be fully processed.

[0063] Preferably, inputting the semantic encoding vector traced by the blade state time series into a blade position predictor based on a decoder to obtain the blade position data at the next moment includes:

[0064] Calculating the square root of the sum of the absolute values and the sum of the squares of all eigenvalues of the semantic encoding vector traced by the blade state time series to obtain the first and second semantic encoding space structure values of the blade state time series, that is:

[0065] w1 = Σ i |f i |

[0066]

[0067] where f i represents the eigenvalue at the i-th position in the semantic encoding vector traced by the blade state time series, and w1 represents the f iThe corresponding semantic encoding spatial structure value of the first blade state time series tracking, where w2 represents the f i The corresponding semantic encoding spatial structure value of the second blade state time series tracking;

[0068] Determine the total number of eigenvalues m of the semantic encoding vector of the blade state time series tracking;

[0069] For each eigenvalue of the semantic encoding vector of the blade state time series tracking, calculate the first blade state time series tracking semantic encoding long-range dependence value x obtained by subtracting the product of the eigenvalue and the total number of eigenvalues from the first blade state time series tracking semantic encoding spatial structure value i = w1 - f i ×m;

[0070] where x i represents the first blade state time series tracking semantic encoding long-range dependence value corresponding to the f i ;

[0071] Calculate the second blade state time series tracking semantic encoding long-range dependence value obtained by subtracting the second blade state time series tracking semantic encoding spatial structure value from the product of the square root of the total number of eigenvalues and the eigenvalue

[0072] where y i represents the second blade state time series tracking semantic encoding long-range dependence value corresponding to the f i ;

[0073] Perform a weighted sum of the exponential value obtained by taking the exponential of the first blade state time series tracking semantic encoding long-range dependence value as the natural constant and the reciprocal of the second blade state time series tracking semantic encoding long-range dependence value to obtain the optimized eigenvalue corresponding to each eigenvalue and

[0074] where f' i represents the optimized eigenvalue corresponding to the f i ; a and b respectively represent different weighting parameters;

[0075] Input the optimized semantic encoding vector of the blade state time series tracking composed of the optimized eigenvalues into the blade position predictor based on the decoder to obtain the blade position data at the next moment.

[0076] Here, when each blade state semantic encoding feature vector in the time queue of the blade state semantic encoding feature vectors represents the image semantics and pose state encoding features of the blade area image frame at a single time point, after performing feature dynamic propagation of the cross-domain multiple gate structure on it, the blade state temporal tracking semantic encoding vector will also have significant cross-domain propagation distribution spatial structure differences due to the differences in local temporal domain structure dynamics, affecting the convergence consistency of the decoder, and thus affecting the accuracy of the blade position data at the next moment obtained by the blade position predictor based on the decoder.

[0077] Based on this, due to the possible lack of spatial structure in the feature set of the blade state temporal tracking semantic encoding vector in the high-dimensional space, the weight matrix of the decoder implicitly infers spatial structure information based on features, resulting in inconsistent convergence. By establishing long-distance feature dependencies based on the overall feature scale of the blade state temporal tracking semantic encoding vector with respect to the spatial structure representation of the blade state temporal tracking semantic encoding vector, to establish the feature local connectivity of the blade state temporal tracking semantic encoding vector, and by predicting the spatial ambiguity information of the object feature values through the unstructured feature value points of the blade state temporal tracking semantic encoding vector, thereby enhancing the spatial inductive bias perception ability of the feature set of the blade state temporal tracking semantic encoding vector, improving the convergence consistency of the decoder, and enhancing the accuracy of the blade position data at the next moment obtained by inputting the blade state temporal tracking semantic encoding vector into the blade position predictor based on the decoder.

[0078] In summary, the infrared radiation de-icing method based on blade tracking according to the embodiments of the present application is elucidated. It obtains the video stream of the blade area through a high-speed camera, extracts the blade state features of each image frame in the video stream of the blade area by using deep learning-based image processing technology, further mines the change pattern of the blade state over time by performing temporal dynamic propagation on the blade state features, captures the motion trajectory and rotation features of the blade, and based on this, predicts the position of the blade at the next moment. Then, based on the blade position data at the next moment, the direction of the infrared radiation is adjusted. In this way, real-time tracking of the high-speed rotating blade and real-time dynamic adjustment of the infrared radiation direction can be achieved, which not only improves the de-icing efficiency, reduces unnecessary energy consumption, but also ensures the stability and reliability of the wind energy conversion system.

[0079] Furthermore, an infrared radiation de-icing system based on blade tracking is also provided.

[0080] Figure 4 is a block diagram of the infrared radiation de-icing system based on blade tracking according to the embodiments of the present application. As Figure 4As shown, the infrared radiation de-icing system 300 based on blade tracking according to an embodiment of the present application includes: a video stream acquisition module 310, configured to acquire a video stream of a blade area collected by a high-speed camera; an image frame extraction module 320, configured to extract image frames from the video stream of the blade area frame by frame to obtain a time queue of blade area image frames; an image preprocessing module 330, configured to preprocess the time queue of the blade area image frames to obtain a time queue of preprocessed blade area image frames; a blade state feature extraction module 340, configured to extract blade state features from the time queue of the preprocessed blade area image frames to obtain a time queue of blade state semantic encoding features; a multi-gated temporal dynamic propagation module 350, configured to perform multi-gated temporal dynamic propagation guided by the blade state on the time queue of the blade state semantic encoding features to obtain blade state temporal tracking semantic encoding features; and an infrared radiation direction adjustment module 360, configured to adjust the direction of infrared radiation based on the blade state temporal tracking semantic encoding features.

[0081] As described above, the infrared radiation de-icing system 300 based on blade tracking according to an embodiment of the present application can be implemented in various wireless terminals, such as a server having an infrared radiation de-icing algorithm based on blade tracking. In a possible implementation manner, the infrared radiation de-icing system 300 based on blade tracking according to an embodiment of the present application can be integrated into a wireless terminal as a software module and / or a hardware module. For example, the infrared radiation de-icing system 300 based on blade tracking can be a software module in the operating system of the wireless terminal, or can be an application program developed for the wireless terminal; of course, the infrared radiation de-icing system 300 based on blade tracking can also be one of many hardware modules of the wireless terminal.

[0082] Alternatively, in another example, the infrared radiation de-icing system 300 based on blade tracking and the wireless terminal can also be separate devices, and the infrared radiation de-icing system 300 based on blade tracking can be connected to the wireless terminal through a wired and / or wireless network and transmit interaction information in accordance with a predefined data format.

[0083] The embodiments of the present disclosure have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to technologies in the market, or to enable other ordinary skill in the art in the technical field to understand the embodiments disclosed herein.

Claims

1. An infrared radiation de-icing method based on blade tracking, characterized in that, Including: Obtain the video stream of the blade area collected by the high-speed camera; Extract each image frame of the video stream of the blade area frame by frame to obtain the time queue of the blade area image frames; Preprocess the time queue of the blade area image frames to obtain the time queue of the preprocessed blade area image frames; Extract the blade state feature from the time queue of the preprocessed blade area image frames to obtain the time queue of the blade state semantic encoding features; Perform multi-gated temporal dynamic propagation guided by the blade state on the time queue of the blade state semantic encoding features to obtain the blade state temporal tracking semantic encoding features; Adjust the direction of the infrared radiation based on the blade state temporal tracking semantic encoding features.

2. The infrared radiation de-icing method for blade tracking according to claim 1, wherein Preprocess the time queue of the blade area image frames to obtain the time queue of the preprocessed blade area image frames, including: Denoise each blade area image frame in the time queue of the blade area image frames to obtain the time queue of the denoised blade area image frames; Enhance the contrast of each denoised blade area image frame in the time queue of the denoised blade area image frames to obtain the time queue of the preprocessed blade area image frames.

3. The infrared radiation de-icing method for blade tracking according to claim 2, wherein Extract the blade state feature from the time queue of the preprocessed blade area image frames to obtain the time queue of the blade state semantic encoding features, including: Use a blade state feature extractor based on the Mobile-Former model to extract the blade state feature from the time queue of the preprocessed blade area image frames to obtain the time queue of the blade state semantic encoding feature vectors as the time queue of the blade state semantic encoding features.

4. The infrared radiation de-icing method for blade tracking according to claim 3, characterized in that, Perform multi-gated temporal dynamic propagation guided by the blade state on the time queue of the blade state semantic encoding features to obtain the blade state temporal tracking semantic encoding features, including: Map the time queue of the blade state semantic encoding feature vectors to the Poincaré space to obtain the time queue of the blade state semantic exotic modulation encoding feature vectors; Calculate the propagation representation of the time queue of the blade state semantic exotic modulation encoding feature vectors, and based on the propagation representation, perform significant propagation aggregation on the time queue of the blade state semantic encoding feature vectors to obtain the blade state temporal tracking semantic encoding vector as the blade state temporal tracking semantic encoding feature.

5. The infrared radiation de-icing method for blade tracking according to claim 4, wherein Calculate the propagation representation of the time queue of the blade state semantic exotic modulation encoding feature vectors, and based on the propagation representation, perform significant propagation aggregation on the time queue of the blade state semantic encoding feature vectors to obtain the blade state temporal tracking semantic encoding vector as the blade state temporal tracking semantic encoding feature, including: Perform forward LSTM sequence encoding on the time queue of the blade state semantic exotic modulation encoding feature vectors to obtain the blade state semantic feature exotic propagation representation vector as the propagation representation; Based on the propagation representation, calculate the weight values of each blade state semantic exotic modulation coding feature vector in the time queue of the blade state semantic exotic modulation coding feature vectors to obtain the time queue of the blade state semantic feature transfer modulation weights; Based on the time queue of the blade state semantic feature transfer modulation weights, calculate the position-weighted sum of the time queue of the blade state semantic exotic modulation coding feature vectors to obtain the blade state semantic exotic feature significant propagation representation vector; Fuse the blade state semantic exotic feature significant propagation representation vector and the blade state semantic feature exotic propagation representation vector to obtain the blade state time-sequence tracking semantic coding vector.

6. The infrared radiation de-icing method for blade tracking according to claim 5, characterized in that Based on the propagation representation, calculating the weight values of each blade state semantic exotic modulation coding feature vector in the time queue of the blade state semantic exotic modulation coding feature vectors to obtain the time queue of the blade state semantic feature transfer modulation weights includes: Calculate the semantic correlation degrees between each blade state semantic exotic modulation coding feature vector in the time queue of the blade state semantic exotic modulation coding feature vectors and the blade state semantic feature exotic propagation representation vector to obtain the time queue of the blade state semantic correlation degrees; Calculate the semantic space jump values of each blade state semantic exotic modulation coding feature vector in the time queue of the blade state semantic exotic modulation coding feature vectors relative to the blade state semantic feature exotic propagation representation vector to obtain the time queue of the blade state semantic space jump values; Perform multi-gated feature modulation on the time queue of the blade state semantic correlation degrees and the time queue of the blade state semantic space jump values to obtain the time queue of the blade state semantic feature transfer modulation weights.

7. The infrared radiation de-icing method for blade tracking according to claim 6, characterized in that, Fusing the blade state semantic exotic feature significant propagation representation vector and the blade state semantic feature exotic propagation representation vector to obtain the blade state time-sequence tracking semantic coding vector includes: Perform residual connection on the blade state semantic exotic feature significant propagation representation vector and the blade state semantic feature exotic propagation representation vector to obtain the blade state semantic exotic feature significant propagation compensation representation vector; Perform Euclidean space mapping on the blade state semantic exotic feature significant propagation compensation representation vector to obtain the blade state time-sequence tracking semantic coding vector.

8. The infrared radiation de-icing method for blade tracking according to claim 7, characterized in that, Based on the blade state time-sequence tracking semantic coding feature, adjust the direction of the infrared radiation, including: Based on the blade state time-sequence tracking semantic coding vector, determine the blade position data at the next moment; Based on the blade position data at the next moment, adjust the direction of the infrared radiation.

9. The infrared radiation de-icing method for blade tracking according to claim 8, wherein Based on the blade state time-sequence tracking semantic coding vector, determining the blade position data at the next moment includes: Input the blade state time-sequence tracking semantic coding vector into a blade position predictor based on a decoder to obtain the blade position data at the next moment.

10. An infrared radiation de-icing system based on blade tracking, characterized in that, Includes: A video stream acquisition module, configured to acquire a video stream of the blade area collected by a high-speed camera; An image frame extraction module, configured to extract image frames from the video stream of the blade area frame by frame to obtain a time queue of blade area image frames; An image preprocessing module for preprocessing the time queue of the blade area image frames to obtain a time queue of preprocessed blade area image frames; A blade state feature extraction module for extracting blade state features from the time queue of the preprocessed blade area image frames to obtain a time queue of blade state semantic encoding features; A multi-gated temporal dynamic propagation module for performing multi-gated temporal dynamic propagation guided by the blade state on the time queue of the blade state semantic encoding features to obtain blade state temporal tracking semantic encoding features; An infrared radiation direction adjustment module for adjusting the direction of infrared radiation based on the blade state temporal tracking semantic encoding features.

Images