Infrared radiation deicing method and system based on blade tracking

Abstract

The invention discloses an infrared radiation deicing method and system based on blade tracking, and the method comprises the steps: obtaining a video stream of a blade region through a high-speed camera, and extracting the blade state features of each image frame in the video stream of the blade region through a deep learning-based image processing technology; and further performing time sequence dynamic propagation on the blade state characteristics to excavate a change mode of the blade state along with time, capturing a movement track and a rotation characteristic of the blade, and realizing prediction of the position of the blade at the next moment based on the movement track and the rotation characteristic. And then, based on the blade position data at the next moment, adjusting the direction of infrared radiation. In this way, real-time tracking of the high-speed rotating blades and real-time dynamic adjustment of the infrared radiation direction can be achieved, the deicing efficiency is improved, unnecessary energy consumption is reduced, and the stability and reliability of a wind energy conversion system are guaranteed.

Description

Technical Field

[0001] The present application relates to the field of intelligent deicing, and more specifically, to an infrared radiation deicing method and system based on blade tracking. Background Art

[0002] In the field of wind power generation technology, the de-icing problem of blades has always been a technical challenge that needs to be solved urgently. Especially for wind turbines operating in cold climates, ice on the blades will not only lead to a significant decrease in wind turbine efficiency, but may also cause safety issues such as unbalanced loads, increased mechanical stress, and potential hazards caused by ice falling off. To this end, engineers have adopted a variety of de-icing methods, including mechanical de-icing, thermal airflow de-icing, and electric heating de-icing. However, these traditional methods often have problems such as slow response speed, high energy consumption, and potential damage to blade materials and structural strength.

[0003] With the development of science and technology, infrared radiation deicing has gradually attracted attention as a new and efficient method. Infrared radiation deicing refers to the use of infrared energy to directly act on the ice layer, so that the ice layer absorbs heat to melt. This method has the advantages of rapid response, energy concentration, and non-contact operation. It is very suitable for large rotating structures such as wind turbine blades. However, traditional infrared deicing technology also faces some difficulties. For example, it is impossible to accurately track high-speed rotating blades and adjust the direction of infrared radiation in real time to ensure effective energy transfer. Due to the large size of wind turbine blades and the fast rotation speed, it is difficult for infrared radiation sources in fixed directions to fully cover each blade, which may lead to poor deicing effect or energy waste.

[0004] Therefore, an infrared radiation deicing method based on blade tracking is expected. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an infrared radiation deicing method and system based on blade tracking, which obtains the video stream of the blade area through a high-speed camera, and uses image processing technology based on deep learning to extract the blade state characteristics of each image frame in the video stream of the blade area, and further performs time-series dynamic propagation of the blade state characteristics to mine the change pattern of the blade state over time, capture the motion trajectory and rotation characteristics of the blade, and based on this, realize the prediction of the position of the blade at the next moment. Then, based on the blade position data at the next moment, adjust the direction of infrared radiation. In this way, real-time tracking of high-speed rotating blades and real-time dynamic adjustment of the infrared radiation direction can be achieved, which not only improves the deicing efficiency, reduces unnecessary energy consumption, but also ensures the stability and reliability of the wind energy conversion system.

[0006] According to one aspect of the present application, a blade tracking-based infrared radiation deicing method is provided, which includes:

[0007] Obtain a video stream of the blade area captured by a high-speed camera;

[0008] Extracting image frames frame by frame from the video stream of the leaf region to obtain a time queue of image frames of the leaf region;

[0009] Preprocessing the time queue of the leaf region image frames to obtain a preprocessed time queue of the leaf region image frames;

[0010] Extracting leaf state features from the time queue of the preprocessed leaf region image frames to obtain a time queue of leaf state semantic coding features;

[0011] Performing leaf state-guided multi-gated timing dynamic propagation on the time queue of the leaf state semantic coding feature to obtain the leaf state timing tracking semantic coding feature;

[0012] Based on the semantic coding features of the blade state temporal tracking, the direction of the infrared radiation is adjusted.

[0013] According to another aspect of the present application, there is provided an infrared radiation deicing system based on blade tracking, comprising:

[0014] A video stream acquisition module is used to acquire a video stream of a blade area captured by a high-speed camera;

[0015] An image frame extraction module, used for extracting image frames frame by frame from the video stream of the leaf region to obtain a time queue of image frames of the leaf region;

[0016] An image preprocessing module, used for preprocessing the time queue of the leaf region image frames to obtain a time queue of the preprocessed leaf region image frames;

[0017] A leaf state feature extraction module, used for extracting leaf state features from the time queue of the preprocessed leaf region image frame to obtain a time queue of leaf state semantic coding features;

[0018] A multi-gated timing dynamic propagation module is used to perform multi-gated timing dynamic propagation guided by leaf states on the time queue of the leaf state semantic coding feature to obtain the leaf state timing tracking semantic coding feature;

[0019] The infrared radiation direction adjustment module is used to adjust the direction of the infrared radiation based on the semantic coding features of the blade state time series tracking.

[0020] Compared with the prior art, the present application provides an infrared radiation deicing method and system based on blade tracking, which obtains the video stream of the blade area through a high-speed camera, and uses image processing technology based on deep learning to extract the blade state characteristics of each image frame in the video stream of the blade area, and further performs time-series dynamic propagation of the blade state characteristics to mine the change pattern of the blade state over time, capture the motion trajectory and rotation characteristics of the blade, and based on this, realize the prediction of the position of the blade at the next moment. Then, based on the blade position data at the next moment, adjust the direction of infrared radiation. In this way, real-time tracking of high-speed rotating blades and real-time dynamic adjustment of the infrared radiation direction can be achieved, which not only improves the deicing efficiency and reduces unnecessary energy consumption, but also ensures the stability and reliability of the wind energy conversion system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 is a flow chart of an infrared radiation deicing method based on blade tracking according to an embodiment of the present application;

[0023] Figure 2 A data flow diagram of an infrared radiation deicing method based on blade tracking according to an embodiment of the present application;

[0024] Figure 3 is a flowchart of sub-step S5 of the infrared radiation deicing method based on blade tracking according to an embodiment of the present application;

[0025] Figure 4 4 is a block diagram of an infrared radiation deicing system based on blade tracking according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0027] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0028] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0029] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0030] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0031] In the technical solution of the present application, an infrared radiation deicing method based on blade tracking is proposed. Figure 1 The figure is a flow chart of an infrared radiation deicing method based on blade tracking according to an embodiment of the present application. Figure 2 FIG. 1 is a data flow diagram of an infrared radiation deicing method based on blade tracking according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the infrared radiation deicing method based on blade tracking according to the embodiment of the present application, the steps include: S1, acquiring a video stream of a blade area captured by a high-speed camera; S2, extracting image frames frame by frame from the video stream of the blade area to obtain a time queue of image frames in the blade area; S3, preprocessing the time queue of image frames in the blade area to obtain a time queue of image frames in the blade area after preprocessing; S4, extracting blade state features from the time queue of image frames in the blade area after preprocessing to obtain a time queue of blade state semantic coding features; S5, performing multi-gated temporal dynamic propagation guided by blade state on the time queue of blade state semantic coding features to obtain blade state temporal tracking semantic coding features; S6, adjusting the direction of infrared radiation based on the blade state temporal tracking semantic coding features.

[0032] In particular, the S1 obtains a video stream of the blade area captured by a high-speed camera. A high-speed camera can capture images at a frame rate much higher than that of a conventional camera device, thereby ensuring high-precision dynamic monitoring of rapidly rotating wind turbine blades. The speed of wind turbine blades is extremely fast when in operation, especially in large wind turbine generator sets, where the rotation speed can reach dozens or even hundreds of revolutions per minute. In this case, it is difficult for a camera of ordinary speed to capture a clear and stable blade image, while a high-speed camera can record video at a higher frame rate (the number of image frames captured per second) than an ordinary camera device, and continuously capture multiple image frames within a very short time interval, thereby providing enough frames to ensure that the position and state of the blades at each moment can be accurately recorded. The continuous video stream captured by the high-speed camera contains not only the spatial information of the blades, but also the information in the time dimension, which is crucial for understanding the state changes of the blades in different time periods. In one embodiment of the present application, a Phantom V2511 high-speed camera is used to capture a video stream of the blade area. The Phantom V2511 can capture up to 36,700 frames per second (fps) in black and white mode, or 24,000 fps in color mode. Such high frame rates are very useful for capturing fast-changing phenomena such as high-speed wind turbine blades.

[0033] It is worth noting that when installing and positioning high-speed cameras, they should be placed in a position that best covers the rotation path of the blades, usually on the wind turbine tower or surrounding stable structures. The installation location must ensure that there are no obstructions in the camera's field of view and minimize external light interference. For large wind turbines, multiple cameras may be required to work together to fully monitor the conditions of all blades. In addition, considering the various severe weather conditions in outdoor environments (such as strong winds, rain and snow, extremely cold temperatures, etc.), the design and protection measures of high-speed cameras also need special consideration. For example, waterproof and dustproof casings, heated glass lenses to prevent frost, and other means are used to ensure normal operation even in extreme environments.

[0034] Synchronization and calibration is another critical step. It is very important to ensure the clock synchronization between the high-speed camera and other system components (such as image processing unit, control system, etc.) to ensure that the timestamp of the collected data is accurate. During the first deployment or regular maintenance, the camera must be calibrated, including focal adjustment, horizontal and vertical angle correction, etc., to ensure that the captured image can truly reflect the actual movement of the blade. At the same time, the system should include a self-diagnostic function that can detect the working status of the camera (such as whether there is a fault, whether it deviates from the preset parameters, etc.), and trigger an alarm to notify relevant personnel to check or repair when necessary. Perform health check procedures regularly to evaluate camera performance and update firmware or hardware components as needed to maintain the optimal operating state of the system.

[0035] In particular, the S2 extracts the image frames of the video stream of the blade area frame by frame to obtain a time queue of the image frames of the blade area. It should be understood that when the high-speed camera captures the video stream of the blade area, these video streams are actually composed of a series of continuous image frames. However, due to the high-speed rotation characteristics of the fan blades, there may be significant position differences between adjacent frames. If the frame extraction interval is too large, important changes in the blade state at certain moments may be ignored, thereby affecting the accuracy of subsequent analysis. Therefore, in the technical solution of the present application, the video stream of the blade area is extracted frame by frame to obtain a time queue of the image frames of the blade area. By extracting frame by frame, independent image frames can be separated from the video stream and arranged in chronological order to form a time queue. The time queue of the image frames of the blade area reflects the movement trajectory of the blade and the pattern of state changes, wherein each frame of the image represents the position and state of the blade at a specific time point, so that the system can more accurately understand the state of the blade changing over time, thereby improving the accuracy of position prediction.

[0036] In particular, the S3 preprocesses the time queue of the blade area image frames to obtain the time queue of the preprocessed blade area image frames. In a specific example of the present application, the process of preprocessing the time queue of the blade area image frames includes: denoising each blade area image frame in the time queue of the blade area image frames to obtain the time queue of the denoised blade area image frames; and contrast enhancing each denoised blade area image frame in the time queue of the denoised blade area image frames to obtain the time queue of the preprocessed blade area image frames. Here, considering that the high-speed camera will encounter changes in ambient light, complex and changeable outdoor weather environment, and inevitable motion blur during shooting, the collected images often contain a lot of noise. And noise may have a serious impact on the accuracy of wind turbine blade state recognition. For example, in ice detection, noise may be mistaken for ice layer, resulting in wrong judgment; and in blade position prediction, noise may cause the model to learn the wrong pattern, affecting the prediction accuracy. Therefore, each blade area image frame in the time queue of the blade area image frames is denoised to reduce the interference of noise on the image, so as to obtain a time queue of denoised blade area image frames. In addition, considering that wind turbine blades are usually in a vast natural environment with different lighting conditions, especially in low light or high contrast conditions, the contrast of the original image may be low, resulting in unclear blade contours and difficulty in distinguishing the blade itself from its background. Therefore, in the technical solution of the present application, each denoised blade area image frame in the time queue of the denoised blade area image frames is further contrast enhanced. Through contrast enhancement, the boundary between the blade and the surrounding environment can be made clearer, which facilitates the subsequent processing steps to identify and track the position of the blade, so as to obtain a time queue of pre-processed blade area image frames.

[0037] In particular, the S4 extracts blade state features from the time queue of the preprocessed blade area image frames to obtain a time queue of blade state semantic coding features. In the technical solution of the present application, a blade state feature extractor based on the Mobile-Former model is used to extract blade state features from the time queue of the preprocessed blade area image frames to obtain a time queue of blade state semantic coding feature vectors as the time queue of the blade state semantic coding features. The blade state feature extractor based on the Mobile-Former model is used to extract blade state features from the time queue of the preprocessed blade area image frames to extract deep semantic features of the blade state to obtain a time queue of blade state semantic coding feature vectors as the time queue of the blade state semantic coding features. Since the wind turbine blades are in a constantly changing environment, their surfaces may be affected by a variety of factors, such as wind speed, temperature, humidity, etc., resulting in different physical phenomena, such as icing. Traditional feature extraction methods often find it difficult to capture these complex patterns. The Mobile-Former model enhances its consistent performance under different conditions by introducing advanced training strategies and technical means, such as self-supervised learning and transfer learning. By using a blade state feature extractor based on the Mobile-Former model to extract blade state features from the time queue of the preprocessed blade area image frames, the most representative blade features can be mined from a large amount of image data, including but not limited to the shape, edge contour and any abnormal changes (such as icing) of the blade, so as to more accurately identify the state of the blade, thereby providing a scientific basis for infrared radiation deicing.

[0038] In one example, while extracting features from a single frame, Mobile-Forme uses temporal information to establish connections between adjacent frames and within the entire sequence. This helps capture the trajectory of leaf motion and its dynamic characteristics, thereby providing richer contextual information for subsequent predictions. The feature maps obtained through the above process contain a lot of information about the state of the leaves, but they are still in a relatively primitive form. In order to better serve downstream tasks (such as position prediction), these features need to be further mapped to a higher-level space, namely, semantic encoding. In this process, Mobile-Former uses an attention mechanism to highlight the most important parts for describing the state of the leaves, while suppressing irrelevant or redundant information to obtain the semantic encoding feature vector of the leaf state.

[0039] As the features of each frame are gradually extracted, all the generated semantic encoding feature vectors of the leaf state are sorted according to the corresponding timestamps to form a continuous time series. This time series not only records the specific state of the leaf at each moment, but also implies any change trend over time, which becomes an important basis for subsequent analysis and decision-making.

[0040] It is worth noting that the working environment of wind turbines is constantly changing, so the feature extraction model used also needs to have a certain degree of flexibility and self-evolution capabilities. Mobile-Former supports online learning and incremental training, which means that it can continuously adjust its parameters as new data arrives to maintain the best working state.

[0041] In particular, the S5 performs a multi-gated time series dynamic propagation guided by the blade state on the time queue of the semantic coding feature of the blade state to obtain the semantic coding feature of the blade state time series tracking. It should be understood that during the high-speed rotation of the wind turbine blade, the motion trajectory and rotation speed of the blade may be affected by many factors, such as wind direction, wind speed, blade shape and mass distribution of the surface ice layer. Although the static feature extraction of the blade state using the Mobile-Former model can describe the blade motion state at a certain moment, it cannot reveal the intrinsic connection between the blade states at each moment. In order to accurately predict the future position of the blade, it is necessary to accurately model the motion trajectory of the blade. Therefore, further, the multi-gated time series dynamic propagation guided by the blade state is performed on the time queue of the semantic coding feature of the blade state, so as to capture the trend of the blade state evolution over time and obtain the semantic coding feature of the blade state time series tracking. Specifically, each gated unit can be regarded as a small memory module, which is responsible for recording and updating the state changes of the wind turbine blade in a specific time period. Through a more complex feature fusion and propagation mechanism, the system can better understand the behavior pattern of the blade at different times, thereby providing support for more accurate state assessment. In a specific example of this application, Figure 3 As shown, the S5 includes: S51, mapping the time queue of the leaf state semantic coding feature vector to the Poincare space to obtain the time queue of the leaf state semantic heterodomain modulation coding feature vector; S52, calculating the propagation representation of the time queue of the leaf state semantic heterodomain modulation coding feature vector, and based on the propagation representation, performing significant propagation aggregation on the time queue of the leaf state semantic coding feature vector to obtain the leaf state timing tracking semantic coding vector as the leaf state timing tracking semantic coding feature.

[0042] Specifically, the S51 maps the time queue of the semantic encoding feature vector of the blade state to the Poincare space to obtain the time queue of the semantic heterogeneous modulation encoding feature vector of the blade state. It should be understood that in the deicing process, the state change of the fan blade is highly nonlinear, and there is a complex hierarchical relationship between the blade states at each moment, and in the Poincare space, this hierarchical structure can be more intuitively represented. By mapping the semantic encoding feature vector of the blade state to the Poincare space, the interaction and dependency between features at different levels can be better captured. Compared with the fixed straight-line distance metric in the Euclidean space, in the Poincare space, the distance between two points changes dynamically according to the geometric relationship between the two points. This nonlinear distance metric is particularly suitable for processing data with complex internal structures. For fan blades, their state changes are often not simple linear processes, but the result of the interweaving of multiple factors. The distance metric of the Poincare space can better reflect the nature of these complex changes, making the model more accurate in evaluating the similarities and differences between different states. In addition, due to the boundary effect of the Poincare space, when two states are very similar, the distance in the Poincare space will be very close, which helps to improve the model's sensitivity to subtle changes, thereby enhancing the ability to recognize changes in leaf states. More specifically, the time queue of the leaf state semantic encoding feature vector is mapped to the Poincare space to obtain the time queue of the leaf state semantic heterogeneous modulation encoding feature vector using the following formula; wherein the formula is:

[0043] V={v1,v2,...,v i ,...,v n}

[0044] h i =W1v i W2

[0045] H={h1,h2,...,h i ,...,h n}

[0046] Where V is the time sequence of the leaf state semantic encoding feature vector, v1, v2, v i and v n are the 1st, 2nd, ith and nth leaf state semantic encoding feature vectors in the time queue of the leaf state semantic encoding feature vector, W1 and W2 are the first weight matrix and the second weight matrix, h1, h2, h i and h nThey are respectively the 1st, 2nd, ith and nth leaf state semantic heterodomain modulation coding feature vectors in the time queue of the leaf state semantic heterodomain modulation coding feature vectors, and H is the time queue of the leaf state semantic heterodomain modulation coding feature vectors.

[0047] Specifically, the S52 calculates the propagation representation of the time queue of the semantic heterogeneous modulation coding feature vector of the blade state, and based on the propagation representation, performs significant propagation aggregation on the time queue of the blade state semantic coding feature vector to obtain the blade state time series tracking semantic coding vector as the blade state time series tracking semantic coding feature. In an embodiment of the present application, first, the time queue of the semantic heterogeneous modulation coding feature vector of the blade state is forward LSTM sequence encoded to obtain the blade state semantic feature heterogeneous propagation representation vector as the propagation representation. It should be understood that for the blade state, some changes may occur instantaneously, such as the rapid adjustment of the blade angle due to a sudden change in wind speed; while other changes may involve a longer time span, such as ice accumulation caused by a gradual decrease in temperature. The forward LSTM sequence encoding can freely adjust the memory content on different time scales through the internal gating structure (input gate, forget gate and output gate), so that overfitting can be avoided when processing long time series while maintaining memory of key information. More specifically, the time queue of the leaf state semantic heterogeneous modulation coding feature vector is forward LSTM sequence encoded using the following sequence encoding formula to obtain the leaf state semantic feature heterogeneous propagation representation vector; wherein the sequence encoding formula is:

[0048] h n+1 =LSTM(H)

[0049] Among them, LSTM is the forward LSTM encoding, h n+1 It is the heterogeneous propagation representation vector of leaf state semantic features.

[0050] Next, based on the propagation representation, the weight values ​​of each leaf state semantic heterodomain modulation coding feature vector in the time queue of the leaf state semantic heterodomain modulation coding feature vector are calculated to obtain the time queue of the leaf state semantic feature transmission modulation weight. Specifically, first, the semantic association between each leaf state semantic heterodomain modulation coding feature vector in the time queue of the leaf state semantic heterodomain modulation coding feature vector and the leaf state semantic feature heterodomain propagation representation vector is calculated to obtain the time queue of the leaf state semantic association; here, the semantic association is used to represent the semantic intrinsic association between the message propagation feature in the Poincare space and each leaf state semantic heterodomain modulation coding feature vector, so as to facilitate the subsequent introduction of the attention mechanism to enhance the model's attention to the leaf state feature. Next, the semantic space jump value of each leaf state semantic heterodomain modulation coding feature vector in the time queue of the leaf state semantic heterodomain modulation coding feature vector relative to the leaf state semantic feature heterodomain propagation representation vector is calculated to obtain the time queue of the leaf state semantic space jump value. This indicator measures the changing trend and rate of feature expression over time by the degree of difference in the spatial span between the message propagation feature in the Poincare space and each leaf state semantic heterodomain modulation coding feature vector. The calculation of the semantic space jump value can not only reveal the dynamic characteristics of the blade state features, but also discover periodic patterns or abnormal events in the data. Then, the time queue of the blade state semantic association and the time queue of the blade state semantic space jump value are modulated by multiple gated features to obtain the time queue of the blade state semantic feature transfer modulation weight. It should be understood that the state change of the wind turbine blade is a complex nonlinear process, and the features at different time points are not equally important for understanding the overall behavior. For example, during the deicing process, the ice layer on the blade surface may suddenly fall off at certain specific moments, thereby changing the motion characteristics of the blade. In order to accurately capture this sudden state change, the semantic association and semantic space jump value are combined through multiple gating mechanisms to adaptively and selectively strengthen or weaken the transmission of specific information according to the relationship between features, and a transfer modulation weight is assigned to the features at each time point to dynamically adjust the direction and intensity of the information flow. This weight reflects the importance of the feature at that moment for subsequent predictions, so that the model can focus more on the key information that really affects the position state of the blade. More specifically, the weight values ​​of each leaf state semantic heterogeneous modulation coding feature vector in the time queue of the leaf state semantic heterogeneous modulation coding feature vector are calculated by the following weight calculation formula to obtain the time queue of the leaf state semantic feature transmission modulation weight; wherein the weight calculation formula is:

[0051]

[0052] α={α1,α2,...,αi ,...,α n}

[0053] β i =||h n+1 -h i ||

[0054] β={β1,β2,...,β i ,...,β n}

[0055]

[0056] Among them, ‖·‖ is the norm of the calculated vector, arccosh(·) is the inverse hyperbolic cosine function, α1, α2, α i and α n are the first, second, i-th and n-th leaf state semantic associations in the time queue of leaf state semantic associations, α is the time queue of the leaf state semantic association, β1, β2, β i and β n are the first, second, i-th and n-th leaf state semantic space jump values ​​in the time queue of the leaf state semantic space jump value, β is the time queue of the leaf state semantic space jump value, W α and W β are the semantic association weight matrix and the semantic space weight matrix respectively. is matrix multiplication, b is the bias vector, sigmoid(·) is the sigmoid function, γ t It is the temporal queue of the modulation weights transferred by the semantic features of leaf states.

[0057] Furthermore, based on the time queue of the modulation weights of the leaf state semantic feature transmission, the position-weighted sum of the time queue of the leaf state semantic heterogeneous modulation coding feature vector is calculated to obtain the leaf state semantic heterogeneous feature significant propagation representation vector. The importance of the feature and the temporal changes are comprehensively considered by weighted summation to obtain a leaf state semantic heterogeneous feature significant propagation representation vector that comprehensively reflects the characteristics of the leaf state change characteristics evolving over time.

[0058] Finally, the leaf state semantic heterogeneous feature significant propagation representation vector and the leaf state semantic feature heterogeneous propagation representation vector are fused to obtain the leaf state time series tracking semantic encoding vector. In the technical solution of the present application, the leaf state semantic heterogeneous feature significant propagation representation vector and the leaf state semantic feature heterogeneous propagation representation vector are residually connected to obtain the leaf state semantic heterogeneous feature significant propagation compensation representation vector; it should be understood that deep neural networks are prone to gradient vanishing or explosion problems during training, especially when processing long time series data, this phenomenon is more obvious. When the number of network layers increases, the gradient of back propagation may gradually approach zero, making it difficult for the model to effectively update parameters, affecting the learning effect. By introducing residual connections, the gradient can be directly transferred from the previous layer to the next layer, avoiding excessive attenuation of information flow in the network. In this technical solution, the residual connection ensures that the model can maintain good training performance even when dealing with complex leaf state changes, thereby improving the stability and reliability of the system. Further, the leaf state semantic heterogeneous feature significant propagation compensation representation vector is Euclidean space mapped to obtain the leaf state time series tracking semantic encoding vector. This process converts the leaf state semantic heterogeneous feature significant propagation compensation representation vector that has undergone complex processing back to the traditional Euclidean space, which is convenient for subsequent standard machine learning algorithms to use. Through appropriate mapping functions and parameter adjustments, effective projection from high-dimensional feature space to low-dimensional space can be achieved, which facilitates feature dimensionality reduction and visualization. More specifically, the leaf state semantic heterogeneous feature significant propagation representation vector and the leaf state semantic feature heterogeneous propagation representation vector are fused with the following fusion formula to obtain the leaf state time series tracking semantic encoding vector; wherein, the fusion formula is:

[0059]

[0060] v n+1 =f euclidean (h n+1 )=W3h n+1 ′W4

[0061] Among them, γ i Yes i The corresponding leaf state semantic feature transfer modulation weight, n is the number of vectors in the time queue of the leaf state semantic heterogeneous modulation coding feature vector, h n+1 ' is the leaf state semantic heterogeneous feature significant propagation compensation representation vector, W3 and W4 are the third and fourth weight matrices respectively, f euclidean (·) is the Euclidean space mapping operation, v n+1 It is the leaf state temporal tracking semantic encoding vector.

[0062] In particular, the S6 adjusts the direction of infrared radiation based on the semantic coding features of the blade state time series tracking. That is, in the technical solution of the present application, first, based on the semantic coding vector of the blade state time series tracking, the blade position data at the next moment is determined. Specifically, the blade state time series tracking semantic coding vector is input into the decoder-based blade position predictor to obtain the blade position data at the next moment. That is, the specific physical quantity, that is, the position data of the blade at the next moment, is recovered from the high-level abstract features (the semantic coding vector of the blade state time series tracking). To achieve this goal, the decoder usually adopts a recursive neural network (RNN) or its variants (such as LSTM, GRU), because these models are good at capturing long-term dependencies in sequence data and can effectively handle nonlinear transformations. Specifically, the decoder parses the received semantic coding of the blade state time series tracking frame by frame through an internal gating mechanism, and gradually builds a prediction of the future position of the blade. The decoding result of each frame depends not only on the input features of the current moment, but also refers to the prediction results of the previous moment, thereby ensuring the consistency and stability of the prediction process. Further based on the blade position data at the next moment, the direction of infrared radiation is adjusted. Here, the blade tracking method can be used to perform directional heating according to actual needs. Since the blade position can be tracked in real time and the radiation direction can be flexibly adjusted, the infrared radiation deicing method based on blade tracking can better adapt to different icing conditions. For areas with severe local icing, the system can increase the radiation intensity when the blades approach, concentrating its strength to quickly melt the ice layer; for slightly frozen parts, lower intensity radiation can be used to avoid material damage caused by excessive heating. In addition, this dynamic adjustment can also effectively prevent the problem of uneven deicing caused by the different rotation speeds of the blades, ensuring that each part is fully treated.

[0063] Preferably, inputting the leaf state time series tracking semantic coding vector into a decoder-based leaf position predictor to obtain leaf position data at the next moment comprises:

[0064] The absolute value sum and the square root of the square sum of all eigenvalues ​​of the leaf state timing tracking semantic coding vector are calculated to obtain the first and second leaf state timing tracking semantic coding space structure values, that is:

[0065] w1=Σ i |f i |

[0066]

[0067] Among them, f i represents the feature value of the i-th position in the leaf state temporal tracking semantic encoding vector, w1 represents the f iThe corresponding first leaf state time series tracking semantic encoding space structure value, w2 represents the f i The corresponding second leaf state temporal tracking semantic encoding spatial structure value;

[0068] Determine the total number m of eigenvalues ​​of the leaf state temporal tracking semantic coding vector;

[0069] For each eigenvalue of the leaf state time series tracking semantic coding vector, the first leaf state time series tracking semantic coding spatial structure value minus the product of the eigenvalue and the total number of eigenvalues ​​is calculated to obtain the first leaf state time series tracking semantic coding long-range dependency value x i =w1-f i ×m;

[0070] Among them, x i Indicates the f i The corresponding first leaf state temporal tracking semantically encodes the long-range dependency value;

[0071] Calculate the second leaf state temporal tracking semantic coding long-range dependency value obtained by multiplying the square root of the total number of eigenvalues ​​by the product of the eigenvalues ​​minus the second leaf state temporal tracking semantic coding spatial structure value

[0072] Among them, y i Indicates the f i The corresponding second leaf state temporal tracking semantically encodes the long-range dependency value;

[0073] The exponential value calculated by taking the long-range dependency value of the first leaf state time series tracking semantic coding as the exponent of the natural constant and the inverse of the long-range dependency value of the second leaf state time series tracking semantic coding are weighted summed to obtain the optimized eigenvalue corresponding to each eigenvalue as well as

[0074] Among them, f' i Indicates the f i The corresponding optimized eigenvalues, a and b represent different weighting parameters;

[0075] The optimized leaf state time-series tracking semantic coding vector composed of the optimized feature values ​​is input into a decoder-based leaf position predictor to obtain leaf position data at the next moment.

[0076] Here, when each leaf state semantic coding feature vector in the time queue of the leaf state semantic coding feature vector represents the image semantics and posture state coding features of the leaf area image frame at a single time point, after the feature dynamic propagation of the cross-domain multiple gate structure is performed on it, the leaf state temporal tracking semantic coding vector will also have significant cross-domain propagation distribution spatial structure differences due to the differences in the structural dynamics between local time domains, affecting the convergence consistency of the decoder, and thus affecting the accuracy of the leaf position data at the next moment obtained by the leaf position predictor based on the decoder.

[0077] Based on this, in view of the possible spatial structure loss of the feature set of the leaf state timing tracking semantic coding vector in the high-dimensional space, which causes the weight matrix of the decoder to implicitly infer the spatial structure information based on the features, resulting in inconsistent convergence. A long-distance feature dependency relationship is established based on the spatial structure representation of the leaf state timing tracking semantic coding vector relative to the overall feature scale of the leaf state timing tracking semantic coding vector to establish the feature local connectivity of the leaf state timing tracking semantic coding vector, and the spatial ambiguous information of the object feature value is captured through the unstructured feature value point prediction of the leaf state timing tracking semantic coding vector, thereby improving the spatial inductive bias perception ability of the feature set of the leaf state timing tracking semantic coding vector, improving the convergence consistency of the decoder, and improving the accuracy of the leaf position data at the next moment obtained by the leaf position predictor based on the decoder input of the leaf state timing tracking semantic coding vector.

[0078] In summary, according to the embodiment of the present application, the infrared radiation deicing method based on blade tracking is explained, which obtains the video stream of the blade area through a high-speed camera, and uses deep learning-based image processing technology to extract the blade state characteristics of each image frame in the video stream of the blade area, and further performs time-series dynamic propagation of the blade state characteristics to mine the change pattern of the blade state over time, capture the motion trajectory and rotation characteristics of the blade, and based on this, realize the prediction of the position of the blade at the next moment. Then, based on the blade position data at the next moment, adjust the direction of infrared radiation. In this way, real-time tracking of high-speed rotating blades and real-time dynamic adjustment of the infrared radiation direction can be achieved, which not only improves the deicing efficiency, reduces unnecessary energy consumption, but also ensures the stability and reliability of the wind energy conversion system.

[0079] Furthermore, an infrared radiation deicing system based on blade tracking is also provided.

[0080] Figure 4 FIG. 1 is a block diagram of an infrared radiation deicing system based on blade tracking according to an embodiment of the present application. Figure 4As shown, according to the embodiment of the present application, the infrared radiation deicing system 300 based on blade tracking includes: a video stream acquisition module 310, which is used to acquire a video stream of a blade area captured by a high-speed camera; an image frame extraction module 320, which is used to extract image frames frame by frame from the video stream of the blade area to obtain a time queue of image frames in the blade area; an image preprocessing module 330, which is used to preprocess the time queue of image frames in the blade area to obtain a time queue of image frames in the blade area after preprocessing; a blade state feature extraction module 340, which is used to extract blade state features from the time queue of image frames in the blade area after preprocessing to obtain a time queue of blade state semantic coding features; a multi-gated temporal dynamic propagation module 350, which is used to perform multi-gated temporal dynamic propagation guided by blade states on the time queue of the blade state semantic coding features to obtain blade state temporal tracking semantic coding features; an infrared radiation direction adjustment module 360, which is used to adjust the direction of infrared radiation based on the blade state temporal tracking semantic coding features.

[0081] As described above, the infrared radiation deicing system 300 based on blade tracking according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an infrared radiation deicing algorithm based on blade tracking. In a possible implementation, the infrared radiation deicing system 300 based on blade tracking according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the infrared radiation deicing system 300 based on blade tracking can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the infrared radiation deicing system 300 based on blade tracking can also be one of the many hardware modules of the wireless terminal.

[0082] Alternatively, in another example, the blade tracking-based infrared radiation deicing system 300 and the wireless terminal may also be separate devices, and the blade tracking-based infrared radiation deicing system 300 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0083] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An infrared radiation deicing method based on blade tracking, characterized in that: include: Obtain a video stream of the blade area captured by a high-speed camera; Extracting image frames frame by frame from the video stream of the leaf region to obtain a time queue of image frames of the leaf region; Preprocessing the time queue of the leaf region image frames to obtain a preprocessed time queue of the leaf region image frames; Extracting leaf state features from the time queue of the preprocessed leaf region image frames to obtain a time queue of leaf state semantic coding features; Performing leaf state-guided multi-gated timing dynamic propagation on the time queue of the leaf state semantic coding feature to obtain the leaf state timing tracking semantic coding feature; Based on the semantic coding features of the blade state temporal tracking, the direction of the infrared radiation is adjusted.

2. The blade tracking infrared radiation deicing method according to claim 1, characterized in that: Preprocessing the time queue of the leaf region image frames to obtain a preprocessed time queue of the leaf region image frames includes: Performing denoising processing on each leaf region image frame in the time queue of the leaf region image frames to obtain a time queue of denoised leaf region image frames; Contrast enhancement is performed on each of the denoised leaf region image frames in the time queue of the denoised leaf region image frames to obtain the time queue of the preprocessed leaf region image frames.

3. The blade tracking infrared radiation deicing method according to claim 2, characterized in that: Extracting leaf state features from the time queue of the preprocessed leaf region image frames to obtain a time queue of leaf state semantic coding features includes: A leaf state feature extractor based on the Mobile-Former model is used to extract leaf state features from the time queue of the preprocessed leaf area image frame to obtain a time queue of leaf state semantic coding feature vectors as the time queue of the leaf state semantic coding features.

4. The blade tracking infrared radiation deicing method according to claim 3, characterized in that: The time queue of the leaf state semantic coding feature is subjected to multi-gated timing dynamic propagation guided by the leaf state to obtain the leaf state timing tracking semantic coding feature, including: Mapping the time queue of the leaf state semantic coding feature vector to the Poincare space to obtain the time queue of the leaf state semantic heterogeneous modulation coding feature vector; The propagation representation of the time queue of the leaf state semantic heterogeneous modulation coding feature vector is calculated, and based on the propagation representation, the time queue of the leaf state semantic coding feature vector is significantly propagated and aggregated to obtain the leaf state timing tracking semantic coding vector as the leaf state timing tracking semantic coding feature.

5. The blade tracking infrared radiation deicing method according to claim 4, characterized in that: Calculating the propagation representation of the time queue of the leaf state semantic heterogeneous modulation coding feature vector, and based on the propagation representation, performing significant propagation aggregation on the time queue of the leaf state semantic coding feature vector to obtain the leaf state timing tracking semantic coding vector as the leaf state timing tracking semantic coding feature, including: Performing forward LSTM sequence encoding on the time queue of the leaf state semantic heterogeneous modulation coding feature vector to obtain a leaf state semantic feature heterogeneous propagation representation vector as the propagation representation; Based on the propagation representation, the weight values ​​of each leaf state semantic heterogeneous modulation coding feature vector in the time queue of the leaf state semantic heterogeneous modulation coding feature vector are calculated to obtain the time queue of the leaf state semantic feature transmission modulation weight; Based on the time queue of the leaf state semantic feature transmission modulation weight, calculating the position-weighted sum of the time queue of the leaf state semantic heterogeneous modulation coding feature vector to obtain the leaf state semantic heterogeneous feature significant propagation representation vector; The leaf state semantic heterogeneous feature significant propagation representation vector and the leaf state semantic feature heterogeneous propagation representation vector are fused to obtain the leaf state temporal tracking semantic encoding vector.

6. The blade tracking infrared radiation deicing method according to claim 5, characterized in that: Based on the propagation representation, the weight values ​​of each leaf state semantic heterogeneous modulation coding feature vector in the time queue of the leaf state semantic heterogeneous modulation coding feature vector are calculated to obtain a time queue of the leaf state semantic feature transmission modulation weight, including: Calculate the semantic association between each leaf state semantic heterodomain modulation coding feature vector in the time queue of the leaf state semantic heterodomain modulation coding feature vector and the leaf state semantic feature heterodomain propagation representation vector to obtain a time queue of leaf state semantic association; Calculate the semantic space jump value of each leaf state semantic heterodomain modulation coding feature vector in the time queue of the leaf state semantic heterodomain modulation coding feature vector relative to the leaf state semantic feature heterodomain propagation representation vector to obtain the time queue of the leaf state semantic space jump value; A multi-gated feature modulation is performed on the time queue of the leaf state semantic association and the time queue of the leaf state semantic space jump value to obtain a time queue of the leaf state semantic feature transfer modulation weight.

7. The blade tracking infrared radiation deicing method according to claim 6, characterized in that: The leaf state semantic heterogeneous feature significant propagation representation vector and the leaf state semantic feature heterogeneous propagation representation vector are integrated to obtain the leaf state temporal tracking semantic encoding vector, including: Residual connection is performed on the leaf state semantics heterogeneous feature significant propagation representation vector and the leaf state semantics feature heterogeneous propagation representation vector to obtain a leaf state semantics heterogeneous feature significant propagation compensation representation vector; The leaf state semantic foreign feature significant propagation compensation representation vector is mapped into Euclidean space to obtain the leaf state temporal tracking semantic encoding vector.

8. The blade tracking infrared radiation deicing method according to claim 7, characterized in that: Adjusting the direction of infrared radiation based on the semantic coding feature of the leaf state temporal tracking includes: Determine the blade position data at the next moment based on the blade state time series tracking semantic coding vector; Based on the blade position data at the next moment, the direction of the infrared radiation is adjusted.

9. The blade tracking infrared radiation deicing method according to claim 8, characterized in that: Based on the leaf state time series tracking semantic coding vector, determining the leaf position data at the next moment includes: The leaf state timing tracking semantic coding vector is input into a decoder-based leaf position predictor to obtain the leaf position data at the next moment.

10. An infrared radiation deicing system based on blade tracking, characterized in that: include: A video stream acquisition module is used to acquire a video stream of a blade area captured by a high-speed camera; An image frame extraction module, used for extracting image frames frame by frame from the video stream of the leaf region to obtain a time queue of image frames of the leaf region; An image preprocessing module, used for preprocessing the time queue of the leaf region image frames to obtain a time queue of the preprocessed leaf region image frames; A leaf state feature extraction module, used for extracting leaf state features from the time queue of the preprocessed leaf region image frame to obtain a time queue of leaf state semantic coding features; A multi-gated timing dynamic propagation module is used to perform multi-gated timing dynamic propagation guided by leaf states on the time queue of the leaf state semantic coding feature to obtain the leaf state timing tracking semantic coding feature; The infrared radiation direction adjustment module is used to adjust the direction of the infrared radiation based on the semantic coding features of the blade state time series tracking.

Images