Wind power plant intelligent inspection system and method based on AR live-action video map

Abstract

The invention provides a wind power plant intelligent inspection system and method based on an AR live-action video map, and the method comprises the steps: obtaining a live-action video of a wind power plant, enabling the live-action video to correspond to the geographic position information of the wind power plant, and generating a wind power plant live-action video map; detecting the running state of the to-be-analyzed fan to obtain a detection result; the detection result corresponds to the number information of the to-be-analyzed fan, and fan state information is generated; and superposing the fan state information on the wind power plant live-action video map to form an AR live-action video map, and displaying the AR live-action video map to inspection personnel. Therefore, an inspector can check the full view of the wind power plant and the state information of each fan on the mobile terminal in real time.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent inspection, and particularly relates to a wind farm intelligent inspection system and method based on an AR real-scene video map. Background Art

[0002] Wind power is a clean and renewable energy source. However, the operation and maintenance of wind farms also face some challenges. Wind farms are usually distributed over a vast area, with a large number of wind turbines. The structure of wind turbines is complex, and the operating state of wind turbines is affected by various factors. Therefore, regular inspection and fault diagnosis of wind farms are important measures to ensure the safe, efficient, and stable operation of wind farms.

[0003] Traditional wind farm inspection methods mainly rely on manual on-site inspections. However, manual inspections are time-consuming and laborious and cannot cover all wind turbines. Therefore, there is an urgent need for an optimized wind farm inspection solution. Summary of the Invention

[0004] The present invention aims to at least solve one of the technical problems existing in the prior art and provides a new technical solution for a wind farm intelligent inspection system and method based on an AR real-scene video map.

[0005] According to a first aspect of the present invention, there is provided a wind farm intelligent inspection method based on an AR real-scene video map, which includes: obtaining a real-scene video of a wind farm, corresponding the real-scene video with geographical location information of the wind farm to generate a wind farm real-scene video map; detecting an operating state of a wind turbine to be analyzed to obtain a detection result; corresponding the detection result with identification information of the wind turbine to be analyzed to generate wind turbine state information; and superimposing the wind turbine state information on the wind farm real-scene video map to form an AR real-scene video map and displaying the AR real-scene video map to an inspector;

[0006] Wherein, detecting an operating state of a wind turbine to be analyzed to obtain a detection result includes:

[0007] obtaining a vibration waveform signal of the wind turbine to be analyzed collected by a vibration sensor during a predetermined time period;

[0008] performing data preprocessing on the vibration waveform signal to obtain a sequence of local vibration waveform signals;

[0009] extracting multi-scale vibration features from the sequence of local vibration waveform signals to obtain a sequence of multi-scale vibration waveform feature maps;

[0010] extracting temporal correlation features from the sequence of multi-scale vibration waveform feature maps to obtain a wind turbine vibration temporal state feature map; and

[0011] Based on the time - series state feature map of the fan vibration, determine whether the operating condition of the fan to be analyzed is abnormal.

[0012] According to the second aspect of the present invention, there is provided an intelligent inspection system for a wind farm based on an AR real - scene video map, which includes:

[0013] A real - scene video acquisition module, configured to acquire the real - scene video of the wind farm and correspond the real - scene video with the geographical location information of the wind farm to generate a real - scene video map of the wind farm;

[0014] A detection result generation module, configured to detect the operating state of the fan to be analyzed to obtain a detection result; a fan state information generation module, configured to correspond the detection result with the number information of the fan to be analyzed to generate fan state information;

[0015] And an AR real - scene video map generation module, configured to superimpose the fan state information on the real - scene video map of the wind farm to form an AR real - scene video map and display the AR real - scene video map to the inspection personnel;

[0016] Among them, the detection result generation module includes:

[0017] A vibration waveform signal acquisition unit, configured to acquire the vibration waveform signal of the fan to be analyzed collected by a vibration sensor within a predetermined time period;

[0018] A data pre - processing unit, configured to perform data pre - processing on the vibration waveform signal to obtain a sequence of local vibration waveform signals;

[0019] A multi - scale vibration feature extraction unit, configured to extract multi - scale vibration features from the sequence of local vibration waveform signals to obtain a sequence of multi - scale vibration waveform feature maps;

[0020] A time - series correlation feature extraction unit, configured to extract time - series correlation features from the sequence of multi - scale vibration waveform feature maps to obtain a fan vibration time - series state feature map; and

[0021] An operating condition determination unit of the fan to be analyzed, configured to determine whether the operating condition of the fan to be analyzed is abnormal based on the fan vibration time - series state feature map.

[0022] One technical effect of the present invention is:

[0023] In the embodiment of the present application, first, a real - scene video of a wind farm is acquired, and the real - scene video is associated with the geographical location information of the wind farm to generate a real - scene video map of the wind farm; then, the operating state of the wind turbine to be analyzed is detected to obtain a detection result; next, the detection result is associated with the serial number information of the wind turbine to be analyzed to generate wind turbine state information; finally, the wind turbine state information is superimposed on the real - scene video map of the wind farm to form an AR real - scene video map, and the AR real - scene video map is displayed to the inspection personnel. In this way, the inspection personnel can view the overall situation of the wind farm and the state information of each wind turbine in real time on the mobile terminal, significantly improving the inspection efficiency of the wind farm. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 FIG. is a flowchart of a wind farm intelligent inspection method based on an AR real - scene video map according to an embodiment of the present application.

[0025] Figure 2 FIG. is a schematic structural diagram of a wind farm intelligent inspection method based on an AR real - scene video map according to an embodiment of the present application.

[0026] Figure 3 FIG. is a block diagram of a wind farm intelligent inspection system based on an AR real - scene video map according to an embodiment of the present application.

[0027] Figure 4 FIG. is a schematic diagram of a scene of a wind farm intelligent inspection method based on an AR real - scene video map according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] Now, various exemplary embodiments of the present application will be described in detail with reference to the accompanying drawings. It should be noted that: unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present application.

[0029] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application and should not be construed as limiting the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0030] Unless otherwise specified, all technical and scientific terms used in the embodiments of the present application have the same meaning as commonly understood by those skilled in the technical field of the present application. The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the scope of the present application.

[0031] Wind power is a clean and renewable energy source that converts wind energy into electrical energy. Through wind turbines, wind energy is transformed into electricity, providing a sustainable power supply for humanity. Although wind power has many advantages, there are also some challenges in the operation and maintenance of wind farms.

[0032] Wind farms usually need to occupy a vast area to deploy a large number of wind turbines. Finding suitable land and carrying out planning and development may face some challenges, especially in densely populated areas or environmentally sensitive regions. A wind farm typically consists of dozens or even hundreds of wind turbines, and each turbine needs to be monitored, maintained, and repaired. Managing the operation and maintenance of a large-scale wind farm requires efficient organizational and management capabilities. Wind turbines are complex mechanical devices composed of many components, such as blades, generators, gearboxes, etc. These components may be subject to wear, corrosion, and damage under long-term operation and harsh environmental conditions. Ensuring the structural safety and reliability of wind turbines is an important challenge. The operating state of wind turbines is affected by various factors, such as wind speed, temperature, humidity, etc. Monitoring and analyzing the operating data of wind turbines can detect potential faults and problems in advance, but for large-scale wind farms, managing and analyzing a large amount of data is also a challenge. The maintenance and repair of wind turbines require regular inspections, lubrication, replacement of parts, etc. Since wind farms are usually located in remote areas, the difficulty of maintenance personnel arriving and repair equipment increases the complexity of maintenance work.

[0033] To address these challenges, the wind power industry has taken a series of measures. For example, by improving wind turbine design and material selection to enhance the reliability and durability of wind turbines. At the same time, using advanced monitoring technologies and data analysis methods to monitor the operating state of wind turbines in real time and give early warnings and diagnose potential faults. Conducting regular inspections and fault diagnoses of wind farms is an important measure to ensure the safe, efficient, and stable operation of wind farms. These activities help to detect potential problems and faults in a timely manner and take corresponding repair and restoration measures to minimize downtime and maintenance costs.

[0034] Traditional wind farm inspection methods mainly rely on manual on-site inspections, but this method has some problems, such as being time-consuming and laborious, and unable to cover all wind turbines. To solve these problems, the wind power industry is actively exploring and adopting new inspection technologies and methods to improve efficiency and accuracy. The following are some introductions: Using drones for wind farm inspections has become a common method. Drones can carry high-resolution cameras, infrared thermal imagers and other devices to quickly and comprehensively inspect wind turbines. They can cover a large range of wind turbines in a short time and provide high-quality images and data to help detect potential problems and damages. Internet of Things (IoT) and sensor technologies, by installing sensors and monitoring devices on wind turbines, can monitor the operating status and performance of wind turbines in real time. These devices can collect data such as wind speed, temperature, vibration, etc. and transmit them to the central control system for analysis and diagnosis. This method can help discover potential faults and anomalies and provide early warning and remote monitoring functions. Data analysis and artificial intelligence (AI), by using data analysis and artificial intelligence technologies, can process and analyze a large amount of inspection data to discover patterns, anomalies and trends. This can help optimize the inspection plan and predict wind turbine failures in advance, thereby reducing the frequency and scope of manual inspections.

[0035] In one embodiment of the present application, Figure 1 It is a flowchart of an intelligent wind farm inspection method based on an AR real-scene video map according to an embodiment of the present application. Figure 2 It is a schematic diagram of the architecture of an intelligent wind farm inspection method based on an AR real-scene video map according to an embodiment of the present application. As Figure 1 and Figure 2 shown, the intelligent wind farm inspection method based on the AR real-scene video map includes:

[0036] Step 110, obtain the real-scene video of the wind farm, and correspond the real-scene video with the geographical location information of the wind farm to generate a wind farm real-scene video map;

[0037] Step 120, detect the operating status of the wind turbine to be analyzed to obtain a detection result;

[0038] Step 130, correspond the detection result with the number information of the wind turbine to be analyzed to generate wind turbine status information;

[0039] And step 140, overlay the wind turbine status information on the wind farm real-scene video map to form an AR real-scene video map, and display the AR real-scene video map to the inspection personnel.

[0040] Among them, detecting the operating status of the wind turbine to be analyzed to obtain a detection result includes:

[0041] Step 121, obtain the vibration waveform signal of the fan to be analyzed collected by the vibration sensor within a predetermined time period;

[0042] Step 122, perform data preprocessing on the vibration waveform signal to obtain a sequence of local vibration waveform signals;

[0043] Step 123, extract multi-scale vibration features from the sequence of local vibration waveform signals to obtain a sequence of multi-scale vibration waveform feature maps;

[0044] Step 124, extract the temporal correlation features from the sequence of multi-scale vibration waveform feature maps to obtain a fan vibration temporal state feature map;

[0045] And, step 125, based on the fan vibration temporal state feature map, determine whether the operating condition of the fan to be analyzed is abnormal.

[0046] In the step 110, ensure that the obtained real-scene video has good quality, can clearly display each part and detail of the wind farm, and accurately record the geographical location information of each video segment, so as to correspond to the fan status information subsequently. Generating a real-scene video map of the wind farm can provide a more intuitive and comprehensive view of the wind farm for the inspection personnel, help them better understand the layout and location of the fans, and improve the inspection efficiency and accuracy.

[0047] In the step 120, select appropriate sensors and monitoring devices to obtain the operating data of the fan to be analyzed, such as vibration sensors, temperature sensors, etc., and ensure that the installation position and method of the sensors are correct and can accurately reflect the operating state of the fan. By detecting the operating state of the fan, potential faults and abnormalities can be discovered in time, and maintenance measures can be taken in advance to avoid downtime and losses.

[0048] In the step 130, ensure that the detection results are accurately corresponded to the number information of the fan to be analyzed, so as to generate fan status information subsequently, establish a complete database or information system, and record and manage the number and status information of the fans. Generating fan status information can provide a comprehensive understanding of the operating state of the fans, including normal operation, abnormal operation, maintenance status, etc., and provide decision-making basis and guidance for the inspection personnel.

[0049] In the step 140, ensure that the fan status information is accurately corresponded to the real-scene video map, and use appropriate technical means to overlay the information on the video, such as augmented reality (AR) technology, and ensure that the AR real-scene video map displayed to the inspection personnel is clear and easy to understand, and can intuitively display the status and location of the fans. The AR real-scene video map can provide more intuitive and real-time fan status information for the inspection personnel, help them quickly locate and judge the operating state of the fans, and improve the inspection efficiency and accuracy.

[0050] By using vibration sensors to collect data and extract multi-scale vibration features, combined with the operating condition judgment algorithm of the fan, the detection and judgment of the operating state of the fan to be analyzed can be realized. This method can provide more accurate and timely fan state information, help inspection personnel to carry out timely repair and maintenance, and ensure the safe operation of the wind farm.

[0051] Among them, in step 120, detecting the operating state of the fan to be analyzed to obtain a detection result, including: step 121, step 122, step 123, step 124 and step 125.

[0052] In the step 121, ensure the correct installation position and parameter settings of the vibration sensor to obtain accurate vibration data, and select an appropriate sampling frequency and sampling duration to fully capture the characteristics of the fan vibration. The vibration waveform signal can provide detailed information about the fan vibration, including vibration amplitude, frequency, period, etc., for subsequent data processing and feature extraction.

[0053] In the step 122, when performing data preprocessing, methods such as filtering, denoising, and downsampling can be adopted to remove noise and interference and extract the local signal of the vibration waveform. Data preprocessing can improve the quality and accuracy of the vibration signal and reduce the errors and interference in subsequent feature extraction.

[0054] In the step 123, multi-scale vibration feature extraction can adopt methods such as wavelet transform and time-frequency analysis to extract vibration features at different frequencies and time scales from the local signal. Multi-scale vibration features can provide more comprehensive and multi-dimensional fan vibration information, including low-frequency, high-frequency, transient and other features, which helps to more accurately describe the operating state of the fan.

[0055] In the step 124, time-series correlation feature extraction can adopt methods such as autocorrelation and cross-correlation to analyze the time-series relationship of the vibration feature map and capture the periodicity and change trend of the vibration signal. Time-series correlation features can provide temporal information of the fan vibration signal, including periodic changes, trend changes, etc., which helps to judge the operating state and abnormal conditions of the fan.

[0056] In the step 125, establish an appropriate judgment algorithm or model, and perform anomaly detection and operating state judgment according to the characteristics and change rules of the fan vibration time-series state feature map. Based on the analysis and judgment of the fan vibration time-series state feature map, it can accurately determine whether the operating condition of the fan is normal, timely detect abnormal conditions and take corresponding repair and maintenance measures.

[0057] The vibration waveform signal is collected by a vibration sensor, and through data preprocessing, multi-scale feature extraction, and time-series correlation feature extraction, a time-series state feature map of the fan vibration can be obtained, and the operating state is judged based on this feature map. This method can achieve accurate monitoring and anomaly detection of the fan operating state, providing strong support for the inspection and maintenance of the wind farm.

[0058] This application provides an intelligent inspection method for a wind farm based on an AR live video map. The specific steps include: Step 110, obtaining the live video of the wind farm and corresponding the live video with the geographical location information of the wind farm to generate a live video map of the wind farm; Step 120, detecting the operating state of the fan to be analyzed to obtain a detection result; Step 130, corresponding the detection result with the number information of the fan to be analyzed to generate fan state information; and Step 140, superimposing the fan state information on the live video map of the wind farm to form an AR live video map and displaying the AR live video map to the inspection personnel.

[0059] Considering that different frequency and amplitude vibration signals are generated during the operation of the fan, these signals reflect the interaction and coupling relationship between various components (such as blades, bearings, gears, etc.) inside the fan. When a component fails or is damaged, it will cause a change in the interaction and coupling with other components, thereby affecting the vibration characteristics of the entire system. Therefore, by analyzing the vibration signal, characteristic information reflecting the health status of the system can be extracted, thereby realizing the detection of the fan operating state.

[0060] However, due to the complexity and variability of the fan operating environment, as well as the nonlinearity and non-stationarity of the fan structure, the fan vibration signal has characteristics such as high dimension, high noise, and multi-scale, making it difficult for traditional vibration signal analysis methods to effectively extract useful characteristic information.

[0061] In particular, detecting the operating state of the fan to be analyzed is an important link in implementing the technical solution of this application. In the technical solution of this application, in order to accurately detect the operating state of the fan to be analyzed, the technical concept of this application is: using the vibration waveform signal generated during the operation of the fan to be analyzed to detect the operating state of the fan.

[0062] Based on this, in the technical solution of this application, first obtain the vibration waveform signal of the fan to be analyzed collected by the vibration sensor during a predetermined time period; and perform data preprocessing on the vibration waveform signal to obtain a sequence of local vibration waveform signals. Here, considering that if data analysis is performed on the entire vibration waveform signal, it is easy to overlook the detailed characteristic information hidden therein. Splitting it into a sequence of local vibration waveform signals can guide the subsequent network model to pay more attention to the local vibration waveform feature distribution.

[0063] In a specific example of the present application, data preprocessing is performed on the vibration waveform signal to obtain a sequence of local vibration waveform signals, including: signal segmentation is performed on the vibration waveform signal to obtain the sequence of local vibration waveform signals.

[0064] Then, multi-scale vibration features in the sequence of local vibration waveform signals are extracted to obtain a sequence of multi-scale vibration waveform feature maps. That is, the waveform feature distributions presented by each of the local vibration waveform signals in different spatial neighborhoods are captured.

[0065] In a specific example of the present application, multi-scale vibration features in the sequence of local vibration waveform signals are extracted to obtain a sequence of multi-scale vibration waveform feature maps, including: a deep learning network model is used to perform feature extraction on the sequence of local vibration waveform signals to obtain the sequence of multi-scale vibration waveform feature maps.

[0066] Further, the deep learning network model is a vibration waveform feature extractor including a first convolutional layer and a second convolutional layer; wherein, using the deep learning network model to perform feature extraction on the sequence of local vibration waveform signals to obtain the sequence of multi-scale vibration waveform feature maps, including: the sequence of local vibration waveform signals respectively passes through the vibration waveform feature extractor including the first convolutional layer and the second convolutional layer to obtain the sequence of multi-scale vibration waveform feature maps.

[0067] In a specific example of the present application, passing the sequence of local vibration waveform signals respectively through the vibration waveform feature extractor including the first convolutional layer and the second convolutional layer to obtain the sequence of multi-scale vibration waveform feature maps, including: inputting the sequence of local vibration waveform signals into the first convolutional layer of the vibration waveform feature extractor to obtain a first-scale vibration waveform feature map, wherein the first convolutional layer has a two-dimensional convolutional kernel of the first scale; inputting the sequence of local vibration waveform signals into the second convolutional layer of the vibration waveform feature extractor to obtain a second-scale vibration waveform feature map, wherein the second convolutional layer has a two-dimensional convolutional kernel of the second scale, and the first scale is different from the second scale; and, cascading the first-scale vibration waveform feature map and the second-scale vibration waveform feature map to obtain the sequence of multi-scale vibration waveform feature maps.

[0068] Next, extract the temporal correlation features in the sequence of the multi-scale vibration waveform feature maps to obtain the fan vibration temporal state feature map. Here, there is a certain correlation in the time dimension for the multi-scale vibration waveform features expressed by the local signals of the vibration waveform. Specifically, when the operating state of the fan to be analyzed is abnormal, its vibration waveform signal may exhibit a form different from the normal state in the time dimension. For example, the vibration waveform signal may have features such as sudden changes and abnormal peaks. These features can reflect changes in the internal structure or external environment of the fan, resulting in unstable operation or interference of the fan.

[0069] In a specific example of the present application, extracting the temporal correlation features in the sequence of the multi-scale vibration waveform feature maps to obtain the fan vibration temporal state feature map includes: passing the sequence of the multi-scale vibration waveform feature maps through a vibration waveform temporal correlation feature extractor based on a three-dimensional convolutional neural network model to obtain the fan vibration temporal state feature map.

[0070] Among them, the vibration waveform temporal correlation feature extractor based on the three-dimensional convolutional neural network model includes: an input layer, a convolutional layer, a pooling layer, an activation layer, and an output layer.

[0071] In a specific embodiment of the present application, determining whether the operating condition of the fan to be analyzed is abnormal based on the fan vibration temporal state feature map includes: performing feature distribution correction on the fan vibration temporal state feature map to obtain a corrected fan vibration temporal state feature map; and passing the corrected fan vibration temporal state feature map through a classifier to obtain a detection result, where the detection result is used to indicate whether the operating condition of the fan to be analyzed is abnormal.

[0072] Here, each multi-scale vibration waveform feature map in the sequence of multi-scale vibration waveform feature maps represents the local neighborhood waveform semantic features of each vibration waveform local signal based on convolutional kernels of different scales, and each feature matrix along the channel dimension of the multi-scale vibration waveform feature maps follows the channel dimension distribution of the vibration waveform feature extractor including the first convolutional layer and the second convolutional layer. Further, when the sequence of multi-scale vibration waveform feature maps passes through the vibration waveform temporal correlation feature extractor based on the three-dimensional convolutional neural network model, the vibration waveform temporal correlation feature extractor based on the three-dimensional convolutional neural network model can use three-dimensional convolutional kernels to capture the context semantic correlation in the sequence of multi-scale vibration waveform feature maps, that is, the context correlation information of the waveform semantic features of each vibration waveform local signal in the temporal dimension. However, due to the scale difference between the three-dimensional convolutional kernel and the two-dimensional convolutional kernel, during three-dimensional convolutional encoding, the convolutional encoding in the temporal dimension and the spatial dimension will cause the local feature distribution of the vibration waveform temporal correlation feature extractor to be sparsified, that is, a sparsified submanifold outside the distribution relative to the overall high-dimensional feature manifold, resulting in poor convergence of the fan vibration temporal state feature map to the predetermined class probability category representation in the probability space when passing through the multi-task classification head module, affecting the accuracy of the classification result.

[0073] Therefore, preferably, when classifying the fan vibration temporal state feature map through a classifier, the feature values of each position of the fan vibration temporal state feature vector after unfolding the fan vibration temporal state feature map are optimized, specifically: the feature values of each position of the fan vibration temporal state feature vector after unfolding the fan vibration temporal state feature map are optimized with the following optimization formula to obtain the corrected fan vibration temporal state feature vector after unfolding the corrected fan vibration temporal state feature map; where the optimization formula is:

[0074]

[0075] where, v i is the feature value at the i-th position of the fan vibration temporal state feature vector v after unfolding the fan vibration temporal state feature map, and v′ i is the feature value at the i-th position of the corrected fan vibration temporal state feature vector after unfolding the corrected fan vibration temporal state feature map, and exp(·) represents the natural exponential function value with the numerical value as the power.

[0076] That is, by processing the sparse distribution in the high-dimensional feature space through heavy-probability-based regularization, the natural distribution transfer of the geometric manifold of the fan vibration time-series state feature vector V in the high-dimensional feature space to the probability space is activated. Thus, by means of heavy-probability-based smoothing regularization of the distribution sparse submanifold of the high-dimensional feature manifold of the fan vibration time-series state feature vector V, the class convergence of the complex high-dimensional feature manifold with high spatial sparsity under a predetermined class probability is improved, thereby enhancing the accuracy of the classification result obtained by the classifier for the fan vibration time-series state feature vector V.

[0077] Subsequently, the corrected fan vibration time-series state feature map is passed through a classifier to obtain a detection result, and the detection result is used to indicate whether the operating condition of the fan to be analyzed is abnormal. By using the classifier to analyze and judge the corrected feature map, the accuracy of detecting the abnormal operating state of the fan can be improved. The classifier can learn and identify the patterns and differences between normal and abnormal states, thereby more reliably judging the operating condition of the fan. The classifier can analyze the fan vibration time-series state feature map in real time and quickly detect abnormal situations, which enables timely discovery of fan faults or abnormalities, reduces downtime, and improves the reliability and stability of the wind farm. By using the classifier for automated operating state detection, the workload and time cost of manual inspections can be reduced. The inspection personnel can carry out targeted repair and maintenance work based on the detection results of the classifier, improving the inspection efficiency and accuracy. By continuously monitoring the operating condition of the fan and detecting abnormalities, preventive maintenance measures can be taken to prevent potential faults in advance, which helps to extend the service life of the fan, reduce maintenance costs, and ensure the sustainable operation of the wind farm. Based on the detection results of the classifier, data-driven decision-making basis can be provided for wind farm managers and operation and maintenance teams, and reasonable maintenance plans and optimization strategies can be formulated according to the detection results to improve the operating efficiency and economic benefits of the wind farm.

[0078] In summary, the intelligent wind farm inspection method based on the AR real-scene video map according to the embodiments of the present application is elucidated, which can accurately detect the operating state of the fan to be analyzed and uses the vibration waveform signal generated during the operation of the fan to be analyzed to detect the operating state of the fan.

[0079] In one embodiment of the present application, Figure 3 is a block diagram of an intelligent wind farm inspection system based on the AR real-scene video map according to the embodiments of the present application. As Figure 3 shown, the intelligent wind farm inspection system 200 based on the AR real-scene video map according to the embodiments of the present application includes:

[0080] The real-scene video acquisition module 210 is configured to acquire the real-scene video of the wind farm, and correspond the real-scene video with the geographical location information of the wind farm to generate a real-scene video map of the wind farm;

[0081] The detection result generation module 220 is configured to detect the operating state of the wind turbine to be analyzed to obtain a detection result;

[0082] The wind turbine status information generation module 230 is configured to correspond the detection result with the number information of the wind turbine to be analyzed to generate wind turbine status information;

[0083] And, the AR real-scene video map generation module 240 is configured to superimpose the wind turbine status information on the real-scene video map of the wind farm to form an AR real-scene video map, and display the AR real-scene video map to the inspection personnel.

[0084] Wherein, the detection result generation module 220 includes:

[0085] The vibration waveform signal acquisition unit 221 is configured to acquire the vibration waveform signal of the wind turbine to be analyzed collected by the vibration sensor in a predetermined time period;

[0086] The data preprocessing unit 222 is configured to perform data preprocessing on the vibration waveform signal to obtain a sequence of local vibration waveform signals;

[0087] The multi-scale vibration feature extraction unit 223 is configured to extract multi-scale vibration features in the sequence of local vibration waveform signals to obtain a sequence of multi-scale vibration waveform feature maps;

[0088] The time-series correlation feature extraction unit 224 is configured to extract time-series correlation features in the sequence of multi-scale vibration waveform feature maps to obtain a wind turbine vibration time-series state feature map;

[0089] And, the operating condition determination unit 225 of the wind turbine to be analyzed is configured to determine whether the operating condition of the wind turbine to be analyzed is abnormal based on the wind turbine vibration time-series state feature map.

[0090] In the wind farm intelligent inspection system based on the AR real-scene video map, the data preprocessing unit is configured to: perform signal segmentation on the vibration waveform signal to obtain a sequence of local vibration waveform signals.

[0091] Here, those skilled in the art can understand that the specific functions and operations of each unit and module in the above-mentioned wind farm intelligent inspection system based on the AR real-scene video map have been introduced in detail in the description of the wind farm intelligent inspection method based on the AR real-scene video map above, and therefore, the repeated description thereof will be omitted. Figures 1 to 2 of the wind farm intelligent inspection method based on the AR real-scene video map, and thus, the repeated description thereof will be omitted.

[0092] As described above, the intelligent wind farm inspection system 200 based on the AR live video map according to the embodiments of the present application can be implemented in various terminal devices, such as a server for intelligent wind farm inspection based on the AR live video map. In one example, the intelligent wind farm inspection system 200 based on the AR live video map according to the embodiments of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the intelligent wind farm inspection system 200 based on the AR live video map can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the intelligent wind farm inspection system 200 based on the AR live video map can also be one of the many hardware modules of the terminal device.

[0093] Alternatively, in another example, the intelligent wind farm inspection system 200 based on the AR live video map and the terminal device can also be separate devices, and the intelligent wind farm inspection system 200 based on the AR live video map can be connected to the terminal device through a wired and / or wireless network, and transmit interaction information in accordance with a predefined data format.

[0094] Figure 4 It is a schematic diagram of the scenario of the intelligent wind farm inspection method based on the AR live video map according to the embodiments of the present application. As Figure 4 shown, in this application scenario, first, a live video of the wind farm is obtained, and the live video is corresponded to the geographical location information of the wind farm to generate a wind farm live video map (for example, C1 as illustrated in Figure 4 ), and the operating state of the wind turbine to be analyzed is detected to obtain a detection result (for example, C2 as illustrated in Figure 4 ); then, the obtained wind farm live video map and the detection result are input into a server (for example, S as illustrated in Figure 4 ) deployed with an intelligent wind farm inspection algorithm based on the AR live video map, where the server can process the wind farm live video map and the detection result based on the intelligent wind farm inspection algorithm based on the AR live video map to form an AR live video map, and display the AR live video map to the inspection personnel.

[0095] It should also be noted that in the devices, equipment and methods of the present application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present application.

[0096] It is understood that the above embodiments are merely exemplary embodiments adopted to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also regarded as the protection scope of the present invention.

Claims

1. A wind farm intelligent inspection method based on AR real-scene video map, comprising: Acquire a real-scene video of the wind farm, and correspond the real-scene video to the geographical location information of the wind farm to generate a real-scene video map of the wind farm; Detecting the operating status of the fan to be analyzed to obtain a detection result; matching the detection result with the serial number information of the fan to be analyzed to generate fan status information; And, superimposing the wind turbine status information on the wind farm real-scene video map to form an AR real-scene video map, and displaying the AR real-scene video map to inspection personnel; The method is characterized in that detecting the operating state of the fan to be analyzed to obtain the detection result includes: Acquiring a vibration waveform signal of the wind turbine to be analyzed in a predetermined time period collected by a vibration sensor; Performing data preprocessing on the vibration waveform signal to obtain a sequence of vibration waveform local signals; Extracting multi-scale vibration features from the sequence of local vibration waveform signals to obtain a sequence of multi-scale vibration waveform feature maps; Extracting the time series correlation features in the sequence of the multi-scale vibration waveform characteristic diagram to obtain a fan vibration time series state characteristic diagram; and Based on the fan vibration time sequence state characteristic diagram, it is determined whether the operating condition of the fan to be analyzed is abnormal.

2. The wind farm intelligent inspection method based on AR real-scene video map according to claim 1 is characterized in that: Performing data preprocessing on the vibration waveform signal to obtain a sequence of vibration waveform local signals includes: The vibration waveform signal is segmented to obtain a sequence of partial vibration waveform signals.

3. The wind farm intelligent inspection method based on AR real-scene video map according to claim 2 is characterized in that: Extracting multi-scale vibration features from the sequence of the vibration waveform local signals to obtain a sequence of multi-scale vibration waveform feature maps, including: A deep learning network model is used to extract features from the sequence of local signals of the vibration waveform to obtain a sequence of multi-scale vibration waveform feature maps.

4. The wind farm intelligent inspection method based on AR real-scene video map according to claim 3 is characterized in that: The deep learning network model is a vibration waveform feature extractor comprising a first convolutional layer and a second convolutional layer; The method of extracting features of the sequence of local signals of the vibration waveform using a deep learning network model to obtain a sequence of multi-scale vibration waveform feature maps includes: The sequence of the vibration waveform local signals is respectively passed through the vibration waveform feature extractor including the first convolution layer and the second convolution layer to obtain the sequence of the multi-scale vibration waveform feature graphs.

5. The wind farm intelligent inspection method based on AR real-scene video map according to claim 4 is characterized in that: The sequence of the vibration waveform local signals is passed through the vibration waveform feature extractor comprising the first convolution layer and the second convolution layer respectively to obtain the sequence of the multi-scale vibration waveform feature graphs, comprising: Inputting the sequence of the vibration waveform local signals into the first convolution layer of the vibration waveform feature extractor to obtain a first-scale vibration waveform feature map, wherein the first convolution layer has a two-dimensional convolution kernel of a first scale; Inputting the sequence of the vibration waveform local signals into a second convolution layer of the vibration waveform feature extractor to obtain a second-scale vibration waveform feature map, wherein the second convolution layer has a two-dimensional convolution kernel of a second scale, and the first scale is different from the second scale; and The first-scale vibration waveform feature map and the second-scale vibration waveform feature map are cascaded to obtain a sequence of the multi-scale vibration waveform feature maps.

6. The wind farm intelligent inspection method based on AR real-scene video map according to claim 5 is characterized in that: Extracting the time series correlation features in the sequence of the multi-scale vibration waveform feature diagram to obtain the fan vibration time series state feature diagram, including: The sequence of the multi-scale vibration waveform feature graphs is passed through a vibration waveform time series correlation feature extractor based on a three-dimensional convolutional neural network model to obtain the fan vibration time series state feature graph.

7. The wind farm intelligent inspection method based on AR real-scene video map according to claim 6 is characterized in that: The vibration waveform temporal correlation feature extractor based on the three-dimensional convolutional neural network model includes: an input layer, a convolution layer, a pooling layer, an activation layer and an output layer.

8. The wind farm intelligent inspection method based on AR real-scene video map according to claim 7 is characterized in that: Determining whether the operating condition of the fan to be analyzed is abnormal based on the fan vibration time sequence state characteristic diagram includes: Performing characteristic distribution correction on the fan vibration time series state characteristic diagram to obtain a corrected fan vibration time series state characteristic diagram; and The corrected fan vibration time series state characteristic diagram is passed through a classifier to obtain a detection result, and the detection result is used to indicate whether the operating condition of the fan to be analyzed is abnormal.

9. An intelligent inspection system for wind farms based on AR real-life video maps, comprising: A real-scene video acquisition module is used to acquire a real-scene video of a wind farm, and to correspond the real-scene video to the geographical location information of the wind farm to generate a real-scene video map of the wind farm; A detection result generating module, used for detecting the operating status of the fan to be analyzed to obtain the detection result; A fan status information generating module, used for matching the detection result with the serial number information of the fan to be analyzed to generate fan status information; and an AR real-scene video map generation module, which is used to superimpose the wind turbine status information on the wind farm real-scene video map to form an AR real-scene video map, and display the AR real-scene video map to inspection personnel; Characterized in that the detection result generating module comprises: A vibration waveform signal acquisition unit, used to acquire the vibration waveform signal of the wind turbine to be analyzed in a predetermined time period collected by a vibration sensor; A data preprocessing unit, used for performing data preprocessing on the vibration waveform signal to obtain a sequence of vibration waveform local signals; A multi-scale vibration feature extraction unit, used to extract multi-scale vibration features from the sequence of vibration waveform local signals to obtain a sequence of multi-scale vibration waveform feature maps; A time series correlation feature extraction unit, used to extract the time series correlation features in the sequence of the multi-scale vibration waveform feature diagram to obtain a fan vibration time series state feature diagram; and The operating condition determination unit of the fan to be analyzed is used to determine whether the operating condition of the fan to be analyzed is abnormal based on the fan vibration time series state characteristic diagram.

10. The wind farm intelligent inspection system based on AR real-scene video map according to claim 9 is characterized in that: The data preprocessing unit is used for: The vibration waveform signal is segmented to obtain a sequence of partial vibration waveform signals.

Images