Wind turbine respirator anomaly detection method and system based on image segmentation

Abstract

The application discloses a wind turbine respirator abnormality detection method and system based on image segmentation, relates to the technical field of respirator abnormality detection, and comprises the following steps: collecting a respirator deployment area according to an image acquisition device to determine a space-time image frame; returning the space-time image frame to trigger a first image processor to determine a corrected image frame; performing pixel scanning on the corrected image frame to trigger a respirator full-life cycle detector to determine a respirator pixel matrix; outputting abnormality detection data; and executing respirator abnormality alarm and replacement instruction issuing according to a central control end of a fan management system. The application solves the technical problems of low accuracy of wind turbine respirator abnormality detection in the prior art, difficulty in adapting to complex working conditions, and low safety and stability of fan major components, and achieves the technical effects of accurate detection of wind turbine respirator abnormalities and improved safety and stability of fan major components.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of respirator anomaly detection, and particularly relates to a wind turbine respirator anomaly detection method and system based on image segmentation. BACKGROUND

[0002] In the operating environment of a wind turbine respirator, complex light field conditions such as dark light and backlight are often encountered, and these environmental factors can easily cause image distortion and affect accurate judgment of the respirator state. Meanwhile, in practical applications, respirators come in various specifications, and respirators of different specifications differ in structure, internal particle variation law, and other aspects, and their states present a complex dynamic trend in the whole life cycle. At present, there is a lack of a systematic solution for anomaly detection of such respirators, and existing methods often cannot adapt to complex light field environments, have insufficient detection generalization for respirators of multiple specifications, cannot realize a standardized detection process, and cannot meet the demand for efficient and accurate detection of respirator abnormal states in actual engineering.

[0003] The prior art has the technical problems of low accuracy of wind turbine respirator anomaly detection, difficulty in adapting to complex working conditions, and low safety and stability of large components of the wind turbine. SUMMARY

[0004] The present application provides a wind turbine respirator anomaly detection method and system based on image segmentation, which is used to solve the technical problems of low accuracy of wind turbine respirator anomaly detection, difficulty in adapting to complex working conditions, and low safety and stability of large components of the wind turbine in the prior art.

[0005] In view of the above problems, the present application provides a wind turbine respirator anomaly detection method and system based on image segmentation.

[0006] In a first aspect of the present application, a wind turbine respirator anomaly detection method based on image segmentation is provided, and the method comprises the following steps:

[0007] According to the image acquisition device, the ventilator deployment area is collected to determine the spatiotemporal image frame; the spatiotemporal image frame is transmitted back to trigger the first image processor to perform polarization compensation based on active structured light projection guided by directional demand to determine the corrected image frame; pixel scanning is performed on the corrected image frame to trigger the ventilator full life cycle detector, and a first-order segmentation is performed by performing a first pixel corner point positioning based on the ventilator area to determine the ventilator pixel matrix; for the ventilator pixel matrix, a second segmentation based on the second pixel corner point positioning of the particle color level inside the ventilator and a ratio calculation based on the corner pixel positioning are performed to output abnormality detection data, wherein the calculation target is the ratio of particles of different color levels inside the ventilator; according to the central control terminal of the fan management system, an alarm message based on the abnormality detection data is generated, and a ventilator abnormality alarm and replacement instruction are issued, wherein the multivariate data mode is displayed on the display interface.

[0008] A second aspect of the present application provides a wind turbine respirator anomaly detection system based on image segmentation, the system comprising:

[0009] The spatiotemporal image frame determination module is used to collect information from the ventilator deployment area according to the image acquisition device and determine the spatiotemporal image frame; the correction image frame determination module is used to transmit the spatiotemporal image frame back, trigger the first image processor, perform polarization compensation based on active structured light projection guided by directional demand, and determine the correction image frame; the first-order detection module is used to perform pixel scanning on the correction image frame, trigger the ventilator full life cycle detector, perform first-order segmentation by performing first pixel corner point positioning based on the ventilator area, and determine the ventilator pixel matrix; the second-order detection module is used to perform second segmentation based on second pixel corner point positioning based on the color level of particles inside the ventilator and ratio calculation based on corner pixel positioning for the ventilator pixel matrix, and output abnormal detection data, wherein the calculation target is the ratio of particles of different color levels inside the ventilator; the abnormal alarm module is used to generate alarm information based on the abnormal detection data according to the central control terminal of the fan management system, execute ventilator abnormality alarm and replacement instruction issuance, wherein the multivariate data mode is displayed on the display interface.

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0011] The image acquisition device captures the ventilator deployment area and determines a spatiotemporal image frame. The spatiotemporal image frame is transmitted back to trigger the first image processor, which performs polarization compensation based on active structured light projection guided by directional demand to determine a corrected image frame. Pixel scanning is performed on the corrected image frame to trigger the ventilator full life cycle detector to determine the ventilator pixel matrix. For the ventilator pixel matrix, a second segmentation based on the second pixel corner point positioning of the ventilator's internal particle color level and a ratio calculation based on the corner pixel positioning are performed to output abnormality detection data. Based on the central control terminal of the wind turbine management system, an alarm message based on the abnormality detection data is generated, and a ventilator abnormality alarm and replacement instruction are issued. This achieves the technical effect of accurately detecting abnormalities in wind turbine ventilators and improving the safety and stability of large wind turbine components. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic flow chart of a method for detecting abnormalities in a wind turbine respirator based on image segmentation according to an embodiment of the present application;

[0014] Figure 2 Schematic diagram of the structure of the wind turbine respirator anomaly detection system based on image segmentation provided in an embodiment of the present application.

[0015] Description of the accompanying drawings: spatiotemporal image frame determination module 10 , corrected image frame determination module 20 , first-order detection module 30 , second-order detection module 40 , abnormality alarm module 50 . DETAILED DESCRIPTION

[0016] This application provides a wind turbine respirator anomaly detection method and system based on image segmentation, which is used to solve the technical problems in the existing technology that the wind turbine respirator anomaly detection accuracy is insufficient and it is difficult to adapt to complex working conditions, resulting in low safety and stability of large components of the wind turbine.

[0017] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of this application.

[0018] Example 1, asFigure 1 As shown, the present application provides a wind turbine respirator anomaly detection method based on image segmentation, which comprises:

[0019] Step S100: According to the image acquisition device, the respirator deployment area is collected to determine the space-time image frame.

[0020] Specifically, the respirator deployment area is collected according to the image acquisition device to determine the space-time image frame, which is the initial link of the wind turbine respirator anomaly detection process. The collected space-time image frame is subsequently fed back to trigger the first image processor, which needs to be constructed based on the light field conditions of the respirator assembly area (at least including dark field and inverse light field), form an extensible processing branch through adversarial supervised training for different light field conditions, and combine the image quality and light field condition prior node to set the input end; at the same time, the space-time image frame after processing will be used to trigger the respirator full life cycle detector, which is constructed based on the attention elements of the respirator area and internal particle color scale, and finally serves the output of the anomaly detection data and the subsequent alarm and instruction issuance of the central control end. It is the basis for image data acquisition in the entire detection method, and provides original image information for subsequent polarization compensation, pixel segmentation, and anomaly judgment steps.

[0021] Step S200: Feed back the space-time image frame to trigger the first image processor to perform polarization compensation based on the active structured light projection of the directional demand orientation to determine the corrected image frame.

[0022] Specifically, the space-time image frame is fed back to trigger the first image processor, which needs to be constructed first: by acquiring the dark field and inverse light field conditions of the respirator assembly area, adversarial supervised training of the image discriminator-projection generator architecture is performed for the two kinds of light field conditions respectively, forming the first processing branch and the second processing branch, and the parallel and extensible branches together constitute the first image processor; after the construction is completed, the first prior node is deployed for image quality and the second prior node is deployed for light field condition, and the processor input end is set accordingly. When triggered, the space-time image frame is first identified, the image quality prior based on the first prior node and the light field condition prior based on the second prior node are executed, if the image quality meets the standard, the processor is empty, and then the corresponding processing branch (multi-channel polarization channel based on polarization-depth joint self-attention deployment) is triggered according to the light field condition prior result, and polarization compensation based on the active structured light projection of the directional demand orientation is executed, and finally the corrected image frame is determined, which provides an optimized image basis for subsequent pixel scanning and segmentation.

[0023] Step S300: Perform pixel scanning on the corrected image frame to trigger the respirator full life cycle detector to perform first-order segmentation based on the first pixel corner positioning of the respirator area to determine the respirator pixel matrix.

[0024] Specifically, pixel scanning is performed on the corrected image frame to trigger the respirator full life cycle detector, the construction of which introduces the respirator region as a first attention element and the color gradient of the particles inside the respirator as a second attention element, and takes the two elements as constraints and the corner pixel positioning as the identification target; after triggering, the detector first scans the corrected image frame to determine the image pixel matrix, and then performs the first corner pixel positioning based on the first attention element and the corner boundary determination on the matrix, wherein the corner positioning is performed with a preset neighborhood pixel difference, and the positioning result is filtered with the first attention element, and finally the image pixel matrix is segmented according to the determined segmentation pixel boundary to obtain the respirator pixel matrix, which is the basis for subsequent second segmentation and proportion calculation based on the color gradient of the particles inside the respirator, and throughout the process, the first image processor and the respirator full life cycle detector are embedded and deployed in the fan management system, and the input end is connected with the image acquisition device to provide support for the continuity of the detection process.

[0025] Step S400: For the respirator pixel matrix, a second segmentation based on the second pixel corner positioning of the color gradient of the particles inside the respirator and a proportion calculation based on the corner pixel positioning are performed, and abnormal detection data is output, wherein the calculation target is the proportion of the particles inside the respirator with different color gradients.

[0026] Specifically, for the determined respirator pixel matrix, a second segmentation based on the second pixel corner positioning of the color gradient of the particles inside the respirator and a proportion calculation based on the corner pixel positioning are performed, and abnormal detection data is output, and the calculation target is the proportion of the particles inside the respirator with different color gradients. The second pixel corner specifically refers to the color gradient transition pixel point of the first color and the second color of the particles inside the respirator. When performing positioning, if the color contrast of the first color and the second color is less than a preset contrast, pixel contrast enhancement processing is performed first, and then the ratio calculation based on pixel segmentation is performed according to the color gradient pixel boundary obtained by positioning; after outputting the abnormal detection data, a first state diagram of the service state of the respirator is generated based on the color gradient pixel boundary, the service state curve is updated with the frame time sequence as the axis, the life cycle progress bar is generated, and the trend time sequence is supplemented based on the curve to perform early warning management. These abnormal detection data and related state diagrams, curves and progress bars together constitute a multi-data mode, which provides a core basis for the generation of alarm information and the display of the interface of the subsequent central control terminal, and the whole process relies on the respirator full life cycle detector constructed with the respirator region and the color gradient of the particles inside the respirator as the attention elements.

[0027] Step S500: According to the central control terminal of the fan management system, alarm information based on the abnormal detection data is generated, respirator abnormality alarm and replacement instruction issuing are performed, and the multi-data mode is displayed on the display interface.

[0028] Specifically, the central control end of the fan management system will generate corresponding alarm information according to the aforementioned output abnormal detection data, and then execute the respiratory device abnormal alarm and replacement instruction issuing, and simultaneously display the information in a multi-element data mode on the display interface. The multi-element data mode here covers the abnormal detection data itself, a respiratory device service state first state graph generated based on a color step pixel boundary, a service state curve updated by the abnormal detection data with a frame time sequence as an axial direction, a life cycle progress bar generated based on the curve, and related content of the curve after the curve is supplemented with a lower time sequence through a situation time sequence prediction. The central control end can realize these functions thanks to the fact that the first image processor and the respiratory device full life cycle detector are both embedded and deployed in the fan management system, and the output ends of the two are connected to the central control end, so that the entire abnormal detection process forms a closed loop, ensuring the timeliness and accuracy of the alarm, instruction issuing and data display, and providing comprehensive support for the staff to master the respiratory device state and take corresponding measures.

[0029] In one possible implementation manner, step S200 further includes:

[0030] Step S210: Obtain a light field condition of a respiratory device assembly area, wherein the light field condition at least includes a dark field and a back light field.

[0031] Step S220: For the light field condition, perform architecture adversarial supervised training of the image discriminator-projection generator as a first processing branch for the dark field condition.

[0032] Step S230: Perform architecture adversarial supervised training of the image discriminator-projection generator as a second processing branch for the back light field condition.

[0033] Step S240: Determine the first image processor in parallel with the first processing branch and the second processing branch, wherein the processing branch of the first image processor is extensible.

[0034] Specifically, relying on the image acquisition device, the light field data of the respiratory device assembly area is collected in multiple time periods and multiple angles in the initial deployment stage, and the light intensity, polarization direction, incident angle and other parameters in different environments are recorded through the built-in light sensor; at the same time, combined with the auxiliary light field monitoring device deployed around the assembly area, the characteristic data of the dark field (such as low light, shadow covered scene) and the back light field (such as strong light direct, light source located behind the shooting subject scene) are continuously captured, and these data are integrated into a light field condition data set, providing original light field feature basis for subsequent construction of the first processing branch and the second processing branch for the dark field and the back light field respectively, and ensuring that the first image processor can accurately adapt to the image processing requirements in different light field environments.

[0035] When performing adversarial supervised training of the image discriminator-projection generator architecture for dark field conditions to construct the first processing branch, a generative adversarial network (GAN) is used as the basic framework. The projection generator takes raw dark field image data as input and, through alternating convolutional and deconvolutional layers, generates dark-field-specific active structured light projection parameters and outputs a polarization-compensated predicted image. The image discriminator receives the predicted image output by the generator and a standard calibrated dark field image (as a real sample), extracts features from both, distinguishes authenticity from falsity using a multi-layer perceptron or convolutional neural network, and outputs a probability of recognition. During training, the generator optimizes network parameters using a loss function (such as cross-entropy loss) to minimize the discriminator's accuracy, making the generated compensated image closer to the real sample. The discriminator simultaneously optimizes its parameters to maximize its accuracy, forming an adversarial game between the two. After multiple rounds of iterative training, when the images generated by the generator can make the discrimination accuracy of the discriminator close to the random level (such as 50%), the training converges. At this time, the model composed of the projection generator and the image discriminator serves as the first processing branch, which has the ability to process polarization compensation for dark field conditions.

[0036] When constructing the second processing branch through adversarial supervised training of the image discriminator-projection generator architecture for backlighting conditions, this is achieved based on a generative adversarial network (GAN) framework: the projection generator receives raw image data under backlighting conditions, extracts image features through convolutional layers, and combines this with deconvolutional layers to generate active structured light projection parameters adapted to the backlighting conditions, thereby outputting a polarization-compensated image. The image discriminator then compares the features of the compensated image output by the generator with a standard calibrated image (real sample) under backlighting conditions, and outputs a discrimination probability (determining whether the image is a generated result or a real sample) through a multi-layer perceptron. During training, the generator aims to minimize the discriminator's recognition accuracy and uses a mean squared error loss function to optimize parameters so that the generated image approximates the standard calibrated image. The discriminator aims to maximize recognition accuracy and adjusts its parameters using a cross-entropy loss function, forming an adversarial iteration between the two. After multiple rounds of training, when the images generated by the generator make the discrimination accuracy of the discriminator approach the random level, the training converges. At this time, the model composed of the generator and the discriminator serves as the second processing branch and can be specifically used for polarization compensation processing under inverse light field conditions.

[0037] The first processing branch trained for dark field conditions and the second processing branch trained for back light field conditions are arranged in parallel to determine the first image processor. This means that the two processing branches can independently and simultaneously respond to image processing requirements under different light field conditions. When the input spatio-temporal image frame involves a dark field environment, the first processing branch can be triggered to operate, and if it involves a back light field environment, the second processing branch can be activated for processing. At the same time, the processing branches of the first image processor have extensible characteristics. In addition to the dark field and back light field processing branches that have been constructed, corresponding processing branches can be added according to other light field conditions (such as side light field, scattered light field, etc.) that may be encountered in actual applications through the same image discriminator-projection generator architecture adversarial supervised training method. Thus, the first image processor can adapt to more complex and diverse light field environments, providing flexible and comprehensive processing capabilities for subsequent precise polarization compensation of spatio-temporal image frames.

[0038] In one possible implementation, step S240 further includes:

[0039] Step S241: deploying a first prior node for image quality and a second prior node for light field conditions.

[0040] Step S242: setting the input end of the first image processor according to the first prior node and the second prior node.

[0041] Specifically, when the first prior node is deployed for image quality, an image sharpness evaluation algorithm based on the Laplacian operator is used to obtain the edge sharpness value of the image by calculating the second derivative change of the image pixel gray value, and the image quality is judged by comparing the edge sharpness value with a preset sharpness threshold. At the same time, a noise detection algorithm based on the Gaussian noise model is used to count the distribution density of noise pixels in the image to assist in verifying the image quality. When the second prior node is deployed for light field conditions, a light field type recognition algorithm based on pixel brightness histogram analysis is used to extract the brightness distribution characteristics of the image. If the brightness mean value is lower than the dark field threshold and the distribution is concentrated, it is determined as a dark field. If the brightness mean value is higher than the back light field threshold and there is an obvious strong light area, it is determined as a back light field. The algorithm realizes rapid recognition of the light field conditions, and provides an algorithm level judgment basis for the triggering of the processing branches of the first image processor.

[0042] The input end of the first image processor is set according to the first prior node and the second prior node, and is implemented by establishing a linkage mechanism with the spatiotemporal image frame recognition. The input end will first identify the spatiotemporal image frame, and then execute the image quality prior based on the first prior node to determine the first prior result, and at the same time execute the light field condition prior based on the second prior node to determine the second prior result; if the first prior result meets the standard, the input end will control the first image processor to idle, and then trigger the corresponding target processing branch (first processing branch or second processing branch) according to the second prior result. Since the processing branch has built-in multi-polarization channels based on the joint self-attention deployment of polarization-depth, the setting of the input end needs to ensure that these channels can operate efficiently with the triggering of the target processing branch, so that the first image processor can perform polarization compensation in a targeted manner to ensure the accuracy of the corrected image frame.

[0043] In one possible implementation, step S242 further includes:

[0044] Step S2421: Identify the spatiotemporal image frame, perform image quality prior based on the first prior node, and determine a first prior result.

[0045] Step S2422: executing the light field condition prior based on the second prior node to determine a second prior result.

[0046] If the first a priori result meets the criteria, the first image processor is idled, and the target processing branch is triggered based on the second a priori result, where the target processing branch is the first processing branch or the second processing branch. The processing branch has built-in multi-polarization channels based on a joint self-attention deployment of polarization-depth.

[0047] Specifically, the returned spatiotemporal image frames are first identified. This identification process mainly extracts the basic features of the image as the basis for subsequent processing. Subsequently, the image quality prior is executed based on the deployed first prior node, that is, the quality of the identified spatiotemporal image frames is evaluated based on the preset standards related to image quality in the first prior node (such as clarity, noise level and other indicators). By comparing the evaluation result with the preset threshold, the first prior result (meets the standard or does not meet the standard) is finally determined, providing a basis for judging the image quality for the subsequent triggering of the processing branch of the first image processor.

[0048] The light field condition prior based on the second prior node is executed, that is, according to the preset standard related to the light field condition in the deployed second prior node, the light field environment where the identified space-time image frame is located is analyzed and judged, and whether it belongs to a defined light field type such as a dark field or a backlight field is focused on. By comparing the analysis result with the preset light field condition characteristics, the second prior result is finally determined, and the judgment basis of the light field environment is provided for the trigger of the subsequent first image processor target processing branch (first processing branch or second processing branch).

[0049] When the first prior result is up to standard, the first image processor enters an idle state, that is, no additional preprocessing operation is performed, and the corresponding target processing branch is triggered according to the second prior result. If the light field condition is a dark field, the first processing branch is triggered; if it is a backlight field, the second processing branch is triggered. The multi-path polarization channel based on the joint self-attention deployment of polarization-depth built in the processing branch is constructed in the following way: first, independent polarization perception sub-channels are set for different polarization directions (such as 0°, 45°, 90°, 135°), and each sub-channel is equipped with a dedicated polarization filter module to capture polarization light information in the corresponding direction; at the same time, a depth estimation sub-channel is integrated to obtain scene depth data through phase difference calculation of structured light projection. On this basis, a joint self-attention mechanism is constructed, which can perform cross-channel correlation analysis on the features of each polarization sub-channel and the features of the depth sub-channel, focusing on the cooperative change of polarization characteristics and depth information in the respirator area, and strengthening the feature response of the key area through weight distribution, and finally forming a multi-path collaborative polarization compensation processing channel to accurately cope with polarization interference under different light field conditions and improve the quality of the corrected image frame.

[0050] In one possible implementation, step S300 further includes:

[0051] Step S310: introducing a first attention element, wherein the first attention element is a respirator area.

[0052] Step S320: introducing a second attention element, wherein the second attention element is the color step of the particles inside the respirator.

[0053] Step S330: according to the constraint of the first attention element and the second attention element, taking the positioning of the corner pixel as the identification target, and constructing the respirator full life cycle detector.

[0054] Specifically, the first attention element is introduced, and the respirator area is specifically set as the content of the element. This means that when building the respirator full life cycle detector, attention is first focused on the area where the respirator is located in the image. By defining the feature range of the respirator area (such as shape, position and other attributes), the detector can focus on this area during subsequent image processing, filter out irrelevant background information, lay the foundation for accurately positioning the pixel corners of the respirator area, and ensure that the subsequent segmentation operation can be accurately carried out around the respirator area.

[0055] The second attention element is introduced, and the color scale of the particles inside the respirator is specifically set as the content of the element. This setting means that when building the respirator full life cycle detector, in addition to focusing on the overall area of the respirator, it will also focus on the color level changes of the particles inside the respirator. By defining the feature range of different color scales (such as brightness, saturation and other attributes), the detector can accurately identify and distinguish the color scale differences of these particles during subsequent processing, provide a targeted attention direction for color scale-based segmentation and proportion calculation, and more accurately capture the state changes inside the respirator.

[0056] The specific implementation means of building the respirator full life cycle detector is: first, build a detection model framework based on deep learning, convert the first attention element (respirator area) and the second attention element (color scale of particles inside the respirator) into constraint parameters for model training, and form a training sample set with constraints by masking the respirator area and classifying the color scale of the internal particles in the training data; then, embed an angle point detection layer in the model, use an improved Harris corner point detection algorithm as the basic operator, and combine the attention mechanism to integrate the weight parameters of the first and second attention elements into the corner point response function, so that the model can preferentially focus on the respirator area and the area with significant internal color scale changes during feature extraction, and strengthen the positioning accuracy of the corner pixel. Through multiple rounds of iterative training to optimize the model parameters, the detector can accurately locate the corner pixel that meets the constraint condition in the input image, and finally build a respirator full life cycle detector that takes corner pixel positioning as the core recognition target and is constrained by the double attention elements.

[0057] In one possible implementation, step S300 further includes:

[0058] Step S340: According to the respirator full life cycle detector, the correction image frame is scanned to determine the image pixel matrix.

[0059] Step S350: For the image pixel matrix, perform first corner pixel positioning based on the first attention element and corner boundary to determine the segmentation pixel boundary, wherein the first corner pixel positioning is performed with a preset neighborhood pixel difference, and the positioning result is screened with the first attention element.

[0060] Step S360: According to the segmentation pixel boundary, the image pixel matrix is segmented to determine the respirator pixel matrix.

[0061] Specifically, according to the respirator full life cycle detector constructed, a comprehensive pixel-level scanning is carried out on the corrected image frame after polarization compensation. During the scanning process, the detector reads the information of each pixel in the corrected image frame line by line and column by column, including the coordinate position, gray value and color parameter of the pixel, and systematically organizes these information into an ordered matrix form, so as to determine the image pixel matrix. The matrix fully presents the pixel distribution of the corrected image frame, and provides basic data support for the subsequent segmentation operation based on corner pixel positioning.

[0062] For the determined image pixel matrix, the first corner pixel positioning and corner boundary determination are performed with the first attention element (respirator region) as the guide. The corner pixels are identified by a preset neighborhood pixel difference, that is, the gray value difference between each pixel and the pixels within a preset range around it is calculated, and when the difference reaches a set threshold, the pixel is marked as a potential corner; then, the potential corners are screened according to the first attention element, and the corner pixels belonging to the respirator region are retained, and then the screened corner pixels are connected to form a corner boundary, and finally the segmentation pixel boundary is determined, which provides accurate boundary basis for the subsequent segmentation of the respirator pixel matrix.

[0063] According to the determined segmentation pixel boundary, the image pixel matrix is segmented, and the pixels in the image pixel matrix within the range defined by the segmentation pixel boundary are screened out, which together constitute the pixel set of the respirator region; at the same time, other pixels outside the boundary are excluded, and finally the pixel matrix containing only the respirator region is formed, that is, the respirator pixel matrix is determined, which provides accurate pixel-level data basis for the further detection of the respirator internal particles.

[0064] In one possible implementation manner, step S400 further includes:

[0065] Step S410: For the respirator pixel matrix, a second corner pixel positioning based on a second attention element is performed to determine a color step pixel boundary, wherein the second corner pixel is a pixel point of color step transition between a first color and a second color of the respirator internal particles, and wherein if the color step contrast between the first color and the second color is less than a preset contrast, a pixel contrast enhancement processing is performed.

[0066] Step S420: According to the color step pixel boundary, a ratio calculation based on pixel segmentation is performed to generate abnormality detection data.

[0067] Specifically, for the respirator pixel matrix, the second corner pixel positioning is performed with the second attention element (the color scale of the respirator internal particle) as the core, and then the color scale pixel boundary is determined. The second corner pixel here specifically refers to the pixel point where the first chroma and the second chroma in the respirator internal particle change in color scale. The color difference between normal particles and abnormal particles is preset, for example, blue is set as the first chroma (representing normal color) and white is set as the second chroma (representing abnormal color); then, based on the algorithm principle of pixel corner point positioning, the distribution characteristics of the two color particles at the pixel level are analyzed to automatically identify the boundaries of the blue particle region and the white particle region. The pixel points on these boundaries are the corner pixels; on this basis, the pixel number ratio of the divided blue particle range and white particle range can be directly calculated to determine whether the state of the respirator is abnormal through the ratio. In addition, if the color scale contrast of the first chroma (blue) and the second chroma (white) is less than the preset contrast threshold, pixel contrast enhancement processing is performed to improve the distinguishability of the two colors and ensure the accuracy of boundary identification and subsequent calculation.

[0068] According to the determined color scale pixel boundary, the pixel-level segmentation of the particle regions of different color scales in the respirator pixel matrix is performed, and the pixel ranges occupied by the first chroma (such as normal color blue) particles and the second chroma (such as abnormal color white) particles are determined. Then, the number of pixels in these two regions is counted, and the ratio between them (such as the ratio of the number of blue particle pixels to the number of white particle pixels, or the proportion of the number of white particle pixels to the total number of particle pixels) is calculated. By comparing the ratio with the preset normal range, detection data reflecting whether the state of the respirator internal particles is abnormal, i.e. abnormal detection data, is generated, which provides a basis for subsequent alarm and instruction issuance.

[0069] In one possible implementation manner, step S420 further includes:

[0070] Step S421: generating a first state map based on the service state of the respirator according to the color scale pixel boundary.

[0071] Step S422: updating the service state curve of the respirator according to the abnormal detection data, wherein the service state curve is axis-oriented with frame timing.

[0072] Step S423: generating a life cycle progress bar of the respirator according to the service state curve.

[0073] Step S424: according to the service state curve, performing situation timing prediction based on the specifications and operation mechanism of the respirator, supplementing the service state curve with lower timing according to the predicted trend, and performing respirator precursor warning management according to the predicted trend.

[0074] Step S425: Wherein, the anomaly detection data, the first state map and the service state curve, the life cycle progress bar are displayed as multi-data patterns in the visual interface.

[0075] Specifically, the specific implementation means of generating the first state map based on the service state of the respirator is: first, analyze the different color scale particle regions defined by the color scale pixel boundary, and perform feature extraction on the blue particle region representing the normal state and the white particle region representing the abnormal state respectively; then, using a simplified atlas form, the blue particle region is presented as a continuous blue filled block, and is assigned a specific code "N" (representing normal), and the white particle region is presented as an interval white grid block, and is assigned a specific code "A" (representing abnormal); at the same time, the pixel proportion data of each region is marked on the edge of the atlas. In this way, the physical acquisition image is converted into an intuitive graph containing color blocks, codes and quantitative data, forming the first state map, which clearly reflects the service state of the respirator.

[0076] The service state curve of the respirator is updated according to the generated anomaly detection data. The curve takes frame time sequence as the axis and records the ratio data of different color scale particles determined by each detection in the entire process of the respirator from installation to the present. These data constitute a curve graph with time sequence frame frequency in time sequence. By corresponding the newly acquired anomaly detection data to the corresponding frame time sequence position, the curve is supplemented or corrected, so that the service state curve can dynamically and continuously reflect the state change of the respirator at different time points and intuitively present the state trend in the service process.

[0077] The life cycle progress bar is generated according to the updated service state curve of the respirator. Based on the state change trend of the respirator from installation to the present reflected by the service state curve, and combined with the expected service life of the respirator, the entire life cycle is divided into several stages. By calculating the position of the current state in the entire life cycle, it is presented in the form of an intuitive progress bar, for example, different colors are used to distinguish normal, warning and abnormal stages, clearly showing the length of time used and the remaining life of the respirator, helping to quickly understand the life cycle stage it is in.

[0078] The long short-term memory network (LSTM) algorithm is used for situation time sequence prediction: first, the abnormal detection data (different color scale particle proportion) arranged in frame time sequence in the service state curve is taken as the input sequence, and a multi-dimensional feature matrix is constructed by combining the respirator specification parameters (such as rated service life, particle capacity threshold) and operation mechanism data (such as daily average operation time, load fluctuation coefficient); the long-term dependence relationship in the data sequence is captured through the gating mechanism of the LSTM network, the model is trained to learn the change rule in the normal state; the trained model is used to predict the future particle proportion change trend for a period of time to obtain a prediction sequence; the sequence is supplemented to the service state curve as a lower time sequence to form a complete trend curve; at the same time, a warning threshold (such as abnormal particle proportion reaching 30%) is set, and if the value of a time sequence point in the prediction trend reaches or exceeds the threshold, the warning mechanism is triggered to generate a precursor warning signal.

[0079] The abnormal detection data, the first state diagram, the service state curve and the life cycle progress bar are integrated into a multi-element data mode and displayed and updated on the visual interface. The abnormal detection data presents the proportion of different color scale particles in the respirator in numerical form, directly reflecting the current abnormality; the first state diagram shows the spatial characteristics of particle distribution through simplified atlas or coding; the service state curve presents the historical and predicted state change trend with frame time sequence as the axis; the life cycle progress bar identifies the life cycle stage of the respirator in the form of progress ratio. These data are arranged in different areas in the interface, and through the linkage interaction function, the user can click any data item to view detailed associated information, realize the synchronous update and comprehensive display of multi-element data, and provide comprehensive and intuitive respirator state reference.

[0080] In one possible implementation, step S500 further includes:

[0081] Step S510: The first image processor and the respirator full life cycle detector are embedded in the fan management system, the input end is connected with the image acquisition device, and the output end is connected with the central control end of the fan management system.

[0082] Specifically, the first image processor and the respirator full life cycle detector are integrated into the fan management system in an embedded manner to form a part of the system. The input ends of the two components are connected with the image acquisition device to receive the space-time image frames about the respirator deployment area collected by the image acquisition device; and the output ends thereof are connected with the central control end of the fan management system to transmit the corrected image frames, abnormal detection data and other information obtained after processing to the central control end, provide data support for the central control end to generate alarm information, execute alarm and issue replacement instructions, and thus realize the integrated operation of the entire abnormal detection process in the fan management system.

[0083] Embodiment two, based on the same inventive concept as the image segmentation-based wind turbine respirator anomaly detection method in the preceding embodiment, as Figure 2 As shown in the preceding embodiment, the present application provides an image segmentation-based wind turbine respirator anomaly detection system, and the system and method embodiments in the present application are based on the same inventive concept. The system comprises:

[0084] The spatiotemporal image frame determination module 10 is configured to collect the respirator deployment area according to the image acquisition device and determine the spatiotemporal image frame.

[0085] The correction image frame determination module 20 is configured to return the spatiotemporal image frame, trigger the first image processor, perform polarization compensation based on directional demand-oriented active structured light projection, and determine the correction image frame.

[0086] The first-order detection module 30 is configured to perform pixel scanning on the correction image frame, trigger the respirator full-life cycle detector, perform first-order segmentation based on respirator region-based first pixel corner positioning, and determine the respirator pixel matrix.

[0087] The second-order detection module 40 is configured to perform second segmentation based on respirator internal particle color step second pixel corner positioning and proportion calculation based on corner pixel positioning on the respirator pixel matrix, and output anomaly detection data, wherein the calculation target is the proportion of respirator internal particles of different color steps.

[0088] The anomaly alarm module 50 is configured to generate alarm information based on the anomaly detection data according to the central control end of the wind turbine management system, perform respirator anomaly alarm and replacement instruction issuance, and display multiple data modes on the display interface.

[0089] Further, the system is also configured to implement the following functions:

[0090] Obtain the light field condition of the respirator assembly area, wherein the light field condition at least includes dark field and inverse light field; for the light field condition, perform architecture adversarial supervised training of the image discriminator-projection generator on the dark field condition as a first processing branch, perform architecture adversarial supervised training of the image discriminator-projection generator on the inverse light field condition as a second processing branch, and determine the first image processor in parallel with the first processing branch and the second processing branch, wherein the processing branch of the first image processor is extensible.

[0091] Further, the system is also configured to implement the following functions:

[0092] Deploy the first prior node based on image quality and the second prior node based on light field condition; and set the input end of the first image processor according to the first prior node and the second prior node.

[0093] Furthermore, the system is also used to implement the following functions:

[0094] Identify the spatiotemporal image frame, execute the image quality prior based on the first prior node, and determine a first prior result; execute the light field condition prior based on the second prior node, and determine a second prior result; wherein, if the first prior result is up to standard, idle the first image processor, and trigger the target processing branch according to the second prior result, wherein the target processing branch is the first processing branch or the second processing branch; wherein the processing branch has a built-in multi-polarization channel based on the joint self-attention deployment of polarization-depth.

[0095] Furthermore, the system is also used to implement the following functions:

[0096] A first attention factor is introduced, wherein the first attention factor is the respirator area; a second attention factor is introduced, wherein the second attention factor is the color level of particles inside the respirator; based on the first and second attention factors as constraints and the positioning of corner pixels as the identification target, the respirator full life cycle detector is constructed.

[0097] Furthermore, the system is also used to implement the following functions:

[0098] According to the respirator full life cycle detector, the corrected image frame is scanned to determine the image pixel matrix; for the image pixel matrix, first corner pixel positioning and corner boundary based on the first attention factor are performed to determine the segmentation pixel boundary, wherein the first corner pixel positioning is performed with a preset neighborhood pixel difference, and the positioning result is screened with the first attention factor; according to the segmentation pixel boundary, the image pixel matrix is ​​segmented to determine the respirator pixel matrix.

[0099] Furthermore, the system is also used to implement the following functions:

[0100] For the respirator pixel matrix, second corner pixel positioning based on the second attention factor is performed to determine the color scale pixel boundary, wherein the second corner pixel is a pixel point where the color scale of the first chromaticity and the second chromaticity in the particles inside the respirator changes gradually, wherein if the color scale contrast between the first chromaticity and the second chromaticity is less than a preset contrast, pixel contrast enhancement processing is performed; according to the color scale pixel boundary, a ratio calculation based on pixel segmentation is performed to generate abnormality detection data.

[0101] Furthermore, the system is also used to implement the following functions:

[0102] According to the color scale pixel boundary, a first state diagram based on the service status of the ventilator is generated; according to the abnormality detection data, the service status curve of the ventilator is updated, wherein the service status curve takes the frame timing as the axis; according to the service status curve, a life cycle progress bar of the ventilator is generated; according to the service status curve, with the ventilator specifications and operation mechanism as the benchmark, by performing situation timing prediction, the service status curve is supplemented with lower-level timing according to the prediction trend, wherein the ventilator precursor warning management is executed according to the prediction trend; wherein the abnormality detection data, the first state diagram and the service status curve, and the life cycle progress bar are used as multivariate data models and displayed and updated on a visual interface.

[0103] Furthermore, the system is also used to implement the following functions:

[0104] The first image processor and the respirator full life cycle detector are embedded and deployed in the fan management system, with the input end connected to the image acquisition device and the output end connected to the central control end of the fan management system.

[0105] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0106] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

[0107] This specification and drawings are merely illustrative of the present application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of this application. Thus, this application is intended to include such modifications and variations as fall within the scope of this application and its equivalents.

Claims

1. A wind turbine respirator anomaly detection method based on image segmentation, characterized in that: The method comprises: According to the image acquisition equipment, the ventilator deployment area is collected to determine the spatiotemporal image frame; Transmitting back the spatiotemporal image frame, triggering a first image processor to perform polarization compensation based on active structured light projection guided by directional requirements, and determining a corrected image frame; Performing pixel scanning on the corrected image frame, triggering a full life cycle detector for the ventilator, performing first-order segmentation by locating a first pixel corner point based on the ventilator region, and determining a ventilator pixel matrix; For the ventilator pixel matrix, performing a second segmentation based on second pixel corner location of the particle color scale inside the ventilator and a ratio calculation based on the corner pixel location, and outputting abnormality detection data, wherein the calculation target is the ratio of particles of different color scales inside the ventilator; According to the central control terminal of the fan management system, an alarm message based on the abnormal detection data is generated, and a ventilator abnormality alarm and replacement instruction are issued, wherein a multivariate data mode is displayed on the display interface; The first-order segmentation is performed based on the first pixel corner location of the respirator area, including: Scanning the calibration image frame according to the respirator full life cycle detector to determine an image pixel matrix; For the image pixel matrix, performing first corner pixel positioning and corner boundary based on the first attention factor to determine the segmentation pixel boundary, wherein the first corner pixel positioning is performed using a preset neighborhood pixel difference, and the positioning results are screened using the first attention factor; Segmenting the image pixel matrix according to the segmented pixel boundaries to determine a respirator pixel matrix; The second segmentation based on the second pixel corner location of the particle color scale inside the respirator and the ratio calculation based on the corner pixel location are performed, including: For the respirator pixel matrix, performing second corner pixel positioning based on the second attention factor to determine a color scale pixel boundary, wherein the second corner pixel is a pixel point where the color scale of the first chromaticity and the second chromaticity in the particles inside the respirator gradually changes, and if the color scale contrast between the first chromaticity and the second chromaticity is less than a preset contrast, performing pixel contrast enhancement processing; According to the color scale pixel boundary, a ratio calculation based on pixel segmentation is performed to generate abnormality detection data.

2. The method according to claim 1, wherein Before triggering the first image processor, the construction of the first image processor includes: Acquiring light field conditions of a respirator assembly area, wherein the light field conditions at least include a dark field and a backlight field; For the light field condition, for the dark field condition, performing adversarial supervised training of the image discriminator-projection generator architecture as a first processing branch; For adverse light field conditions, perform adversarial supervised training of the image discriminator-projection generator architecture as the second processing branch; The first processing branch and the second processing branch are performed in parallel to determine the first image processor, wherein the processing branch of the first image processor is scalable.

3. The method according to claim 2, wherein After determining the first image processor, the method includes: Deploy the first prior node based on image quality and the second prior node based on light field conditions; An input terminal of the first image processor is set according to the first a priori node and the second a priori node.

4. The method according to claim 3, wherein Triggering the first image processor, including: Identifying the spatiotemporal image frame, performing an image quality prior based on the first prior node, and determining a first prior result; executing a light field condition prior based on the second prior node to determine a second prior result; If the first a priori result is up to standard, the first image processor is idle, and a target processing branch is triggered according to the second a priori result, wherein the target processing branch is the first processing branch or the second processing branch; Among them, the processing branch has built-in multi-polarization channels based on polarization-depth joint self-attention deployment.

5. The method according to claim 1, wherein Before triggering the respirator full life cycle detector, the construction of the respirator full life cycle detector includes: Introducing a first element of attention, wherein the first element of attention is a respirator area; Introducing a second attention factor, wherein the second attention factor is the color scale of particles inside the respirator; The respirator full life cycle detector is constructed based on the first and second attention factors as constraints and the positioning of corner pixels as the recognition target.

6. The method according to claim 1, wherein After generating anomaly detection data, including: generating a first state map based on the service state of the respirator according to the color scale pixel boundary; updating a service status curve of the respirator according to the abnormality detection data, wherein the service status curve is oriented with a frame time sequence as an axis; generating a life cycle progress bar for the respirator according to the service status curve; Based on the service status curve, taking the specifications and operation mechanism of the respirator as a benchmark, by performing a situation time series prediction, the service status curve is supplemented with a lower time series according to the prediction trend, wherein the respirator precursor warning management is performed according to the prediction trend; The abnormality detection data, the first state diagram and the service state curve, and the life cycle progress bar are used as a multivariate data model and updated on a visual interface.

7. The wind turbine respirator anomaly detection method based on image segmentation according to claim 1, characterized in that: The first image processor and the respirator full life cycle detector are embedded and deployed in the fan management system, with the input end connected to the image acquisition device and the output end connected to the central control end of the fan management system.

8. Wind turbine respirator anomaly detection system based on image segmentation, characterized in that: The system is used to implement the wind turbine respirator anomaly detection method based on image segmentation according to any one of claims 1 to 7, and the system comprises: a spatiotemporal image frame determination module, configured to collect images of the ventilator deployment area according to an image acquisition device and determine spatiotemporal image frames; a correction image frame determination module, configured to transmit back the spatiotemporal image frame, trigger the first image processor, perform polarization compensation based on active structured light projection guided by directional requirements, and determine the correction image frame; a first-order detection module configured to perform pixel scanning on the corrected image frame, trigger a full-lifecycle detector for the respirator, and perform first-order segmentation by locating a first pixel corner point based on the respirator region to determine a respirator pixel matrix; The second-order detection module is used to perform a second segmentation based on the second pixel corner point positioning of the particle color level inside the respirator and a ratio calculation based on the corner pixel positioning for the respirator pixel matrix, and output abnormality detection data, wherein the calculation target is the ratio of particles of different color levels inside the respirator; the abnormality alarm module is used to generate alarm information based on the abnormality detection data according to the central control terminal of the fan management system, execute the ventilator abnormality alarm and replacement instruction, wherein the multivariate data pattern is displayed on the display interface.

Images