Waterfall plot enhancement method, and vibration signal identification method and device based on waterfall plot
By decomposing and optimizing the brightness flux component of the waterfall diagram, an enhanced waterfall diagram is generated, which solves the problem of poor waterfall diagram recognition accuracy in the prior art, and achieves higher recognition accuracy and lower false alarm rate.
Patent Information
- Application Number
- CN202411788037.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-03-28
AI Technical Summary
In the prior art, when the waterfall diagram is directly input to the signal recognition model, the recognition accuracy is poor, resulting in a high false alarm rate of monitoring alarms.
By decomposing the brightness channel components of the waterfall diagram, the illumination and reflection components are optimized, including adaptive dynamic gamma changes and multi-scale passivation masking, followed by weighted fusion and merging to generate an enhanced waterfall diagram.
The image characteristics of the enhanced waterfall diagram are improved, the recognition accuracy is ensured when vibration recognition of the waterfall diagram is detected, and the monitoring alarm false alarm rate is reduced.
Smart Images
Figure CN119722517A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of waterfall graph processing, and in particular to a waterfall graph enhancement method, and a vibration signal recognition method and device based on the waterfall graph. Background Art
[0002] In terms of monitoring and identifying vibration signals around long-distance perimeters and pipelines, the combination of fiber optic sensing technology and deep learning methods has brought revolutionary breakthroughs in this field, greatly reducing the cost of manpower and material resources and improving the level of intelligence. Specifically, when the vibration caused by external disturbances on or near the optical cable causes the phase of the Rayleigh scattered light in the optical fiber to change, the waterfall diagram composed of the backward Rayleigh scattered signal is obtained, and the waterfall diagram is identified to determine the vibration position, which can effectively reduce the false alarm rate of monitoring alarms.
[0003] Since waterfall charts use colors to represent data changes, these colors are usually pseudo colors or gradient colors, which lack details and layers compared to natural pictures, and are not conducive to image feature extraction. In other words, directly inputting waterfall charts into signal recognition models will lead to poor recognition results due to the small number of image features in waterfall charts, which in turn leads to a high false alarm rate for monitoring alarms.
[0004] Therefore, there is an urgent need to provide a waterfall diagram enhancement method, a vibration signal recognition method and device based on the waterfall diagram to enhance the waterfall diagram and thereby ensure the accuracy of the vibration signal recognition results. Summary of the invention
[0005] In view of this, it is necessary to provide a waterfall chart enhancement method, a vibration signal recognition method and device based on the waterfall chart, so as to solve the technical problem of poor recognition accuracy when vibration recognition is directly performed on the acquired waterfall chart in the prior art.
[0006] On the one hand, in order to solve the above technical problems, the present invention provides a waterfall chart enhancement method, comprising: Obtain a waterfall chart; the waterfall chart includes a hue channel component, a lightness channel component, and a saturation channel component; Decomposing the brightness channel component to obtain an illumination component and a reflection component; Performing adaptive dynamic gamma change processing on the illumination components to obtain optimized illumination components; Performing multi-scale passivation and masking processing on the reflection component to obtain an optimized reflection component; Performing weighted fusion on the optimized illumination component and the optimized reflection component to obtain an optimized brightness component; The optimized brightness component, the hue channel component and the saturation channel component are combined to obtain an enhanced waterfall graph.
[0007] In a possible implementation, the method further includes: Performing contrast-limited adaptive histogram equalization processing on the hue channel component to obtain an optimized hue component; Then, the step of merging the optimized brightness component, the hue channel component, and the saturation channel component to obtain an enhanced waterfall graph includes: The optimized brightness component, the optimized hue component and the saturation channel component are combined to obtain an enhanced waterfall chart.
[0008] In a possible implementation, the method further includes: Performing adaptive saturation enhancement processing on the saturation channel component to obtain an optimized saturation component; Then, the step of merging the optimized brightness component, the hue channel component, and the saturation channel component to obtain an enhanced waterfall graph includes: The optimized brightness component, the hue channel component and the optimized saturation component are combined to obtain an enhanced waterfall graph.
[0009] In a possible implementation, the method further includes: The enhanced waterfall image is subjected to color enhancement processing based on a multi-scale retinal enhancement algorithm with color restoration to obtain an optimized waterfall image.
[0010] On the other hand, the present invention also provides a vibration signal recognition method based on a waterfall diagram, comprising: The waterfall chart is enhanced based on the waterfall chart enhancement method to obtain an enhanced waterfall chart; Inputting the enhanced waterfall chart into a fully trained vibration signal recognition model to obtain a vibration signal recognition result; The vibration signal recognition model includes a preliminary recognition module and a gated axial attention module; The preliminary recognition module is used to perform multi-scale feature extraction on the enhanced waterfall image, obtain at least two image features of different scales, and determine a candidate frame based on the at least two image features; The gated axial attention module is used to perform height axial, width axial and depth axial attention learning on the at least two image features and the candidate frame to obtain attention features, and determine the vibration signal recognition result based on the attention features; Among them, the waterfall chart enhancement method is the waterfall chart enhancement method described in any one of the possible implementation methods mentioned above.
[0011] In a possible implementation, the preliminary recognition module includes a feature extraction unit, a feature fusion unit and a preliminary recognition unit; The feature extraction unit includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a fourth feature extraction layer; the first feature extraction layer is used to downsample the enhanced waterfall image to obtain a first image feature, the second feature extraction layer is used to downsample the first image feature to obtain a second image feature, the third feature extraction layer is used to downsample the second image feature to obtain a third image feature, and the fourth feature extraction layer is used to downsample the third image feature to obtain a fourth image feature; The feature fusion unit includes a first feature fusion layer, a second feature fusion layer and a third feature fusion layer; the first feature fusion layer is used to convolve the fourth image feature to obtain a first fusion feature, the second feature fusion layer is used to convolve the third image feature to obtain a first convolution feature, upsample the first fusion feature to obtain a first upsampled feature, and fuse the first convolution feature and the first upsampled feature to obtain a second fusion feature, the third feature fusion layer is used to convolve the second image feature to obtain a second convolution feature, upsample the second fusion feature to obtain a second upsampled feature, and fuse the second convolution feature and the second upsampled feature to obtain a third fusion feature; The preliminary recognition unit is used to recognize the first fusion feature, the second fusion feature and the third fusion feature to obtain the candidate frame.
[0012] In one possible implementation, the gated axial attention module includes a region of interest pooling layer, a feature splicing layer, a parallel height axial attention unit, a width axial attention unit and a depth axial attention unit, a maximum pooling layer, a convolution layer, a gating layer and a recognition unit; The region of interest pooling layer is used to pool the third image feature, the fourth image and the image in the candidate frame respectively, and obtain the first pooling feature, the second pooling feature and the third pooling feature accordingly; The feature splicing layer is used to perform feature splicing on the first pooling feature, the second pooling feature and the third pooling feature to obtain a splicing feature; The high-axis attention unit is used to perform high-axis attention learning on the splicing feature to obtain a high-axis feature; The width axial attention unit is used to perform width axial attention learning on the splicing feature to obtain a width axial feature; The deep axial attention unit is used to perform deep axial attention learning on the splicing features to obtain deep axial features; The maximum pooling layer is used to perform maximum pooling processing on the height axial feature, the width axial feature and the depth axial feature to obtain the maximum pooling feature; The convolution layer is used to perform convolution processing on the maximum pooling feature to obtain a convolution pooling feature; The gating layer is used to screen the convolutional pooling features to obtain effective features; The recognition unit is used to recognize the effective features and obtain the vibration signal recognition result.
[0013] In a possible implementation, the splicing feature is a four-dimensional feature, including the number of channels, depth feature, height feature and width feature; the gated axial attention module also includes a feature rearrangement unit arranged between the feature splicing layer and the parallel height axial attention unit, width axial attention unit and depth axial attention unit; The feature rearrangement unit is used to rearrange the spliced features to generate a first rearranged feature corresponding to the height axial attention unit, a second rearranged feature corresponding to the width axial attention unit, and a third rearranged feature corresponding to the depth axial attention unit; The four-dimensional feature order of the first rearranged feature is: number of channels, height feature, depth feature and width feature; The four-dimensional feature order of the second rearrangement feature is: number of channels, width feature, height feature and depth feature; The four-dimensional feature order of the third rearranged feature is: number of channels, depth feature, height feature and width feature.
[0014] On the other hand, the present invention also provides a waterfall chart enhancement device, comprising: A waterfall graph acquisition unit, used to acquire a waterfall graph; the waterfall graph includes a hue channel component, a lightness channel component and a saturation channel component; A brightness channel component decomposition unit, used for decomposing the brightness channel component to obtain an illumination component and a reflection component; An illumination component optimization unit, used for performing adaptive dynamic gamma change processing on the illumination component to obtain an optimized illumination component; A reflection component optimization unit, used for performing multi-scale passivation and masking processing on the reflection component to obtain an optimized reflection component; A brightness component optimization unit, used for weighted fusion of the optimized illumination component and the optimized reflection component to obtain an optimized brightness component; The component merging unit is used to merge the optimized brightness component, the hue channel component and the saturation channel component to obtain an enhanced waterfall chart.
[0015] On the other hand, the present invention also provides a vibration signal recognition method based on a waterfall diagram, comprising: An image enhancement unit, used for enhancing the waterfall graph based on a waterfall graph enhancement method to obtain an enhanced waterfall graph; A signal recognition unit, used for inputting the enhanced waterfall diagram into a well-trained vibration signal recognition model to obtain a vibration signal recognition result; The vibration signal recognition model includes a preliminary recognition module and a gated axial attention module; The preliminary recognition module is used to perform multi-scale feature extraction on the enhanced waterfall image, obtain at least two image features of different scales, and determine a candidate frame based on the at least two image features; The gated axial attention module is used to perform height axial, width axial and depth axial attention learning on the at least two image features and the candidate frame to obtain attention features, and determine the vibration signal recognition result based on the attention features; Among them, the waterfall chart enhancement method is the waterfall chart enhancement method described in any one of the possible implementation methods mentioned above.
[0016] The beneficial effect of the present invention is that the waterfall enhancement method provided by the present invention optimizes the illumination component and the reflection component decomposed from the brightness channel component respectively, specifically, performs adaptive dynamic gamma change processing on the illumination component to enhance the overall brightness of the waterfall, and performs multi-scale passivation masking processing on the reflection component to enhance the edge details of the waterfall at different scales. That is, the image features of the generated enhanced waterfall are improved, and then when the waterfall is recognized, the recognition result can be determined by the enhanced image features, ensuring the recognition accuracy of vibration recognition of the waterfall. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0018] Figure 1 A schematic flow chart of an embodiment of the waterfall chart enhancement method provided by the present invention; Figure 2 A schematic flow chart of an embodiment of a vibration signal recognition method based on a waterfall diagram provided by the present invention; Figure 3 A schematic diagram of the structure of an embodiment of the vibration signal recognition model provided by the present invention; Figure 4A schematic diagram of the structure of an embodiment of a gated axial attention module provided by the present invention; Figure 5 A schematic diagram of the structure of an embodiment of a waterfall graph enhancement device provided by the present invention; Figure 6 A schematic structural diagram of an embodiment of a vibration signal recognition device based on a waterfall diagram provided by the present invention. DETAILED DESCRIPTION
[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0020] It should be understood that the schematic drawings are not drawn to scale. The flowchart used in the present invention shows the operations implemented according to some embodiments of the present invention. It should be understood that the operations of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present invention, can add one or more other operations to the flowchart, and can also remove one or more operations from the flowchart. Some of the block diagrams shown in the accompanying drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.
[0021] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0022] The present invention provides a waterfall diagram enhancement method, a waterfall diagram-based vibration signal recognition method and device, which are described below respectively.
[0023] Figure 1 A schematic flow chart of an embodiment of the waterfall chart enhancement method provided by the present invention is shown in FIG. Figure 1 As shown, the waterfall chart enhancement method includes: S101, obtaining a waterfall chart; the waterfall chart includes a hue channel (H) component, a lightness channel (V) component, and a saturation channel (S) component; S102, decomposing the brightness channel component to obtain an illumination component and a reflection component; S103, performing adaptive dynamic gamma change processing on the illumination component to obtain an optimized illumination component; S104, performing multi-scale passivation masking processing on the reflection component to obtain an optimized reflection component; S105, performing weighted fusion on the optimized illumination component and the optimized reflection component to obtain an optimized brightness component; S106: Merge the optimized brightness component, hue channel component, and saturation channel component to obtain an enhanced waterfall graph.
[0024] The waterfall graph in step S101 is an image in the HSV space, which avoids the color cast problem of the image in the RGB format when extracting image features, and further ensures the feature extraction performance of the enhanced waterfall graph.
[0025] It should be understood that if the format of the initial waterfall image is RGB, the RGB image is converted to the HSV space. The conversion of the RGB image and the HSV image is a prior art and will not be described in detail here.
[0026] The formula after the adaptive dynamic gamma change processing in step S103 is:
[0027] In the formula, To optimize the lighting component; is the light component; is the adaptive gamma coefficient.
[0028] Among them, step S104 is specifically as follows: first, blurring the reflection component at different scales to obtain reflection blur components of multiple scales; second, calculating the difference between the reflection component and the reflection blur components of multiple scales to obtain multiple difference components; taking the sum of the multiple difference components and the reflection component as the passivation masking components of multiple scales; finally, weighted fusion of the passivation masking components of multiple scales to obtain the optimized reflection component.
[0029] The embodiment of the present invention can extract edge features at different scales by blurring the reflection components at different scales, thereby strengthening the edge features of the waterfall graph.
[0030] It should be noted that the input of the recognition model is usually an image in RGB format. Therefore, after step S106, the process further includes: converting the enhanced waterfall graph into an enhanced waterfall graph in RGB format.
[0031] Compared with the prior art, the waterfall enhancement method provided by the embodiment of the present invention optimizes the illumination component and the reflection component decomposed from the brightness channel component respectively. Specifically, the illumination component is subjected to adaptive dynamic gamma change processing to enhance the overall brightness of the waterfall, and the reflection component is subjected to multi-scale passivation masking processing to enhance the edge details of the waterfall at different scales. That is, the image features of the generated enhanced waterfall are improved, and when the waterfall is recognized, the recognition result can be determined by the enhanced image features, thereby ensuring the recognition accuracy of vibration recognition of the waterfall.
[0032] To further improve the features of the enhanced waterfall graph, in some embodiments of the present invention, the waterfall graph enhancement method further includes: The hue channel component is processed by contrast-limited adaptive histogram equalization (CLAHE) to obtain the optimized hue component; Then step S106 is specifically as follows: The optimized brightness component, the optimized hue component, and the saturation channel component are combined to obtain an enhanced waterfall chart.
[0033] The embodiment of the present invention improves the color distribution of an image and makes the image color more balanced by performing contrast-limited adaptive histogram equalization (CLAHE) processing on a hue channel component.
[0034] Furthermore, in some embodiments of the present invention, the waterfall chart enhancement method further includes: Performing adaptive saturation enhancement processing on the saturation channel component to obtain an optimized saturation component; Then step S106 is specifically as follows: The optimized brightness component, the hue channel component, and the optimized saturation component are combined to obtain an enhanced waterfall chart.
[0035] The embodiment of the present invention can further enhance the color of the image by performing adaptive saturation enhancement processing on the saturation channel component.
[0036] In a specific embodiment of the present invention, the adaptive saturation enhancement process is specifically as follows: Obtain the correspondence between the saturation interval and the saturation enhancement ratio to ensure the saturation channel component of each pixel in the waterfall diagram, then determine the saturation interval where the saturation channel component is located, and then determine the saturation enhancement ratio of the saturation channel component of this pixel through the correspondence, enhance the saturation channel component of the pixel based on this saturation enhancement ratio, and then obtain the optimized saturation component.
[0037] It should be noted that: in order to optimize the enhancement effect of the enhanced waterfall chart, the saturation channel component and the hue channel can be optimized at the same time, and the optimized saturation optimized component, hue optimized component and optimized brightness component are merged to obtain an enhanced waterfall chart.
[0038] In order to further enhance the enhancement effect, in some embodiments of the present invention, after step S106, the following steps are further included: The enhanced waterfall image is color enhanced based on the Multi-Scale Retinex with Color Restoration (MSRCR) algorithm to obtain the optimized waterfall image.
[0039] The embodiment of the present invention performs color enhancement processing on the enhanced waterfall image through the MSRCR algorithm, which can achieve a good sharpening effect while avoiding color distortion through the color restoration factor, thereby achieving the purpose of restoring the color ratio.
[0040] In a specific embodiment of the present invention, the expression of the MSRCR algorithm is:
[0041] In the formula, To enhance the waterfall chart; To optimize the waterfall chart; is the pixel coordinate; is the Gaussian convolution function; is the weighting coefficient; is the filter operator; k is the number of scales.
[0042] The sum of the weighted coefficients of all scales is 1.
[0043] In summary, the waterfall image enhancement method proposed in the embodiment of the present invention performs contrast-limited adaptive histogram equalization processing on the hue channel component to improve the color distribution of the image and make the image color more balanced; decomposes the lightness channel component into an illumination component and a reflection component, and performs an adaptive dynamic gamma transform on the illumination component to enhance the overall brightness of the image. At the same time, a multi-scale passivation masking algorithm is used to enhance the reflection component and improve edge details; an adaptive saturation enhancement algorithm is used for the saturation channel component to further enhance the image color. Finally, the three channels are merged and converted into an RGB image, which is combined with the color restoration factor in the MSRCR to obtain the final enhanced image, thereby improving the enhancement effect of the enhanced waterfall image, that is, the image quality in the enhanced waterfall image is enhanced, which facilitates the identification of the enhanced waterfall image.
[0044] The waterfall graph enhancement method proposed in the above embodiment enhances the features in the waterfall graph. The accuracy of the recognition result of the waterfall graph is not only affected by the image quality of the waterfall graph, but also by the feature extraction capability of the vibration signal recognition model. The stronger the feature extraction capability of the vibration signal recognition model, the more accurate the recognition result. Therefore, the embodiment of the present invention also provides a vibration signal recognition method based on the waterfall graph, such as Figure 2 and Figure 3 As shown in the figure, the vibration signal recognition based on the waterfall diagram includes: S201, enhancing the waterfall chart based on a waterfall chart enhancement method to obtain an enhanced waterfall chart; S202, inputting the enhanced waterfall chart into a fully trained vibration signal recognition model to obtain a vibration signal recognition result; The vibration signal recognition model includes a preliminary recognition module and a gated axial attention module; The preliminary recognition module is used to perform multi-scale feature extraction on the enhanced waterfall image, obtain at least two image features of different scales, and determine a candidate frame based on the at least two image features; The gated axial attention module is used to perform height-axial, width-axial and depth-axial attention learning on at least two image features and candidate frames, obtain attention features, and determine the vibration signal recognition result based on the attention features.
[0045] The waterfall graph enhancement method is the waterfall graph enhancement method in any one of the above embodiments.
[0046] It should be noted that: Figure 3 The dotted lines in the figure are candidate boxes of multi-scale features input to the gated axial attention module.
[0047] The embodiment of the present invention sets a gated axial attention module to perform height-axis, width-axis and depth-axis attention learning on at least two image features and candidate frames, accurately captures and intensifies the feature changes in height, width and depth, thereby enhancing the feature representation capability and improving the accuracy of vibration signal recognition results. In addition, the embodiment of the present invention sets a gated axial attention module to perform attention learning on two image features and candidate frames of different scales, ensuring the scale diversity of the input features, thereby further improving the accuracy of vibration signal recognition results.
[0048] It should be understood that before step S102, the vibration signal recognition model needs to be trained, tested and verified to improve the model performance of the vibration signal recognition model.
[0049] In some embodiments of the present invention, Figure 3 As shown, the preliminary recognition module includes a feature extraction unit, a feature fusion unit and a preliminary recognition unit; The feature extraction unit includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a fourth feature extraction layer; the first feature extraction layer is used to downsample the enhanced waterfall image to obtain a first image feature, the second feature extraction layer is used to downsample the first image feature to obtain a second image feature, the third feature extraction layer is used to downsample the second image feature to obtain a third image feature, and the fourth feature extraction layer is used to downsample the third image feature to obtain a fourth image feature; The feature fusion unit includes a first feature fusion layer, a second feature fusion layer and a third feature fusion layer; the first feature fusion layer is used to convolve the fourth image feature to obtain a first fusion feature, the second feature fusion layer is used to convolve the third image feature to obtain a first convolution feature, upsample the first fusion feature to obtain a first upsampled feature, and fuse the first convolution feature and the first upsampled feature to obtain a second fusion feature, the third feature fusion layer is used to convolve the second image feature to obtain a second convolution feature, upsample the second fusion feature to obtain a second upsampled feature, and fuse the second convolution feature and the second upsampled feature to obtain a third fusion feature; The preliminary recognition unit is used to recognize the first fusion feature, the second fusion feature and the third fusion feature to obtain a candidate frame.
[0050] In a specific embodiment of the present invention, Figure 4 As shown, the gated axial attention module includes a region of interest pooling layer, a feature concatenation layer, a parallel height axial attention unit, a width axial attention unit, and a depth axial attention unit, a maximum pooling layer, a convolution layer, a gating layer, and a recognition unit; The region of interest pooling layer is used to pool the third image feature, the fourth image, and the image in the candidate frame, respectively, to obtain the first pooling feature, the second pooling feature, and the third pooling feature accordingly; The feature concatenation layer is used to concatenate the first pooling feature, the second pooling feature and the third pooling feature to obtain a concatenated feature; The high axial attention unit is used to perform high axial attention learning on the spliced features to obtain high axial features; The width axial attention unit is used to perform width axial attention learning on the splicing features to obtain width axial features; The deep axial attention unit is used to perform deep axial attention learning on the spliced features to obtain deep axial features; The maximum pooling layer is used to perform maximum pooling processing on the height axial features, the width axial features and the depth axial features to obtain the maximum pooling features; The convolution layer is used to perform convolution processing on the maximum pooling feature to obtain the convolution pooling feature; The gating layer is used to filter the convolutional pooling features to obtain effective features; The recognition unit is used to recognize effective features and obtain vibration signal recognition results.
[0051] The embodiment of the present invention extracts a single-scale pooling representation from the third image feature, the fourth image feature and the candidate box by setting a region of interest pooling layer, thereby achieving small-scale alignment of features, which is beneficial for fine positioning of the waterfall chart and classification of small targets, and ensures the accuracy of the recognition results.
[0052] Among them, the purpose of the maximum pooling layer performing maximum pooling processing on the height axial features, the width axial features and the depth axial features is to weight the multi-directional features.
[0053] In a specific embodiment of the present invention, the stitching feature is a four-dimensional feature, including the number of channels, depth feature, height feature and width feature, namely (C, D, H, W).
[0054] In order to make the splicing features more adaptable to different axial attention units, in some embodiments of the present invention, such as Figure 4 As shown, the gated axial attention module also includes a feature rearrangement unit arranged between the feature concatenation layer and the parallel height axial attention unit, width axial attention unit, and depth axial attention unit; The feature rearrangement unit is used to rearrange the concatenated features to generate a first rearranged feature corresponding to the height axial attention unit, a second rearranged feature corresponding to the width axial attention unit, and a third rearranged feature corresponding to the depth axial attention unit; The four-dimensional feature order of the first rearranged feature is: number of channels, height feature, depth feature and width feature; The four-dimensional feature order of the second rearrangement feature is: number of channels, width feature, height feature and depth feature; The four-dimensional feature order of the third rearrangement feature is: number of channels, depth feature, height feature and width feature.
[0055] The embodiment of the present invention rearranges the spliced features to adapt to the attention calculation requirements in different directions. Specifically, the second dimension is designated as the focus dimension (focus_dim), and the remaining dimensions are regarded as non-focus dimensions. In order to effectively focus the features, the rearranged feature tensor is dimensionalized, and the non-focus dimensions are merged into one dimension, so that the feature shape is transformed into (C, focus_dim, combined_dims).
[0056] In addition, the maximum pooling layer provided in the embodiment of the present invention can reduce the feature shape from (C, focus_dim, combined_dims) to (C, focus_dim), making the output feature representation more compact. In this process, each position focus_dim in each channel C will select the maximum value of the corresponding combined_dims dimension, thereby outputting a feature representation with reduced dimensions. The maximum pooling operation helps to highlight the strongest signal in the area, capture key features, and effectively ignore secondary information.
[0057] It should be noted that after obtaining the feature maps (C, focus_dim, 1) of height, width, and depth, the output of each dimension needs to be expanded back to the original three-dimensional shape. The output of the height axis is expanded to (C, 1, H, 1), the output of the width axis is expanded to (C, 1, 1, W), and the output of the depth axis is expanded to (C, D, 1, 1). Then, the expanded features are multiplied to achieve effective feature fusion. This fusion strategy can effectively integrate information from different spatial dimensions, thereby improving the expressiveness of the features.
[0058] In order to better implement the waterfall graph enhancement method in the embodiment of the present invention, based on the waterfall graph enhancement method, the embodiment of the present invention also provides a waterfall graph enhancement device, such as Figure 5 As shown, the waterfall diagram enhancement device 500 includes: A waterfall graph acquisition unit 501 is used to acquire a waterfall graph; the waterfall graph includes a hue channel component, a lightness channel component and a saturation channel component; A brightness channel component decomposition unit 502, used to decompose the brightness channel component to obtain an illumination component and a reflection component; The illumination component optimization unit 503 is used to perform adaptive dynamic gamma change processing on the illumination component to obtain an optimized illumination component; A reflection component optimization unit 504, configured to perform multi-scale passivation and masking processing on the reflection component to obtain an optimized reflection component; The brightness component optimization unit 505 is used to perform weighted fusion of the optimized illumination component and the optimized reflection component to obtain the optimized brightness component; The component merging unit 506 is used to merge the optimized brightness component, the hue channel component and the saturation channel component to obtain an enhanced waterfall graph.
[0059] The waterfall chart enhancement device 500 provided in the above embodiment can implement the technical solution described in the above waterfall chart enhancement method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above waterfall chart enhancement method embodiment, which will not be repeated here.
[0060] In order to better implement the vibration signal recognition method based on the waterfall diagram in the embodiment of the present invention, on the basis of the vibration signal recognition method based on the waterfall diagram, the embodiment of the present invention also provides a vibration signal recognition device based on the waterfall diagram, such as Figure 6 As shown, the vibration signal recognition device 600 based on the waterfall diagram includes: An image enhancement unit 601 is used to enhance the waterfall graph based on a waterfall graph enhancement method to obtain an enhanced waterfall graph; The signal recognition unit 602 is used to input the enhanced waterfall diagram into the fully trained vibration signal recognition model to obtain a vibration signal recognition result; The vibration signal recognition model includes a preliminary recognition module and a gated axial attention module; The preliminary recognition module is used to perform multi-scale feature extraction on the enhanced waterfall image, obtain at least two image features of different scales, and determine a candidate frame based on the at least two image features; The gated axial attention module is used to perform height-axial, width-axial and depth-axial attention learning on at least two image features and candidate frames, obtain attention features, and determine the vibration signal recognition result based on the attention features.
[0061] The waterfall graph enhancement method is the waterfall graph enhancement method in any one of the above embodiments.
[0062] The waterfall chart-based vibration signal identification device 600 provided in the above embodiment can implement the technical solution described in the above waterfall chart-based vibration signal identification method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above waterfall chart-based vibration signal identification method embodiment, which will not be repeated here.
[0063] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.
[0064] The waterfall diagram enhancement method, the vibration signal recognition method and the device based on the waterfall diagram provided by the present invention are introduced in detail above. The principle and implementation mode of the present invention are explained by using specific examples in this article. The description of the above embodiments is only used to help understand the method and the core idea of the present invention. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation mode and the scope of application. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A waterfall chart enhancement method, characterized in that: include: Obtain a waterfall chart; the waterfall chart includes a hue channel component, a lightness channel component, and a saturation channel component; Decomposing the brightness channel component to obtain an illumination component and a reflection component; Performing adaptive dynamic gamma change processing on the illumination components to obtain optimized illumination components; Performing multi-scale passivation and masking processing on the reflection component to obtain an optimized reflection component; Performing weighted fusion on the optimized illumination component and the optimized reflection component to obtain an optimized brightness component; The optimized brightness component, the hue channel component and the saturation channel component are combined to obtain an enhanced waterfall graph.
2. The waterfall graph enhancement method according to claim 1, characterized in that: The method further comprises: Performing contrast-limited adaptive histogram equalization processing on the hue channel component to obtain an optimized hue component; Then, the step of merging the optimized brightness component, the hue channel component, and the saturation channel component to obtain an enhanced waterfall graph includes: The optimized brightness component, the optimized hue component and the saturation channel component are combined to obtain an enhanced waterfall chart.
3. The waterfall graph enhancement method according to claim 1, characterized in that: The method further comprises: Performing adaptive saturation enhancement processing on the saturation channel component to obtain an optimized saturation component; Then, the step of merging the optimized brightness component, the hue channel component, and the saturation channel component to obtain an enhanced waterfall graph includes: The optimized brightness component, the hue channel component and the optimized saturation component are combined to obtain an enhanced waterfall graph.
4. The waterfall graph enhancement method according to claim 1, characterized in that: The method further comprises: The enhanced waterfall image is subjected to color enhancement processing based on a multi-scale retinal enhancement algorithm with color restoration to obtain an optimized waterfall image.
5. A vibration signal recognition method based on waterfall diagram, characterized in that: include: The waterfall chart is enhanced based on the waterfall chart enhancement method to obtain an enhanced waterfall chart; Inputting the enhanced waterfall chart into a fully trained vibration signal recognition model to obtain a vibration signal recognition result; The vibration signal recognition model includes a preliminary recognition module and a gated axial attention module; The preliminary recognition module is used to perform multi-scale feature extraction on the enhanced waterfall image, obtain at least two image features of different scales, and determine a candidate frame based on the at least two image features; The gated axial attention module is used to perform height axial, width axial and depth axial attention learning on the at least two image features and the candidate frame to obtain attention features, and determine the vibration signal recognition result based on the attention features; Wherein, the waterfall chart enhancement method is the waterfall chart enhancement method described in any one of claims 1-4.
6. The vibration signal recognition method based on waterfall diagram according to claim 5 is characterized in that: The preliminary recognition module includes a feature extraction unit, a feature fusion unit and a preliminary recognition unit; The feature extraction unit includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a fourth feature extraction layer; the first feature extraction layer is used to downsample the enhanced waterfall image to obtain a first image feature, the second feature extraction layer is used to downsample the first image feature to obtain a second image feature, the third feature extraction layer is used to downsample the second image feature to obtain a third image feature, and the fourth feature extraction layer is used to downsample the third image feature to obtain a fourth image feature; The feature fusion unit includes a first feature fusion layer, a second feature fusion layer and a third feature fusion layer; the first feature fusion layer is used to convolve the fourth image feature to obtain a first fusion feature, the second feature fusion layer is used to convolve the third image feature to obtain a first convolution feature, upsample the first fusion feature to obtain a first upsampled feature, and fuse the first convolution feature and the first upsampled feature to obtain a second fusion feature, the third feature fusion layer is used to convolve the second image feature to obtain a second convolution feature, upsample the second fusion feature to obtain a second upsampled feature, and fuse the second convolution feature and the second upsampled feature to obtain a third fusion feature; The preliminary recognition unit is used to recognize the first fusion feature, the second fusion feature and the third fusion feature to obtain the candidate frame.
7. The vibration signal recognition method based on waterfall diagram according to claim 6 is characterized in that: The gated axial attention module includes a region of interest pooling layer, a feature splicing layer, a parallel height axial attention unit, a width axial attention unit and a depth axial attention unit, a maximum pooling layer, a convolution layer, a gating layer and a recognition unit; The region of interest pooling layer is used to pool the third image feature, the fourth image and the image in the candidate frame respectively, and obtain the first pooling feature, the second pooling feature and the third pooling feature accordingly; The feature splicing layer is used to perform feature splicing on the first pooling feature, the second pooling feature and the third pooling feature to obtain a splicing feature; The high-axis attention unit is used to perform high-axis attention learning on the splicing feature to obtain a high-axis feature; The width axial attention unit is used to perform width axial attention learning on the splicing feature to obtain a width axial feature; The deep axial attention unit is used to perform deep axial attention learning on the splicing features to obtain deep axial features; The maximum pooling layer is used to perform maximum pooling processing on the height axial feature, the width axial feature and the depth axial feature to obtain the maximum pooling feature; The convolution layer is used to perform convolution processing on the maximum pooling feature to obtain a convolution pooling feature; The gating layer is used to screen the convolutional pooling features to obtain effective features; The recognition unit is used to recognize the effective features and obtain the vibration signal recognition result.
8. The vibration signal recognition method based on waterfall diagram according to claim 7 is characterized in that: The splicing feature is a four-dimensional feature, including the number of channels, depth feature, height feature and width feature; the gated axial attention module also includes a feature rearrangement unit arranged between the feature splicing layer and the parallel height axial attention unit, width axial attention unit and depth axial attention unit; The feature rearrangement unit is used to rearrange the spliced features to generate a first rearranged feature corresponding to the height axial attention unit, a second rearranged feature corresponding to the width axial attention unit, and a third rearranged feature corresponding to the depth axial attention unit; The four-dimensional feature order of the first rearranged feature is: number of channels, height feature, depth feature and width feature; The four-dimensional feature order of the second rearrangement feature is: number of channels, width feature, height feature and depth feature; The four-dimensional feature order of the third rearranged feature is: number of channels, depth feature, height feature and width feature.
9. A waterfall chart enhancement device, characterized in that: include: A waterfall graph acquisition unit, used to acquire a waterfall graph; the waterfall graph includes a hue channel component, a lightness channel component and a saturation channel component; A brightness channel component decomposition unit, used for decomposing the brightness channel component to obtain an illumination component and a reflection component; An illumination component optimization unit, used for performing adaptive dynamic gamma change processing on the illumination component to obtain an optimized illumination component; A reflection component optimization unit, used for performing multi-scale passivation and masking processing on the reflection component to obtain an optimized reflection component; A brightness component optimization unit, used for weighted fusion of the optimized illumination component and the optimized reflection component to obtain an optimized brightness component; The component merging unit is used to merge the optimized brightness component, the hue channel component and the saturation channel component to obtain an enhanced waterfall chart.
10. A vibration signal recognition method based on waterfall diagram, characterized in that: include: An image enhancement unit, used for enhancing the waterfall graph based on a waterfall graph enhancement method to obtain an enhanced waterfall graph; A signal recognition unit, used for inputting the enhanced waterfall diagram into a well-trained vibration signal recognition model to obtain a vibration signal recognition result; The vibration signal recognition model includes a preliminary recognition module and a gated axial attention module; The preliminary recognition module is used to perform multi-scale feature extraction on the enhanced waterfall image, obtain at least two image features of different scales, and determine a candidate frame based on the at least two image features; The gated axial attention module is used to perform height axial, width axial and depth axial attention learning on the at least two image features and the candidate frame to obtain attention features, and determine the vibration signal recognition result based on the attention features; Wherein, the waterfall chart enhancement method is the waterfall chart enhancement method described in any one of claims 1-4.