Sonar detection method and device for gravel distribution of offshore wind power pile foundation

Abstract

The invention discloses an offshore wind power pile foundation gravel distribution sonar detection method and device, and the method comprises the steps: obtaining an initial feature map according to a sonar image; based on a texture roughness algorithm, according to the initial feature map, obtaining a target fractal dimension matrix; performing nearest neighbor interpolation operation on each target fractal dimension element in the target fractal dimension matrix to obtain a prior fractal matrix; performing element-by-element multiplication on the initial feature map and the prior fractal matrix to obtain an initial fractal feature; obtaining a target fractal feature according to the initial feature map and the initial fractal feature; and inputting the target fractal feature into a sonar image monitoring network to obtain a monitoring result. The broken stone monitoring efficiency and accuracy can be improved, and the method can be widely applied to the technical field of image processing.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a sonar detection method and device for gravel distribution in offshore wind power pile foundations. Background Art

[0002] The existing offshore wind power pile foundation gravel distribution monitoring technology is mainly divided into two categories: optical imaging and acoustic imaging. Compared with optical imaging, acoustic imaging technology can effectively penetrate in underwater environments, is not limited by light conditions, and is not affected by underwater suspended particles and plankton. It is suitable for marine environments with insufficient light and containing plankton. Sonar imaging can be used to analyze the distribution of underwater cables and gravel. The characteristics of cables and gravel under sonar imaging are obviously different, but at this stage, the analysis of sonar images is mostly based on manual resolution, which has the limitations of low efficiency and the accuracy of resolution is greatly affected by personnel experience. More importantly, manual analysis has serious lags and cannot monitor the dynamic situation of gravel in time, burying safety hazards for cables, pile foundations, etc. Summary of the invention

[0003] In view of this, the main purpose of the embodiments of the present invention is to provide a method and device for sonar detection of gravel distribution in offshore wind power pile foundations, in order to solve at least one of the problems of the prior art. The present invention can improve the efficiency and accuracy of gravel monitoring.

[0004] To achieve the above object, an embodiment of the present invention provides a method for detecting the distribution of gravel in offshore wind power pile foundations by sonar, the method comprising the following steps:

[0005] According to the sonar image, an initial feature map is obtained;

[0006] Based on the texture roughness algorithm, obtaining the target fractal dimension matrix according to the initial feature map;

[0007] Performing a nearest neighbor interpolation operation on each target fractal dimension element in the target fractal dimension matrix to obtain a priori fractal matrix;

[0008] Multiplying the initial feature map and the prior fractal matrix element by element to obtain an initial fractal feature;

[0009] Acquire target fractal features according to the initial feature map and the initial fractal features;

[0010] The target fractal features are input into a sonar image monitoring network to obtain monitoring results.

[0011] In some embodiments, obtaining an initial feature map according to a sonar image comprises the following steps:

[0012] The sonar image is passed into the input layer, and the sonar image is resized and vectorized to obtain an intermediate input matrix;

[0013] Passing the intermediate input matrix into the convolutional feature extraction layer to obtain an initial input image;

[0014] The initial input image is passed into the fractal feature embedding network to obtain the initial feature map.

[0015] In some embodiments, the method of obtaining a target fractal dimension matrix based on the initial feature map based on a texture roughness algorithm comprises the following steps:

[0016] Performing a grid segmentation operation on the initial feature map to obtain an initial feature block;

[0017] Based on the fractal dimension scale, obtaining the total number of fractal boxes covering each of the initial feature blocks;

[0018] According to the fractal dimension scale and the total number of fractal boxes, obtaining initial fractal dimension elements through linear estimation;

[0019] Arranging the initial fractal dimension elements to obtain an initial fractal dimension matrix;

[0020] The initial fractal dimension matrix is ​​mapped to obtain a target fractal dimension matrix.

[0021] In some embodiments, the initial fractal dimension element is obtained by linear estimation according to the fractal dimension scale and the total number of fractal boxes, and the formula used includes:

[0022]

[0023] Among them, DF nij represents the initial fractal dimension element; r1 and r2 represent the scales of each fractal dimension; Represents the total number of fractal boxes required to cover the initial feature block at each fractal dimension scale.

[0024] In some embodiments, mapping the initial fractal dimension matrix to obtain a target fractal dimension matrix comprises the following steps:

[0025] According to the initial fractal dimension element, mapping the initial fractal dimension matrix to obtain a target fractal dimension element;

[0026] The target fractal dimension elements are arranged to obtain the target fractal dimension matrix.

[0027] In some embodiments, the nearest neighbor interpolation operation is performed on each target fractal dimension element in the target fractal dimension matrix to obtain a priori fractal matrix, and the formula used includes:

[0028]

[0029] Where, NEI(·): R N×S×S →R N×A×A

[0030] in, represents the prior fractal matrix; FM represents the target fractal dimension matrix; NEI(·) represents the nearest neighbor interpolation operation; R represents the real number set; N represents the number of feature channels; S×S and A×A represent the spatial dimensions.

[0031] In some embodiments, the initial feature map and the priori fractal matrix are multiplied element by element to obtain the initial fractal feature, and the formula used includes:

[0032]

[0033] in, represents the initial fractal feature; G i represents the initial feature map; Represents the prior fractal matrix.

[0034] In some embodiments, obtaining the target fractal feature according to the initial feature map and the initial fractal feature comprises the following steps:

[0035] According to the initial fractal feature, obtaining each element value of the feature channel;

[0036] Averaging all the element values ​​to obtain a vector average value of the feature channel;

[0037] Performing a nearest neighbor interpolation operation on the vector average value to obtain an intermediate feature;

[0038] A fractal fusion operation is performed on the initial feature map, the initial fractal feature and the intermediate feature to obtain the target fractal feature.

[0039] To achieve the above-mentioned purpose, another aspect of the embodiment of the present invention provides a sonar detection device for gravel distribution in offshore wind power pile foundations, the device comprising:

[0040] The first module is used to obtain an initial feature map based on the sonar image;

[0041] The second module is used to obtain a target fractal dimension matrix according to the initial feature map based on a texture roughness algorithm;

[0042] The third module is used to perform a nearest neighbor interpolation operation on each target fractal dimension element in the target fractal dimension matrix to obtain a priori fractal matrix;

[0043] The fourth module is used to perform element-by-element multiplication on the initial feature map and the prior fractal matrix to obtain an initial fractal feature;

[0044] A fifth module is used to obtain a target fractal feature according to the initial feature map and the initial fractal feature;

[0045] The sixth module is used to input the target fractal features into the sonar image monitoring network to obtain monitoring results.

[0046] To achieve the above-mentioned purpose, another aspect of an embodiment of the present invention provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned offshore wind power pile foundation gravel distribution sonar detection method.

[0047] To achieve the above-mentioned purpose, another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned offshore wind power pile foundation gravel distribution sonar detection method is implemented.

[0048] To achieve the above object, another aspect of an embodiment of the present invention provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned offshore wind power pile foundation gravel distribution sonar detection method.

[0049] The embodiments of the present invention include at least the following beneficial effects: the present invention provides a sonar detection method and device for gravel distribution of offshore wind power pile foundations, which obtains an initial feature map through sonar images; obtains a target fractal dimension matrix based on the initial feature map based on a texture roughness algorithm; performs a nearest neighbor interpolation operation on each target fractal dimension element in the target fractal dimension matrix to obtain a priori fractal matrix; performs element-by-element multiplication of the initial feature map and the priori fractal matrix to obtain an initial fractal feature; obtains a target fractal feature based on the initial feature map and the initial fractal feature; and inputs the target fractal feature into a sonar image monitoring network to obtain a monitoring result. The present invention can improve the efficiency and accuracy of gravel monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0051] Figure 1 This is a flow chart of a sonar detection method for gravel distribution in offshore wind power pile foundations provided by an embodiment of the present invention;

[0052] Figure 2 It is a sonar image of an underwater cable in the ocean provided by an embodiment of the present invention;

[0053] Figure 3 This is a sonar image of gravel in the ocean provided by an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of the operation flow of the roughness embedded measurement mechanism (REM) provided by an embodiment of the present invention;

[0055] Figure 5 is a schematic diagram of the structure of a feature embedding deep learning model (TES-Net) for sonar provided in an embodiment of the present invention;

[0056] Figure 6 is a schematic diagram of the structure of a sonar image monitoring network provided by an embodiment of the present invention;

[0057] Figure 7 is a schematic diagram of the monitoring result of gravel based on TES-Net provided by an embodiment of the present invention;

[0058] Figure 8 is a schematic diagram of a monitoring result of a cable based on TES-Net provided in an embodiment of the present invention;

[0059] Fig. 9 It is a schematic diagram of the work flow of monitoring gravel distribution of offshore wind power pile foundation based on TES-Net provided by an embodiment of the present invention;

[0060] Fig.10 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention, and they are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0062] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the flow chart. The terms "first / S100" and "second / S200" in the specification and claims and the above-mentioned drawings may be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiment of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0063] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0065] Before describing the embodiments of the present invention in detail, some nouns and terms involved in the embodiments of the present invention are first described. The nouns and terms involved in the embodiments of the present invention are subject to the following explanations.

[0066] ResNet-18, a type of residual neural network, is one of the most widely used backbone model structures in the field of deep learning.

[0067] TRC, the full name of which is Texture roughness calculation, is a texture roughness calculation method. In order to quantify the characteristics of texture and roughness and their distribution in sonar images, based on fractal dimension calculation, the present invention proposes a texture roughness algorithm, which will be described in detail later.

[0068] REM, the full name of which is Roughness embedding measurement, is designed based on the TRC algorithm and adapted for deep learning networks.

[0069] TES-Net, the full name of which is Feature embedding for sonar, is a feature embedding deep learning model for sonar, built based on the REM mechanism.

[0070] At present, the riprap method is a construction technology used to prevent scouring of single pile foundations in offshore wind farms. This method is mainly used to protect the foundation of the wind turbine from the impact of ocean current scouring and ensure the stability and safety of the wind turbine. In the construction of offshore wind farms, due to the instability of the seabed, especially under the action of tides and waves, the soil around the single pile foundation may be washed away, causing the seabed to drop, thus affecting the stability of the wind turbine. In order to solve this problem, the riprap method is widely used. The construction process of the riprap method includes the use of graded blocks and crushed stones for filling to achieve the purpose of protecting the single pile foundation. The ultimate goal of this method is that after the scouring protection is completed, the soil around the pile will not be hollowed out by the water flow, and the range and depth of the scouring pit will no longer increase, thereby avoiding the steel pipe pile from tilting due to the loss of soil around the pile or other non-operating loads.

[0071] In practical applications, riprap is used to repair scour pits to ensure the stable operation of wind turbines. In addition, researchers have invented the anti-scour technology of bonded riprap, which forms a bonded riprap body by pouring cement-based bonding materials into the riprap body to improve its anti-scour ability. It is worth noting that although the riprap method is an effective means of pile foundation maintenance, there are related cable equipment distributed around the pile foundation of the offshore wind power platform. The thrown stones may squeeze, rub, or have other forms of adverse effects on the cables, which is related to the normal operation and safety of offshore wind power generation facilities. Therefore, it is necessary to monitor the distribution of gravel on the pile foundation after riprap treatment.

[0072] Existing underwater monitoring technologies are mainly divided into two categories: optical imaging and acoustic imaging. Among them, sonar imaging can be used to analyze the distribution of underwater cables and gravel, such as Figure 2 Shown is a sonar image of an underwater cable in the ocean. Figure 3 The sonar image shows gravel in the ocean. The cable and gravel features are clearly different in sonar imaging, but traditional sonar image analysis often relies on manual identification, which is not only time-consuming but also prone to errors.

[0073] There is a large amount of texture and roughness information in sonar images. To quantify the texture and roughness and their distribution characteristics in sonar images, the embodiment of the present invention designs a TRC (Texture roughness calculation) algorithm based on fractal dimension calculation. In order to introduce the TRC algorithm into the neural network, the embodiment of the present invention designs a roughness embedded measurement mechanism (REM) based on the TRC algorithm. The calculation process of the mechanism is as follows: Figure 4 As shown, it includes three parts: fractal dimension calculation, fractal expansion and fractal fusion. The REM mechanism already has the ability to embed high-level semantic information of sonar images into neural networks. In order to fully adapt to the sonar monitoring tasks of seabed gravel and cables, a sonar-oriented feature embedding deep learning model TES-Net (hereinafter referred to as the sonar image monitoring model) is designed based on the REM mechanism, and then the TRC algorithm is introduced into the TES-Net model. Through the TES-Net model, the marine pile foundation gravel distribution monitoring task can be automatically performed, and the distribution of gravel can be monitored in real time and the location of cables can be marked, and the efficiency and accuracy of gravel monitoring can be improved, so as to help engineers accurately judge the current situation and make accurate and timely maintenance decisions. Figure 5 As shown, Figure 5 It is an optional structural diagram of a sonar image monitoring model provided by an embodiment of the present invention, which may include but is not limited to an input layer, a convolutional feature extraction layer, a REM mechanism, a sonar image monitoring network and an output layer in the TES-Net model.

[0074] Optionally, the sonar image monitoring network can be a ResNet-18 network. ResNet-18 serves as the backbone network in TES-Net, responsible for learning the features processed by the REM mechanism and summarizing them as the prior knowledge of the model, and finally predicting and outputting the detection results. ResNet-18 is mainly composed of a residual structure, with a total of four residual blocks, and its residual block distribution is as follows Figure 6 As shown:

[0075] Block 1: Contains two 3x3 convolutional layers with 64 filters, each followed by batch normalization and ReLU activation. The input of the residual block is added to the output of the convolutional layer through a 1x1 convolutional layer to achieve residual connection.

[0076] Block 2: Contains two 3x3 convolutional layers with 128 filters and stride 2 for dimensionality reduction. The input of the residual block is added to the output of the convolutional layer through a 1x1 convolutional layer (stride 2) to achieve residual connection.

[0077] Block 3: Contains two 3x3 convolutional layers with 256 filters and stride 2 for dimensionality reduction. The input of the residual block is added to the output of the convolutional layer through a 1x1 convolutional layer (stride 2) to achieve residual connection.

[0078] Block 4: Contains two 3x3 convolutional layers with 512 filters. The input of the residual block is added to the output of the convolutional layer through a 1x1 convolutional layer to achieve residual connection. There is batch normalization and ReLU activation function between the convolutional layers in each residual block, except for the last convolutional layer which is not followed by ReLU.

[0079] In the output part, ResNet-18 has an output layer that outputs the position and size coordinates of the recognized target, and the result conversion program will use the target box to mark the recognized target object. The recognition result is as follows: Figure 7 and Figure 8 As shown. Figure 7 As shown in Figure 1, TES-Net can mark the size and position of the gravel in the field of view, which will be conducive to the effective real-time monitoring of the position of the gravel. Figure 8 As shown in the figure, TES-Net can mark the size and position of the cables within the field of view. After obtaining the above results, the result judgment program determines the relative position of the red marking box representing the gravel and the green marking box representing the cable within the field of view, and gives whether there is a situation where the gravel affects the safe operation of the cable based on the indicator of whether there is an overlapping area, and finally outputs the monitoring result at this time.

[0080] For example, reference Figure 5 , the sonar image X is input into the TES-Net model, and the complete feature vector of the sonar image is obtained after the input layer performs operations such as scaling and vector matrixing. Then, the REM mechanism is used to fully embed the high-level semantic information into the feature vector of the sonar image X through fractal dimension calculation, fractal expansion, and fractal fusion. Then, the feature vector will be used as the input of the classification network ResNet-18, and finally ResNet-18 will output the result vector. The result vector is converted by the output layer to output the monitoring result Y, and finally the judgment of objects such as seabed gravel or cables is realized.

[0081] refer to Figure 1 , Figure 1 This is an optional flow chart of a sonar detection method for offshore wind power pile foundation gravel distribution provided by an embodiment of the present invention. The method may include but is not limited to steps S100 to S600:

[0082] Step S100, obtaining an initial feature map according to the sonar image;

[0083] Step S200, obtaining a target fractal dimension matrix according to the initial feature map based on a texture roughness algorithm;

[0084] Step S300, performing a nearest neighbor interpolation operation on each target fractal dimension element in the target fractal dimension matrix to obtain a priori fractal matrix;

[0085] Step S400, performing element-by-element multiplication on the initial feature map and the priori fractal matrix to obtain an initial fractal feature;

[0086] Step S500, obtaining a target fractal feature according to the initial feature map and the initial fractal feature;

[0087] Step S600: input the target fractal features into a sonar image monitoring network to obtain a monitoring result.

[0088] In some embodiments, step S100 may include but is not limited to steps S110 to S130:

[0089] Step S110, passing the sonar image into the input layer, resizing the sonar image and performing vector matrix operations on the sonar image to obtain an intermediate input matrix;

[0090] Step S120, passing the intermediate input matrix into the convolutional feature extraction layer to obtain an initial input image;

[0091] Step S130, passing the initial input image into the fractal feature embedding network to obtain the initial feature map.

[0092] In step S110 of some embodiments, the sonar image X is input into the TES-Net model and passed into the input layer, and the sonar image is resized and vectorized for processing by the subsequent convolution layer. The input layer is the starting point of the TES-Net model and is responsible for receiving the original image data. These data can be color images, grayscale images, or any other type of images, which are not limited here. In the input layer, the original data is usually converted into a format suitable for network processing, that is, the original data is vectorized. For the original image data, considering the color channel, the input layer can organize the pixel values ​​of the image into a two-dimensional matrix or a three-dimensional matrix. For example, a color image may be represented as a three-dimensional matrix, where the dimensions are height×width×color channel.

[0093] In step S120 of some embodiments, the intermediate input matrix is ​​passed to the convolutional feature extraction layer, and the convolutional feature extraction layer can capture the local features of the intermediate input matrix, such as edges, textures, etc., through the weighted summation of the convolution kernel and the local area, and generate a feature map that can capture different features of the input data. For example, Figure 5As shown in the figure, under the conditions of convolution kernel size of 3 and stride of 1, the convolution feature extraction layer extracts the local features of the intermediate input matrix by sliding the 3x3 convolution kernel and performing weighted summation at each position to generate an initial input image that can capture different features in the intermediate input matrix.

[0094] In step S130 of some embodiments, in order to fully extract the semantic feature information of the image and further explore the deep implicit features of the image, a fractal feature embedding network is constructed. By passing the initial input image into the fractal feature embedding network, the semantic feature information of the initial input image is fully extracted and the deep implicit features of the initial input image are explored, and an initial feature map can be obtained. The calculation formula is as follows:

[0095] G i =Net(X i )

[0096] Where Net(·): X i →G i ∈R N×A×A

[0097] Among them, G i represents the initial feature map; Net(·) represents the fractal feature embedding network; X i Represents the initial input image; R represents a real number set; N represents the number of feature channels; A×A represents the spatial dimension.

[0098] In some embodiments, step S200 may include but is not limited to steps S210 to S250:

[0099] Step S210, performing a grid segmentation operation on the initial feature map to obtain an initial feature block;

[0100] Step S220, based on the fractal dimension scale, obtaining the total number of fractal boxes covering each of the initial feature blocks;

[0101] Step S230, obtaining initial fractal dimension elements through linear estimation according to the fractal dimension scale and the total number of fractal boxes;

[0102] Step S240, arranging the initial fractal dimension elements to obtain an initial fractal dimension matrix;

[0103] Step S250, mapping the initial fractal dimension matrix to obtain a target fractal dimension matrix.

[0104] In step S210 of some embodiments, the segmented features are extracted, that is, a grid segmentation operation is performed on the initial feature map to obtain an initial feature block. i The feature F in GEach channel of is divided into S×S non-overlapping initial feature blocks, S nij represents the initial feature block of the i-th row and j-th column in the n-th channel. The size of each initial feature block is B×B. Then the regional feature The calculation definition of is as follows:

[0105]

[0106] A=S×B

[0107] Where, θ(·): R N×A×A →R N×S×S×B×B

[0108] in, Represents regional characteristics; F G represents the features in the initial feature map; θ(·) represents the grid segmentation operation; A, S, B represent the respective sizes.

[0109] In step S220 of some embodiments, the number of pixels covering each initial feature block S at different scales r where the fractal dimension needs to be calculated is obtained. nij The total number of fractal boxes L nijr , for the regional features on the nth channel Each initial feature block S nij will be further divided into And satisfying 2≤S≤B / 2, the initial sub-feature block can be obtained, where represents the initial sub-feature block of p rows and q columns. The fractal dimension scale of the block is r, which can be calculated Calculate the total number of fractal boxes L nijr The following definitions can be used:

[0110]

[0111]

[0112] in, Represents the height of each fractal box; B represents size; G represents the grayscale value of the sonar image; L nijr (p, q) represents the initial sub-feature block The number of fractal boxes required at the fractal dimension scale r; g nijpq max Represents the maximum pixel value of the initial sub-feature block; g nijpq min Represents the minimum pixel value of the initial sub-feature block.

[0113] Since the pixel values ​​of the data set in the neural network are distributed in [0, 1] and conform to the Gaussian distribution with a mean of 0, and the value range of the fractal dimension matrix is ​​[1, 3], directly using the numbers in the matrix will cause a lot of information loss. Therefore, it is necessary to use 1 / H for scaling instead of directly applying the grayscale value of the image to the calculation, where H is the size, and the total number of fractal boxes L nijr The expression is:

[0114]

[0115] In step S230 of some embodiments, each initial feature block S is obtained nij The fractal dimension of the initial fractal dimension element DF nij Since the fractal conditions at different fractal dimension scales are similar, the difference in fractal dimension values ​​at different fractal dimension scales is very small. Therefore, it is necessary to obtain the initial fractal dimension element DF by linear fitting according to different fractal dimension scales and the total number of fractal boxes required to cover a certain initial feature block at different fractal dimension scales. nij . Then the calculation formula for approximating the initial fractal dimension element by linear estimation is:

[0116]

[0117] Among them, DF nij represents the initial fractal dimension element; r1 and r2 represent the fractal dimension scales. Represents the total number of fractal boxes required to cover the initial feature block at each fractal dimension scale.

[0118] Compared with the common least squares method, linear estimation is used to approximate each initial fractal dimension element, which can improve computational efficiency. The least squares method involves matrix inversion and multiplication when calculating parameter estimates. When the amount of data is very large, the computational cost of these operations is very high, especially the matrix inversion step, which has a high time complexity. The linear estimation method has a relatively low computational cost and higher computational efficiency, which can improve the overall work efficiency of the model.

[0119] In step S240 of some embodiments, since each initial feature block S is obtained in the above steps, nij The corresponding fractal dimension, that is, each initial fractal dimension element DF nij , then the initial fractal dimension elements can be arranged according to the row and column numbers of each initial feature block to obtain the initial fractal dimension matrix DF∈R N×S×S .

[0120] In some embodiments, step S250 may include but is not limited to steps S251 to S252:

[0121] Step S251, mapping the initial fractal dimension matrix according to the initial fractal dimension element to obtain a target fractal dimension element;

[0122] Step S252: Arrange the target fractal dimension elements to obtain the target fractal dimension matrix.

[0123] In some embodiments, in step S251 to step S252, the initial fractal dimension matrix is ​​mapped into a probabilistic form that conforms to the neural network parameters, and the target fractal dimension matrix FM∈R N×S×S . Then the formula for mapping each initial fractal dimension element is as follows:

[0124]

[0125] Among them, FM nij Represents the deep feature matrix corresponding to the initial feature block in the i-th row and j-th column in the n-th channel, that is, the target fractal dimension element; DF nij Represents the initial fractal dimension element; DF npj Represents the elements of the pth row in the initial fractal dimension matrix; DF niq Represents the elements of the qth column in the initial fractal dimension matrix; p∈{1, 2, ..., i}; q∈{1, 2, ..., j}. Arranging the obtained target fractal dimension elements, we can get the target fractal dimension matrix FM. Each element in the matrix has a certain value, representing the corresponding feature block S nij The texture, roughness and other deep features of the image within the range. In the same type of image, the values ​​of the feature blocks at the corresponding positions of different images are similar, so we can know the initial feature map G i It reflects the prior knowledge of a class of images. The TRC method uses the target fractal dimension matrix as a measure of image roughness and uses linear estimation to improve the model's working efficiency.

[0126] In step S300 of some embodiments, for each target fractal dimension element FM in the target fractal dimension matrix nij The interpolation value is related to the B×B feature block. The purpose is to obtain the value of a single pixel. The specific method is to use FM nij After mapping to the source image, find the nearest point and perform interpolation operation before outputting, that is, each element of the target fractal dimension matrix FM is interpolated by the nearest neighbor to obtain the prior fractal matrix Then there is the following formula:

[0127]

[0128] Where, NEI(·): R N×S×S →R N×A×A

[0129] in, represents the prior fractal matrix; FM represents the target fractal dimension matrix; NEI(·) represents the nearest neighbor interpolation operation; R represents the real number set; N represents the number of feature channels; S×S and A×A represent the spatial dimensions.

[0130] In step S400 of some embodiments, in order to obtain the initial fractal feature, the Hadamard product (also called Hadamard product or element-by-element product) method may be used to multiply the initial feature map and the prior fractal matrix element-by-element, and the following expression is obtained:

[0131]

[0132] in, represents the initial fractal feature; G i represents the initial feature map; Represents the prior fractal matrix.

[0133] In some embodiments, step S500 may include but is not limited to steps S510 to S540:

[0134] Step S510, obtaining each element value of the feature channel according to the initial fractal feature;

[0135] Step S520, averaging all the element values ​​to obtain a vector average value of the feature channel;

[0136] Step S530, performing a nearest neighbor interpolation operation on the vector average value to obtain an intermediate feature;

[0137] Step S540, performing a fractal fusion operation on the initial feature map, the initial fractal feature and the intermediate feature to obtain the target fractal feature.

[0138] The initial fractal feature represents the initial feature map G under the guidance of image prior knowledge. iIn the specific expression form, neural networks are often composed of multiple superimposed layers. Changes in the parameters of a certain layer of the network will cause a chain reaction in a series of layers behind it. After layers of superposition, a small disturbance may have a huge impact. In order to reduce the adverse effects of data distribution on the final result, the embodiment of the present invention normalizes the initial fractal features, that is, the fractal feature vector on each channel will be subtracted from the batch average under the channel. Exemplarily, based on the initial fractal features, the values ​​of each element of the feature channel are obtained, and all element values ​​are averaged to obtain the vector average of the feature channel. Similar to The nearest neighbor interpolation operation is performed on the vector average of the feature channel, and a similar operation is performed on the vector average of the feature channel to finally obtain the interpolation, and a scalar η is used to scale it to obtain the intermediate feature, which is used in the subsequent fractal fusion.

[0139] In some embodiments, according to the initial fractal features By obtaining the element values ​​of each feature channel, we can get the element value G of the i-th row and j-th column under the n-th feature channel. nij Next, all element values ​​are averaged to obtain the vector average value of the feature channel, which is calculated as follows:

[0140]

[0141] in, Represents the vector average of the corresponding batch under the nth channel; G nij Represents the element value of the i-th row and j-th column under the n-th feature channel.

[0142] Then, the nearest neighbor interpolation operation is performed on the vector average and scaled by a scalar η to obtain the intermediate feature When the scalar η is 1, That is to The intermediate features obtained after the interpolation operation are calculated as follows:

[0143]

[0144] Furthermore, for the initial feature map G i , initial fractal features And the intermediate features By performing fractal fusion operation, the final fractal embedding feature, i.e. the target fractal feature, can be obtained. The calculation formula is as follows:

[0145]

[0146] Therefore, the texture, roughness and their distribution patterns in sonar images can be fully quantified and embedded into the neural network architecture as high-level semantic features, which will help the neural network improve its ability to understand the semantic information of complex sonar images, thereby improving the quality of sonar image recognition tasks.

[0147] In step S600 of some embodiments, the target fractal feature is input into the sonar image monitoring network, in which there is an output layer, which outputs the position and size coordinates of the identified target, and the result conversion program will use the target frame to mark the identified target object, for example, mark the size and position of the gravel in the field of view, the size and position of the cable in the field of view, etc. After obtaining the above recognition results, the result judgment program judges the relative position of the marked frame representing the gravel and the marked frame representing the cable in the field of view, and gives whether there is a situation where the gravel affects the safe operation of the cable according to the indicator of whether there is an overlapping area, and finally outputs the monitoring result of the safety of the cable operation at this time.

[0148] refer to Fig. 9 ,by Fig. 9 For example, a TES-Net-based monitoring workflow for offshore wind power pile foundation gravel distribution is as follows:

[0149] Step 1: Sonar deployment: Arrange imaging sonar equipment around the offshore wind power platform and ensure that the sonar equipment is accurately calibrated and positioned so that high-quality sonar images can be obtained.

[0150] Step 2, data acquisition: Start the sonar equipment, scan the distribution of gravel, and transmit the sonar image data back to the computer system via wired mode. The sonar image is pre-processed through the sonar's built-in imaging software.

[0151] Step 3: TES-Net model operation: The preprocessed sonar image is input into the TES-Net neural network model. The TES-Net model analyzes the image through the features learned through its training.

[0152] Step 4. Result output: The TES-Net model will output the recognition results, which may include information such as the location and size of the gravel, as well as the location and size information of the cable. It will also determine whether the current gravel affects the safety of the cable and output the monitoring results.

[0153] Based on the analysis results of TES-Net, maintenance personnel can further assess the impact of the current debris on the cables and the health of the cables, and formulate corresponding maintenance or construction plans, such as adjusting the debris around the wind turbine platform and repairing or replacing the cables.

[0154] In some embodiments, the texture roughness algorithm (TRC) proposed by the present invention based on fractal dimension calculation has the following steps:

[0155] Step 1: Extract the segmented features, that is, perform grid segmentation operation on the initial feature map to obtain the initial feature block.

[0156] Step 2: Obtain the total number of fractal boxes covering each initial feature block at different scales where the fractal dimension needs to be calculated.

[0157] Step 3: According to different fractal dimension scales and the total number of fractal boxes required to cover a certain initial feature block at different fractal dimension scales, the initial fractal dimension elements are obtained by linear fitting to obtain the initial fractal dimension matrix.

[0158] Step 4: Map the initial fractal dimension matrix into a probabilistic form that conforms to the neural network parameters to obtain the target fractal dimension elements and the target fractal dimension matrix.

[0159] In some embodiments, the roughness embedded measurement mechanism (REM) designed based on the TRC algorithm of the present invention has the following three parts:

[0160] I. Fractal dimension calculation

[0161] In order to fully extract the semantic feature information of the image and further explore the deep implicit features of the image, a fractal feature embedding network was constructed. According to the initial input image, the initial feature map was extracted through the fractal feature embedding network. Then, the fractal dimension was calculated using the TRC algorithm based on the initial feature map to obtain the target fractal dimension matrix. Then, each target fractal dimension element in the target fractal dimension matrix was interpolated to obtain the prior fractal matrix.

[0162] II. Fractal expansion

[0163] According to the initial feature map and the prior fractal matrix, the Hadamard product method is used to obtain the initial fractal features. Then, the initial fractal features are normalized. The fractal feature vector on each channel will be subtracted from the batch average under the channel. This normalization method can also be called zero mean normalization or mean subtraction normalization. The purpose is to eliminate the mean difference of the feature data so that the data is on the same scale for subsequent processing. In addition, according to the initial fractal features, the values ​​of each element of the feature channel are obtained, and all element values ​​are averaged to obtain the vector average of the feature channel. Then the vector average of the feature channel is subjected to the nearest neighbor interpolation operation, and is scaled using a scalar to obtain the intermediate features used in subsequent fractal fusion.

[0164] III. Fractal Fusion

[0165] The fractal fusion operation is performed on the initial feature map, the initial fractal feature and the intermediate feature to obtain the final fractal embedding feature, that is, the target fractal feature.

[0166] The embodiment of the present invention further provides an offshore wind power pile foundation gravel distribution sonar detection device, which can implement the above-mentioned offshore wind power pile foundation gravel distribution sonar detection method, and the device includes:

[0167] The first module is used to obtain an initial feature map based on the sonar image;

[0168] The second module is used to obtain a target fractal dimension matrix according to the initial feature map based on a texture roughness algorithm;

[0169] The third module is used to perform a nearest neighbor interpolation operation on each target fractal dimension element in the target fractal dimension matrix to obtain a priori fractal matrix;

[0170] The fourth module is used to perform element-by-element multiplication on the initial feature map and the prior fractal matrix to obtain an initial fractal feature;

[0171] A fifth module is used to obtain a target fractal feature according to the initial feature map and the initial fractal feature;

[0172] The sixth module is used to input the target fractal features into the sonar image monitoring network to obtain monitoring results.

[0173] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0174] The embodiment of the present invention further provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the above-mentioned offshore wind power pile foundation gravel distribution sonar detection method is implemented. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0175] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0176] refer to Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0177] The processor 701 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0178] The memory 702 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 702 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 702, and the processor 701 calls and executes a sonar detection method for gravel distribution of offshore wind power pile foundations according to an embodiment of the present invention;

[0179] Input / output interface 703, used to implement information input and output;

[0180] Communication interface 704, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0181] A bus 705 that transmits information between the various components of the device (e.g., the processor 701, the memory 702, the input / output interface 703, and the communication interface 704);

[0182] The processor 701 , the memory 702 , the input / output interface 703 and the communication interface 704 are connected to each other in communication within the device via a bus 705 .

[0183] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned offshore wind power pile foundation gravel distribution sonar detection method.

[0184] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0185] The embodiment of the present invention also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the aforementioned offshore wind power pile foundation gravel distribution sonar detection method.

[0186] In summary, the offshore wind power pile foundation gravel distribution sonar detection method and device according to the embodiment of the present invention have the following advantages:

[0187] 1. The stability of the offshore wind power platform is crucial to the safe operation of the entire wind farm. The TES-Net proposed in this invention can accurately identify and analyze the distribution of gravel and the location information of submarine cables, helping engineers to timely evaluate the stability of the platform foundation and the health of the underwater cables, and prevent potential safety hazards.

[0188] 2. Traditional sonar image analysis often relies on manual recognition, which is time-consuming and error-prone. The proposed TES-Net can automate this process to reduce human resource costs. At the same time, accurate information on the distribution of gravel and cable locations can help to develop more effective maintenance plans, thereby reducing long-term operation and maintenance costs.

[0189] 3. The data analysis results provided by the automated monitoring solution designed based on TES-Net in the present invention can be used as a decision support tool. The automated detection solution can provide all-weather uninterrupted monitoring results, realize all-weather monitoring of gravel distribution and the health of submarine cables, help managers and engineers make scientific and reasonable decisions more timely, and improve the speed and accuracy of data processing.

[0190] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0191] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0192] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0193] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0194] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0195] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0196] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0197] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0198] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A sonar detection method for gravel distribution of offshore wind power pile foundation, characterized in that: According to the sonar image, an initial feature map is obtained; Based on the texture roughness algorithm, obtaining the target fractal dimension matrix according to the initial feature map; Performing a nearest neighbor interpolation operation on each target fractal dimension element in the target fractal dimension matrix to obtain a priori fractal matrix; Multiplying the initial feature map and the prior fractal matrix element by element to obtain an initial fractal feature; Acquire target fractal features according to the initial feature map and the initial fractal features; The target fractal features are input into a sonar image monitoring network to obtain monitoring results.

2. The offshore wind power pile foundation gravel distribution sonar detection method according to claim 1 is characterized in that: The method of obtaining an initial feature map according to the sonar image comprises the following steps: The sonar image is passed into the input layer, and the sonar image is resized and vectorized to obtain an intermediate input matrix; Passing the intermediate input matrix into the convolutional feature extraction layer to obtain an initial input image; The initial input image is passed into the fractal feature embedding network to obtain the initial feature map.

3. The offshore wind power pile foundation gravel distribution sonar detection method according to claim 1 is characterized in that: The method of obtaining a target fractal dimension matrix based on the texture roughness algorithm according to the initial feature map comprises the following steps: Performing a grid segmentation operation on the initial feature map to obtain an initial feature block; Based on the fractal dimension scale, obtaining the total number of fractal boxes covering each of the initial feature blocks; According to the fractal dimension scale and the total number of fractal boxes, obtaining initial fractal dimension elements through linear estimation; Arranging the initial fractal dimension elements to obtain an initial fractal dimension matrix; The initial fractal dimension matrix is ​​mapped to obtain a target fractal dimension matrix.

4. The offshore wind power pile foundation gravel distribution sonar detection method according to claim 3 is characterized in that: The initial fractal dimension element is obtained by linear estimation according to the fractal dimension scale and the total number of fractal boxes, and the formula used includes: Among them, DF nij represents the initial fractal dimension element; r1 and r2 represent the scales of each fractal dimension; Represents the total number of fractal boxes required to cover the initial feature block at each fractal dimension scale.

5. The offshore wind power pile foundation gravel distribution sonar detection method according to claim 3 is characterized in that: Mapping the initial fractal dimension matrix to obtain a target fractal dimension matrix comprises the following steps: According to the initial fractal dimension element, mapping the initial fractal dimension matrix to obtain a target fractal dimension element; The target fractal dimension elements are arranged to obtain the target fractal dimension matrix.

6. The offshore wind power pile foundation gravel distribution sonar detection method according to claim 1 is characterized in that: The nearest neighbor interpolation operation is performed on each target fractal dimension element in the target fractal dimension matrix to obtain a priori fractal matrix, and the formula used includes: In the formula, NEI(·):R N×S×S →R N×A×A in, represents the prior fractal matrix; FM represents the target fractal dimension matrix; NEI(·) represents the nearest neighbor interpolation operation; R represents the real number set; N represents the number of feature channels; S×S and A×A represent the spatial dimensions.

7. The offshore wind power pile foundation gravel distribution sonar detection method according to claim 1 is characterized in that: The initial feature map and the priori fractal matrix are multiplied element by element to obtain the initial fractal feature, and the formula used includes: in, represents the initial fractal feature; G i represents the initial feature map; Represents the prior fractal matrix.

8. The offshore wind power pile foundation gravel distribution sonar detection method according to claim 1 is characterized in that: The step of obtaining the target fractal feature according to the initial feature map and the initial fractal feature comprises the following steps: According to the initial fractal feature, obtaining each element value of the feature channel; Averaging all the element values ​​to obtain a vector average value of the feature channel; Performing a nearest neighbor interpolation operation on the vector average value to obtain an intermediate feature; A fractal fusion operation is performed on the initial feature map, the initial fractal feature and the intermediate feature to obtain the target fractal feature.

9. A sonar detection device for gravel distribution of offshore wind power pile foundation, characterized in that: include: The first module is used to obtain an initial feature map based on the sonar image; The second module is used to obtain a target fractal dimension matrix according to the initial feature map based on a texture roughness algorithm; The third module is used to perform a nearest neighbor interpolation operation on each target fractal dimension element in the target fractal dimension matrix to obtain a priori fractal matrix; The fourth module is used to perform element-by-element multiplication on the initial feature map and the prior fractal matrix to obtain an initial fractal feature; A fifth module is used to obtain a target fractal feature according to the initial feature map and the initial fractal feature; The sixth module is used to input the target fractal features into the sonar image monitoring network to obtain monitoring results.

10. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 8.

Images