Wall recognition method, system, device and storage medium based on sonar images

Abstract

The present application relates to the field of sonar technology and discloses a method, system, device, and storage medium for wall recognition using sonar images. The method comprises: inputting target sonar image data into a pre-trained anomaly detection model to perform classification prediction on whether each pixel is a wall, thereby obtaining a wall recognition result; the training step of the anomaly detection model comprises: obtaining training data, using the training data, training an initial model to perform classification prediction on whether each pixel is a wall, thereby obtaining an anomaly detection model, wherein the initial model uses an anomaly detection network based on a convolutional neural network, and the training data includes: sonar image sample data and a target label matrix. The anomaly detection model of the present application directly classifies each pixel as a wall. Through pixel-level classification, it can more accurately identify walls, more meticulously outline the contours of the wall, and accurately display various details of the wall, thereby improving the accuracy of wall recognition.

Description

Technical Field

[0001] The present application relates to the fields of sonar technology and artificial intelligence technology, and in particular to a method, system, device and storage medium for wall recognition using sonar images. Background Art

[0002] Sonar detection technology is widely used in underwater environmental monitoring and obstacle detection. It transmits and receives reflected signals, generating images of underwater objects based on the reflected signals. Changes in grayscale values ​​in the images can be used to identify various underwater targets. However, the complex underwater environment causes sonar images to be affected by noise, blur, and other factors, resulting in unstable image quality during wall recognition, making it difficult to locate and identify walls. In recent years, convolutional neural networks (CNNs) have made significant progress in image recognition and are being used to process sonar images for wall detection. They can automatically extract image features, making detection more accurate. However, current CNN-based wall recognition methods use bounding boxes to mark wall locations. This approach can only roughly identify walls and struggles to accurately represent the contours and details of complex wall structures, resulting in low wall recognition accuracy. Summary of the Invention

[0003] Based on this, it is necessary to use bounding boxes to mark the wall position in the existing CNN-based wall recognition method. This method can only roughly identify the wall and is difficult to accurately present the outline and details of complex wall shapes, resulting in the technical problem that the accuracy of the wall recognition results is not high. A sonar image wall recognition method, system, device and storage medium are proposed.

[0004] In a first aspect, a method for wall recognition using sonar images is provided, the method comprising:

[0005] Acquire target sonar image data;

[0006] Input the target sonar image data into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, thereby obtaining a wall recognition result;

[0007] Among them, the training step of the anomaly detection model includes: obtaining training data, using the training data to train the initial model to classify and predict whether each pixel point is a wall, and obtaining the anomaly detection model. The initial model adopts an anomaly detection network based on a convolutional neural network, and the training data includes: sonar image sample data and a target label matrix.

[0008] Furthermore, the initial model includes, in sequence: an input layer, at least two feature extraction units, a feature fusion unit, a decoding unit, and a classification unit, wherein each of the feature extraction units is arranged in parallel;

[0009] The feature extraction unit includes, in sequence: a first convolutional layer, a first activation layer, a first batch normalization layer, a first maximum pooling layer, a second convolutional layer, a second activation layer, a second batch normalization layer and a second maximum pooling layer;

[0010] The decoding unit sequentially includes: a first deconvolution layer, a third activation layer, a third batch normalization layer and a second deconvolution layer.

[0011] Furthermore, the number of the feature extraction units is four;

[0012] The convolution kernels of the first convolution layer and the second convolution layer of the first feature extraction unit both have a size of 3×3;

[0013] The size of the convolution kernel of the first convolution layer of the second feature extraction unit is 7×7, and the size of the convolution kernel of the second convolution layer of the second feature extraction unit is 3×3;

[0014] The size of the convolution kernel of the first convolution layer of the third feature extraction unit is 15×15, and the size of the convolution kernel of the second convolution layer of the third feature extraction unit is 5×5;

[0015] The size of the convolution kernel of the first convolution layer of the fourth feature extraction unit is 25×25, and the size of the convolution kernel of the second convolution layer of the fourth feature extraction unit is 7×7;

[0016] The sizes of the convolution kernels of the first deconvolution layer and the second deconvolution layer are both 2×2.

[0017] Furthermore, before the step of obtaining training data, the method further includes:

[0018] Obtain sonar echo signal sample data;

[0019] Performing two-dimensional image matrix reshaping, normalization processing, jet color map processing, and visualization according to the sonar echo signal sample data, and obtaining abnormal point marks based on the visualization, to obtain a wall coordinate set corresponding to each wall;

[0020] generating a multi-order polynomial equation according to each of the wall coordinate sets;

[0021] generating a two-dimensional label matrix according to the sonar echo signal sample data and each of the multi-order polynomial equations;

[0022] Perform label expansion on the two-dimensional label matrix to obtain the target label matrix.

[0023] Furthermore, the multi-order polynomial equation adopts a quartic polynomial equation.

[0024] Furthermore, the step of generating a two-dimensional label matrix according to the sonar echo signal sample data and each of the multi-order polynomial equations includes:

[0025] A double-layer loop traversal method is used to traverse all data points in the sonar echo signal sample data, and the absolute value of the difference between the y-axis coordinate of the position corresponding to each data point and each equation value corresponding to the data point is calculated to obtain the target distance, wherein the equation value is the value obtained by substituting the x-axis coordinate of the position corresponding to the data point into the multi-order polynomial equation;

[0026] If the target distance between the data point and the target is less than a preset threshold, the label value corresponding to the data point in the two-dimensional label matrix is ​​set to a first value;

[0027] If the data point does not exist and the target distance is less than a preset threshold, the label value corresponding to the data point in the two-dimensional label matrix is ​​set to a second value.

[0028] Furthermore, after the step of obtaining sonar echo signal sample data, the method further includes:

[0029] Performing three-dimensional image reconstruction on the sonar echo signal sample data to obtain sample data to be processed;

[0030] The sample data to be processed is subjected to geometric transformation and Gamma correction in sequence to obtain the sonar image sample data.

[0031] In a second aspect, a sonar image wall recognition system is provided, the system comprising: an electronic device configured to implement the steps of any of the above-mentioned sonar image wall recognition methods.

[0032] In a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned sonar image wall recognition methods when executing the computer program.

[0033] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned sonar image wall recognition methods are implemented.

[0034] The wall recognition method, system, device and storage medium of the sonar image of the present application obtains the wall recognition result by obtaining the target sonar image data and inputting the pre-trained anomaly detection model to perform wall classification prediction of the pixel points, wherein the anomaly detection model adopts an anomaly detection network based on a convolutional neural network and is trained with training data containing sonar image sample data and a target label matrix. The traditional box labeling method can only roughly mark the wall and it is difficult to accurately present the outline and details of the complex wall. The anomaly detection model of the present application directly classifies each pixel as a wall. Through pixel-level classification, the wall can be identified more accurately. This means that when facing complex underwater wall structures, the present application can overcome the limitations of the traditional box labeling method, more carefully outline the outline of the wall, accurately display the various details of the wall, and improve the accuracy of wall recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0036] in:

[0037] Figure 1 A diagram illustrating an application environment of a method for wall recognition using sonar images according to an embodiment;

[0038] Figure 2 is a flow chart of a method for wall recognition using sonar images in one embodiment;

[0039] Figure 3 is a flow chart of a method for wall recognition using sonar images in one embodiment;

[0040] Figure 4 is a flow chart of a method for wall recognition using sonar images in one embodiment;

[0041] Figure 5 This is a schematic diagram of the network structure of the initial model of this application;

[0042] Figure 6 A corresponding diagram of the sonar image sample data and the target label matrix of this application;

[0043] Figure 7 FIG. 4 is a structural block diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0045] The wall recognition method of the sonar image provided in the embodiment of the present application can be applied to Figure 1 In an application environment, the client 110 communicates with the server 120 through a network.

[0046] The server 120 is configured to implement the steps of the sonar image wall recognition method of the present application: obtaining target sonar image data through the client 110; inputting the target sonar image data into a pre-trained anomaly detection model to perform classification prediction on whether each pixel is a wall, thereby obtaining a wall recognition result; wherein the anomaly detection model training step includes: obtaining training data, and using the training data to train an initial model to perform classification prediction on whether each pixel is a wall, thereby obtaining the anomaly detection model, wherein the initial model uses an anomaly detection network based on a convolutional neural network, and the training data includes sonar image sample data and a target label matrix. The wall recognition result is obtained by obtaining target sonar image data and inputting it into the pre-trained anomaly detection model to perform wall classification prediction on the pixels, wherein the anomaly detection model uses an anomaly detection network based on a convolutional neural network and is trained with training data including sonar image sample data and a target label matrix. The traditional frame marking method can only roughly identify the wall and it is difficult to accurately present the outline and details of the complex wall. The anomaly detection model of this application directly classifies each pixel as a wall. Through pixel-level classification, it can more accurately identify the wall. This means that when faced with complex underwater wall structures, this application can overcome the limitations of the traditional frame marking method, outline the wall more carefully, accurately display the various details of the wall, and improve the accuracy of wall recognition.

[0047] The client 110 (i.e., electronic device) may be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. The server 120 (i.e., electronic device) may be implemented as a standalone server or a server cluster consisting of multiple servers. In other words, both the client 110 and the server 120 may be computer devices.

[0048] The present application is described in detail below through specific embodiments.

[0049] See also Figure 2 As shown, Figure 2A flowchart of a method for wall recognition using sonar images provided in an embodiment of the present application includes the following steps:

[0050] S1: Acquire target sonar image data;

[0051] First, a sonar detection device is used to perform a sonar scan of the target underwater environment. Then, the scanned sonar echo signal data is reconstructed into a three-dimensional image, and the reconstructed image is used as the target sonar image data.

[0052] The target underwater environment is an underwater environment in which wall recognition is desired.

[0053] Sonar echo signal data is a .dat file. It contains the echo intensity of each location (also called a data point) in the target underwater environment. In other words, sonar echo signal data is one-dimensional data.

[0054] It can be understood that the pixel points in the target sonar image data correspond one-to-one to the data points in the sonar echo signal data.

[0055] It is understandable that when reshaping a 3D image, the content of the data is not changed in essence, but the dimension of the data is increased. The specific implementation method of the 3D image reshaping can be selected from the existing technology and will not be described in detail here.

[0056] Specifically, the target sonar image data input by the user can be obtained through the client, the target sonar image data can be obtained from the preset storage space, and the target sonar image data sent by a third-party application can also be obtained.

[0057] S2: Input the target sonar image data into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, thereby obtaining a wall recognition result;

[0058] Specifically, the target sonar image data is input into a pre-trained anomaly detection model to perform classification prediction on whether each pixel point is a wall, and the vector obtained by the classification prediction is used as the wall recognition result.

[0059] Each vector value in the wall recognition result is the probability value of a wall.

[0060] It can be understood that the vector value in the i-th row and j-th column in the wall recognition result and the pixel value in the i-th row and j-th column in the target sonar image data correspond to the same position point, and i and j are both positive integers greater than 0.

[0061] Among them, the training step of the anomaly detection model includes: obtaining training data, using the training data to train the initial model to classify and predict whether each pixel point is a wall, and obtaining the anomaly detection model. The initial model adopts an anomaly detection network based on a convolutional neural network, and the training data includes: sonar image sample data and a target label matrix.

[0062] An anomaly detection model is a model that performs anomaly detection tasks. Anomaly detection is a supervised learning task (that is, it uses sonar image sample data as input, a target label matrix as the target for supervision, and trains the initial model to output an increasingly close match to the target label matrix). It involves identifying data points in the data that do not conform to the norm (i.e., data points corresponding to walls). Anomaly detection tasks can be categorized as binary classification or multi-classification problems. Binary classification problems typically classify data into anomaly (walls) and normal (objects other than walls). In other words, using the training data to train the initial model to predict whether each pixel is a wall is actually a classification model. Specific training methods can be selected from existing technologies and are not detailed here.

[0063] The target label matrix is ​​a three-dimensional matrix. When the element value (also called label value) in the target label matrix is ​​the first value, the data point corresponding to the element value is a wall. When the element value in the target label matrix is ​​the second value, the data point corresponding to the element value is not a wall.

[0064] The element value in the mth row and kth column of the target label matrix corresponds to the same location point as the pixel value in the mth row and kth column of the sonar image sample data, and both m and k are positive integers greater than 0.

[0065] It is understandable that the training data is used to train the initial model for classification prediction of whether each pixel is a wall. When the training is completed, the trained initial model is used as the anomaly detection model.

[0066] This embodiment obtains wall recognition results by acquiring target sonar image data and inputting it into a pre-trained anomaly detection model to perform pixel-by-pixel wall classification prediction, wherein the anomaly detection model uses an anomaly detection network based on a convolutional neural network and is trained with training data containing sonar image sample data and a target label matrix. The traditional box labeling method can only roughly identify the wall and has difficulty accurately presenting the outline and details of complex wall shapes. The anomaly detection model of the present application directly classifies each pixel as a wall, and can more accurately identify the wall through pixel-level classification. This means that when faced with complex underwater wall structures, the present application can overcome the limitations of the traditional box labeling method, more meticulously outline the wall's outline, accurately display the various details of the wall, and improve the accuracy of wall recognition.

[0067] Please participate Figure 5 In one embodiment, the initial model includes, in sequence: an input layer, at least two feature extraction units, a feature fusion unit, a decoding unit, and a classification unit, wherein each of the feature extraction units is arranged in parallel;

[0068] The feature extraction unit includes, in sequence: a first convolutional layer, a first activation layer, a first batch normalization layer, a first maximum pooling layer, a second convolutional layer, a second activation layer, a second batch normalization layer and a second maximum pooling layer;

[0069] The decoding unit sequentially includes: a first deconvolution layer, a third activation layer, a third batch normalization layer and a second deconvolution layer.

[0070] The input layer is used to receive data input into the initial model, and then input the received data into each feature extraction unit.

[0071] Each of the feature extraction units constitutes an encoder. Each of the feature extraction units performs feature extraction at different scales.

[0072] The feature fusion unit is used to splice the features extracted by each of the feature extraction units in the channel dimension to form a feature map that fuses multi-scale features, and input the feature map into the decoding unit.

[0073] For example, if there are four feature extraction units, the feature fusion unit concatenates these four feature maps (one per feature extraction unit) along the channel dimension. The specific principle is: concatenation is performed along the channel dimension. In PyTorch (an open-source deep learning framework for machine learning and deep learning), in the NCHW (N: batchsize; C: number of channels; H: feature map height; W: feature map width) format, torch.cat (the function used to concatenate tensors) concatenates along the channel dimension (dim=1). It is necessary to ensure that all dimensions other than the number of channels are identical; otherwise, direct concatenation will fail. Only when the N, H / 4, and W / 4 dimensions are identical can cat (concatenation) along the first dimension (the channel dimension) be successfully performed. The first and second feature extraction units have 128 dimensions, and the third and fourth feature extraction units have 64 dimensions. Therefore, after concatenation, there are 384 channels (that is, the feature fusion unit receives 384 channels of data). In other words, the feature fusion unit realizes the fusion of parallel features.

[0074] In the fields of machine learning and deep learning, batch size refers to the number of data samples used to train the model in one iteration.

[0075] From a data structure perspective, the parallel feature aggregation operation "expands" horizontally along the channel direction, allowing each convolution kernel in the subsequent network to see features from different branches. This can be thought of as the network initially splitting the image into multiple branches with different receptive fields (each feature extraction unit as a branch), each completing its own convolution and pooling path, and then "converging" at the same depth level before entering the decoding unit.

[0076] The decoding unit is used to decode the feature map input by the feature fusion unit and finally output a single-channel image.

[0077] The classification unit is a fully connected layer that uses the Sigmoid activation function. The classification unit is used to classify the input of the decoding unit.

[0078] The sigmoid activation function is a common S-shaped function in biology, also known as the S-shaped growth curve. In information science, due to its monotonic increasing properties and the monotonic increasing properties of its inverse function, the sigmoid activation function is often used as an activation function in neural networks to map variables between 0 and 1.

[0079] The first convolutional layer and the second convolutional layer both use convolutional layers.

[0080] The first activation layer, the second activation layer, and the third activation layer all use the ReLU activation function.

[0081] The ReLU (Rectified Linear Unit) activation function is one of the most commonly used activation functions in convolutional neural networks (CNNs) and many deep learning models. Its main function is to introduce nonlinearity, enabling the model to learn and express more complex features.

[0082] The features extracted by the feature extraction unit, after being processed by the ReLU activation function, can map linearly inseparable data to a new space, enabling the initial model to learn more complex patterns and relationships, thereby enhancing the nonlinear expression ability of the model.

[0083] The first batch normalization layer, the second batch normalization layer, and the third batch normalization layer all use batch normalization layers.

[0084] Both the first and second max pooling layers utilize max pooling. During the initial model run, the max pooling layer gradually reduces the spatial dimensions of the feature map. This operation directly reduces the amount of computation, as a smaller feature map size means a smaller input size for subsequent layers, thereby reducing the computational effort required to calculate the connection weights between neurons. Simultaneously, this simplification of the feature map helps prevent overfitting by removing local details and noise from the data, preventing the model from becoming overly dependent on special cases in the training data and thereby improving the model's generalization capabilities. More importantly, the max pooling layer retains the most significant features while reducing the spatial dimensions of the feature map. By selecting the maximum value in a local region, it preserves the most representative feature information within each local region. These retained significant features are crucial for accurate analysis, recognition, or classification tasks in subsequent processing layers, laying the foundation for the efficient and accurate operation of the entire model.

[0085] The first and second deconvolution layers both use deconvolution layers, also known as convolution transpose layers. Deconvolution layers gradually restore the spatial resolution of the image, ensuring that image details are restored.

[0086] This multi-branch approach (each feature extraction unit is considered a branch) enables a multi-scale feature extraction design. This multi-branch design effectively integrates image features at different scales, enabling the initial model to learn and enhance detail from complex images. The resulting enhanced image output makes walls more prominent and enhances their detectability. This design is particularly well-suited for tasks involving complex backgrounds and multi-scale features, such as sonar image sample data, which is subject to significant background noise and target variation.

[0087] Optionally, the sizes of the convolution kernels of the first convolutional layers are different.

[0088] It can be understood that the first convolution layer of the feature extraction unit is followed by a first activation layer, a first batch of normalization layers, and a first maximum pooling layer to gradually extract image features and reduce spatial resolution.

[0089] This embodiment improves the model, resulting in an improved initial model that more precisely outlines the wall's contours and accurately displays its details. This initial model maintains high recognition accuracy in underwater applications characterized by high noise and poor image quality. It demonstrates greater robustness than traditional methods and can cope with the challenges of complex underwater conditions.

[0090] In one embodiment, the number of the feature extraction units is four;

[0091] The convolution kernels of the first convolution layer and the second convolution layer of the first feature extraction unit both have a size of 3×3;

[0092] The size of the convolution kernel of the first convolution layer of the second feature extraction unit is 7×7, and the size of the convolution kernel of the second convolution layer of the second feature extraction unit is 3×3;

[0093] The size of the convolution kernel of the first convolution layer of the third feature extraction unit is 15×15, and the size of the convolution kernel of the second convolution layer of the third feature extraction unit is 5×5;

[0094] The size of the convolution kernel of the first convolution layer of the fourth feature extraction unit is 25×25, and the size of the convolution kernel of the second convolution layer of the fourth feature extraction unit is 7×7;

[0095] The sizes of the convolution kernels of the first deconvolution layer and the second deconvolution layer are both 2×2.

[0096] The first convolutional layer of the first feature extraction unit is used to extract the edges of the wall; the first convolutional layer of the second feature extraction unit is used to capture local shapes; the first convolutional layer of the third feature extraction unit is used to extract the contours and complex structures of the corresponding wall area; and the first convolutional layer of the fourth feature extraction unit is used to obtain the overall shape and background information of the wall. Specifically, a small convolution kernel (3×3) extracts low-level image details, such as the wall's edges; a small to medium convolution kernel (7×7) captures the local shape of the wall; a medium to large convolution kernel (15×15) extracts the contours and object structure of the corresponding wall area; and a large convolution kernel (25×25) extracts global features, such as the general shape of the wall and background information. By integrating these features in the feature fusion unit, the initial model can comprehensively analyze multi-scale information in the image, from local details to the overall structure (such as the wall's contour), achieving accurate wall detection and recognition.

[0097] See also Figure 3 As shown, in one embodiment, before the step of obtaining training data, the method further includes:

[0098] S31: Obtain sonar echo signal sample data;

[0099] Sonar detection equipment is used to perform sonar scanning of the underwater environment, and the sonar echo signal data obtained by the scanning is used as sonar echo signal sample data.

[0100] Specifically, the sonar echo signal sample data input by the user can be obtained through the client, the sonar echo signal sample data can be obtained from the preset storage space, and the sonar echo signal sample data sent by a third-party application can also be obtained.

[0101] S32: performing two-dimensional image matrix reconstruction, normalization processing, jet color map processing, visualization, and obtaining abnormal point marks based on the visualization according to the sonar echo signal sample data to obtain a wall coordinate set corresponding to each wall;

[0102] A jet colormap is a color map commonly used in fields like data visualization. A jet colormap covers a continuous color range from blue to green to red to yellow. It typically maps the range of data values ​​to this color range, with lower values ​​at the blue end, higher values ​​at the yellow end, and intermediate values ​​in the green and red areas. Jet colormaps are designed to provide an intuitive way to represent the range of data. By using different colors, the human eye can more easily distinguish between different ranges of values ​​in the data, helping people quickly understand the distribution and changing trends of the data.

[0103] Specifically, a two-dimensional image matrix is ​​reshaped based on the sonar echo signal sample data, the reshaped data is normalized, and the normalized data is used to enhance the visibility of reflection intensity using a jet color map. This two-dimensional image matrix reshaping and normalization ensures that the data is suitable for subsequent processing while ensuring stable learning of the initial model during training, preventing size and range differences in the input data from affecting the initial model's performance.

[0104] The sonar echo signal sample data is reshaped into a two-dimensional image matrix and normalized to ensure stable data operation in subsequent steps. Through the matrix reshaping operation, the sonar echo signal sample data is converted from a one-dimensional vector to a two-dimensional image matrix (for example, if the original data is 100 frames, it has a total of 30,720,000 data points, and we reshape it into 100*600*512, that is, 100 two-dimensional image matrices) for subsequent image processing and feature extraction. The normalization process ensures the consistency of the data scale, preventing data of different scales from negatively impacting the initial model training.

[0105] The data obtained by jet color map processing is displayed on the interface for visualization.

[0106] For example, Figure 6 As shown, Figure 6 The child Figure 6 a is a visualization of the data input into the initial model. Figure 6 The child Figure 6 b is the interface display of the data obtained by processing the jet color map to achieve a visual effect diagram, Figure 6 Wall q1 and wall q2 are shown in both a and 6b.

[0107] The specific steps for visually marking anomalies for each wall include: Based on the displayed content, when we identify an anomaly in a certain area (i.e., a wall), we interactively mark it with the mouse. When the mouse is pressed, the starting position is recorded and the coordinate record is initialized. When the mouse is dragged, the current mouse position is recorded and the coordinate list is updated. When the mouse is released, all recorded coordinates are saved. Note that pressing the mouse again clears the previously recorded coordinate list. Finally, all coordinates are saved to a text file in a simple format, with one coordinate point (x, y) per line. This text file serves as the coordinate set for that wall.

[0108] For example, if we believe that there are anomalies in two areas of the read image (that is, there are walls), then we need to first drag the mouse in one of the areas (do not release the mouse during this period) and record as many coordinates as possible while ensuring accuracy. After completion, we need to modify the path of the saved file and use the file as a wall coordinate set. Then re-execute the program and drag the mouse in another area and record as many coordinates as possible while ensuring accuracy. After completion, we need to modify the path of the saved file and use the file as a wall coordinate set.

[0109] S33: generating a multi-order polynomial equation according to each of the wall coordinate sets;

[0110] Specifically, a multi-order polynomial equation is fitted for each of the wall coordinate sets.

[0111] The multi-order polynomial equation can be a quadratic polynomial equation, a cubic polynomial equation, or a quartic polynomial equation. In other words, the multi-order polynomial equation of the present application is a polynomial equation of at least quadratic order.

[0112] Since the wall is relatively regular and its position is relatively fixed, the multi-order polynomial equation obtained by fitting is used to describe the wall.

[0113] S34: generating a two-dimensional label matrix according to the sonar echo signal sample data and each of the multi-order polynomial equations;

[0114] Specifically, a two-dimensional label matrix is ​​generated by traversing all data points through the multi-order polynomial equation of all walls, wherein the element value in the two-dimensional label matrix is ​​a first value or a second value, the first value is an abnormal point (the data point corresponding to the wall), and the second value is a normal point (the data point corresponding to the background outside the wall).

[0115] S35: Perform label expansion on the two-dimensional label matrix to obtain the target label matrix.

[0116] Specifically, label expansion is used to adjust the dimension of the tensor of the two-dimensional label matrix so that the target label matrix obtained after label expansion matches the shape of the data of the input initial model, which allows the initial model to compare images and label values ​​(element values ​​in the two-dimensional label matrix) during training.

[0117] This embodiment uses color marking (i.e., jet colormap processing) to accurately display the shape and edges of the wall, avoiding the drawback of traditional box marking methods that cannot accurately depict wall details. This not only improves recognition accuracy but also enhances the visualization of the results.

[0118] In one embodiment, the multi-order polynomial equation is a quartic polynomial equation.

[0119] For example, for Figure 6 The wall q1 in the figure is fitted with a quartic polynomial equation: ,

[0120] against Figure 6 The wall q2 in the figure is fitted with a quartic polynomial equation: , x is the x-axis coordinate of the data point.

[0121] It can be understood that the degree in the quartic polynomial equation is the largest power of x.

[0122] See also Figure 4 As shown, in one embodiment, the step of generating a two-dimensional label matrix based on the sonar echo signal sample data and each of the multi-order polynomial equations includes:

[0123] S341: traversing all data points in the sonar echo signal sample data using a double-layer loop traversal method, and calculating the absolute value of the difference between the y-axis coordinate of the position corresponding to each data point and each equation value corresponding to the data point, to obtain the target distance, wherein the equation value is the value obtained by substituting the x-axis coordinate of the position corresponding to the data point into the multi-order polynomial equation;

[0124] Specifically, a double-layer loop traversal method is used to traverse all data points in the sonar echo signal sample data, and the y-axis coordinate of the position corresponding to the data point is subtracted from the equation value obtained by substituting the x-axis coordinate of the position corresponding to the data point into the multi-order polynomial equation, and the absolute value of the subtracted data is used as a target distance.

[0125] That is, a target distance is calculated for each data point for each of the multi-order polynomial equations.

[0126] S342: If the target distance between the data point and the target is less than a preset threshold, setting the label value corresponding to the data point in the two-dimensional label matrix to a first value;

[0127] Specifically, if the target distance between the data point and the target is less than a preset threshold, it means that the data point is a wall. Therefore, in the two-dimensional label matrix, the label value corresponding to the data point is set to the first value.

[0128] S343: If the data point does not exist and the target distance is less than a preset threshold, the label value corresponding to the data point in the two-dimensional label matrix is ​​set to a second value.

[0129] Specifically, if the data point does not exist and the target distance is less than a preset threshold, it means that the data point is not a wall. Therefore, in the two-dimensional label matrix, the label value corresponding to the data point is set to the second value.

[0130] Optionally, the first value is set to 1 and the second value is set to 0.

[0131] This embodiment determines the target distance by calculating the absolute value of the difference between the y-axis coordinate of each data point's corresponding location and the equation value obtained by substituting it into a multi-order polynomial equation, effectively screening and classifying data points. When a data point exists whose target distance is less than a preset threshold, its label value is set to the first value in the two-dimensional label matrix. This helps accurately mark data points that fit the polynomial equation well, potentially indicating that these data points meet certain expected characteristics or patterns. Data points whose target distance is not less than the preset threshold are set to the second value, distinguishing data points that do not fit the polynomial equation well. This helps distinguish data points of different nature in the analysis of sonar echo signal data, such as distinguishing signals corresponding to walls from signals corresponding to points outside the wall, thereby improving the accuracy of the understanding, analysis, and subsequent processing of the sonar echo signal data.

[0132] See also Figure 3 As shown, in one embodiment, after the step of obtaining sonar echo signal sample data, the method further includes:

[0133] S36: reconstructing the sonar echo signal sample data into a three-dimensional image to obtain sample data to be processed;

[0134] Specifically, the structure and relevant parameters of the sonar echo signal sample data, such as the number of data points and their distribution across different dimensions, must be determined. Then, according to pre-defined rules, these discrete data points are spatially arranged, and the coordinate positions of each data point in the new three-dimensional space are determined for the x, y, and z dimensions. Mathematical methods such as interpolation can be used to fill in the gaps between data points to ensure the integrity and coherence of the three-dimensional image. Next, based on the characteristics of the sonar echo signal sample data, each data point is assigned a corresponding color or grayscale value to represent the echo intensity attribute. Finally, using specialized graphics rendering techniques and tools, these processed data points are combined to form a complete and intuitive three-dimensional image that visually presents the key characteristics of the sonar echo signal sample data (e.g., distribution and echo intensity).

[0135] S37: Performing geometric transformation and Gamma correction on the sample data to be processed in sequence to obtain the sonar image sample data.

[0136] Geometric transformations include rotation, translation, scaling, and flipping. For example, a three-dimensional image can be transformed by a rotation matrix. The edge of the wall is rotated 30° and moved to the edge of the 3D image to simulate different viewing angles. Here, cos is the cosine function and sin is the sine function.

[0137] Gamma correction is an image processing technique that applies nonlinear transformations to image pixel values ​​to improve image display and visual quality. Gamma correction adjusts the brightness curve to emphasize the brightness difference between walls and backgrounds, thereby enhancing the robustness and recognition capabilities of the initial model for complex scenes.

[0138] Specifically, the sample data to be processed is first geometrically transformed, and then the data obtained by the geometric transformation is gamma-corrected, and the data obtained by the gamma correction is used as the sonar image sample data.

[0139] In this embodiment, geometric transformation and gamma correction are performed on the sample data to be processed in sequence, so that the originally blurred wall becomes clear and bright after processing, and the originally strong noise is also suppressed after processing.

[0140] In one embodiment, a wall recognition system using sonar images is provided, the system comprising: an electronic device configured to implement the steps of any one of the above-mentioned wall recognition methods using sonar images.

[0141] This embodiment obtains wall recognition results by acquiring target sonar image data and inputting it into a pre-trained anomaly detection model to perform pixel-by-pixel wall classification prediction, wherein the anomaly detection model uses an anomaly detection network based on a convolutional neural network and is trained with training data containing sonar image sample data and a target label matrix. The traditional box labeling method can only roughly identify the wall and has difficulty accurately presenting the outline and details of complex wall shapes. The anomaly detection model of the present application directly classifies each pixel as a wall, and can more accurately identify the wall through pixel-level classification. This means that when faced with complex underwater wall structures, the present application can overcome the limitations of the traditional box labeling method, more meticulously outline the wall's outline, accurately display the various details of the wall, and improve the accuracy of wall recognition.

[0142] In one embodiment, a computer device is provided. The computer device may be a server 120, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client 110 via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the server 120 side of a sonar image wall recognition method.

[0143] In one embodiment, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0144] Acquire target sonar image data;

[0145] Input the target sonar image data into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, thereby obtaining a wall recognition result;

[0146] Among them, the training step of the anomaly detection model includes: obtaining training data, using the training data to train the initial model to classify and predict whether each pixel point is a wall, and obtaining the anomaly detection model. The initial model adopts an anomaly detection network based on a convolutional neural network, and the training data includes: sonar image sample data and a target label matrix.

[0147] This embodiment obtains wall recognition results by acquiring target sonar image data and inputting it into a pre-trained anomaly detection model to perform pixel-by-pixel wall classification prediction, wherein the anomaly detection model uses an anomaly detection network based on a convolutional neural network and is trained with training data containing sonar image sample data and a target label matrix. The traditional box labeling method can only roughly identify the wall and has difficulty accurately presenting the outline and details of complex wall shapes. The anomaly detection model of the present application directly classifies each pixel as a wall, and can more accurately identify the wall through pixel-level classification. This means that when faced with complex underwater wall structures, the present application can overcome the limitations of the traditional box labeling method, more meticulously outline the wall's outline, accurately display the various details of the wall, and improve the accuracy of wall recognition.

[0148] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0149] Acquire target sonar image data;

[0150] Input the target sonar image data into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, thereby obtaining a wall recognition result;

[0151] Among them, the training step of the anomaly detection model includes: obtaining training data, using the training data to train the initial model to classify and predict whether each pixel point is a wall, and obtaining the anomaly detection model. The initial model adopts an anomaly detection network based on a convolutional neural network, and the training data includes: sonar image sample data and a target label matrix.

[0152] This embodiment obtains wall recognition results by acquiring target sonar image data and inputting it into a pre-trained anomaly detection model to perform pixel-by-pixel wall classification prediction, wherein the anomaly detection model uses an anomaly detection network based on a convolutional neural network and is trained with training data containing sonar image sample data and a target label matrix. The traditional box labeling method can only roughly identify the wall and has difficulty accurately presenting the outline and details of complex wall shapes. The anomaly detection model of the present application directly classifies each pixel as a wall, and can more accurately identify the wall through pixel-level classification. This means that when faced with complex underwater wall structures, the present application can overcome the limitations of the traditional box labeling method, more meticulously outline the wall's outline, accurately display the various details of the wall, and improve the accuracy of wall recognition.

[0153] It should be noted that the above functions or steps that can be implemented by computer-readable storage media or electronic devices can be found in the corresponding descriptions of the server 120 side and the client 110 side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0154] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0155] Those skilled in the art will clearly understand that for the sake of convenience and brevity in description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0156] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A wall recognition method using sonar images, characterized in that: The method comprises: Acquire target sonar image data; Input the target sonar image data into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, thereby obtaining a wall recognition result; The training step of the anomaly detection model includes: obtaining training data, using the training data to train the initial model to classify and predict whether each pixel is a wall, thereby obtaining the anomaly detection model, wherein the initial model uses an anomaly detection network based on a convolutional neural network, and the training data includes sonar image sample data and a target label matrix; Before the step of obtaining training data, the method further includes: Obtain sonar echo signal sample data; Performing two-dimensional image matrix reshaping, normalization processing, jet color map processing, and visualization according to the sonar echo signal sample data, and obtaining abnormal point marks based on the visualization, to obtain a wall coordinate set corresponding to each wall; generating a multi-order polynomial equation according to each of the wall coordinate sets; generating a two-dimensional label matrix according to the sonar echo signal sample data and each of the multi-order polynomial equations; Perform label expansion on the two-dimensional label matrix to obtain the target label matrix.

2. The wall recognition method of sonar images according to claim 1, characterized in that: The initial model includes, in sequence: an input layer, at least two feature extraction units, a feature fusion unit, a decoding unit, and a classification unit, wherein each of the feature extraction units is arranged in parallel; The feature extraction unit includes, in sequence: a first convolutional layer, a first activation layer, a first batch normalization layer, a first maximum pooling layer, a second convolutional layer, a second activation layer, a second batch normalization layer and a second maximum pooling layer; The decoding unit sequentially includes: a first deconvolution layer, a third activation layer, a third batch normalization layer and a second deconvolution layer.

3. The wall recognition method of sonar images according to claim 2, characterized in that: The number of the feature extraction units is four; The convolution kernels of the first convolution layer and the second convolution layer of the first feature extraction unit both have a size of 3×3; The size of the convolution kernel of the first convolution layer of the second feature extraction unit is 7×7, and the size of the convolution kernel of the second convolution layer of the second feature extraction unit is 3×3; The size of the convolution kernel of the first convolution layer of the third feature extraction unit is 15×15, and the size of the convolution kernel of the second convolution layer of the third feature extraction unit is 5×5; The size of the convolution kernel of the first convolution layer of the fourth feature extraction unit is 25×25, and the size of the convolution kernel of the second convolution layer of the fourth feature extraction unit is 7×7; The sizes of the convolution kernels of the first deconvolution layer and the second deconvolution layer are both 2×2.

4. The wall recognition method using sonar images according to claim 1, wherein: The multi-order polynomial equation adopts a quartic polynomial equation.

5. The wall recognition method using sonar images according to claim 1, characterized in that: The step of generating a two-dimensional label matrix according to the sonar echo signal sample data and each of the multi-order polynomial equations includes: A double-layer loop traversal method is used to traverse all data points in the sonar echo signal sample data, and the absolute value of the difference between the y-axis coordinate of the position corresponding to each data point and each equation value corresponding to the data point is calculated to obtain the target distance, wherein the equation value is the value obtained by substituting the x-axis coordinate of the position corresponding to the data point into the multi-order polynomial equation; If the target distance between the data point and the target is less than a preset threshold, the label value corresponding to the data point in the two-dimensional label matrix is ​​set to a first value; If the data point does not exist and the target distance is less than a preset threshold, the label value corresponding to the data point in the two-dimensional label matrix is ​​set to a second value.

6. The wall recognition method using sonar images according to claim 1, characterized in that: After the step of obtaining sonar echo signal sample data, the method further includes: Performing three-dimensional image reconstruction on the sonar echo signal sample data to obtain sample data to be processed; The sample data to be processed is subjected to geometric transformation and Gamma correction in sequence to obtain the sonar image sample data.

7. A sonar image wall recognition system, characterized in that: The system includes: an electronic device configured to implement the steps of the wall recognition method using sonar images according to any one of claims 1 to 6.

8. An electronic device, characterized in that: The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the wall recognition method of the sonar image according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the wall recognition method of sonar images according to any one of claims 1 to 6 are implemented.

Images