Method and system for wall recognition in sonar image, device, and storage medium

Abstract

The present application relates to the technical field of sonar, and discloses a method and system for wall recognition in a sonar image, a device, and a storage medium. The method comprises: inputting target sonar image data into a pre-trained anomaly detection model to perform classification prediction of whether each pixel is a wall, so as to obtain a wall recognition result. Training steps of the anomaly detection model comprise: acquiring training data, and using the training data to train an initial model for classification prediction of whether each pixel is a wall, so as to obtain the anomaly detection model, wherein the initial model uses an anomaly detection network based on a convolutional neural network, and the training data comprises sonar image sample data and a target label matrix. The anomaly detection model in the present application directly performs classification recognition of whether each pixel is a wall. By means of pixel-level classification, a wall can be recognized more accurately, the contour of the wall can be outlined more precisely, and various details of the wall are accurately presented, thereby improving the accuracy of wall recognition.

Description

Methods, systems, equipment and storage media for wall recognition from sonar images Technical Field

[0001] This application relates to the fields of sonar technology and artificial intelligence technology, and in particular to a method, system, device and storage medium for wall recognition from sonar images. Background Technology

[0002] Sonar detection technology is widely used in underwater environment monitoring and obstacle detection. It transmits and receives reflected signals, generating images of underwater objects based on these signals. Changes in grayscale values ​​within the images can be used to identify various underwater targets. However, the complex underwater environment causes sonar images to be affected by noise, blur, and other factors, resulting in unstable image quality during wall recognition and making wall localization and identification difficult. In recent years, Convolutional Neural Networks (CNNs) have seen significant development in image recognition and have been used in sonar image processing for wall detection, automatically extracting image features for more accurate detection. However, current CNN-based wall recognition methods use bounding boxes to mark wall locations. This method can only provide a rough identification of the wall and struggles to accurately represent the contours and details of complex wall shapes, leading to low accuracy in wall recognition results. Summary of the Invention

[0003] Based on this, it is necessary to address the technical problem that existing CNN-based wall recognition methods use bounding boxes to mark the location of walls. This method can only roughly identify the walls and is difficult to accurately present the outline and details of complex wall shapes, resulting in low accuracy of wall recognition results. Therefore, a wall recognition method, system, device, and storage medium based on sonar images are proposed.

[0004] In a first aspect, a method for wall recognition from sonar images is provided, the method comprising:

[0005] Acquire target sonar image data;

[0006] The target sonar image data is input into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, thus obtaining the wall recognition result.

[0007] The training steps of the anomaly detection model include: acquiring training data, using the training data to train an initial model to predict whether each pixel is a wall, thereby obtaining the anomaly detection model. The initial model uses an anomaly detection network based on a convolutional neural network. The training data includes sonar image sample data and a target label matrix.

[0008] Furthermore, the initial model sequentially includes: an input layer, at least two feature extraction units, a feature fusion unit, a decoding unit, and a classification unit, with each feature extraction unit connected in parallel;

[0009] The feature extraction unit sequentially includes: a first convolutional layer, a first activation layer, a first batch normalization layer, a first max pooling layer, a second convolutional layer, a second activation layer, a second batch normalization layer, and a second max pooling layer;

[0010] The decoding unit sequentially includes: a first deconvolution layer, a third activation layer, a third batch normalization layer, and a second deconvolution layer.

[0011] Furthermore, the number of feature extraction units is four;

[0012] The kernel size of the first convolutional layer and the second convolutional layer of the first feature extraction unit is 3×3;

[0013] The kernel size of the first convolutional layer of the second feature extraction unit is 7×7, and the kernel size of the second convolutional layer of the second feature extraction unit is 3×3.

[0014] The kernel size of the first convolutional layer of the third feature extraction unit is 15×15, and the kernel size of the second convolutional layer of the third feature extraction unit is 5×5.

[0015] The kernel size of the first convolutional layer of the fourth feature extraction unit is 25×25, and the kernel size of the second convolutional layer of the fourth feature extraction unit is 7×7.

[0016] The kernel size of both the first and second deconvolution layers is 2×2.

[0017] Furthermore, prior to the step of acquiring training data, the method further includes:

[0018] Acquire sonar echo signal sample data;

[0019] Based on the sonar echo signal sample data, two-dimensional image matrix reshaping, normalization processing, JET color map processing, visualization, and abnormal point marking based on visualization are performed to obtain the wall coordinate set corresponding to each wall.

[0020] Based on each set of wall coordinates, generate multiple polynomial equations;

[0021] A two-dimensional tag matrix is ​​generated based on the sonar echo signal sample data and each of the multiple polynomial equations.

[0022] The target label matrix is ​​obtained by expanding the label matrix.

[0023] Furthermore, the multiple polynomial equations are quartic polynomial equations.

[0024] Further, the step of generating a two-dimensional tag matrix based on the sonar echo signal sample data and each of the multiple polynomial equations includes:

[0025] The target distance is obtained by iterating through all data points in the sonar echo signal sample data using a double-loop traversal method, and calculating the absolute value of the difference between the y-axis coordinate of the corresponding position of each data point and the equation value corresponding to the data point. The equation value is the value obtained by substituting the x-axis coordinate of the corresponding position of the data point into the polynomial equation.

[0026] If the target distance of a data point is less than a preset threshold, then in the two-dimensional label matrix, the label value corresponding to the data point is set to the first value;

[0027] If the target distance of the data point is less than a preset threshold, then the label value corresponding to the data point is set to the second value in the two-dimensional label matrix.

[0028] Furthermore, after the step of acquiring sonar echo signal sample data, the method further includes:

[0029] The sonar echo signal sample data is reconstructed into a three-dimensional image to obtain the sample data to be processed.

[0030] The sample data to be processed is subjected to geometric transformation and Gamma correction in sequence to obtain the sonar image sample data.

[0031] In a second aspect, a wall recognition system based on sonar images is provided, the system comprising: an electronic device configured to implement the steps of the wall recognition method based on sonar images described in any of the preceding claims.

[0032] Thirdly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described methods for wall recognition of sonar images.

[0033] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for wall recognition of sonar images.

[0034] This application discloses a method, system, device, and storage medium for wall recognition from sonar images. The method acquires target sonar image data and inputs it into a pre-trained anomaly detection model to predict wall classification at each pixel, thus obtaining the wall recognition result. The anomaly detection model employs a convolutional neural network-based anomaly detection network trained with training data including sonar image sample data and a target label matrix. Traditional bounding box marking methods can only roughly identify walls, failing to accurately represent the contours and details of complex wall shapes. In contrast, the anomaly detection model of this application directly classifies each pixel as a wall, enabling more precise wall identification through pixel-level classification. This means that when facing complex underwater wall structures, this application overcomes the limitations of traditional bounding box marking methods, more meticulously outlining the wall's contours and accurately displaying various details, thereby improving the accuracy of wall recognition. Attached Figure Description

[0035] Figure 1 shows the application environment of a wall recognition method based on sonar images in one embodiment;

[0036] Figure 2 is a flowchart of a wall recognition method based on sonar images in one embodiment;

[0037] Figure 3 is a flowchart of a wall recognition method based on sonar images in one embodiment;

[0038] Figure 4 is a flowchart of a wall recognition method based on sonar images in one embodiment;

[0039] Figure 5 is a schematic diagram of the network structure of the initial model of this application;

[0040] Figure 6 shows the correspondence between the sonar image sample data and the target label matrix of this application;

[0041] Figure 7 is a structural block diagram of an electronic device in one embodiment. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] The wall recognition method for sonar images provided in this application embodiment can be applied in the application environment shown in Figure 1, wherein the client 110 communicates with the server 120 through the network.

[0044] The server 120 is configured to implement the wall recognition method for sonar images according to this application in the following steps: acquiring target sonar image data through the client 110; inputting the target sonar image data into a pre-trained anomaly detection model to perform classification prediction of whether each pixel is a wall, thereby obtaining a wall recognition result; wherein, the training step of the anomaly detection model includes: acquiring training data, using the training data to train an initial model to perform classification prediction of whether each pixel is a wall, thereby obtaining the anomaly detection model, wherein the initial model adopts an anomaly detection network based on a convolutional neural network, and the training data includes: sonar image sample data and a target label matrix. The wall recognition result is obtained by acquiring target sonar image data and inputting it into a pre-trained anomaly detection model to perform wall classification prediction of pixels, wherein the anomaly detection model adopts an anomaly detection network based on a convolutional neural network and is trained with training data containing sonar image sample data and a target label matrix. Traditional bounding box marking methods can only roughly identify walls, making it difficult to accurately represent the outline and details of complex wall shapes. However, the anomaly detection model in this application directly classifies each pixel to determine whether it belongs to a wall. Through pixel-level classification, walls can be identified more accurately. This means that when faced with complex underwater wall structures, this application can overcome the limitations of traditional bounding box marking methods, more meticulously outline the wall's contours, accurately display various details of the wall, and improve the accuracy of wall recognition.

[0045] The client 110 (i.e., the electronic device) can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server 120 (i.e., the electronic device) can be implemented using a standalone server or a server cluster consisting of multiple servers. In other words, both the client 110 and the server 120 can be computer devices.

[0046] The present application will now be described in detail through specific embodiments.

[0047] Please refer to Figure 2, which is a flowchart illustrating a wall recognition method for sonar images provided in an embodiment of this application, including the following steps:

[0048] S1: Acquire target sonar image data;

[0049] First, a sonar detection device is used to scan the underwater environment of the target. Then, the sonar echo signal data obtained from the scan is used to reconstruct a three-dimensional image, and the reconstructed image is used as the target sonar image data.

[0050] The target underwater environment is the underwater environment in which wall identification is desired.

[0051] Sonar echo signal data is a .dat file that includes the echo intensity at each location point (also called a data point) in the target underwater environment. In other words, sonar echo signal data is one-dimensional data.

[0052] It is understandable that the pixels in the target sonar image data correspond one-to-one with the data points in the sonar echo signal data.

[0053] It is understandable that 3D image reconstruction does not fundamentally change the content of the data; it merely adds dimensionality. Specific implementation methods for 3D image reconstruction can be chosen from existing technologies, which will not be elaborated upon here.

[0054] Specifically, target sonar image data can be obtained through the client inputting the target sonar image data, or from a preset storage space, or from target sonar image data sent by a third-party application.

[0055] S2: Input the target sonar image data into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, and obtain the wall recognition result;

[0056] Specifically, the target sonar image data is input into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, and the vector obtained from the classification and prediction is used as the wall recognition result.

[0057] Each vector value in the wall recognition result is a probability value for the wall.

[0058] It is understandable that the vector value in the i-th row and j-th column of the wall recognition result corresponds to the same position point as the pixel value in the i-th row and j-th column of the target sonar image data, where i and j are both positive integers greater than 0.

[0059] The training steps of the anomaly detection model include: acquiring training data, using the training data to train an initial model to predict whether each pixel is a wall, thereby obtaining the anomaly detection model. The initial model uses an anomaly detection network based on a convolutional neural network. The training data includes sonar image sample data and a target label matrix.

[0060] An anomaly detection model is a model used for anomaly detection tasks. Anomaly detection is a supervised learning task (i.e., using sonar image sample data as input to the model and the target label matrix as the supervision target, training the initial model's output to increasingly approximate the target label matrix). It involves identifying data points in the data that do not conform to the norm (i.e., data points corresponding to walls). Anomaly detection tasks can be divided into binary classification and multi-class classification problems. Binary classification problems typically divide the data into anomaly classes (walls) and normal classes (the environment outside the walls). In other words, training the initial model using the aforementioned training data to predict whether each pixel represents a wall is training a classification model. Specific training methods can be chosen from existing technologies and will not be elaborated upon here.

[0061] The target label matrix is ​​a three-dimensional matrix. When the element value (also called the label value) in the target label matrix is ​​the first value, the data point corresponding to that element value is a wall. When the element value in the target label matrix is ​​the second value, the data point corresponding to that element value is not a wall.

[0062] The element value in the m-th row and k-th column of the target label matrix corresponds to the pixel value in the m-th row and k-th column of the sonar image sample data at the same location point, where m and k are both positive integers greater than 0.

[0063] It is understood that the training data is used to train the initial model to predict whether each pixel is a wall. When the training is completed, the initial model is used as the anomaly detection model.

[0064] This embodiment obtains wall recognition results by acquiring target sonar image data and inputting it into a pre-trained anomaly detection model to perform pixel-by-pixel wall classification prediction. The anomaly detection model employs a convolutional neural network-based anomaly detection network trained with training data including sonar image sample data and a target label matrix. Traditional bounding box marking methods can only roughly identify walls, making it difficult to accurately represent the contours and details of complex wall shapes. In contrast, the anomaly detection model of this application directly classifies each pixel to determine whether it represents a wall. Pixel-level classification enables more accurate wall identification. This means that when facing complex underwater wall structures, this application can overcome the limitations of traditional bounding box marking methods, more meticulously outlining the wall's contours, accurately displaying various details, and improving the accuracy of wall recognition.

[0065] Please refer to Figure 5. In one embodiment, the initial model sequentially includes: an input layer, at least two feature extraction units, a feature fusion unit, a decoding unit, and a classification unit, with each feature extraction unit connected in parallel.

[0066] The feature extraction unit sequentially includes: a first convolutional layer, a first activation layer, a first batch normalization layer, a first max pooling layer, a second convolutional layer, a second activation layer, a second batch normalization layer, and a second max pooling layer;

[0067] The decoding unit sequentially includes: a first deconvolution layer, a third activation layer, a third batch normalization layer, and a second deconvolution layer.

[0068] The input layer is used to receive the data input to the initial model, and then input the received data into each feature extraction unit.

[0069] The aforementioned feature extraction units constitute an encoder. Each feature extraction unit performs feature extraction at different scales.

[0070] The feature fusion unit is used to concatenate the features extracted by each of the feature extraction units along the channel dimension to form a feature map that fuses multi-scale features, and then inputs the feature map into the decoding unit.

[0071] For example, if there are four feature extraction units, the feature fusion unit will concatenate these four feature maps (each feature extraction unit outputs one feature map) along the channel dimension. Specifically, the concatenation occurs along the channel dimension: In PyTorch's NCHW (N: batch size; C: number of channels; H: feature map height; W: feature map width) format, when `torch.cat` (the function for concatenating tensors) concatenates along the channel dimension (dim=1), all dimensions except the number of channels must be identical. Otherwise, direct concatenation is impossible. Only when their N, H / 4, and W / 4 dimensions are completely identical can they be successfully concatenated along the first channel dimension. The first and second feature extraction units have 128 dimensions, and the third and fourth feature extraction units have 64 dimensions, resulting in 384 channels after concatenation (meaning the feature fusion unit inputs data with 384 channels). In other words, the feature fusion unit achieves the fusion of parallel features.

[0072] In the fields of machine learning and deep learning, batch size refers to the number of data samples used to train a model in one iteration.

[0073] From a data structure perspective, the aggregation of parallel features involves a "lateral expansion" in the channel direction, allowing each convolutional kernel of the subsequent network to access features from different branches. This can be understood as the network splitting the image into multiple branches with different receptive fields (each feature extraction unit being a branch), each completing its own convolution and pooling paths, and then "merging" at the same depth level before entering the decoding unit.

[0074] The decoding unit is used to decode the feature map input by the feature fusion unit and finally output a single-channel image.

[0075] The classification unit is a fully connected layer that uses the sigmoid activation function. The classification unit is used to classify data based on the input of the decoding unit.

[0076] The sigmoid activation function is a common sigmoid function in biology, also known as an sigmoid growth curve. In information science, due to its monotonically increasing and inversely monotonically increasing properties, the sigmoid activation function is often used as the activation function of neural networks, mapping variables to the range of 0 to 1.

[0077] Both the first and second convolutional layers are convolutional layers.

[0078] The first, second, and third activation layers all use the ReLU activation function.

[0079] The ReLU (Rectified Linear Unit) activation function is one of the most commonly used activation functions in Convolutional Neural Networks (CNNs) and many deep learning models. Its main function is to introduce non-linearity, enabling the model to learn and represent more complex features.

[0080] The features extracted by the feature extraction unit, after being processed by the ReLU activation function, can map linearly inseparable data to a new space, enabling the initial model to learn more complex patterns and relationships, thereby enhancing the model's nonlinear expressive power.

[0081] The first, second, and third batches of normalized layers all adopted batch normalization.

[0082] Both the first and second max-pooling layers employ max-pooling. During the initial model execution, the max-pooling layers progressively reduce the spatial dimension of the feature maps. This operation directly reduces computational cost, as a smaller feature map size means a smaller input scale for subsequent layers, thus reducing the computational cost of connection weights between neurons. Simultaneously, this simplification of the feature maps helps prevent overfitting by removing some local details and noise from the data, avoiding excessive reliance on special cases in the training data and improving the model's generalization ability. More importantly, the max-pooling layers retain the most salient features while reducing the spatial dimension of the feature maps. By selecting the maximum value in local regions, they preserve the most representative feature information within each local region. These preserved salient features are crucial for subsequent processing layers to accurately analyze, identify, or classify tasks, laying the foundation for the efficient and accurate operation of the entire model.

[0083] Both the first and second deconvolution layers employ deconvolution, also known as convolution transpose layers. Deconvolution layers progressively restore the spatial resolution of the image, ensuring that image details are preserved.

[0084] This multi-branch design (treating each feature extraction unit as a branch) enables multi-scale feature extraction. The multi-branch approach effectively fuses image features at different scales, allowing the initial model to learn from complex images and enhance detail information. The resulting enhanced image makes the wall more prominent and improves its detectability. The advantage of this design lies in its ability to effectively handle complex backgrounds and multi-scale features, making it particularly suitable for tasks involving sonar image sample data with significant background noise and target variation.

[0085] Optionally, the kernel size of each first convolutional layer can be different.

[0086] It is understandable that the first convolutional layer of the feature extraction unit is followed by a first activation layer, a first batch of normalization layers, and a first max pooling layer to progressively extract image features and reduce spatial resolution.

[0087] This embodiment improves the model, resulting in an initial model that more precisely outlines the wall's contours and accurately displays its various details. The initial model maintains high recognition accuracy in underwater environments with high noise and poor image quality, exhibiting stronger robustness than traditional methods and capable of handling the challenges of complex underwater conditions.

[0088] In one embodiment, the number of feature extraction units is four;

[0089] The kernel size of the first convolutional layer and the second convolutional layer of the first feature extraction unit is 3×3;

[0090] The kernel size of the first convolutional layer of the second feature extraction unit is 7×7, and the kernel size of the second convolutional layer of the second feature extraction unit is 3×3.

[0091] The kernel size of the first convolutional layer of the third feature extraction unit is 15×15, and the kernel size of the second convolutional layer of the third feature extraction unit is 5×5.

[0092] The kernel size of the first convolutional layer of the fourth feature extraction unit is 25×25, and the kernel size of the second convolutional layer of the fourth feature extraction unit is 7×7.

[0093] The kernel size of both the first and second deconvolution layers is 2×2.

[0094] The first convolutional layer of the first feature extraction unit is used to extract the edges of the wall; the first convolutional layer of the second feature extraction unit is used to capture local shapes; the first convolutional layer of the third feature extraction unit is used to extract the contours and complex structures of the corresponding area of ​​the wall; and the first convolutional layer of the fourth feature extraction unit is used to obtain the overall shape and background information of the wall. In other words, a small-sized convolutional kernel (3×3) extracts low-level detail features of the image, such as the edges of the wall; a medium-sized convolutional kernel (7×7) captures the local shape of the wall; a medium-sized convolutional kernel (15×15) extracts the contours and object structures of the corresponding area of ​​the wall; and a large-sized convolutional kernel (25×25) extracts global features, such as the approximate shape of the wall and background information. By integrating these features through a feature fusion unit, the initial model can comprehensively analyze multi-scale information of the image, from the local details of the wall to its overall structure (such as the contour of the wall), achieving accurate detection and recognition of the wall.

[0095] Please refer to Figure 3. In one embodiment, before the step of acquiring training data, the method further includes:

[0096] S31: Acquire sonar echo signal sample data;

[0097] Sonar detection equipment is used to scan the underwater environment, and the obtained sonar echo signal data is used as sonar echo signal sample data.

[0098] Specifically, sonar echo signal sample data can be obtained through the client, from a preset storage space, or from a third-party application.

[0099] S32: Based on the sonar echo signal sample data, perform two-dimensional image matrix reshaping, normalization processing, JET color map processing, visualization, and obtain anomaly point markers based on visualization to obtain the wall coordinate set corresponding to each wall;

[0100] Jet color maps are a type of color mapping commonly used in fields such as data visualization. A Jet color map contains a continuous color gradient from blue to green to red to yellow. It typically maps a range of data values ​​to this color range, with lower values ​​corresponding to the blue end, higher values ​​to the yellow end, and intermediate values ​​distributed in the green and red areas. The purpose of Jet color maps is to provide an intuitive way to represent the range of data variation. By using different colors, the human eye can more easily distinguish different numerical ranges in the data, thus helping people quickly understand the distribution and trends of the data.

[0101] Specifically, a two-dimensional image matrix is ​​reconstructed based on the sonar echo signal sample data. The reconstructed data is then normalized, and a JET color map is used to enhance the visibility of the reflection intensity. This two-dimensional image matrix reconstructing and normalization process ensures that the data is suitable for subsequent processing while also ensuring that the initial model can learn stably during training, avoiding the impact of size and range differences in the input data on the initial model's performance.

[0102] The sonar echo signal sample data undergoes two-dimensional image matrix reshaping and normalization to ensure stable operation in subsequent steps. Through matrix reshaping, the sonar echo signal sample data is transformed from a one-dimensional vector into a two-dimensional image matrix (for example, if the original data consists of 100 frames with 30,720,000 data points, it is reshaped into 100*600*512, or 100 two-dimensional image matrices), facilitating subsequent image processing and feature extraction. Normalization ensures data scale consistency, preventing data of different scales from negatively impacting initial model training.

[0103] The data obtained from Jet color mapping is displayed on the interface to achieve visualization.

[0104] For example, as shown in Figure 6, sub-figure 6a in Figure 6 is a visualization of the input initial model data, and sub-figure 6b in Figure 6 is a visualization of the data obtained by processing the Jet color map. Both Figure 6a and 6b show wall q1 and wall q2.

[0105] The specific steps for visually identifying and marking anomaly points for each wall include: Based on the interface display, when we perceive an anomaly (i.e., a wall) in a certain area, we interactively mark it using the mouse. When the mouse is pressed, the starting position is recorded, and the coordinate record is initialized; when the mouse is dragged, the current mouse position is recorded, and the coordinate list is updated; when the mouse is released, all recorded coordinate points are saved. It's important to note that pressing the mouse again will clear the previously recorded coordinate list. Finally, all coordinate points are saved to a text file with a simple format, where each line records one coordinate point (x, y). This text file serves as the wall coordinate set corresponding to that wall.

[0106] For example, if we believe that there are two abnormal areas (i.e., walls) in the image we read, we need to first drag the mouse in one area (without releasing it) and record as many coordinates as possible while ensuring accuracy. After completion, we need to change the path of the saved file and use that file as a set of wall coordinates. Then, we need to re-run the program, drag the mouse in the other area, and record as many coordinates as possible while ensuring accuracy. After completion, we need to change the path of the saved file and use that file as a set of wall coordinates.

[0107] S33: Generate multiple polynomial equations based on each set of wall coordinates;

[0108] Specifically, a polynomial equation of multiple degrees is fitted for each set of wall coordinates.

[0109] The polynomial equation of degree can be a quadratic polynomial equation, a cubic polynomial equation, or a quartic polynomial equation. That is to say, the polynomial equation of degree in this application is a polynomial equation of at least degree.

[0110] Because the wall is relatively regular and its position is relatively fixed, a multinomial equation obtained through fitting is used to describe the wall.

[0111] S34: Generate a two-dimensional tag matrix based on the sonar echo signal sample data and each of the multiple polynomial equations;

[0112] Specifically, a two-dimensional label matrix is ​​generated by traversing all data points through the polynomial equations of all walls. The element values ​​in the two-dimensional label matrix are either a first value or a second value. The first value represents an outlier (data point corresponding to the wall) and the second value represents a normal point (data point corresponding to the background outside the wall).

[0113] S35: Expand the two-dimensional label matrix to obtain the target label matrix.

[0114] Specifically, label expansion is used to adjust the dimension of the tensor of the two-dimensional label matrix so that the target label matrix obtained after label expansion matches the shape of the input initial pattern data. This allows the initial model to be compared with the image and label values ​​(element values ​​in the two-dimensional label matrix) during training.

[0115] This embodiment accurately displays the shape and edges of the wall using color marking (i.e., JET color mapping), avoiding the shortcomings of traditional box marking methods that cannot depict wall details in detail. This not only improves the accuracy of recognition but also enhances the visualization of the results.

[0116] In one embodiment, the multiple polynomial equation is a quartic polynomial equation.

[0117] For example, for wall q1 in Figure 6, the fitted fourth-order polynomial equation is:

[0118] For the wall q2 in Figure 6, the fourth-order polynomial equation is obtained by fitting: , where x is the x-axis coordinate of the data point.

[0119] It is understandable that the degree in a quartic polynomial equation is the largest power of x.

[0120] Referring to Figure 4, in one embodiment, the step of generating a two-dimensional tag matrix based on the sonar echo signal sample data and each of the multiple polynomial equations includes:

[0121] S341: Use a double-layer loop traversal method to traverse all data points in the sonar echo signal sample data, and calculate the absolute value of the difference between the y-axis coordinate of the corresponding position of each data point and the equation value corresponding to the data point, to obtain the target distance. The equation value is the value obtained by substituting the x-axis coordinate of the corresponding position of the data point into the polynomial equation.

[0122] Specifically, a double-loop traversal method is used to traverse all data points in the sonar echo signal sample data. The y-axis coordinate of the data point is subtracted from the x-axis coordinate of the data point and substituted into the polynomial equation to calculate the equation value. The absolute value of the subtraction is used as a target distance.

[0123] In other words, a target distance will be calculated for each data point for each of the aforementioned polynomial equations.

[0124] S342: If the target distance of a data point is less than a preset threshold, then in the two-dimensional label matrix, the label value corresponding to the data point is set to the first value;

[0125] Specifically, if the target distance of a data point is less than a preset threshold, it means that the data point is a wall. Therefore, in the two-dimensional label matrix, the label value corresponding to the data point is set as the first value.

[0126] S343: If the target distance of the data point is less than a preset threshold, then in the two-dimensional label matrix, the label value corresponding to the data point is set to the second value.

[0127] Specifically, if the target distance of the data point is less than a preset threshold, it means that the data point is not a wall. Therefore, in the two-dimensional label matrix, the label value corresponding to the data point is set as the second value.

[0128] Optionally, the first value can be set to 1 and the second value to 0.

[0129] This embodiment determines the target distance by calculating the absolute value of the difference between the y-axis coordinate of each data point and the equation value obtained by substituting it into multiple polynomial equations. This effectively filters and classifies data points. When there are data points with a target distance less than a preset threshold, their label value is set to the first value in the two-dimensional label matrix. This helps to accurately mark data points that fit the polynomial equation well, possibly indicating that these data points conform to a certain expected characteristic or pattern. For data points without a target distance less than the preset threshold, a second value is set, which can distinguish data points that do not fit the polynomial equation well. This helps to differentiate data points of different natures in the analysis of sonar echo signal data, such as distinguishing signals corresponding to walls from signals corresponding to points outside the walls, thereby improving the accuracy of understanding, analyzing, and subsequent processing of sonar echo signal data.

[0130] Please refer to Figure 3. In one embodiment, after the step of acquiring sonar echo signal sample data, the method further includes:

[0131] S36: Perform three-dimensional image reconstruction on the sonar echo signal sample data to obtain sample data to be processed;

[0132] Specifically, first, the structure and relevant parameters of the sonar echo signal sample data need to be determined, such as the number of data points and their distribution across different dimensions. Then, according to pre-defined rules, the spatial layout of these discrete data points is planned, determining the coordinates of each data point in the new three-dimensional space for each of the x, y, and z dimensions. Mathematical methods such as interpolation can be used to fill in gaps between data points to ensure the integrity and coherence of the three-dimensional image. Next, based on the characteristics of the sonar echo signal sample data, each data point is assigned a corresponding color or grayscale value to represent the echo intensity attribute. Finally, using specialized graphics rendering techniques and tools, these processed data points are combined to form a complete and intuitive three-dimensional image that visually presents the important features of the sonar echo signal sample data (such as distribution and echo intensity).

[0133] S37: Perform geometric transformation and Gamma correction on the sample data to be processed in sequence to obtain the sonar image sample data.

[0134] Geometric transformations include rotation, translation, scaling, and flipping. For example, a 3D image can be rotated by 30° to move the edge of a wall to the edge of the 3D image using a rotation matrix to simulate different perspectives. Here, cos is the cosine function and sin is the sine function.

[0135] Gamma correction is an image processing technique that improves image display and visual quality by applying a non-linear transformation to the pixel values ​​of an image. Gamma correction adjusts the brightness curve to further highlight the brightness difference between walls and the background, thereby enhancing the robustness and recognition ability of the initial model in complex scenes.

[0136] Specifically, the sample data to be processed is first subjected to geometric transformation, and then the data obtained by geometric transformation is subjected to Gamma correction. The data obtained by Gamma correction is used as the sonar image sample data.

[0137] This embodiment performs geometric transformation and Gamma correction on the sample data to be processed in sequence, making the originally blurry wall clear and bright after processing, while the originally strong noise is also suppressed after processing.

[0138] In one embodiment, a wall recognition system for sonar images is provided, the system comprising: an electronic device configured to implement the steps of the wall recognition method for sonar images described above.

[0139] This embodiment obtains wall recognition results by acquiring target sonar image data and inputting it into a pre-trained anomaly detection model to perform pixel-by-pixel wall classification prediction. The anomaly detection model employs a convolutional neural network-based anomaly detection network trained with training data including sonar image sample data and a target label matrix. Traditional bounding box marking methods can only roughly identify walls, making it difficult to accurately represent the contours and details of complex wall shapes. In contrast, the anomaly detection model of this application directly classifies each pixel to determine whether it represents a wall. Pixel-level classification enables more accurate wall identification. This means that when facing complex underwater wall structures, this application can overcome the limitations of traditional bounding box marking methods, more meticulously outlining the wall's contours, accurately displaying various details, and improving the accuracy of wall recognition.

[0140] In one embodiment, a computer device is provided, which may be a server 120, and its internal structure diagram is shown in Figure 7. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used to communicate with an external client 110 via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a sonar image wall recognition method on the server 120 side.

[0141] In one embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, performs the following steps:

[0142] Acquire target sonar image data;

[0143] The target sonar image data is input into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, thus obtaining the wall recognition result.

[0144] The training steps of the anomaly detection model include: acquiring training data, using the training data to train an initial model to predict whether each pixel is a wall, thereby obtaining the anomaly detection model. The initial model uses an anomaly detection network based on a convolutional neural network. The training data includes sonar image sample data and a target label matrix.

[0145] This embodiment obtains wall recognition results by acquiring target sonar image data and inputting it into a pre-trained anomaly detection model to perform pixel-by-pixel wall classification prediction. The anomaly detection model employs a convolutional neural network-based anomaly detection network trained with training data including sonar image sample data and a target label matrix. Traditional bounding box marking methods can only roughly identify walls, making it difficult to accurately represent the contours and details of complex wall shapes. In contrast, the anomaly detection model of this application directly classifies each pixel to determine whether it represents a wall. Pixel-level classification enables more accurate wall identification. This means that when facing complex underwater wall structures, this application can overcome the limitations of traditional bounding box marking methods, more meticulously outlining the wall's contours, accurately displaying various details, and improving the accuracy of wall recognition.

[0146] In one embodiment, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, performs the following steps:

[0147] Acquire target sonar image data;

[0148] The target sonar image data is input into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, thus obtaining the wall recognition result.

[0149] The training steps of the anomaly detection model include: acquiring training data, using the training data to train an initial model to predict whether each pixel is a wall, thereby obtaining the anomaly detection model. The initial model uses an anomaly detection network based on a convolutional neural network. The training data includes sonar image sample data and a target label matrix.

[0150] This embodiment obtains wall recognition results by acquiring target sonar image data and inputting it into a pre-trained anomaly detection model to perform pixel-by-pixel wall classification prediction. The anomaly detection model employs a convolutional neural network-based anomaly detection network trained with training data including sonar image sample data and a target label matrix. Traditional bounding box marking methods can only roughly identify walls, making it difficult to accurately represent the contours and details of complex wall shapes. In contrast, the anomaly detection model of this application directly classifies each pixel to determine whether it represents a wall. Pixel-level classification enables more accurate wall identification. This means that when facing complex underwater wall structures, this application can overcome the limitations of traditional bounding box marking methods, more meticulously outlining the wall's contours, accurately displaying various details, and improving the accuracy of wall recognition.

[0151] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or electronic device described above can be referred to the relevant descriptions of the server 120 side and the client 110 side in the aforementioned method embodiments. To avoid repetition, they will not be described one by one here.

[0152] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0153] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0154] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for wall recognition from sonar images, characterized in that, The method includes: Acquire target sonar image data; The target sonar image data is input into a pre-trained anomaly detection model to classify and predict whether each pixel is a wall, thus obtaining the wall recognition result. The training steps of the anomaly detection model include: acquiring training data, using the training data to train an initial model to predict whether each pixel is a wall, thereby obtaining the anomaly detection model. The initial model uses an anomaly detection network based on a convolutional neural network. The training data includes sonar image sample data and a target label matrix.

2. The wall recognition method for sonar images according to claim 1, characterized in that, The initial model sequentially includes: an input layer, at least two feature extraction units, a feature fusion unit, a decoding unit, and a classification unit, with each feature extraction unit connected in parallel; The feature extraction unit sequentially includes: a first convolutional layer, a first activation layer, a first batch normalization layer, a first max pooling layer, a second convolutional layer, a second activation layer, a second batch normalization layer, and a second max pooling layer; The decoding unit sequentially includes: a first deconvolution layer, a third activation layer, a third batch normalization layer, and a second deconvolution layer.

3. The wall recognition method for sonar images according to claim 2, characterized in that, The number of feature extraction units is four; The kernel size of the first convolutional layer and the second convolutional layer of the first feature extraction unit is 3×3; The kernel size of the first convolutional layer of the second feature extraction unit is 7×7, and the kernel size of the second convolutional layer of the second feature extraction unit is 3×3. The kernel size of the first convolutional layer of the third feature extraction unit is 15×15, and the kernel size of the second convolutional layer of the third feature extraction unit is 5×5. The kernel size of the first convolutional layer of the fourth feature extraction unit is 25×25, and the kernel size of the second convolutional layer of the fourth feature extraction unit is 7×7. The kernel size of both the first and second deconvolution layers is 2×2.

4. The wall recognition method for sonar images according to claim 1, characterized in that, Before the step of acquiring training data, the method further includes: Acquire sample data of sonar echo signals; Based on the sonar echo signal sample data, two-dimensional image matrix reshaping, normalization processing, JET color map processing, visualization, and abnormal point marking based on visualization are performed to obtain the wall coordinate set corresponding to each wall. Based on each set of wall coordinates, generate multiple polynomial equations; A two-dimensional tag matrix is ​​generated based on the sonar echo signal sample data and each of the multiple polynomial equations. The target label matrix is ​​obtained by expanding the label matrix.

5. The wall recognition method for sonar images according to claim 4, characterized in that, The multiple polynomial equations are quartic polynomial equations.

6. The wall recognition method for sonar images according to claim 4, characterized in that, The step of generating a two-dimensional label matrix based on the sonar echo signal sample data and each of the multiple polynomial equations includes: The target distance is obtained by iterating through all data points in the sonar echo signal sample data using a double-loop traversal method, and calculating the absolute value of the difference between the y-axis coordinate of the corresponding position of each data point and the equation value corresponding to the data point. The equation value is the value obtained by substituting the x-axis coordinate of the corresponding position of the data point into the polynomial equation. If the target distance of a data point is less than a preset threshold, then in the two-dimensional label matrix, the label value corresponding to the data point is set to the first value; If the target distance of the data point is less than a preset threshold, then the label value corresponding to the data point is set to the second value in the two-dimensional label matrix.

7. The wall recognition method for sonar images according to claim 4, characterized in that, After the step of acquiring sonar echo signal sample data, the method further includes: The sonar echo signal sample data is reconstructed into a three-dimensional image to obtain the sample data to be processed. The sample data to be processed is subjected to geometric transformation and Gamma correction in sequence to obtain the sonar image sample data.

8. A wall recognition system based on sonar images, characterized in that, The system includes an electronic device configured to implement the steps of the wall recognition method for sonar images as described in any one of claims 1 to 7.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the wall recognition method for sonar images as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the wall recognition method for sonar images as described in any one of claims 1 to 7.

Images