Underwater acoustic image classification method and system based on edge calculation, and electronic equipment

The edge computing underwater acoustic image classification method addresses challenges of unclear images and high energy consumption by using adaptive preprocessing and AI models for accurate target detection in complex underwater environments.

CN119068258APending Publication Date: 2024-12-03SHANGHAI OCEAN UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411197663.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing underwater imaging technologies face challenges in complex underwater environments due to light attenuation, scattering, and absorption, leading to unclear images, accuracy issues in target detection, high energy consumption, and exposure concerns.

Method used

An edge computing underwater acoustic image classification method using sensors, sonar equipment, and AI models like SRGAN, AUGGAN, U2Net, and VGG for adaptive image preprocessing and hybrid ranging, with SIFT feature matching and transfer learning to enhance image classification.

Benefits of technology

Enables accurate and efficient underwater acoustic image classification, reducing energy consumption and concealment issues, while maintaining image quality and enhancing target detection in complex underwater conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068258A_ABST
    Figure CN119068258A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of marine intelligent analysis, and particularly discloses an underwater acoustic image classification method and system based on edge calculation and electronic equipment, and the method adopts a sonar technology and carries Nvidia Jetson nano edge equipment. The edge device realizes data acquisition and processing in an underwater environment, and has strong computing capability and processing capability. According to the method, acoustic image data can be acquired, and multi-level processing including image enhancement, augmentation, segmentation and classification is carried out through an AI algorithm, so that intelligent identification and classification of acoustic images are realized. Meanwhile, the system also has multiple sensor functions, can obtain the information of the posture of the equipment and the surrounding environment, carries out the unified processing of the data, and outputs and displays the data, so as to achieve all the functions of the system. Through the technical means, the scheme can realize accurate detection and classification of underwater targets, and provides important technical support for underwater environment research and monitoring.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of marine intelligent analysis technology, and more specifically, to an edge computing underwater acoustic image classification method, system and electronic equipment. Background Art

[0002] Existing technologies cannot adapt to special underwater scenes, including edge computing devices and optical imaging devices, which cannot be applied to complex underwater environments. Although the imaging of some optical devices is more intuitive at close range, optical imaging technology will be affected by many factors, and even optical imaging technology will not work properly if it encounters a complex underwater environment. In addition, edge computing devices cannot remain stable underwater, which will have an adverse effect on image acquisition and subsequent image processing, including the following problems:

[0003] 1. Adaptability of optical images: In underwater environments, optical images are affected by factors such as light attenuation, scattering, and absorption, so it is difficult to obtain clear images in deep water or low visibility conditions.

[0004] 2. Accuracy issues in target detection and recognition: Targets in underwater environments usually have complex shapes and appearances, and traditional optical images have difficulty accurately identifying targets in low visibility.

[0005] 3. Energy consumption problem: Traditional optical image sensors usually require a lot of energy to illuminate the target, which causes energy consumption problems.

[0006] 4. Concealment and concealment issues: In some application scenarios, such as underwater reconnaissance and marine resource exploration, concealment and concealment are very important, while optical sensors are easily exposed.

[0007] In view of this, the present invention provides an edge computing underwater acoustic image classification method, system and electronic device to solve the above problems. Summary of the invention

[0008] In order to overcome the problems in the prior art, the present invention proposes an edge computing underwater acoustic image classification method, system and electronic equipment, which can cope with complex underwater environments in real time through the application of adaptive image preprocessing and hybrid ranging algorithm.

[0009] The present invention provides an underwater acoustic image classification method for edge computing, which is applied to an edge computing device and includes the following steps:

[0010] Step A1: Use sensors and sonar equipment to collect data on the underwater environment, and match the collected data to obtain acoustic image data;

[0011] Step A2: Perform image processing on the acoustic image data based on the AI ​​model to obtain acoustic image classification.

[0012] As a preferred technical solution of the present invention, the sensors include water quality sensors, pressure sensors, flow rate sensors and attitude sensors. The data of sensors and sonar equipment are connected to the system control center. The system control center is based on access to external data for extended data collection. The system control center is integrated on an edge computing device.

[0013] As a preferred technical solution of the present invention, water quality data, pressure data, flow rate data and posture data corresponding to the current underwater detection equipment are obtained based on sensors, and acoustic images are obtained based on the sonar frequency and beam pattern of the sonar equipment. The water quality data, pressure data, flow rate data and acoustic images are feature matched to generate acoustic image data.

[0014] As a preferred technical solution of the present invention, the logic for acquiring the acoustic image data is:

[0015] Align the collected data of different sensors in time and space to ensure that all sensors collect data at the same timestamp and location information;

[0016] SI FT feature matching technology is used to match the visual features of the acoustic image with the physical features of the sensor based on the timestamp and location information.

[0017] As a preferred technical solution of the present invention, the specific steps of SI FT feature matching technology are as follows:

[0018] Convolve the acoustic image using Gaussian kernels of different scales to construct a scale space;

[0019] In scale space, local extreme points are detected by comparing images of adjacent scales;

[0020] The detected extreme points are accurately located. By fitting a three-dimensional quadratic function, the position and scale of the key points can be determined, and the key points with low contrast and edge response points can be removed.

[0021] Assign one or more directions to each key point to make it rotation invariant; by calculating the gradient direction histogram in the neighborhood of the key point, the main direction and auxiliary direction of the key point can be determined;

[0022] Generate a feature descriptor based on the position, scale, and direction of the key point; the feature descriptor includes a 128-dimensional feature vector, which represents the gradient information in the neighborhood of the key point based on the 128-dimensional feature vector;

[0023] The generated feature descriptors are used to match the key points between different acoustic images. The Euclidean distance between two feature descriptors is calculated and matching key point pairs are found according to preset rules.

[0024] As a preferred technical solution of the present invention, the AI ​​model includes an SRGAN algorithm model, an AUGGAN algorithm model, a U2Net algorithm model and a VGG algorithm, wherein: acoustic image enhancement is performed based on the SRGAN algorithm model, and acoustic image data augmentation is performed based on the AUGGAN algorithm model; the U2Net algorithm model performs image segmentation, and the VGG algorithm performs image classification.

[0025] As a preferred technical solution of the present invention, the processing logic of the acoustic image classification is:

[0026] The acoustic image data is resized to an image size of 224x224 pixels;

[0027] The acoustic image data is processed by the SRGAN algorithm model and the AUGGAN algorithm model to update and expand the acoustic image data set; the acoustic image data set is divided into a training set, a validation set, and a test set;

[0028] Transfer learning is adopted to load a pre-trained VGG model on the acoustic image dataset. The VGG model includes: freezing the convolution layer, so that the convolution layer remains frozen and does not participate in training, and using the convolution layer as a feature extractor to extract features; adding a custom fully connected layer on top of the convolution layer of the VGG model, and updating the weights in the fully connected layer.

[0029] Use the validation set to evaluate the performance of the model and adjust the hyperparameters.

[0030] The test set is used for final performance evaluation.

[0031] As a preferred technical solution of the present invention, the steps of updating weights in the fully connected layer are:

[0032] Determine the structural composition of the VGG model, including convolutional layers for extracting local features of the image and fully connected layers for integrating the features extracted by all previous layers for classification and regression;

[0033] Calculate the error between the output of the fully connected layer and the actual label based on the loss function, and propagate the error back to the fully connected layer; count the contribution of each weight to the output error;

[0034] Specifically, the error signal will be distributed to different neurons according to the size of the weight. The larger the weight, the greater the impact on the error; conversely, the smaller the weight, the smaller the impact on the error.

[0035] Based on the error obtained through back propagation, the weights of the fully connected layer are updated using an optimization algorithm;

[0036] Multiple iterations are performed over the acoustic image dataset to adjust the weights and model parameters.

[0037] In a second aspect, the present invention provides an underwater acoustic image classification system for edge computing, based on the implementation of the first aspect, comprising a system control center and an algorithm function module, wherein each module is connected by wire or wirelessly;

[0038] The system control center uses sensors and sonar equipment to collect data about the underwater environment and matches the collected data to obtain acoustic image data;

[0039] The algorithm function module processes the acoustic image data based on the AI ​​model to obtain acoustic image classification.

[0040] In a third aspect, the present invention provides an electronic device comprising: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0041] The processor executes the first aspect by calling a computer program stored in the memory.

[0042] The specific advantages of the present invention are as follows:

[0043] The present invention performs acoustic detection on underwater fish organisms, and performs AI algorithm processing on the collected acoustic image data, including image enhancement, image augmentation, image segmentation and image classification, so as to achieve the function of intelligent recognition and classification of acoustic images. It can also be equipped with a variety of sensors to obtain the device posture and surrounding environment information, and output and display them uniformly, ultimately forming the full functions of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is an architecture diagram of the underwater acoustic image classification system of the present invention;

[0045] Figure 2 This is a flow chart of the underwater acoustic image classification method of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Example 1

[0048] See also Figure 1 This embodiment provides an edge computing underwater acoustic image classification method, using a JetsonNano development board as an edge computing device, and includes the following steps:

[0049] Step A1: Use sensors and sonar equipment to collect data on the underwater environment, and match the collected data to obtain acoustic image data;

[0050] Specifically, the sensors include water quality sensors, pressure sensors, flow rate sensors and attitude sensors. The data of sensors and sonar equipment are connected to the system control center, and the system control center is connected to external data for extended data collection. Water quality data, pressure data, flow rate data and attitude data corresponding to the current underwater detection equipment are obtained based on the sensors. Acoustic images are obtained based on the sonar frequency and beam mode of the sonar equipment. Acoustic image data is generated by feature matching of water quality data, pressure data, flow rate data and acoustic images.

[0051] Specifically, the logic for acquiring the acoustic image data is:

[0052] Align the collected data of different sensors in time and space to ensure that all sensors collect data at the same timestamp and location information;

[0053] SI FT feature matching technique is used to match the visual features of the acoustic image with the physical features of the sensor based on the timestamp and location information.

[0054] More specifically, feature matching technology is a means of extracting multimodal features. SIFT feature extraction is one of the feature extraction methods for image modality in multimodal systems. This embodiment takes SIFT feature matching technology as an example. The image features extracted by SIFT can be combined with features of other modalities (such as TF-IDF features of text or spectral features of audio) to provide richer data representation, enhance the model's ability to understand data, and improve task performance, such as in applications such as image annotation, visual question answering, or cross-modal retrieval. Make more full use of the complementarity of data to improve the expression ability and performance of the model.

[0055] The specific steps of SI FT feature matching technology are as follows:

[0056] Convolve the acoustic image using Gaussian kernels of different scales to construct a scale space;

[0057] In scale space, local extreme points are detected by comparing images of adjacent scales;

[0058] The detected extreme points are accurately located. By fitting a three-dimensional quadratic function, the position and scale of the key points can be determined, and the key points with low contrast and edge response points can be removed.

[0059] Assign one or more directions to each key point to make it rotation invariant; by calculating the gradient direction histogram in the neighborhood of the key point, the main direction and auxiliary direction of the key point can be determined;

[0060] Generate a feature descriptor based on the position, scale, and direction of the key point; the feature descriptor includes a 128-dimensional feature vector, which represents the gradient information in the neighborhood of the key point based on the 128-dimensional feature vector;

[0061] Specifically, the area around the key point is divided into multiple sub-regions, and the gradient direction and magnitude are calculated in each sub-region, and the gradient directions and magnitudes in all sub-regions are summarized into a feature vector;

[0062] The generated feature descriptors are used to match the key points between different acoustic images. The Euclidean distance between two feature descriptors is calculated and matching key point pairs are found according to preset rules.

[0063] Specifically, the preset rule is generally that the descriptor pair with the smallest Euclidean distance is considered to be matched; however, in actual applications, the range of the Euclidean distance can be set according to actual conditions. In addition, SI FT feature extraction is a technology widely used in the field of image processing. It can detect local features in images and generate feature descriptors, and the feature descriptors are invariant to scale scaling, rotation and brightness changes. In the introduction, image features are extracted by combining the steps of constructing a scale space, detecting extreme points, filtering feature points, assigning direction values ​​and generating feature descriptions. Features are extracted from different types of data sources (such as images, text, audio, etc.), and these features are further fused to obtain a more comprehensive representation.

[0064] Among them: the feature extraction method of the visual features of the acoustic image and the physical features of the sensor can be parallel, extracting features from data of different modalities separately and then fusing them; it can also be serial, that is, the data is extracted features through different networks in turn and then fused; or it can be embedded, mapping data of different modalities to a shared space for fusion, and these fusion methods are not limited.

[0065] Step A2: performing image processing on the acoustic image data based on the AI ​​model to obtain acoustic image classification;

[0066] It should be noted that: this embodiment uses sonar technology, artificial intelligence algorithms and edge computing devices to provide strong technical support for underwater detection; according to the depth and complexity of the underwater environment, the sonar frequency and beam mode of the sonar device are adaptively selected to ensure the best detection range and resolution. For the above algorithm model, if there are other similar artificial intelligence algorithms, as long as they can achieve similar effects, they can actually be replaced with each other; real-time data processing and analysis based on edge computing devices will promote the development and application of edge computing technology, improve the performance and efficiency of edge intelligent devices, and promote the innovation of underwater monitoring technology, providing more possibilities for scientific research and engineering applications in the marine field.

[0067] The edge computing device is an AI computer designed for edge computing; in this embodiment, the edge computing device uses an NVIDIA Jetson nano development board. The NVIDIA Jetson nano development board includes an NVIDIA GPU; the NVIDIA GPU includes a system control center for real-time acquisition of acoustic image data and an algorithm function module for real-time processing of acoustic image data:

[0068] As an edge computing device, the Jetson Nano development board obtains the storage module of acoustic image information in the system control center for processing, which reduces the need to transmit data to the cloud and reduces latency. Therefore, the corresponding storage module has sufficient storage space to save temporary data and model parameters, which is very helpful for continuous data processing and model training. In addition, the Jetson Nano development board is used for long-term operation in an underwater environment, so it needs to operate at low power consumption and have an effective heat dissipation mechanism, so that when it is integrated into underwater detection equipment, it will not have much impact on the size and weight of the equipment.

[0069] Specifically, the AI ​​models include the SRGAN algorithm model, the AUGGAN algorithm model, the U2Net algorithm model and the VGG algorithm, among which: acoustic image enhancement is performed based on the SRGAN algorithm model, and acoustic image data augmentation is performed based on the AUGGAN algorithm model to increase data diversity and prevent overfitting; the U2Net algorithm model is used for image segmentation, and the VGG algorithm is used for image classification.

[0070] The SRGAN algorithm model mainly uses adversarial loss and perceptual loss, which helps to maintain the texture details of the image to generate super-resolution images that are more in line with human visual perception.

[0071] The AUGGAN algorithm model generates sufficiently realistic data through the generator to deceive the discriminator. At the same time, the discriminator is responsible for distinguishing between real data and fake data generated by the generator. By alternately training the generator and the discriminator, AUGGAN is able to generate diverse images, thereby increasing the diversity of the data and preventing overfitting.

[0072] More specifically, the processing logic of acoustic image classification is:

[0073] The acoustic image data is resized to an image size of 224x224 pixels;

[0074] The acoustic image data is processed by the SRGAN algorithm model and the AUGGAN algorithm model to update and expand the acoustic image data set; the acoustic image data set is divided into a training set, a validation set, and a test set;

[0075] Transfer learning is adopted to load a pre-trained VGG model on the acoustic image dataset. The VGG model includes: freezing the convolution layer, so that the convolution layer remains frozen and does not participate in training, using the convolution layer as a feature extractor to extract features, and the pooling layer further compresses the size of the feature map during the feature extraction process while maintaining important feature information; adding a custom fully connected layer on top of the convolution layer of the VGG model, and updating the weights in the fully connected layer.

[0076] Among them: the training set data is passed through the convolution layer of the VGG model to extract features, and the extracted features are sent to the custom fully connected layer, and the weights are updated in the fully connected layer.

[0077] Use the validation set to evaluate the performance of the model and adjust the hyperparameters.

[0078] The test set is used for final performance evaluation.

[0079] It should be noted that the VGG algorithm is used to classify the acoustic image, wherein the VGG network is mainly composed of multiple convolutional layers, fully connected layers and pooling layers, and multiple 3x3 convolutional kernels are stacked to replace large-scale convolutional kernels, so that the 3x3 filter can provide more spatial coverage with the same number of parameters. By increasing the depth of the network, VGG is able to learn richer feature representations. Continuous 3x3 convolutional layers can effectively capture more local information and integrate it into global features. Due to the relatively simple structure of VGG, it is easier to train and has strong generalization ability. This makes it have shown superior performance in various computer vision tasks.

[0080] It is further explained that the steps for updating weights in the fully connected layer are:

[0081] Determine the structural composition of the VGG model, including convolutional layers for extracting local features of the image and fully connected layers for integrating the features extracted by all previous layers for classification and regression;

[0082] Calculate the error between the output of the fully connected layer and the actual label based on the loss function, and propagate the error back to the fully connected layer; count the contribution of each weight to the output error;

[0083] Specifically, the error signal will be distributed to different neurons according to the size of the weight. The larger the weight, the greater the impact on the error; conversely, the smaller the weight, the smaller the impact on the error.

[0084] Based on the error obtained through back propagation, the weights of the fully connected layer are updated using an optimization algorithm;

[0085] Specifically, the optimization algorithm includes the gradient descent method, and the goal of weight update is to reduce the prediction error and thus improve the prediction accuracy of the model.

[0086] Multiple iterations are performed over the acoustic image dataset to adjust the weights and model parameters.

[0087] Specifically, based on the training set, the update weights are determined to reduce prediction errors and improve the prediction accuracy of the model; the validation set is then used to evaluate the performance of the model and adjust the model parameters; finally, the test set is used for the final performance evaluation.

[0088] Example 2

[0089] like Figure 2 As shown, for the parts not described in detail in this embodiment, please refer to the description of embodiment 1. This embodiment provides an underwater acoustic image classification system for edge computing, including a system control center and an algorithm function module; each module is connected by wire or wirelessly;

[0090] The system control center uses sensors and sonar equipment to collect data about the underwater environment and matches the collected data to obtain acoustic image data;

[0091] Specifically, the sensors include water quality sensors, pressure sensors, flow rate sensors and attitude sensors. The data of sensors and sonar equipment are connected to the system control center, and the system control center is connected to external data for extended data collection. Water quality data, pressure data, flow rate data and attitude data corresponding to the current underwater detection equipment are obtained based on the sensors. Acoustic images are obtained based on the sonar frequency and beam mode of the sonar equipment. Acoustic image data is generated by feature matching of water quality data, pressure data, flow rate data and acoustic images.

[0092] Specifically, the logic for acquiring the acoustic image data is:

[0093] Align the collected data of different sensors in time and space to ensure that all sensors collect data at the same timestamp and location information;

[0094] SI FT feature matching technique is used to match the visual features of the acoustic image with the physical features of the sensor based on the timestamp and location information.

[0095] The specific steps of SI FT feature matching technology are as follows:

[0096] Convolve the acoustic image using Gaussian kernels of different scales to construct a scale space;

[0097] In scale space, local extreme points are detected by comparing images of adjacent scales;

[0098] The detected extreme points are accurately located. By fitting a three-dimensional quadratic function, the position and scale of the key points can be determined, and the key points with low contrast and edge response points can be removed.

[0099] Assign one or more directions to each key point to make it rotation invariant; by calculating the gradient direction histogram in the neighborhood of the key point, the main direction and auxiliary direction of the key point can be determined;

[0100] Generate a feature descriptor based on the position, scale, and direction of the key point; the feature descriptor includes a 128-dimensional feature vector, which represents the gradient information in the neighborhood of the key point based on the 128-dimensional feature vector;

[0101] The generated feature descriptors are used to match the key points between different acoustic images. The Euclidean distance between two feature descriptors is calculated and matching key point pairs are found according to preset rules.

[0102] Algorithm function module, which processes acoustic image data based on AI model to obtain acoustic image classification;

[0103] It should be noted that: this embodiment uses sonar technology, artificial intelligence algorithms and edge computing devices to provide strong technical support for underwater detection; according to the depth and complexity of the underwater environment, the sonar frequency and beam mode of the sonar device are adaptively selected to ensure the best detection range and resolution. For the above algorithm model, if there are other similar artificial intelligence algorithms, as long as they can achieve similar effects, they can actually be replaced with each other; real-time data processing and analysis based on edge computing devices will promote the development and application of edge computing technology, improve the performance and efficiency of edge intelligent devices, and promote the innovation of underwater monitoring technology, providing more possibilities for scientific research and engineering applications in the marine field.

[0104] The edge computing device is an AI computer designed for edge computing; in this embodiment, the edge computing device uses an NVIDIA Jetson nano development board. The NVIDIA Jetson nano development board includes an NVIDIA GPU; the NVIDIA GPU includes a system control center for real-time acquisition of acoustic image data and an algorithm function module for real-time processing of acoustic image data:

[0105] As an edge computing device, the Jetson Nano development board obtains the storage module of acoustic image information in the system control center for processing, which reduces the need to transmit data to the cloud and reduces latency. Therefore, the corresponding storage module has sufficient storage space to save temporary data and model parameters, which is very helpful for continuous data processing and model training. In addition, the Jetson Nano development board is used for long-term operation in an underwater environment, so it needs to operate at low power consumption and have an effective heat dissipation mechanism, so that when it is integrated into underwater detection equipment, it will not have much impact on the size and weight of the equipment.

[0106] Specifically, the AI ​​models include the SRGAN algorithm model, the AUGGAN algorithm model, the U2Net algorithm model and the VGG algorithm, among which: acoustic image enhancement is performed based on the SRGAN algorithm model, and acoustic image data augmentation is performed based on the AUGGAN algorithm model to increase data diversity and prevent overfitting; the U2Net algorithm model is used for image segmentation, and the VGG algorithm is used for image classification.

[0107] The SRGAN algorithm model mainly uses adversarial loss and perceptual loss, which helps to maintain the texture details of the image to generate super-resolution images that are more in line with human visual perception.

[0108] The AUGGAN algorithm model generates sufficiently realistic data through the generator to deceive the discriminator. At the same time, the discriminator is responsible for distinguishing between real data and fake data generated by the generator. By alternately training the generator and the discriminator, AUGGAN is able to generate diverse images, thereby increasing the diversity of the data and preventing overfitting.

[0109] More specifically, the processing logic of acoustic image classification is:

[0110] The acoustic image data is resized to an image size of 224x224 pixels;

[0111] The acoustic image data is processed by the SRGAN algorithm model and the AUGGAN algorithm model to update and expand the acoustic image data set; the acoustic image data set is divided into a training set, a validation set, and a test set;

[0112] Transfer learning is adopted to load a pre-trained VGG model on the acoustic image dataset. The VGG model includes: freezing the convolution layer, so that the convolution layer remains frozen and does not participate in training, using the convolution layer as a feature extractor to extract features, and the pooling layer further compresses the size of the feature map during the feature extraction process while maintaining important feature information; adding a custom fully connected layer on top of the convolution layer of the VGG model, and updating the weights in the fully connected layer.

[0113] Among them: the training set data is passed through the convolution layer of the VGG model to extract features, and the extracted features are sent to the custom fully connected layer, and the weights are updated in the fully connected layer.

[0114] Use the validation set to evaluate the performance of the model and adjust the hyperparameters.

[0115] The test set is used for final performance evaluation.

[0116] It should be noted that the VGG algorithm is used to classify the acoustic image, wherein the VGG network is mainly composed of multiple convolutional layers, fully connected layers and pooling layers, and multiple 3x3 convolutional kernels are stacked to replace large-scale convolutional kernels, so that the 3x3 filter can provide more spatial coverage with the same number of parameters. By increasing the depth of the network, VGG is able to learn richer feature representations. Continuous 3x3 convolutional layers can effectively capture more local information and integrate it into global features. Due to the relatively simple structure of VGG, it is easier to train and has strong generalization ability. This makes it have shown superior performance in various computer vision tasks.

[0117] It is further explained that the steps for updating weights in the fully connected layer are:

[0118] Determine the structural composition of the VGG model, including convolutional layers for extracting local features of the image and fully connected layers for integrating the features extracted by all previous layers for classification and regression;

[0119] Calculate the error between the output of the fully connected layer and the actual label based on the loss function, and propagate the error back to the fully connected layer; count the contribution of each weight to the output error;

[0120] Specifically, the error signal will be distributed to different neurons according to the size of the weight. The larger the weight, the greater the impact on the error; conversely, the smaller the weight, the smaller the impact on the error.

[0121] Based on the error obtained through back propagation, the weights of the fully connected layer are updated using an optimization algorithm;

[0122] Specifically, the optimization algorithm includes the gradient descent method, and the goal of weight update is to reduce the prediction error and thus improve the prediction accuracy of the model.

[0123] Multiple iterations are performed over the acoustic image dataset to adjust the weights and model parameters.

[0124] Specifically, based on the training set, the update weights are determined to reduce prediction errors and improve the prediction accuracy of the model; the validation set is then used to evaluate the performance of the model and adjust the model parameters; finally, the test set is used for the final performance evaluation.

[0125] Example 3

[0126] For the parts not described in detail in this embodiment, please refer to the description of Embodiment 1. This embodiment provides an electronic device including: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0127] The processor executes the above-mentioned edge computing underwater acoustic image classification method by calling the computer program stored in the memory.

[0128] This embodiment also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the above-mentioned edge computing underwater acoustic image classification method.

[0129] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0130] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An underwater acoustic image classification method based on edge computing, It is characterized in that Applied to edge computing devices, it includes the following steps: Step A1: Use sensors and sonar equipment to collect data on the underwater environment, and match the collected data to obtain acoustic image data; Step A2: Perform image processing on the acoustic image data based on the AI ​​model to obtain acoustic image classification.

2. According to the underwater acoustic image classification method of edge computing in claim 1, It is characterized in that The sensors include water quality sensors, pressure sensors, flow rate sensors and attitude sensors. The data of sensors and sonar equipment are connected to the system control center. The system control center is based on the access to external data for extended data collection. The system control center is integrated on the edge computing device.

3. The underwater acoustic image classification method based on edge computing according to claim 2, It is characterized in that Water quality data, pressure data, flow rate data and posture data corresponding to the current underwater detection equipment are obtained based on the sensor, and acoustic images are obtained based on the sonar frequency and beam mode of the sonar equipment. Acoustic image data is generated by feature matching of water quality data, pressure data, flow rate data and acoustic images.

4. The underwater acoustic image classification method of edge computing according to claim 3, It is characterized in that The acquisition logic of the acoustic image data is: Align the collected data of different sensors in time and space to ensure that all sensors collect data at the same timestamp and location information; SIFT feature matching technology is used to match the visual features of the acoustic image with the physical features of the sensor based on the timestamp and location information.

5. The underwater acoustic image classification method based on edge computing according to claim 4, It is characterized in that The specific steps of SIFT feature matching technology are as follows: Convolve the acoustic image using Gaussian kernels of different scales to construct a scale space; In scale space, local extreme points are detected by comparing images of adjacent scales; The detected extreme points are accurately located. By fitting a three-dimensional quadratic function, the position and scale of the key points can be determined, and the key points with low contrast and edge response points can be removed. Assign one or more directions to each key point to make it rotation invariant; by calculating the gradient direction histogram in the neighborhood of the key point, the main direction and auxiliary direction of the key point can be determined; Generate a feature descriptor based on the position, scale, and direction of the key point; the feature descriptor includes a 128-dimensional feature vector, which represents the gradient information in the neighborhood of the key point based on the 128-dimensional feature vector; The generated feature descriptors are used to match the key points between different acoustic images. The Euclidean distance between two feature descriptors is calculated and matching key point pairs are found according to preset rules.

6. The underwater acoustic image classification method of edge computing according to claim 5, It is characterized in that The AI ​​models include SRGAN algorithm model, AUGGAN algorithm model, U2Net algorithm model and VGG algorithm, among which: acoustic image enhancement is performed based on the SRGAN algorithm model, and acoustic image data augmentation is performed based on the AUGGAN algorithm model; the U2Net algorithm model performs image segmentation, and the VGG algorithm performs image classification.

7. The underwater acoustic image classification method of edge computing according to claim 6, It is characterized in that The processing logic of the acoustic image classification is: The acoustic image data is resized to an image size of 224x224 pixels; The acoustic image data is processed by the SRGAN algorithm model and the AUGGAN algorithm model to update and expand the acoustic image data set; the acoustic image data set is divided into a training set, a validation set, and a test set; Using transfer learning, a pre-trained VGG model is loaded on the acoustic image dataset, wherein the VGG model includes: freezing the convolution layer, so that the convolution layer remains frozen and does not participate in training, and using the convolution layer as a feature extractor to extract features; Add a custom fully connected layer on top of the convolutional layer of the VGG model and update the weights within the fully connected layer. Use the validation set to evaluate the performance of the model and adjust the hyperparameters. The test set is used for final performance evaluation.

8. The underwater acoustic image classification method of edge computing according to claim 7, It is characterized in that The steps for updating weights in a fully connected layer are: Determine the structural composition of the VGG model, including convolutional layers for extracting local features of the image and fully connected layers for integrating the features extracted by all previous layers for classification and regression; Calculate the error between the output of the fully connected layer and the actual label based on the loss function, and backpropagate the error to the fully connected layer; Count the contribution of each weight to the output error; Specifically, the error signal will be distributed to different neurons according to the size of the weight. The larger the weight, the greater the impact on the error; conversely, the smaller the weight, the smaller the impact on the error. Based on the error obtained through back propagation, the weights of the fully connected layer are updated using an optimization algorithm; Multiple iterations are performed over the acoustic image dataset to adjust the weights and model parameters.

9. An edge computing underwater acoustic image classification system, based on an edge computing underwater acoustic image classification method according to any one of claims 1 to 8, It is characterized in that It includes system control center and algorithm function modules, and each module is connected by wire or wireless; The system control center uses sensors and sonar equipment to collect data about the underwater environment and matches the collected data to obtain acoustic image data; The algorithm function module processes the acoustic image data based on the AI ​​model to obtain acoustic image classification.

10. An electronic device, include: A processor and a memory, wherein the memory stores a computer program that can be called by the processor; The processor executes the edge computing underwater acoustic image classification method described in any one of claims 1 to 8 by calling the computer program stored in the memory.

Citation Information

Patent Citations

  • Website error-reporting screenshot classification method based on feature fusion

    CN110414571A

  • AI underwater target detection and positioning method based on sonar image

    CN115496997A

  • Intelligent work reporting and quality inspection method and system based on computer vision technology

    CN118037091A

  • Detection, capture and tracking system for remote small target

    CN118154646A

  • Underwater target identification method based on multi-modal fusion

    CN118485908A