Underwater target detection method and system based on edge calculation, and related equipment

Through edge computing and adaptive image preprocessing, combined with hybrid ranging algorithm, the target detection accuracy and real-time problems in underwater environments are solved, and efficient underwater target detection is achieved, suitable for fields such as marine research and military reconnaissance.

CN119068325APending Publication Date: 2024-12-03SHANGHAI OCEAN UNIV

Patent Information

Application Number
CN202411197659.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The complex light refraction, scattering and suspended particles in the underwater environment affect image quality, resulting in a decrease in the accuracy of the target detection algorithm, high computing resources consumption, poor real-time performance, low binocular ranging positioning accuracy, unintuitive user interface, and complex operation.

Method used

Using edge computing technology, combining adaptive image preprocessing and hybrid ranging algorithms, the YOLOv5 model is loaded for real-time image processing and ranging through embedded AI development boards such as NVIDIA Jetson Nano and binocular cameras, adjust the preprocessing parameters and optimize the image. Quality and positioning accuracy provide an intuitive user interface.

Benefits of technology

It reduces dependence on high-end computing equipment, improves the accuracy and real-time nature of underwater target detection, reduces system costs, is suitable for fields such as marine research and military reconnaissance, and promotes the development of marine scientific research.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068325A_ABST
    Figure CN119068325A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of ocean intelligent analysis, and particularly discloses an underwater target detection method and system based on edge computing and related equipment.The method comprises the following steps that edge computing equipment is configured, and the edge computing equipment comprises an embedded AI development board and a binocular camera; an underwater target image is extracted based on a binocular camera, and the underwater target image is processed based on an embedded AI development board; obtaining an image processing model adaptive to the real-time change of the underwater target image based on model training and optimization; a YOLOv5 model is deployed on an NVIDIA Jetson Nano development board, an NVIDIA CUDA model is utilized to accelerate the reasoning process, an underwater target image is analyzed in real time, and preprocessing parameters are adjusted to adapt to different underwater environment conditions; binocular ranging is carried out on the underwater target image based on a binocular ranging algorithm, and the target position of the underwater target is obtained; visual display is carried out through a visual user interface.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of marine intelligent analysis technology, and more specifically, to an edge computing underwater target detection method, system and related equipment. Background Art

[0002] The underwater environment is complex, and factors such as light refraction, scattering, and suspended particles in the water can significantly affect image quality, resulting in reduced accuracy of target detection algorithms. Traditional target detection algorithms are prone to false detection and missed detection when processing low-quality images; there are also the following problems:

[0003] 1. When processing a large number of high-resolution video streams, the computing resources are consumed a lot and the real-time performance is poor.

[0004] 2. The computing power of embedded devices (such as NVIDIA Jetson Nano) is limited and it is difficult to process complex deep learning models.

[0005] 3. Binocular ranging technology is easily affected by light and suspended objects in underwater environments, resulting in reduced positioning accuracy. The existing binocular ranging algorithm is complex in calculation and has poor real-time performance.

[0006] 4. The visualization of the test results in the user interface is not intuitive enough and the operation is complicated, which makes it difficult for users to understand and use the test results, and is not conducive to users to quickly get started and use it.

[0007] In view of this, the present invention provides an edge computing underwater target detection method, system and related equipment to solve the above problems. Summary of the invention

[0008] In order to overcome the problems in the prior art, the present invention proposes an edge computing underwater target detection method, system and related equipment, which can cope with complex underwater environments in real time through the application of adaptive image preprocessing and hybrid ranging algorithm.

[0009] The present invention provides an underwater target detection method for edge computing, comprising the following steps:

[0010] Step S1: Loading and initializing the image processing model based on the edge computing device;

[0011] Step S2: Integrate and deploy the image processing model described in step S1 on the embedded AI development board, perform real-time edge computing of underwater target images based on the accelerated reasoning process, and adjust preprocessing parameters;

[0012] Step S3: performing binocular ranging on the underwater target image based on a binocular ranging algorithm to obtain the target position of the underwater target;

[0013] Step S4: Visually analyze and display the detection results based on the visual user interface.

[0014] As a preferred technical solution of the present invention, the edge computing device includes an embedded AI development board and a binocular camera; the underwater target image is extracted based on the binocular camera, and the underwater target image is simultaneously acquired from the same underwater target in different directions by two cameras, and the underwater target image includes a first target image and a second target image; the embedded AI development board is an embedded image processing device for loading and initializing an image processing model, and the underwater target image is processed by the image processing model.

[0015] As a preferred technical solution of the present invention, step S1 includes:

[0016] Step S11: obtaining an image processing model that is adaptive to real-time changes of underwater target images based on model training and optimization;

[0017] Step S12: Initializing the image processing model based on the underwater target images under different optical conditions, and performing image detection on the underwater target images based on the initialized image processing model.

[0018] As a preferred technical solution of the present invention, the embedded AI development board includes an NVIDIA Jetson Nano development board, the image processing model includes a YOLOv5 model, and the NVIDIA CUDA model is used to accelerate the reasoning process, analyze underwater target images in real time, and adjust preprocessing parameters to adapt to different underwater environmental conditions;

[0019] Based on the NVIDIA Jetson Nano development board, edge computing is performed on underwater target images to obtain light detection results. The shooting environment is characterized based on the light detection results. The brightness, contrast, and saturation of the underwater target images are adaptively adjusted based on the light detection results.

[0020] As a preferred technical solution of the present invention, the calculation logic of the binocular ranging algorithm is:

[0021] Step S31: adjusting the internal and external parameters of the binocular camera based on the calibration plate, wherein the internal parameters include the focal length, principal point coordinates and distortion coefficient corresponding to each camera; and the external parameters include the rotation matrix and translation vector corresponding to the underwater target image.

[0022] Step S32: Match the first target image and the second target image using a matching algorithm to obtain a depth value of the underwater target in space.

[0023] As a preferred technical solution of the present invention, the logic for obtaining the internal and external parameters of the binocular camera is:

[0024] Place the calibration plate in front of the binocular cameras and ensure that the calibration plate is within the field of view of both cameras;

[0025] Adjust the distance and angle between the camera and the calibration plate to capture multiple sets of images from different perspectives;

[0026] Harris corner point detection is used to extract the coordinates of the corner points on the calibration plate, and a binocular camera imaging model is established. The perspective projection corresponding to the binocular camera imaging model determines the internal parameters and the external parameters, and the calibrated internal parameters and external parameters are saved.

[0027] As a preferred technical solution of the present invention, the logic for obtaining the depth value of the underwater target in space is:

[0028] The binocular cameras are marked as a first camera and a second camera respectively, the first camera collects a first target image, and the second camera collects a second target image;

[0029] The underwater target image includes a first target image and a second target image, wherein the first target image and the second target image are target images of the same target simultaneously viewed from different perspectives by a binocular camera;

[0030] By comparing the first target image and the second target image to perform stereo matching calculation, the disparity values ​​of the same target at different viewing angles are determined, the depth value of the target in three-dimensional space is obtained, and the target position of the underwater target is characterized based on the depth value.

[0031] In a second aspect, the present invention proposes an underwater target detection system for edge computing, which implements the method described in the first aspect, including a model loading module, an edge computing integration module, a target positioning module, and a result visualization module. The modules are connected by wire or wirelessly, and the detection, positioning, and analysis of underwater targets are achieved through the collaborative work of the modules.

[0032] Model loading module, which loads and initializes the image processing model based on the edge computing device;

[0033] The edge computing integration module integrates and deploys the image processing model described in step S1 on the embedded AI development board, performs real-time edge computing of underwater target images based on the accelerated reasoning process, and adjusts preprocessing parameters;

[0034] The target positioning module performs binocular ranging on the underwater target image based on the binocular ranging algorithm to obtain the target position of the underwater target;

[0035] The result visualization module performs visual analysis and display of the test results based on a visual user interface.

[0036] In a third aspect, the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0037] The processor executes the above-mentioned edge computing underwater target detection method by calling the computer program stored in the memory.

[0038] In a fourth aspect, the present invention provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the above-mentioned edge computing method for underwater target detection.

[0039] The specific advantages of the present invention are as follows:

[0040] The present invention reduces the dependence on high-end computing equipment and reduces the overall system cost by using adaptive image preprocessing and hybrid ranging algorithms, edge computing and optimized deep learning models. It can be widely used for underwater detection and monitoring in the fields of marine research, environmental monitoring, military reconnaissance, etc., thereby helping researchers and environmentalists to effectively monitor marine life, protect the marine ecological environment, and promote the development of marine scientific research. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of the underwater target detection method of the present invention;

[0042] Figure 2 This is a diagram of the underwater target detection system architecture of the present invention;

[0043] Figure 3 This is the YOLOv5 model architecture diagram of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Example 1

[0046] See also Figure 1 , this embodiment provides an edge computing underwater target detection method, including the following steps:

[0047] Step S1: Loading and initializing the image processing model based on the edge computing device;

[0048] Specifically, the edge computing device includes an embedded AI development board and a binocular camera; the underwater target image is extracted based on the binocular camera, the embedded AI development board is an embedded image processing device that loads and initializes the image processing model, and the underwater target image is processed through the image processing model.

[0049] More specifically, the binocular camera includes two high-resolution underwater cameras equipped in the system, and the underwater target images are simultaneously acquired from the two cameras at different viewing angles for the same underwater target. The underwater target images include a first target image and a second target image, which provide a data basis for subsequent stereo matching and three-dimensional position calculation; the high resolution is a resolution of not less than 1080p, so as to achieve clear image capture;

[0050] The embedded AI development board includes an embedded image processing device for processing high-resolution video streams and data calculations based on a machine learning model. In this embodiment, the embedded AI development board takes the NVIDIA Jetson Nano development board as an example to process underwater target images. The NVIDIA Jetson Nano development board has powerful GPU computing capabilities, which is suitable for AI and deep learning applications to meet real-time processing requirements; the NVIDIA Jetson Nano development board is configured with 4GB of memory to ensure that there is enough running memory to process complex underwater target images; in addition, a 128GB microSD card is used as the system's storage medium to save the operating system, applications, and collected underwater target images, thereby smoothly processing high-resolution video streams and complex computing tasks.

[0051] It should be noted that the embedded AI development board can also be replaced by other hardware platforms, such as a widely used single-board computer, such as Raspberry Pi 4, which can be used with NCS2 (Neural Compute Stick 2) to achieve AI acceleration. Or the AI ​​development board launched by Google, such as Coral Dev Board, which has a built-in Edge TPU and is suitable for edge AI applications.

[0052] The step S1 comprises:

[0053] Step S11: obtaining an image processing model that is adaptive to real-time changes of underwater target images based on model training and optimization;

[0054] Specifically, the image processing models include the YOLOv5 model, the Faster R-CNN model and the SSD model for implementing the target detection algorithm; different target detection algorithm models have different advantages and disadvantages, and different image processing models or model combinations are used based on application scenarios, real-time requirements, computing resource limitations and corresponding target features of underwater target images; wherein:

[0055] The YOLOv5 model has a very fast detection speed while maintaining high accuracy, making it suitable for real-time detection tasks. Compared with two-stage detection algorithms such as Faster R-CNN, the YOLOv5 model uses a single-stage detection, which reduces the amount of calculation and inference time. In addition, the YOLOv5 model has a relatively simple structure and is easy to deploy on embedded devices.

[0056] However, in complex underwater environments, due to light refraction and scattering, the YOLOv5 model requires additional image preprocessing to improve detection accuracy, and more training data is needed to improve the model's generalization ability.

[0057] The Faster R-CNN model provides higher accuracy in target detection tasks and uses a region proposal network to generate candidate target regions to improve the accuracy of target positioning detection.

[0058] However, compared with the YOLOv5 model, the Faster R-CNN model has a slower calculation speed, which affects the real-time performance. In addition, it has a higher demand for computing resources and is not suitable for resource-constrained embedded devices.

[0059] The SSD model provides a good balance between speed and accuracy and is suitable for applications that require fast and relatively accurate detection. However, the SSD model has high requirements for image quality and performs poorly when detecting small targets. Compared with the other two models, it requires more optimization to run effectively on embedded devices.

[0060] In summary, the refraction and scattering of light in the underwater environment will affect the clarity of the collected underwater target images, thereby affecting the detection accuracy. Therefore, an adaptive image processing model is needed to improve the detection capability in the underwater environment, further expand the data set collected and annotated in the underwater environment, and train based on a large amount of image data to improve the generalization ability of the model. Therefore, the YOLOv5 model is preferred in this embodiment, which can achieve higher detection accuracy when processing optical images in complex underwater environments.

[0061] Step S12: initializing the image processing model based on the underwater target images under different optical conditions, and performing image detection on the underwater target images based on the initialized image processing model;

[0062] Specifically, the processing logic for initializing the image processing model is:

[0063] Data collection and preprocessing: Collect the underwater target images in step S1 under different optical conditions, and preprocess the underwater target images to obtain an enhanced processing data set. The preprocessing includes rotation, flipping, brightness adjustment, and noise addition.

[0064] It should be noted that the enhanced processed dataset not only enhances the image contrast through preprocessing to help the model better distinguish between the target and the background; it can also expand the number of datasets. Each preprocessed underwater target image is used as a new underwater target image for image analysis, which can effectively cope with changes in different optical conditions and improve the robustness of the model in various environments, thereby reducing false detections and missed detections, and further improving the accuracy of the model in extracting image features.

[0065] Image processing model initialization settings: The image processing model is trained based on the enhanced processing dataset. The cross entropy loss function and Adam optimizer are used in the training process to determine the initialization settings of the image processing model: learning rate is 0.001, batch size is 16, pruning ratio is set to 30%, and quantization is 8-bit.

[0066] Specifically, the processing logic for image detection of underwater target images is:

[0067] Dataset segmentation: The collected underwater image dataset is segmented into three parts: training set, validation set, and test set.

[0068] It should be noted that the training set is used for model training, the validation set is used for performance evaluation and model adjustment during the training process, and the test set is used for performance evaluation of the final model.

[0069] Target calibration: The location and category of the detected target are marked in the collected underwater image data. This data will be used to train and evaluate the YOLOv5 model.

[0070] Constructing the YOLOv5 model: The underwater target detection model is constructed based on the YOLOv5 model. Figure 3 The underwater target detection model shown includes a feature extraction network Backbone module, a feature fusion network Neck module and a feature recognition Head.

[0071] The Backbone module contains the Focus structure and the CSP structure, wherein the Focus structure is a convolutional neural network layer for feature extraction; the CSP structure is used to build the Backbone, and the structure contains multiple CSP modules, each module divides the input feature map into two parts, one part is processed by the sub-network, and the other part is directly processed by the next layer. Then the two parts of the feature map are spliced ​​together as the input of the next layer; each CSP module in the Backbone contains multiple CBL (convolution, BatchNorm, Leaky ReLU) layers, and may perform feature extraction and enhancement through different downsampling methods.

[0072] The Neck module includes the FPN (Feature Pyramid Network) structure and the PAN (Path Aggregation Network) structure. Specifically:

[0073] The FPN structure is mainly used to fuse the feature maps extracted by Backbone at different scales to enhance the model's ability to detect multi-scale targets. FPN upsamples the deep feature maps through upsampling operations and splices them with shallow feature maps, allowing the model to better identify small targets.

[0074] The PAN structure further enhances feature fusion and improves information flow through a bottom-up path. It fuses and transfers feature maps of different levels through additional CBL layers and upsampling operations, ensuring that features of all levels can be fully utilized before the Head module.

[0075] The Head module includes three independent detection heads, each used to detect objects of different scales. The three detection heads process the feature maps of three different sizes output from the Neck module (e.g., detection of large, medium, and small objects), and use convolutional layers for final classification and bounding box regression. Each detection head detects objects of a specific scale through multiple CSP modules and CBL layers, and finally outputs the predicted category, position, and confidence.

[0076] Training model: The YOLOv5 model is trained using the preprocessed training set. During the training process, the cross entropy loss function and Adam optimizer are used, as well as the predetermined initialization settings, including a learning rate of 0.001, a batch size of 16, a pruning ratio of 30%, and quantization of 8-bit.

[0077] Validation model: Use the validation set to validate the model and evaluate its performance. If the performance of the model is not satisfactory, you can adjust the model and retrain it.

[0078] Test model: After the model is trained and validated, the final model is evaluated using the reserved test set. The test set can accurately measure the performance of the model on unseen data to ensure its generalization ability and practical application effect.

[0079] Step S2: Integrate and deploy the image processing model described in step S1 on the embedded AI development board, perform real-time edge computing of underwater target images based on the accelerated reasoning process, and adjust preprocessing parameters;

[0080] It should be noted that edge computing is a distributed computing architecture that transfers data processing and analysis tasks from the cloud to the edge of the network, that is, close to the data source. This computing model can significantly reduce data transmission latency, increase processing speed, and optimize bandwidth usage.

[0081] On the edge computing device, i.e., the AI ​​development board, the underwater target image is calculated in real time at the edge based on the accelerated reasoning process and the preprocessing parameters are adjusted to adapt to different underwater environmental conditions;

[0082] Specifically, the implementation logic of the model deployed on edge computing devices is as follows:

[0083] Taking the embedded AI development board as NVIDIA Jetson Nano development board and the image processing model as YOLOv5 model for illustration, the NVIDIA CUDA model is used to accelerate the reasoning process, analyze underwater target images in real time and adjust preprocessing parameters to adapt to different underwater environmental conditions; Among them: NVIDIA CUDA is a parallel computing platform and programming model, which uses graphics processing unit (GPU) for edge computing to accelerate the reasoning process of YOLOv5 model, and the specific calculation and image processing are performed by YOLOv5 model for image processing, so that the YOLOv5 model can run at a higher speed and efficiency when performing the task of predicting underwater target detection; Actual tests show that on 1080p video stream, the system can stably run at a frame rate of 30FPS to achieve efficient real-time processing.

[0084] More specifically, the NVIDIA Jetson Nano development board is used as the edge computing hardware platform; its powerful GPU computing capabilities and hardware accelerated reasoning functions (such as NVIDIA CUDA cores) are used to achieve efficient edge computing processing. Through model pruning and quantization technology, the computational complexity is reduced, the processing speed is increased, and real-time target detection is achieved.

[0085] Convert the trained YOLOv5 model into a format that can be recognized by TensorRT, deploy the YOLOv5 model to the NVIDIA Jetson Nano development board, and use NVIDIA CUDA accelerated reasoning technology to perform parallel computing on the GPU to improve the reasoning speed of the model.

[0086] Use NVIDIA TensorRT tools to optimize the YOLOv5 model; TensorRT can quantize the model weights and activations, reduce the model size, and improve reasoning efficiency. At the same time, TensorRT can also optimize the operation graph, including operation fusion and tensor fusion, to reduce the amount of calculation and memory access during the reasoning process, achieve real-time target detection, and ensure that the YOLOv5 model quickly outputs light detection results after receiving input data on the edge computing device to meet real-time requirements.

[0087] Optimize the resource allocation of NVIDIA Jetson Nano, assign compute-intensive tasks to the GPU, assign I / O-intensive tasks to the CPU, and ensure that GPU and CPU resources are effectively utilized through reasonable task scheduling strategies.

[0088] Perform edge computing on underwater target images based on the NVIDIA Jetson Nano development board to obtain light detection results, characterize the shooting environment based on the light detection results, and adaptively adjust the brightness, contrast, and saturation of underwater target images based on the light detection results;

[0089] The light detection results include low brightness environment, high contrast environment, and color temperature deviation environment. The adaptive adjustment logic is:

[0090] In water, the absorption and scattering of light will cause color temperature deviation of underwater images. Insufficient light and turbid water will also affect the brightness and contrast of the image. In order to meet these challenges, a series of image processing technologies are needed to optimize image quality and ensure the accuracy and robustness of subsequent processing such as feature extraction and target detection. These technologies include histogram equalization, contrast adjustment, white balance correction, denoising, sharpening, and HDR technology to optimize image quality and ensure the accuracy and robustness of subsequent processing such as feature extraction and target detection. At the same time, considering the needs of image content and processing resources, the image resolution is dynamically adjusted to achieve the best processing effect.

[0091] Step S3: Perform binocular ranging on the underwater target image based on a binocular ranging algorithm to obtain the target position of the underwater target.

[0092] Specifically, the calculation logic of the binocular ranging algorithm is:

[0093] Step S31: adjusting the internal parameters and external parameters of the binocular cameras based on the calibration plate, wherein the internal parameters include the focal length, principal point coordinates and distortion coefficient corresponding to each camera; and the external parameters include the rotation matrix and translation vector corresponding to the underwater target image;

[0094] Specifically, the logic for obtaining the internal and external parameters of the binocular camera is:

[0095] Place the calibration plate in front of the binocular cameras and ensure that the calibration plate is within the field of view of both cameras;

[0096] Adjust the distance and angle between the camera and the calibration plate to capture multiple sets of images from different perspectives;

[0097] Harris corner point detection is used to extract the coordinates of the corner points on the calibration plate, and a binocular camera imaging model is established. The perspective projection corresponding to the binocular camera imaging model determines the internal parameters and the external parameters, and the calibrated internal parameters and external parameters are saved.

[0098] Step S32: Matching the first target image and the second target image by a matching algorithm, obtaining a depth value of the underwater target in space, and characterizing a target position of the underwater target based on the depth value;

[0099] The logic for obtaining the depth value of the underwater target in space is as follows:

[0100] The binocular cameras are marked as a first camera and a second camera respectively, the first camera collects a first target image, and the second camera collects a second target image;

[0101] The underwater target image includes a first target image and a second target image, wherein the first target image and the second target image are target images of the same target simultaneously viewed from different perspectives by a binocular camera;

[0102] By comparing the first target image and the second target image to perform stereo matching calculation, the spatial position difference of the same target under different viewing angles is determined, and the depth value of the target in three-dimensional space is obtained.

[0103] More specifically, a matching cost table is created based on the first feature point in the first target image, and the matching cost between each possible position and the first feature point is calculated by sliding a window in the corresponding search area in the second target image; the position with the smallest matching cost is the best matching point of the second feature point, which represents the corresponding position of the first feature point in the second target image.

[0104] The disparity value is calculated based on the second matching point and the first matching point, that is, the position difference in the horizontal direction between the corresponding points in the first image and the second image. Then, the depth value of the current underwater target is calculated through a conversion formula using the obtained internal and external parameters to achieve accurate positioning of the underwater target.

[0105] The specific conversion formulas include:

[0106]

[0107] Where: depth is the depth value of the underwater target, f is the focal length, i.e. the pixel focal length, b is the baseline length, d is the parallax value, c is the distance between the pixels and the horizon. xr With c xl are the column coordinates of the two camera principal points.

[0108] It should be noted that the hybrid ranging algorithm combines deep learning and traditional geometric measurement methods, and can maintain high positioning accuracy in complex underwater environments; in actual tests, the positioning error was reduced by an average of 15%, showing high stability and reliability in diverse underwater environments. The optimized binocular ranging algorithm significantly improves computational efficiency, enabling the system to process images captured by the camera in real time and achieve fast and accurate target positioning.

[0109] In addition, binocular ranging can also replace ranging with laser radar and sonar to achieve target positioning; among them:

[0110] LiDAR: uses laser ranging technology to achieve high-precision target positioning;

[0111] Sonar: Detects and measures the distance to underwater objects by using the reflection of sound waves.

[0112] Step S4: Visually analyzing and displaying the detection results based on a visual user interface;

[0113] Specifically: Rich visualization functions: The system provides a detailed display of detection results, including target bounding boxes, target category annotations, and confidence information. Users can intuitively understand the detection results. The simple and intuitive user interface makes operation more convenient, allowing users to quickly get started. User feedback displays improve operational efficiency and reduce learning costs.

[0114] It also includes data management and analysis, providing import and export functions and statistical analysis functions for test results; the test results are imported and exported by supporting CSV and JSON formats; users can view historical data and trend charts, and perform statistical analysis on the test results.

[0115] Example 2: Figure 2 As shown, for the parts not described in detail in this embodiment, please refer to the description of Example 1. This embodiment provides an edge computing underwater target detection system, including a model loading module, an edge computing integration module, a target positioning module and a result visualization module. The modules are connected to each other through wired or wireless connections, and the collaborative work of the modules realizes the detection, positioning and analysis of underwater targets.

[0116] A model loading module, which loads and initializes the image processing model based on the edge computing device, and the model loading module includes a model selection unit and an initialization model unit, including;

[0117] A model selection unit, which obtains an image processing model that is adaptive to real-time changes of underwater target images based on model training and optimization, wherein the image processing model is preferably a YOLOv5 model;

[0118] Specifically, the image processing models include the YOLOv5 model, the Faster R-CNN model and the SSD model for implementing the target detection algorithm; different target detection algorithm models have different advantages and disadvantages, and different image processing models or model combinations are used based on application scenarios, real-time requirements, computing resource limitations and corresponding target features of underwater target images; wherein:

[0119] The YOLOv5 model has a very fast detection speed while maintaining high accuracy, making it suitable for real-time detection tasks. Compared with two-stage detection algorithms such as Faster R-CNN, the YOLOv5 model uses a single-stage detection, which reduces the amount of calculation and inference time. In addition, the YOLOv5 model has a relatively simple structure and is easy to deploy on embedded devices.

[0120] However, in complex underwater environments, due to light refraction and scattering, the YOLOv5 model requires additional image preprocessing to improve detection accuracy, and more training data is needed to improve the model's generalization ability.

[0121] The Faster R-CNN model provides higher accuracy in target detection tasks and uses a region proposal network to generate candidate target regions to improve the accuracy of target positioning detection.

[0122] However, compared with the YOLOv5 model, the Faster R-CNN model has a slower calculation speed, which affects the real-time performance. In addition, it has a higher demand for computing resources and is not suitable for resource-constrained embedded devices.

[0123] The SSD model provides a good balance between speed and accuracy and is suitable for applications that require fast and relatively accurate detection. However, the SSD model has high requirements for image quality and performs poorly when detecting small targets. Compared with the other two models, it requires more optimization to run effectively on embedded devices.

[0124] In summary, the refraction and scattering of light in the underwater environment will affect the clarity of the collected underwater target images, thereby affecting the detection accuracy. Therefore, an adaptive image processing model is needed to improve the detection capability in the underwater environment, further expand the data set collected and annotated in the underwater environment, and train based on a large amount of image data to improve the generalization ability of the model. Therefore, the YOLOv5 model is preferred in this embodiment, which can achieve higher detection accuracy when processing optical images in complex underwater environments.

[0125] The model initialization unit initializes the image processing model based on underwater target images under different optical conditions, and performs image detection on the underwater target images based on the initialized image processing model.

[0126] Specifically, the processing logic for initializing the image processing model is:

[0127] Data collection and preprocessing: Collect the underwater target images in step S1 under different optical conditions, and preprocess the underwater target images to obtain an enhanced processing data set. The preprocessing includes rotation, flipping, brightness adjustment, and noise addition.

[0128] Image processing model initialization settings: The image processing model is trained based on the enhanced processing dataset. The cross entropy loss function and Adam optimizer are used in the training process to determine the initialization settings of the image processing model: learning rate is 0.001, batch size is 16, pruning ratio is set to 30%, and quantization is 8-bit.

[0129] The edge computing integration module integrates and deploys the image processing model described in step S1 on the embedded AI development board, performs real-time edge computing of underwater target images based on the accelerated reasoning process, and adjusts preprocessing parameters;

[0130] The logic for obtaining the preprocessing parameters:

[0131] The embedded AI development board is NVIDIA Jetson Nano development board and the image processing model is YOLOv5 model. The NVIDIA CUDA model is used to accelerate the reasoning process, analyze underwater target images in real time and adjust preprocessing parameters to adapt to different underwater environmental conditions;

[0132] Based on the NVIDIA Jetson Nano development board, edge computing is performed on underwater target images to obtain light detection results. The shooting environment is characterized based on the light detection results. The brightness, contrast, and saturation of the underwater target images are adaptively adjusted based on the light detection results.

[0133] The target positioning module performs binocular ranging on the underwater target image based on the binocular ranging algorithm to obtain the target position of the underwater target;

[0134] Specifically, the calculation logic of the binocular ranging algorithm is:

[0135] Step S31: adjusting the internal parameters and external parameters of the binocular cameras based on the calibration plate, wherein the internal parameters include the focal length, principal point coordinates and distortion coefficient corresponding to each camera; and the external parameters include the rotation matrix and translation vector corresponding to the underwater target image;

[0136] Specifically, the logic for obtaining the internal and external parameters of the binocular camera is:

[0137] Place the calibration plate in front of the binocular cameras and ensure that the calibration plate is within the field of view of both cameras;

[0138] Adjust the distance and angle between the camera and the calibration plate to capture multiple sets of images from different perspectives;

[0139] Harris corner point detection is used to extract the coordinates of the corner points on the calibration plate, and a binocular camera imaging model is established. The perspective projection corresponding to the binocular camera imaging model determines the internal parameters and the external parameters, and the calibrated internal parameters and external parameters are saved.

[0140] Step S32: performing bidirectional matching on the first target image and the second target image by using a matching algorithm to obtain a depth value of the underwater target in space;

[0141] The logic for obtaining the depth value of the underwater target in space is as follows:

[0142] The binocular cameras are marked as a first camera and a second camera respectively, the first camera collects a first target image, and the second camera collects a second target image;

[0143] The underwater target image includes a first target image and a second target image, wherein the first target image and the second target image are target images of the same target simultaneously viewed from different perspectives by a binocular camera;

[0144] By comparing the first target image and the second target image to perform stereo matching calculation, the disparity values ​​of the same target at different viewing angles are determined, and the depth value of the target in three-dimensional space is obtained.

[0145] The result visualization module performs visual analysis and display of the test results based on a visual user interface.

[0146] Embodiment 3: For the parts not described in detail in this embodiment, please refer to the description of Embodiment 1. This embodiment provides an electronic device, including: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0147] The processor executes the above-mentioned edge computing underwater target detection method by calling the computer program stored in the memory.

[0148] Embodiment 4: This embodiment provides a computer-readable storage medium storing instructions. When the instructions are executed on a computer, the computer executes the above-mentioned edge computing method for underwater target detection.

[0149] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0150] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An underwater target detection method based on edge computing, characterized in that: The following steps are involved: Step S1: Loading and initializing the image processing model based on the edge computing device; Step S2: Integrate and deploy the image processing model described in step S1 on the embedded AI development board, perform real-time edge computing of underwater target images based on the accelerated reasoning process, and adjust preprocessing parameters; Step S3: performing binocular ranging on the underwater target image based on a binocular ranging algorithm to obtain the target position of the underwater target; Step S4: Visually analyze and display the detection results based on the visual user interface.

2. The underwater target detection method of edge computing according to claim 1, characterized in that: The edge computing device includes an embedded AI development board and a binocular camera; the underwater target image is extracted based on the binocular camera, and the underwater target image is simultaneously acquired from the same underwater target at different perspectives by two cameras, and the underwater target image includes a first target image and a second target image; the embedded AI development board is an embedded image processing device for loading and initializing an image processing model, and the underwater target image is processed by the image processing model.

3. The underwater target detection method of edge computing according to claim 2, characterized in that: The step S1 comprises: Step S11: obtaining an image processing model that is adaptive to real-time changes of underwater target images based on model training and optimization; Step S12: Initializing the image processing model based on the underwater target images under different optical conditions, and performing image detection on the underwater target images based on the initialized image processing model.

4. The underwater target detection method of edge computing according to claim 2, characterized in that: The embedded AI development board includes an NVIDIA Jetson Nano development board, and the image processing model includes a YOLOv5 model, which uses an NVIDIA CUDA model to accelerate the reasoning process, analyze underwater target images in real time, and adjust preprocessing parameters to adapt to different underwater environmental conditions; Based on the NVIDIA Jetson Nano development board, edge computing is performed on underwater target images to obtain light detection results. The shooting environment is characterized based on the light detection results. The brightness, contrast, and saturation of the underwater target images are adaptively adjusted based on the light detection results.

5. The underwater target detection method of edge computing according to claim 4, characterized in that: The calculation logic of the binocular ranging algorithm is: Step S31: adjusting the internal parameters and external parameters of the binocular cameras based on the calibration plate, wherein the internal parameters include the focal length, principal point coordinates and distortion coefficient corresponding to each camera; and the external parameters include the rotation matrix and translation vector corresponding to the underwater target image; Step S32: Match the first target image and the second target image through a matching algorithm to obtain a depth value of the underwater target in space, and characterize the target position of the underwater target based on the depth value.

6. The underwater target detection method of edge computing according to claim 5, characterized in that: The logic for obtaining the internal and external parameters of the binocular camera is as follows: Place the calibration plate in front of the binocular cameras and ensure that the calibration plate is within the field of view of both cameras; Adjust the distance and angle between the camera and the calibration plate to capture multiple sets of images from different perspectives; Harris corner point detection is used to extract the coordinates of the corner points on the calibration plate, and a binocular camera imaging model is established. The perspective projection corresponding to the binocular camera imaging model determines the internal parameters and the external parameters, and the calibrated internal parameters and external parameters are saved.

7. The underwater target detection method of edge computing according to claim 6, characterized in that: The logic for obtaining the depth value of the underwater target in space is as follows: The binocular cameras are marked as a first camera and a second camera respectively, the first camera collects a first target image, and the second camera collects a second target image; The underwater target image includes a first target image and a second target image, wherein the first target image and the second target image are target images of the same target simultaneously viewed from different perspectives by a binocular camera; By comparing the first target image and the second target image to perform stereo matching calculation, the disparity values ​​of the same target at different viewing angles are determined, and the depth value of the target in three-dimensional space is obtained.

8. An underwater target detection system for edge computing, an underwater target detection method for edge computing according to any one of claims 1 to 7, characterized in that: It includes a model loading module, an edge computing integration module, a target positioning module, and a result visualization module. Each module is connected via wired or wireless connections. Through the collaborative work of each module, the detection, positioning, and analysis of underwater targets are realized. Model loading module, which loads and initializes the image processing model based on the edge computing device; The edge computing integration module integrates and deploys the image processing model described in step S1 on the embedded AI development board, performs real-time edge computing of underwater target images based on the accelerated reasoning process, and adjusts preprocessing parameters; The target positioning module performs binocular ranging on the underwater target image based on the binocular ranging algorithm to obtain the target position of the underwater target; The result visualization module performs visual analysis and display of the test results based on a visual user interface.

9. An electronic device comprising: include: A processor and a memory, wherein the memory stores a computer program that can be called by the processor; The processor executes the edge computing underwater target detection method described in any one of claims 1 to 7 by calling the computer program stored in the memory.

10. A computer-readable storage medium storing instructions, which, when executed on a computer, enables the computer to execute the edge computing underwater target detection method described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Submarine cable mechanical arm autonomous obstacle avoidance grabbing system and grabbing method

    CN116214532A

  • Image processing method and system for industrial environment

    CN117710267A

  • Autonomous underwater operation robot based on binocular vision and underwater operation method

    CN118124760A

  • Underwater structure ranging system based on computer vision

    CN118424232A

  • Image processing apparatus, image processing system, image processing method, and program

    JP2024056504A

Cited By

  • Pipeline shallow profile image automatic identification system based on deep learning

    CN119992306A

  • An automatic recognition system for pipeline shallow section images based on deep learning

    CN119992306B

  • Binocular three-dimensional target detection method and system, readable storage medium and program product

    CN120852757A

  • Aircraft height measurement method and device based on binocular vision

    CN121330043A