Fishery resource assessment method, device, system and electronic equipment

By combining preprocessing of side-sweep sonar images and biomass evaluation models, automated fishery resource evaluation is achieved, solving the problems of low efficiency and poor accuracy in the existing technology, and efficient and accurate fishery resource distribution information is obtained.

CN116466353BActive Publication Date: 2024-11-22CHINA AGRI UNIV
View PDF 18 Cites 0 Cited by

Patent Information

Application Number
CN202310259316.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-09
Publication Date
2024-11-22
Estimated Expiration
2043-03-09

AI Technical Summary

Technical Problem

In the prior art, fishery resource biomass assessment based on side-swept sonar images relies on manual counting, which is inefficient and difficult to obtain accurate fishery resource distribution information.

Method used

By preprocessing the side-swept sonar image, including oblique distance correction and image enhancement processing, and automated evaluation combined with biomass evaluation model, fishery resource distribution maps are generated.

Benefits of technology

It improves the efficiency and accuracy of fishery resource assessment and can intuitively understand the distribution of fishery resource in waters.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116466353B_ABST
    Figure CN116466353B_ABST
Patent Text Reader

Abstract

The present invention provides a method, device, system and electronic equipment for assessing fishery resources, and belongs to the technical field of marine resource assessment. The method comprises: obtaining multiple echo data of a target water area collected by a side-scan sonar, and generating a side-scan sonar image; preprocessing each side-scan sonar image, including slant range correction processing and / or side-scan sonar image enhancement processing; performing biomass assessment on each preprocessed side-scan sonar image, and obtaining fishery resource distribution images of the same size as each side-scan sonar image; determining the geographical coordinates of each pixel point on the fishery resource distribution image according to the geographical coordinates corresponding to each pixel point on the side-scan sonar image, and determining a fishery resource distribution map; splicing all fishery resource distribution maps corresponding to the target water area, and obtaining a target fishery resource distribution map of the target water area. The method has a high accuracy rate in assessing fishery resources, and can intuitively and clearly understand the distribution of fishery resources in the water area.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of marine resource assessment, and in particular to a fishery resource assessment method, device, system and electronic equipment. Background Art

[0002] Fish and shrimp and other aquatic products provide humans with abundant high-quality protein. However, with the impact of overfishing, environmental changes and human behavior, fishery resources are declining year by year. In order to ensure the sustainable development of fisheries, it is crucial to immediately and accurately understand the biomass of fishery resources. Biomass assessment can quickly obtain the status of fishery resources in target waters and guide subsequent fishing decisions and ecological protection measures.

[0003] At present, the biomass assessment of fishery resources mostly adopts traditional sampling methods, including trawling sampling, electrofishing sampling, mark-recapture method, etc., combined with mathematical models for assessment. Traditional methods are usually time-consuming and labor-intensive, easy to cause damage to fish, and the estimation results are biased. In recent years, in order to improve the efficiency of assessment, a variety of non-destructive assessment methods have emerged, including machine vision and hydroacoustic methods. Machine vision uses optical cameras and artificial intelligence algorithms to assess the biomass of monitored fish schools, but it is limited by the short propagation distance of light underwater and the influence of visibility. It is only suitable for small-scale, clear underwater resource assessment. Hydroacoustics is not affected by visibility, and the propagation distance of sound waves is much greater than that of light, so it is used for fishery resource assessment in complex underwater environments.

[0004] In related technologies, side-scan sonar uses sound waves to detect targets and can scan a large area, providing a way to assess the biomass of fishery resources. However, due to the low signal-to-noise ratio and blurred target outlines of side-scan sonar images, biomass assessment based on side-scan sonar images still relies on manual counting by professionals with experience in interpreting acoustic images. This technical method is time-consuming and labor-intensive, and it is difficult to obtain accurate information on the distribution of fishery resources. Summary of the invention

[0005] The present invention provides a fishery resource assessment method, device, system and electronic equipment, which are used to solve the defects of low assessment efficiency and difficulty in obtaining accurate fishery resource distribution information caused by manually counting fishery resource biomass according to side-scan sonar images in the prior art, and realize a method for automatically performing fishery resource assessment after preprocessing side-scan sonar images. The method has high assessment efficiency and can obtain a relatively accurate fishery distribution map, so as to intuitively and clearly understand the distribution of fishery resources in the water area.

[0006] The present invention provides a fishery resource assessment method, comprising:

[0007] Acquire multiple echo data of the target water area collected by the side-scan sonar, and generate a side-scan sonar image according to each of the echo data;

[0008] Preprocessing each of the side-scan sonar images, wherein the preprocessing includes slant range correction processing and / or side-scan sonar image enhancement processing;

[0009] Based on the biomass assessment model, the biomass of each of the preprocessed side-scan sonar images is assessed to obtain fishery resource distribution images of the same size as each of the side-scan sonar images;

[0010] Determine the geographical coordinates of each pixel point on the fishery resource distribution image according to the geographical coordinates corresponding to each pixel point on the side scan sonar image, and determine the corresponding fishery resource distribution map based on the geographical coordinates of each pixel point on the fishery resource distribution image;

[0011] All fishery resource distribution maps corresponding to the target waters are spliced ​​together to obtain a target fishery resource distribution map of the target waters.

[0012] According to a fishery resource assessment method provided by the present invention, the step of acquiring a plurality of echo data of a target water area collected by a side-scan sonar and generating a side-scan sonar image according to each of the echo data comprises:

[0013] Acquiring in real time the echo data received by the side-scan sonar after it emits sound waves in the target waters;

[0014] The echo data are quantified to obtain a side-scan sonar image of the sampling area corresponding to each echo data.

[0015] According to a fishery resource assessment method provided by the present invention, the preprocessing of each side-scan sonar image comprises:

[0016] Acquire the height of the towed fish of the side scan sonar from the bottom of the target water area based on the side scan sonar image;

[0017] performing slant range correction on the side scan sonar image according to the height;

[0018] Image enhancement processing is performed on the side-scan sonar image after slant range correction based on an image enhancement model.

[0019] According to a fishery resource assessment method provided by the present invention, the biomass assessment model is used to perform biomass assessment on each of the pre-processed side-scan sonar images, and fishery resource distribution images with the same size as each of the side-scan sonar images are obtained, including:

[0020] Performing fishery resource identification processing on each of the pre-processed side-scan sonar images to obtain a target side-scan sonar image and / or a blank fishery resource distribution image;

[0021] Based on the biomass assessment model, biomass assessment is performed on each of the target side-scan sonar images, and a target fishery resource distribution image having the same size as each of the target side-scan sonar images is obtained;

[0022] The target fishery resource distribution image and the blank fishery resource distribution image are determined as fishery resource distribution images, the target side-scan sonar image is the side-scan sonar image including fishery resources, and the blank fishery resource distribution image is obtained based on the side-scan sonar image excluding fishery resources; the biomass assessment model includes an encoder and a decoder, the encoder and the decoder are connected through multiple layers of atrous convolutional layers, the encoder includes an Inception structure and a maximum pooling layer, and the decoder includes multiple convolution-deconvolution modules.

[0023] According to a fishery resource assessment method provided by the present invention, the fishery resource identification processing is performed on each of the pre-processed side-scan sonar images to obtain a target side-scan sonar image and / or a blank fishery resource distribution image, including:

[0024] Assigning weight information to the target of interest in the preprocessed side-scan sonar image to obtain an attention weight image;

[0025] According to the attention weight image, cropping an image block containing at least one target of interest in the preprocessed side scan sonar image;

[0026] Extracting features from the image block, and fusing the features extracted from the image block with the corresponding attention weight image to obtain a fused feature map;

[0027] The side-scan sonar image is classified based on the fusion feature map to obtain the target side-scan sonar image and / or the blank fishery resource distribution image.

[0028] The present invention also provides a fishery resource assessment device, comprising:

[0029] An image generation module is used to obtain a plurality of echo data of the target water area collected by the side scan sonar, and generate a side scan sonar image according to each of the echo data;

[0030] An image preprocessing module, used for preprocessing each of the side-scan sonar images, wherein the preprocessing includes slant range correction processing and / or side-scan sonar image enhancement processing;

[0031] A biomass assessment module, for performing biomass assessment on each of the preprocessed side-scan sonar images based on a biomass assessment model, and obtaining fishery resource distribution images of the same size as each of the side-scan sonar images;

[0032] A distribution map generation module, used to determine the geographical coordinates of each pixel point on the corresponding fishery resource distribution image according to the geographical coordinates corresponding to each pixel point on the side scan sonar image, and determine the corresponding fishery resource distribution map based on the geographical coordinates of each pixel point on the fishery resource distribution image;

[0033] The distribution map splicing module is used to splice all fishery resource distribution maps corresponding to the target water area to obtain the target fishery resource distribution map of the target water area.

[0034] The present invention also provides a fishery resource assessment system, comprising: an unmanned ship, a side-scan sonar, an unmanned ship control device and the above-mentioned fishery resource assessment device; wherein:

[0035] The unmanned ship includes a hull, a first attitude sensor installed on the hull, an ultrasonic sensor, a positioning module, a first communication module, a main control module and a driving module, wherein the first attitude sensor is used to obtain navigation information of the unmanned ship, and the navigation information includes ship speed, acceleration and angular velocity; the positioning module is used to obtain positioning information of the unmanned ship; the ultrasonic sensor is used to detect obstacles around the unmanned ship to obtain detection information; the first communication module is used to send the navigation information, the positioning information and the detection information, and is also used to receive route information and / or control instructions; the main control module is used to control the driving module to drive according to the route information and / or the control instructions based on the navigation information and positioning information of the unmanned ship;

[0036] The side scan sonar comprises a towed fish, a tow cable, a deck unit and a second communication module, wherein the deck unit is arranged on the hull of the unmanned ship; the towed fish is placed in the target waters and connected to the deck unit and the unmanned ship through the tow cable; the second communication module is used to send the echo data collected by the towed fish and the positioning information to the fishery resource assessment device;

[0037] The unmanned ship control device comprises an unmanned ship control module and a third communication module, wherein the unmanned ship control module is used to generate the route information and / or the control instruction, and the third communication module is used to receive the navigation information, the positioning information and the detection information, and is also used to issue the route information and / or the control instruction;

[0038] The fishery resource assessment device also includes a fourth communication module for receiving the echo data and the positioning information.

[0039] A fishery resource assessment system provided by the present invention also includes:

[0040] A remote controller is used to send remote control instructions to the unmanned ship.

[0041] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned fishery resource assessment methods is implemented.

[0042] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the fishery resource assessment method as described in any one of the above is implemented.

[0043] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the fishery resource assessment method as described above is implemented.

[0044] The fishery resource assessment method, device, system and electronic device provided by the present invention reduce the lateral distortion and water column area on the side-scan sonar image by performing slant range correction processing on the side-scan sonar image, and remove the noise points of the side-scan sonar image, refine the texture features and enhance the contrast of the side-scan sonar image to a certain extent by performing enhancement processing on the side-scan sonar. The above-mentioned preprocessing method improves the quality of the side-scan sonar image, so that the recognition accuracy of fishery resource assessment based on the preprocessed side-scan sonar image is higher and the assessment efficiency is higher. After the fishery resource identification is performed, the method can also obtain a fishery resource distribution map according to the geographic coordinates, so that the biomass and distribution of fishery resources in the water area can be intuitively and clearly understood. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 It is a schematic diagram of the flow of the fishery resources assessment method provided by the present invention;

[0047] Figure 2 It is a schematic diagram of the relationship between the pixel coordinates on the side scan sonar image and the towed fish coordinates provided by the present invention;

[0048] Figure 3 It is a flow chart of a method for generating a side-scan sonar image provided by the present invention;

[0049] Figure 4 It is a flow chart of a method for preprocessing a side-scan sonar image provided by the present invention;

[0050] Figure 5 It is a schematic diagram of a side-scan sonar image provided by the present invention;

[0051] Figure 6 It is a flow chart of a method for biomass assessment on side-scan sonar images provided by the present invention;

[0052] Figure 7 It is a schematic diagram of the structure of the biomass assessment model provided by the present invention;

[0053] Figure 8 It is a flow chart of the method for fishery resource identification and processing provided by the present invention;

[0054] Fig. 9 It is a schematic diagram of the classification network structure provided by the present invention;

[0055] Fig.10 It is a structural schematic diagram of the fishery resources assessment device provided by the present invention;

[0056] Fig.11 It is a structural schematic diagram of the fishery resources assessment system provided by the present invention;

[0057] Fig.12 This is a schematic diagram of the route of the unmanned ship provided by the present invention;

[0058] Fig.13 This is a schematic diagram of the navigation of the unmanned boat and the towed fish provided by the present invention;

[0059] Fig.14 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Combine the following Figure 1-Figure 9 The fishery resource assessment method of the present invention is described, the method comprising:

[0062] S101: Acquire a plurality of echo data of a target water area collected by a side-scan sonar, and generate a side-scan sonar image according to each of the echo data.

[0063] Specifically, the side scan sonar moves in the target waters according to a predetermined route, and emits sound waves at a preset frequency during the movement. After emitting sound waves, the side scan sonar will receive returned echo data. A target area water area needs to emit multiple sound waves for detection, so multiple echo data will be returned. At this time, the echo data received by the side scan sonar is obtained, and a waterfall chart, i.e., a side scan sonar image, is generated based on the echo data.

[0064] S102: Preprocessing each of the side-scan sonar images, wherein the preprocessing includes slant range correction processing and / or side-scan sonar image enhancement processing.

[0065] Specifically, since each waterfall image, i.e., each side-scan sonar image, generated according to each echo data may have lateral distortion and water column areas, a complete side-scan sonar image without distortion is obtained through slant range correction. Furthermore, since side-scan sonar images have many noise points, blurred outlines, low contrast, etc., image enhancement can be used to remove noise points on the side-scan sonar image as much as possible, refine the texture features on the side-scan sonar image to make the outline clear, and improve the contrast of the side-scan sonar image, thereby obtaining a high-quality side-scan sonar image.

[0066] S103: Based on the biomass assessment model, biomass assessment is performed on each of the preprocessed side-scan sonar images to obtain fishery resource distribution images of the same size as each of the side-scan sonar images.

[0067] Specifically, after preprocessing each side-scan sonar image, the fishery resources in the preprocessed side-scan sonar image are identified and estimated based on the biomass assessment model, the number of fishery resources in the sampling area corresponding to the side-scan sonar image is obtained, and a fishery resource distribution image of the same size as the side-scan sonar image is obtained. The fishery resource distribution image is the same size as the side-scan sonar image, which facilitates the subsequent determination of the geographic coordinates of each pixel point on the fishery resource distribution image based on the side-scan sonar image. For all fishery resources in the target waters, the number of fishery resources in multiple sampling areas estimated for each side-scan sonar image can be obtained by superimposing.

[0068] S104: Determine the geographical coordinates of each pixel point on the fishery resource distribution image according to the geographical coordinates corresponding to each pixel point on the side scan sonar image, and determine the corresponding fishery resource distribution map based on the geographical coordinates of each pixel point on the fishery resource distribution image.

[0069] Specifically, the side scan sonar is connected to a ship and moves in the target waters under the drag of the ship. A positioning module is set on the hull, which can be a Beidou receiver or a GPS (Global Positioning System). The geographical information of the hull can be obtained through the positioning module, and then the geographical coordinates of each pixel on the side scan sonar image can be obtained through coordinate transformation.

[0070] For example, a Beidou receiver is installed on the hull, and the geographic information of the ship is obtained by receiving Beidou signals or GPS signals. The geographic information is converted via the Beidou receiver to the towing point where the ship is connected to the towed fish of the side-scan sonar, the towed point to the towed fish, and the towed fish to the side-scan sonar image, thereby obtaining the geographic coordinates of each pixel point on the side-scan sonar image.

[0071] The position of the drag point is calculated as follows:

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] Among them, (x tp ,y tp ) represents the geographic coordinates of the drag point; (x G ,y G ) represents the geographic coordinates of the Beidou receiver on board; (Δx, Δy) represents the offset of the tow point relative to the Beidou receiver; A represents the heading of the ship; p and r represent the attitude of the ship, p represents the pitch angle, and r represents the roll angle; (x tps ,y tps ) represents the coordinates of the drag point when it is stationary; (x gs ,y gs ) represents the coordinates of the Beidou receiver when stationary.

[0078] Since the towed fish and the towing point are both in the direction of the route, that is, on the same straight line, after knowing the geographical coordinates of the towing point, the geographical coordinates of the towed fish are calculated as follows:

[0079]

[0080]

[0081] Among them, (x tf ,y tf) represents the geographical coordinates of the towfish; d layback represents the horizontal distance between the towing point and the towed fish; α represents a hyperparameter, which is determined by the tightness of the towline; l represents the length of the towline from the towing point to the towed fish; h tp It indicates the height from the towing point to the water surface, and d indicates the depth of the towed fish.

[0082] On the side scan sonar image, Figure 2 As shown in , the towfish is located on the route in the center of the side scan sonar image. The geographic coordinates of pixel i (x ig ,y ig )for:

[0083]

[0084] Among them, d h Represents the projection point P from the pixel point Pi on the preprocessed side-scan sonar image to the towed fish on the side-scan sonar image t The actual horizontal distance.

[0085] Since the size of the generated fishery resource distribution map is consistent with the size of the preprocessed sonar image, the geographic coordinates of each pixel point on the side-scan sonar image are directly mapped to the geographic coordinates of each pixel point on the fishery resource distribution map, thereby obtaining the fishery resource distribution map.

[0086] S105: splicing all fishery resource distribution maps corresponding to the target water area to obtain a target fishery resource distribution map of the target water area.

[0087] Specifically, after all fishery resource distribution maps of the target waters are obtained, all fishery resource distribution maps are spliced ​​according to the geographic coordinates of each pixel point to obtain a target fishery resource distribution map of the target waters.

[0088] The fishery resource assessment method, device, system and electronic device provided by the present invention reduce the lateral distortion and water column area on the side-scan sonar image by performing slant range correction processing on the side-scan sonar image, and remove the noise points of the side-scan sonar image, refine the texture features and enhance the contrast of the side-scan sonar image to a certain extent by performing enhancement processing on the side-scan sonar. The above-mentioned preprocessing method improves the quality of the side-scan sonar image, so that the recognition accuracy of fishery resource assessment based on the preprocessed side-scan sonar image is higher and the assessment efficiency is higher. After the fishery resource identification is performed, the method can also obtain a fishery resource distribution map according to the geographic coordinates, so that the biomass and distribution of fishery resources in the water area can be intuitively and clearly understood.

[0089] In one embodiment, Figure 3As shown in , the step of acquiring a plurality of echo data of a target water area collected by a side-scan sonar and generating a side-scan sonar image according to each of the echo data includes:

[0090] S301: Acquire in real time the echo data received by the side scan sonar after emitting sound waves in the target water area.

[0091] Specifically, real-time acquisition means that after the side scan sonar emits a scanning sound wave, the echo data is immediately acquired when the echo data is collected. Acquiring the echo data in real time means that the fishery resources of the scanned area in the target waters can be calculated in real time according to the acquired echo data, and the fishery resource distribution map of the scanned area in the target waters can be obtained and output in real time.

[0092] S302: quantizing the echo data to obtain a side-scan sonar image of the sampling area corresponding to each echo data.

[0093] For example, the returned echo data collected by the side-scan sonar may be 11-bit or 64-bit data, and the echo data can be integrated into 8-bit integer data to obtain a waterfall diagram of the sampling area corresponding to the echo data, i.e., a side-scan sonar image, thereby reducing subsequent calculations and storage costs.

[0094] For example, the echo data can be integrated into 8-bit integer data using the following formula:

[0095]

[0096] Among them, image represents the re-quantized data, that is, the side scan sonar image; C represents the custom hyperparameter; n represents the number of bits after quantization, which can be 8 in this example; m represents the number of bits of the original data, that is, the echo data; data represents the echo data.

[0097] In one embodiment, Figure 4 As shown in , the preprocessing of each of the side scan sonar images includes:

[0098] S401: Acquire the height of the towed fish of the side scan sonar from the bottom of the target water area based on the side scan sonar image.

[0099] Specifically, refer to Figure 5 As shown in Figure 5 is a side scan sonar image, the white solid line is the route of the towed fish of the side scan sonar, and the white dotted line is the seabed line. The height of the towed fish from the seabed is the distance from the seabed line to the route of the towed fish. For example, since the side scan sonar image obtained by scanning the echo data once is narrow, the seabed line at any location can be taken to calculate the height of the towed fish from the seabed, or the average of the seabed lines can be taken to calculate the height of the towed fish from the seabed.

[0100] S402: Perform slant range correction on the side scan sonar image according to the height.

[0101] Specifically, the lateral distance on the side-scan sonar image, that is, the vertical distance between the fishing target and the towfish route, is the slant distance of the sound waves emitted by the towfish reaching the seabed, rather than the actual distance between the fishing target and the towfish route. This causes lateral distortion. In addition, the water column area formed by the towfish divides the area directly below the towfish into two independent areas. Therefore, slant distance correction is required to obtain a complete and distortion-free side-scan sonar image.

[0102] Slope distance correction is performed according to the following formula:

[0103]

[0104] Among them, d h is the pixel P i The actual horizontal distance, that is, the pixel point P i The horizontal distance from the route, r is the pixel point P on the side scan sonar image without preprocessing i The lateral distance, i.e. the slant distance, h is the height from the towed fish to the bottom of the water.

[0105] S403: Performing image enhancement processing on the side scan sonar image after slant range correction based on an image enhancement model.

[0106] Specifically, since clear side-scan sonar images are difficult to obtain, it is difficult to use the commonly used low-quality-high-quality image pairs to supervise the learning of image enhancement models. Therefore, by converting the image enhancement problem of side-scan sonar images into an image style transfer problem, that is, migrating low-quality side-scan sonar images into high-quality single-channel optical images, the style of the side-scan sonar images is changed while keeping the content of the side-scan sonar images unchanged, thereby achieving the purpose of refining textures, increasing contrast, and reducing noise.

[0107] Exemplarily, the image enhancement model can use an unsupervised non-matching image generation adversarial network, in which the generator is an encoder-decoder symmetric framework, the encoder uses multiple convolution modules for downsampling, the decoder uses a deconvolution module for upsampling, and the convolution and deconvolution modules corresponding to the decoder and encoder are jump-connected to eliminate overfitting and gradient vanishing problems. The convolution module consists of a convolution layer + batch normalization + activation function, and the number of convolution modules and channels is determined according to actual conditions. The discriminator consists of a convolution module, which downsamples the image and finally performs global average pooling.

[0108] The training process of the above image enhancement model is:

[0109] The side scan sonar image is input into the generator to obtain an enhanced image. The single-channel optical image and the enhanced image are randomly input into the discriminator to distinguish true from false. The discriminator is supervised by the loss function so that the discriminator can recognize the single-channel optical image and the enhanced image to the maximum extent, and the generator needs to generate an enhanced image that is similar in style to the single-channel optical image as much as possible. In this way, after the confrontation between the discriminator and the generator, the optimal image enhancement model can be obtained when the two reach a balance. Exemplarily, color images with content similar to that of the sonar image, such as fish, sand dunes, rocks, etc., can be collected and then converted into grayscale images to obtain the single-channel optical image used in training.

[0110] After the optimal image enhancement model is obtained through training, the side-scan sonar image after slant range correction is input into the optimal image enhancement model to obtain the enhanced side-scan sonar image.

[0111] In one embodiment, Figure 6 As shown in, the biomass assessment model is used to perform biomass assessment on each of the pre-processed side-scan sonar images, and fishery resource distribution images of the same size as each of the side-scan sonar images are obtained, including:

[0112] S601: Perform fishery resource identification processing on each of the pre-processed side-scan sonar images to obtain a target side-scan sonar image and / or a blank fishery resource distribution image.

[0113] Specifically, in order to save the computing power of the subsequent biomass assessment model and improve the computing speed to facilitate the rapid assessment of biomass, fishery resources are first identified before the biomass assessment of the side-scan sonar image, and the target side-scan sonar images with fishery resources and the non-target side-scan sonar images without fishery resources are identified. Since the non-target side-scan sonar images do not need to be evaluated for biomass, for the non-target side-scan sonar images, a blank fishery resource distribution image can be directly output after identification, and the blank fishery resource distribution image can be a blank image of the same size as the side-scan sonar image.

[0114] S602: Based on the biomass assessment model, a biomass assessment is performed on each of the target side-scan sonar images, and a target fishery resource distribution image of the same size as each of the target side-scan sonar images is obtained; wherein the target fishery resource distribution image and the blank fishery resource distribution image are determined as the fishery resource distribution image, the target side-scan sonar image is the side-scan sonar image including fishery resources, and the blank fishery resource distribution image is obtained based on the side-scan sonar image excluding fishery resources; the biomass assessment model includes an encoder and a decoder, the encoder and the decoder are connected by multiple layers of atrous convolutional layers, the encoder includes an Inception structure and a maximum pooling layer, and the decoder includes multiple convolution-deconvolution modules.

[0115] Exemplary, reference Figure 7 As shown in , the biomass assessment model is implemented by a counting network based on a density map. The counting network adopts an improved SANet network framework, which is an encoder-decoder structure. Since side-scan sonar images are remote sensing images, fish and other fishery resources usually account for a small proportion of them and belong to the category of small targets. Therefore, in order to increase the accuracy of fish recognition and thus improve the counting accuracy, multiple layers of hole convolution layers are inserted between the encoder and the decoder. The hole convolution layer can obtain a larger feature map under the same receptive field, which is helpful for the recognition of small targets such as fish. At the same time, the hole convolution layer can also obtain the same receptive field as the ordinary convolution with a smaller number of parameters, which is beneficial to the real-time detection of fish distribution. Due to different species or distances, fish often appear in different sizes in side-scan sonar images, so different sizes of convolution kernels are needed to extract multi-scale features to ensure the accuracy of the model. The Inception structure (deep learning structure) uses parallel convolution kernels of different sizes, which can efficiently extract richer spatial features while increasing the network width, so the Inception structure is used in the encoder. In order to reduce the number of parameters, save computing power, and achieve real-time prediction, multiple smaller convolution kernels are used to replace larger convolution kernels. Convolution kernels larger than 5*5 are defined as larger convolution kernels. For example, two 3*3 convolution kernels are used to replace one 5*5 convolution kernel, and three 3*3 convolution kernels are used to replace one 7*7 convolution kernel. At the same time, a 1*1 convolution kernel is inserted before each parallel convolution to reduce the dimension of the feature map, further reducing the number of parameters.

[0116] The decoder uses upsampling with multiple convolution-deconvolution modules to output a fish distribution density map of the same size as the original image, which is beneficial for subsequent geographic coordinate conversion.

[0117] When training the above biomass assessment model, the mean square error (L2 loss) is used as the loss function to minimize the error between the generated fishery resource distribution map and the true density map. The mean square error loss function formula is as follows:

[0118]

[0119] Where N is the number of side scan sonar images in the training set; X k is the kth side-scan sonar image; θ is the network parameter to be determined; D(·) is the predicted density map, i.e., the distribution map of fishery resources; D k is the true value density map corresponding to the kth side scan sonar image.

[0120] However, since the L2 loss measures the difference between each pixel, it is very sensitive to noise and it is difficult to obtain accurate spatial distribution information of fish. Therefore, the structural similarity index measure (SSIM) is added to the L2 loss to obtain the total loss function to measure the similarity between the generated fish distribution density map and the true density map, forcing the biomass assessment model to generate a fishery resource distribution map that is more similar to the true density map, thereby obtaining a high-quality fish distribution density map.

[0121] The calculation formula of structural similarity index SSIM is as follows:

[0122]

[0123] Among them, E and G represent the fishery resource distribution map and the true value density map respectively; μ E and μ G They represent the average values ​​of the fishery resource distribution map and the true density map respectively; and Respectively represent the variance of the fishery resource distribution map and the true value density map; σ EG Represents the covariance between the fishery resource distribution map and the true density map; C1 and C2 represent hyperparameters, which can be set to relatively small values ​​to avoid the denominator being zero.

[0124] In summary, the total loss function is:

[0125] L=L E +α(1-SSIM)

[0126] Among them, α is the coefficient of structural similarity loss, which is used to adjust the proportion of structural similarity loss in the total loss function.

[0127] In one embodiment, Figure 8 As shown in, the fishery resource identification processing is performed on each of the pre-processed side-scan sonar images to obtain a target side-scan sonar image and / or a blank fishery resource distribution image, including:

[0128] S801: Assigning weight information to the target of interest in the preprocessed side-scan sonar image to obtain an attention weight image.

[0129] S802: Crop an image block including at least one target of interest from the preprocessed side-scan sonar image according to the attention weight image.

[0130] S803: Extract features from the image block, and fuse the features extracted from the image block with the corresponding attention weight image to obtain a fused feature map.

[0131] S804: Classify the side-scan sonar image based on the fusion feature map to obtain the target side-scan sonar image and / or the blank fishery resource distribution image.

[0132] For example, fishery resource identification can be achieved using a classification model. Currently, the commonly used classification network generally extracts features from the image first to obtain features with large differences between different targets, and then classifies the features to achieve image classification. However, since side-scan sonar images cover a very wide range, the targets therein are often small, and it is difficult to distinguish between fishery resources such as fish and shrimp and non-fishery resources such as stones. If a traditional classification network is used to extract features from the side-scan sonar image as a whole, the features will often be submerged in the larger bottom features, making them difficult to identify, because non-bottom targets such as fish, shrimp, and stones are small.

[0133] Therefore, the strategy of first enlarging the region of interest containing the target of interest and then extracting and classifying the features of the region of interest is adopted to improve the accuracy of identifying fish and shrimp and other fishery resources. Fig. 9 As shown in , the specific implementation of this strategy is based on a classification network, which first uses a saliency detection convolutional neural network a Θ (·) is used to detect targets of interest that are obviously different from the bottom in the side scan sonar image, such as stones, fish and shrimp, and assign higher weights to the targets of interest to obtain the attention weight image. When there is no target of interest in the side scan sonar image, the attention weights of all locations on the attention weight image are the same.

[0134] According to the attention weight image, multiple image blocks containing at least one target of interest are cropped on the original side scan sonar image, namely the region of interest. The region of interest is extracted through the feature extraction network f Θ (·) Feature extraction. Since the space occupied by the objects of interest such as fish, shrimp or stones in the cropped area of ​​interest is significantly increased, its unique features that are different from the bottom can be effectively extracted. Among them, there is no cropped image block for the side scan sonar image without the object of interest.

[0135] In order to combine the global and local features of the side scan sonar image, the attention weight image is used as the global feature to fuse the features extracted from the region of interest to obtain a fused feature map, which is then input into the classifier g Θ (·) to classify, so as to know which side scan sonar images are target side scan sonar images, where g Θ (·) is implemented by a classification neural network. The above classification network can extract more representative features of non-bottom targets such as fish and shrimp, thus achieving higher classification accuracy and recognition precision. Among them, for side-scan sonar images without fishery resources, the classifier g Θ(·) directly outputs a blank fishery resource distribution image, which can be a blank image with the same size as the side scan sonar image.

[0136] The fishery resource assessment device provided by the present invention is described below. The fishery resource assessment device described below and the fishery resource assessment method described above can be referenced to each other.

[0137] like Fig.10 As shown, the fishery resource assessment device includes: an image generation module 1001, an image preprocessing module 1002, a biomass assessment module 1003, a distribution map generation module 1004 and a distribution map splicing module 1005; the image generation module 1001 is used to obtain the echo data of the target waters collected by the side scan sonar, and generate a side scan sonar image according to the echo data; the image preprocessing module 1002 is used to preprocess each of the side scan sonar images, and the preprocessing includes slant range correction processing and side scan sonar image enhancement processing; the biomass assessment module 1003 is used to, based on the biomass assessment model, The pre-processed side-scan sonar images are subjected to biomass assessment to obtain fishery resource distribution images of the same size as the side-scan sonar images; the distribution map generation module 1004 is used to determine the geographical coordinates of each pixel point on the corresponding fishery resource distribution image according to the geographical coordinates corresponding to each pixel point on the side-scan sonar image, and determine the corresponding fishery resource distribution map based on the geographical coordinates of each pixel point on the fishery resource distribution image; the distribution map splicing module 1005 splices all fishery resource distribution maps corresponding to the target waters to obtain a target fishery resource distribution map of the target waters.

[0138] The fishery resource assessment device provided by the present invention performs slant range correction processing on the side-scan sonar image, thereby weakening the lateral distortion and water column area on the side-scan sonar image, and performs enhancement processing on the side-scan sonar, thereby removing the noise points of the side-scan sonar image to a certain extent, refining the texture features, and enhancing the contrast of the side-scan sonar image. The above-mentioned preprocessing method improves the quality of the side-scan sonar image, so that the recognition accuracy of fishery resource assessment based on the biomass assessment model through the preprocessed side-scan sonar image is higher, and the assessment efficiency is higher; after the fishery resource identification is performed, the method can also obtain a fishery resource distribution map according to the geographical coordinates, so that the biomass and distribution of fishery resources in the water area can be intuitively and clearly understood.

[0139] In one embodiment, the image generation module 1001 includes: a real-time acquisition submodule and a quantification processing submodule; the real-time acquisition submodule is used to acquire in real time the echo data received by the side-scan sonar after the side-scan sonar emits sound waves in the target waters; the quantification processing submodule is used to perform quantification processing on each echo data to obtain a side-scan sonar image of the sampling area corresponding to each echo data.

[0140] In one embodiment, the image preprocessing module 1002 includes: a height acquisition submodule, a correction submodule and an image enhancement submodule; the height acquisition submodule is used to acquire the height of the towed fish of the side scan sonar from the bottom of the target water area based on the side scan sonar image; the correction submodule is used to perform slant range correction on the side scan sonar image according to the height; the image enhancement submodule is used to perform image enhancement processing on the side scan sonar image after slant range correction based on an image enhancement model.

[0141] In one embodiment, the biomass assessment module 1003 includes: a fishery identification submodule and a biomass assessment submodule; the fishery identification submodule is used to perform fishery resource identification processing on each of the pre-processed side-scan sonar images to obtain a target side-scan sonar image and / or a non-blank fishery resource distribution image. The biomass assessment submodule is used to perform biomass assessment on the target side-scan sonar image based on the biomass assessment model, and obtain a target fishery resource distribution image of the same size as each of the target side-scan sonar images; wherein the target fishery resource distribution image and the blank fishery resource distribution image are determined as fishery resource distribution images, the target side-scan sonar image is the side-scan sonar image including fishery resources, and the blank fishery resource distribution image is obtained based on the side-scan sonar image not including fishery resources; the biomass assessment model includes an encoder and a decoder, the encoder and the decoder are connected by multiple layers of atrous convolution layers, the encoder includes an Inception structure and a maximum pooling layer, and the decoder includes multiple convolution-deconvolution modules.

[0142] In one embodiment, the fishery identification submodule is specifically used to: assign weight information to the target of interest in the preprocessed side-scan sonar image to obtain an attention weight image; crop an image block containing at least one target of interest in the preprocessed side-scan sonar image according to the attention weight image; perform feature extraction on the image block, and perform feature fusion on the features extracted from the image block and the corresponding attention weight image to obtain a fused feature map; classify the side-scan sonar image based on the fused feature map to obtain the target side-scan sonar image and / or a blank fishery resource distribution image.

[0143] The fishery resource assessment system provided by the present invention is described below. Fig.11 As shown in, the system includes: an unmanned ship, a side-scan sonar, an unmanned ship control device and the above-mentioned fishery resource assessment device; wherein the unmanned ship includes a hull, a first attitude sensor installed on the hull, an ultrasonic sensor, a positioning module, a first communication module, a main control module and a drive module, the first attitude sensor is used to obtain the navigation information of the unmanned ship, and the navigation information includes ship speed, acceleration and angular velocity; the positioning module is used to obtain the positioning information of the unmanned ship; the ultrasonic sensor is used to detect obstacles around the unmanned ship to obtain detection information; the first communication module is used to send the navigation information, the positioning information and the detection information, and is also used to receive route information and / or control instructions; the main control module is used to, based on the navigation information and positioning information of the unmanned ship, according to the route information and / or the The control instruction controls the driving module to drive; the side scan sonar includes a towed fish, a tow cable, a deck unit and a second communication module, and the deck unit is sealed on the hull of the unmanned ship; the towed fish is placed in the target waters and connected to the deck unit and the unmanned ship through the tow cable; the second communication module is used to send the echo data and the positioning information collected by the towed fish to the fishery resource assessment device; the unmanned ship control device includes an unmanned ship control module and a third communication module, the unmanned ship control module is used to generate the route information and / or the control instruction, the third communication module is used to receive the navigation information, the positioning information and the detection information, and is also used to issue the route information and / or the control instruction; the fishery resource assessment device also includes a fourth communication module for receiving the echo data and the positioning information.

[0144] Exemplarily, the fishery resource assessment system includes a shore-based subsystem and a sea-based subsystem. The shore-based subsystem includes a fishery resource assessment device and an unmanned vessel control device. The fishery resource assessment device and the unmanned vessel control device can be set in the same computer or server. The sea-based subsystem includes an unmanned vessel and a side-scan sonar.

[0145] The unmanned ship control device is used to plan the route information and / or control instructions of the unmanned ship, and the control instructions are used to control the unmanned ship to stop, return, continue sailing, etc. Fig.12 As shown in , the route can be a straight line going back and forth in the target waters, and the distance between two adjacent straight lines is less than or equal to the single-side width of the side-scan sonar, that is, Fig.13 Considering the problems of sea waves and yaw, it is best to make the distance between two adjacent straight lines smaller than the unilateral width of the side scan sonar to avoid scanning gaps. Since the unilateral width of the side scan sonar is determined by the height and slant distance of the towed fish, considering the overlap rate between two adjacent scanning strips of the side scan sonar, the distance between two adjacent straight lines in the route can be:

[0146]

[0147] Among them, d line It is expressed as the distance between two adjacent straight lines; R is the slant distance (i.e., range) of the side-scan sonar; H is the water depth, which can be the maximum water depth in the target area; h tow is the distance between the towed fish and the water surface, which is determined according to the position of the towed fish relative to the unmanned vessel; σ is the overlap coefficient.

[0148] The driving module on the unmanned ship can include two drivers, two motors connected to the two drivers respectively, and the motor drive end is connected to the propeller of the unmanned ship. The heading of the unmanned ship can be controlled by differential drive. After the main control module of the unmanned ship receives the route information, it controls the driving module based on the current position information and navigation information of the unmanned ship, thereby driving the unmanned ship to navigate according to the route. Among them, the main control module needs to be sealed and set on the unmanned ship.

[0149] When it is necessary to assess the fishery resources in the target waters, the deck unit of the side scan sonar is installed in the cabin of the unmanned ship, the towed fish is connected to the deck unit through the tow cable and all the connection ports are sealed. To ensure that the connection between the towed fish and the deck unit is not stressed, the tow cable and the unmanned ship are fixed at the towing point of the unmanned ship through a fixed rope. Fig.13 As shown, the towfish scans the target waters according to the route by being towed by an unmanned boat. Transducers are symmetrically arranged on both sides of the towfish for emitting sound waves and receiving echoes, thereby collecting underwater echo data. The echo data is transmitted to the deck unit via the tow cable, and the second communication module on the deck unit sends the echo data to the fishery resource assessment device of the shore-based subsystem.

[0150] It is understandable that in order to improve the real-time performance of data transmission, 4G or 5G communication technology can be used for communication. To facilitate communication between the shore-based subsystem and the sea-based subsystem, a first communication module is provided on the unmanned ship, a second communication module is provided on the deck unit, a third communication module is provided on the unmanned ship control device, and a fourth communication module is provided on the fishery resource assessment device.

[0151] Exemplarily, the third communication module and the fourth communication module on the fishery resource assessment device and the unmanned boat control device can be set with different and unique IP addresses. When the sea-based subsystem and the shore-based subsystem communicate, they can first obtain the IP address of the corresponding communication module through intranet penetration technology, and then use the TCP / IP communication protocol to form a socket socket according to the IP address and port number of each corresponding communication module for communication transmission.

[0152] The fishery resource assessment system provided by the present invention performs slant range correction processing on the side-scan sonar image, thereby weakening the lateral distortion and water column area on the side-scan sonar image, and performs enhancement processing on the side-scan sonar, thereby removing the noise points of the side-scan sonar image to a certain extent, refining the texture features, and enhancing the contrast of the side-scan sonar image. The above-mentioned preprocessing method improves the quality of the side-scan sonar image, so that the recognition accuracy of fishery resource assessment based on the biomass assessment model through the preprocessed side-scan sonar image is higher, and the assessment efficiency is higher; after the fishery resource identification is performed, the method can also obtain a fishery resource distribution map according to the geographic coordinates, so that the biomass and distribution of fishery resources in the water area can be intuitively and clearly understood; further, the system can transmit data in real time through the setting of multiple communication modules, and drive the side-scan sonar to perform collection work through an unmanned ship, thereby saving human resources.

[0153] The speed of the unmanned vessel is related to the scanning frequency of the towed fish. After determining the scanning frequency of the towed fish, the speed of the unmanned vessel needs to be set according to the scanning frequency of the towed fish to prevent longitudinal distortion of the side-scan sonar image as much as possible.

[0154] In one embodiment, it further includes: a remote controller, wherein the remote controller is used to send remote control instructions to the unmanned ship.

[0155] Specifically, the unmanned boat can be switched from automatic mode to manual mode, and the unmanned boat operator can flexibly and autonomously control the unmanned boat through a remote control, thereby enhancing the human-computer interaction of the unmanned boat and improving the safety of the unmanned boat's navigation.

[0156] Fig.14 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.14As shown, the electronic device may include: a processor (processor) 1410, a communication interface (Communications Interface) 1420, a memory (memory) 1430 and a communication bus 1440, wherein the processor 1410, the communication interface 1420, and the memory 1430 communicate with each other through the communication bus 1440. The processor 1410 can call the logic instructions in the memory 1430 to execute a fishery resource assessment method, which includes: obtaining multiple echo data of the target waters collected by the side-scan sonar, and generating a side-scan sonar image according to each of the echo data; preprocessing each of the side-scan sonar images, and the preprocessing includes slant range correction processing and / or side-scan sonar image enhancement processing; based on the biomass assessment model, performing biomass assessment on each of the preprocessed side-scan sonar images, and obtaining fishery resource distribution images of the same size as each of the side-scan sonar images; determining the geographical coordinates of each pixel point on the corresponding fishery resource distribution image according to the geographical coordinates corresponding to each pixel point on the side-scan sonar image, and determining the corresponding fishery resource distribution map based on the geographical coordinates of each pixel point on the fishery resource distribution image; splicing all fishery resource distribution maps corresponding to the target waters to obtain a target fishery resource distribution map of the target waters.

[0157] In addition, the logic instructions in the above-mentioned memory 1430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0158] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the fishery resource assessment method provided by the above-mentioned methods, which includes: obtaining multiple echo data of the target waters collected by the side-scan sonar, and generating a side-scan sonar image according to each of the echo data; preprocessing each of the side-scan sonar images, and the preprocessing includes slant range correction processing and / or side-scan sonar image enhancement processing; based on a biomass assessment model, performing biomass assessment on each of the preprocessed side-scan sonar images, and obtaining fishery resource distribution images of the same size as each of the side-scan sonar images; determining the geographical coordinates of each pixel point on the corresponding fishery resource distribution image according to the geographical coordinates corresponding to each pixel point on the side-scan sonar image, and determining the corresponding fishery resource distribution map based on the geographical coordinates of each pixel point on the fishery resource distribution image; splicing all fishery resource distribution maps corresponding to the target waters to obtain a target fishery resource distribution map of the target waters.

[0159] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the fishery resource assessment method provided by the above-mentioned methods, the method comprising: acquiring multiple echo data of the target waters collected by a side-scan sonar, and generating a side-scan sonar image based on each of the echo data; preprocessing each of the side-scan sonar images, the preprocessing comprising slant range correction processing and / or side-scan sonar image enhancement processing; based on a biomass assessment model, performing a biomass assessment on each of the preprocessed side-scan sonar images, and obtaining fishery resource distribution images of the same size as each of the side-scan sonar images; determining the geographical coordinates of each pixel point on the corresponding fishery resource distribution image based on the geographical coordinates corresponding to each pixel point on the side-scan sonar image, and determining the corresponding fishery resource distribution map based on the geographical coordinates of each pixel point on the fishery resource distribution image; splicing all fishery resource distribution maps corresponding to the target waters to obtain a target fishery resource distribution map of the target waters.

[0160] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0161] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for assessing fishery resources, It is characterized in that include: Acquire multiple echo data of the target water area collected by the side-scan sonar, and generate a side-scan sonar image according to each of the echo data; Preprocessing each of the side-scan sonar images, wherein the preprocessing includes slant range correction processing and side-scan sonar image enhancement processing; Based on the biomass assessment model, the biomass of each of the preprocessed side-scan sonar images is assessed to obtain fishery resource distribution images of the same size as each of the side-scan sonar images; Wherein, based on the biomass assessment model, the biomass assessment is performed on each of the preprocessed side-scan sonar images to obtain fishery resource distribution images of the same size as each of the side-scan sonar images, including: Performing fishery resource identification processing on each of the pre-processed side-scan sonar images to obtain a target side-scan sonar image and a blank fishery resource distribution image; The step of performing fishery resource identification processing on each of the pre-processed side-scan sonar images to obtain a target side-scan sonar image and a blank fishery resource distribution image includes: Assigning weight information to the target of interest in the preprocessed side-scan sonar image to obtain an attention weight image; According to the attention weight image, cropping an image block containing at least one target of interest in the preprocessed side scan sonar image; Extracting features from the image block, and fusing the features extracted from the image block with the corresponding attention weight image to obtain a fused feature map; Classifying the side-scan sonar image based on the fusion feature map to obtain the target side-scan sonar image and a blank fishery resource distribution image; Based on the biomass assessment model, biomass assessment is performed on each of the target side-scan sonar images, and a target fishery resource distribution image having the same size as each of the target side-scan sonar images is obtained; The target fishery resource distribution image and the blank fishery resource distribution image are determined as fishery resource distribution images, the target side-scan sonar image is the side-scan sonar image including fishery resources, and the blank fishery resource distribution image is obtained based on the side-scan sonar image not including fishery resources; the biomass assessment model includes an encoder and a decoder, the encoder and the decoder are connected through multiple layers of atrous convolution layers, the encoder includes an Inception structure and a maximum pooling layer, and the decoder includes multiple convolution-deconvolution modules; Determine the geographical coordinates of each pixel point on the fishery resource distribution image according to the geographical coordinates corresponding to each pixel point on the side scan sonar image, and determine the corresponding fishery resource distribution map based on the geographical coordinates of each pixel point on the fishery resource distribution image; All fishery resource distribution maps corresponding to the target waters are spliced ​​together to obtain a target fishery resource distribution map of the target waters.

2. The fishery resource assessment method according to claim 1, It is characterized in that The step of acquiring a plurality of echo data of the target water area collected by the side-scan sonar and generating a side-scan sonar image according to each of the echo data comprises: Acquiring in real time the echo data received by the side-scan sonar after it emits sound waves in the target waters; The echo data are quantified to obtain a side-scan sonar image of the sampling area corresponding to each echo data.

3. The fishery resource assessment method according to claim 1, It is characterized in that The preprocessing of each of the side scan sonar images comprises: Acquire the height of the towed fish of the side scan sonar from the bottom of the target water area based on the side scan sonar image; performing slant range correction on the side scan sonar image according to the height; Image enhancement processing is performed on the side-scan sonar image after slant range correction based on an image enhancement model.

4. A fishery resource assessment device, It is characterized in that include: An image generation module is used to obtain a plurality of echo data of the target water area collected by the side scan sonar, and generate a side scan sonar image according to each of the echo data; An image preprocessing module, used for preprocessing each of the side-scan sonar images, wherein the preprocessing includes slant range correction processing and side-scan sonar image enhancement processing; A biomass assessment module, for performing biomass assessment on each of the preprocessed side-scan sonar images based on a biomass assessment model, and obtaining fishery resource distribution images of the same size as each of the side-scan sonar images; The biomass assessment model is also used to perform fishery resource identification processing on each of the preprocessed side-scan sonar images to obtain a target side-scan sonar image and a blank fishery resource distribution image, including: assigning weight information to the target of interest in the preprocessed side-scan sonar image to obtain an attention weight image; cropping an image block containing at least one target of interest in the preprocessed side-scan sonar image according to the attention weight image; extracting features from the image block, and fusing the features extracted from the image block with the corresponding attention weight image to obtain a fused feature map; classifying the side-scan sonar image based on the fused feature map to obtain the target side-scan sonar image and a blank fishery resource distribution map. image; performing biomass assessment on each of the target side-scan sonar images based on the biomass assessment model, and obtaining a target fishery resource distribution image of the same size as each of the target side-scan sonar images; wherein the target fishery resource distribution image and the blank fishery resource distribution image are determined as the fishery resource distribution image, the target side-scan sonar image is the side-scan sonar image including fishery resources, and the blank fishery resource distribution image is obtained based on the side-scan sonar image excluding fishery resources; the biomass assessment model includes an encoder and a decoder, the encoder and the decoder are connected by multiple layers of atrous convolutional layers, the encoder includes an Inception structure and a maximum pooling layer, and the decoder includes multiple convolution-deconvolution modules; A distribution map generation module, used to determine the geographical coordinates of each pixel point on the corresponding fishery resource distribution image according to the geographical coordinates corresponding to each pixel point on the side scan sonar image, and determine the corresponding fishery resource distribution map based on the geographical coordinates of each pixel point on the fishery resource distribution image; The distribution map splicing module is used to splice all fishery resource distribution maps corresponding to the target water area to obtain the target fishery resource distribution map of the target water area.

5. A fishery resource assessment system, It is characterized in that It comprises an unmanned boat, a side-scan sonar, an unmanned boat control device and a fishery resource assessment device as claimed in claim 4; wherein, The unmanned ship includes a hull, a first attitude sensor installed on the hull, an ultrasonic sensor, a positioning module, a first communication module, a main control module and a driving module, wherein the first attitude sensor is used to obtain navigation information of the unmanned ship, and the navigation information includes ship speed, acceleration and angular velocity; the positioning module is used to obtain positioning information of the unmanned ship; the ultrasonic sensor is used to detect obstacles around the unmanned ship to obtain detection information; the first communication module is used to send the navigation information, the positioning information and the detection information, and is also used to receive route information and control instructions; the main control module is used to control the driving module to drive according to the route information and the control instructions based on the navigation information and positioning information of the unmanned ship; The side scan sonar comprises a towed fish, a tow cable, a deck unit and a second communication module, wherein the deck unit is arranged on the hull of the unmanned ship; the towed fish is placed in the target waters and connected to the deck unit and the unmanned ship through the tow cable; the second communication module is used to send the echo data collected by the towed fish and the positioning information to the fishery resource assessment device; The unmanned ship control device comprises an unmanned ship control module and a third communication module, wherein the unmanned ship control module is used to generate the route information and the control instruction, and the third communication module is used to receive the navigation information, the positioning information and the detection information, and is also used to issue the route information and the control instruction; The fishery resource assessment device also includes a fourth communication module for receiving the echo data and the positioning information.

6. The fishery resource assessment system according to claim 5, It is characterized in that Also includes: A remote controller is used to send remote control instructions to the unmanned ship.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the fishery resource assessment method as described in any one of claims 1 to 3 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the fishery resource assessment method according to any one of claims 1 to 3 is implemented.

Citation Information

Patent Citations

  • Classification method, system and device for fish fry

    CN109086800A

  • Fish identification method and device based on convolutional neural network

    CN110766013A

  • Submarine target positioning device based on side-scan sonar

    CN111522013A

  • Marine ranching seabed telemetering and identification method based on acoustic data

    CN112882042A

  • Channel coding blind identification method based on one-dimensional multi-input convolutional neural network

    CN113206808A