Underwater sonar image target detection method, device, electronic equipment and storage medium

Abstract

The present invention relates to an underwater sonar image target detection method, device, electronic device and storage medium, belonging to the field of image processing technology, wherein the underwater sonar image target detection method comprises: obtaining a plurality of continuous sonar image frames before the current moment, performing threshold segmentation on each frame of the plurality of continuous sonar image frames based on the maximum inter-class variance method, and performing expansion, connected domain division and connected domain screening on each frame of the image after threshold segmentation to obtain a target image set; determining the target contour coordinates in each frame of the image in the target image set, performing coordinate transformation on the target contour coordinates, and using a Kalman filter algorithm to fuse each frame of the image in the target image set after coordinate transformation to obtain the predicted sonar image at the current moment. The present invention improves the accuracy of sonar image detection while also reducing the cost of sonar image detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an underwater sonar image target detection method, device, electronic device and storage medium. Background Art

[0002] Underwater environment perception plays a very important role in the underwater task planning and collision avoidance of remotely operated vehicles (ROVs). In a complex underwater environment, the illumination is low, and the perception and display effect of visual images are poor. Under the same conditions, sonar devices can perceive the environment based on the transmission and reception of ultrasonic waves without visible light, are not easily interfered, and have a far detection range, making them more suitable for monitoring complex underwater environments. However, there are also some problems with sonar detection. The complex underwater environment causes a large amount of noise in sonar images, and these noises result in a large number of stray signal interferences in sonar images. In addition, the sonar itself also generates multipath noise and near-field noise, which cause great interference to sonar images.

[0003] Current sonar noise removal methods are costly and large in volume, making it difficult to apply them to small devices such as ROVs. Therefore, how to improve the accuracy of ROV sonar detection has become an urgent technical problem to be solved. Summary of the Invention

[0004] In view of this, it is necessary to provide an underwater sonar image target detection method, device, electronic device and storage medium to solve the problem of low accuracy of current ROV sonar detection.

[0005] To solve the above problems, the present invention provides an underwater sonar image target detection method, including:

[0006] Obtain a continuous multi-frame sonar image before the current moment, perform threshold segmentation on each frame of the continuous multi-frame sonar image based on the maximum inter-class variance method, and perform dilation, connected component division and connected component screening on each frame of the image after threshold segmentation to obtain a target image set;

[0007] Determine the target contour coordinates in each frame of the target image set, perform coordinate transformation on the target contour coordinates, and use the Kalman filter algorithm to fuse each frame of the target image set after coordinate transformation to obtain the predicted sonar image at the current moment.

[0008] In a possible implementation manner, the performing threshold segmentation on each frame of the continuous multi-frame sonar image based on the maximum inter-class variance method includes:

[0009] Determine the segmentation threshold based on the Otsu method, and segment the pixels of the target object and the background pixels in each frame of the continuous multi-frame sonar images.

[0010] In a possible implementation manner, the method further includes:

[0011] After completing the threshold segmentation of each frame of the continuous multi-frame sonar images, adjust the brightness of the background to 0 and adjust the brightness of the target object to 255.

[0012] In a possible implementation manner, the dilating, connected component division, and connected component screening of each frame of the image after threshold segmentation include:

[0013] Perform dilation and connected component division processing on each frame of the image after threshold segmentation;

[0014] Perform connected component screening based on the total number of pixels with brightness greater than 0 in the connected components divided from each frame of the image after threshold segmentation.

[0015] In a possible implementation manner, the performing connected component screening based on the total number of pixels with brightness greater than 0 in the connected components divided from each frame of the image after threshold segmentation includes:

[0016] Perform connected component screening based on the following formula:

[0017]

[0018] where I ccp represents the connected component selected from each frame of the image after threshold segmentation, represents the i-th connected component in each frame of the image after threshold segmentation, represents the total number of pixels with brightness greater than 0 in each frame of the image after threshold segmentation, represents the total number of pixels in each frame of the image after threshold segmentation, τ t represents the pixel ratio threshold, μ t represents the effective pixel number threshold, represents the first value of the pixel ratio threshold, represents the second value of the pixel ratio threshold, θ (i) represents the aspect ratio of the i-th connected component in each frame of the image after threshold segmentation, θ t represents the aspect ratio threshold of the connected component.

[0019] In a possible implementation manner, the determining the target contour coordinates in each frame of the target image set includes:

[0020] Obtain the lower boundary of the effective pixels of the connected components in each frame of the target image set;

[0021] Determine the target contour coordinates within each frame of the target image set based on the lower boundary of the effective pixels of the connected regions in each frame of the target image set.

[0022] In a possible implementation manner, the method further includes:

[0023] Before fusing each frame of the target image set after coordinate transformation using the Kalman filtering algorithm, perform multipath noise elimination on each frame of the target image set.

[0024] The present invention also provides an underwater sonar image target detection device, including:

[0025] An acquisition module, configured to acquire a continuous plurality of frames of sonar images before the current moment, perform threshold segmentation on each frame of the continuous plurality of frames of sonar images based on the Otsu method, and perform dilation, connected region division, and connected region screening on each frame of the image after threshold segmentation to obtain a target image set;

[0026] A determination module, configured to determine the target contour coordinates within each frame of the target image set, perform coordinate transformation on the target contour coordinates, and fuse each frame of the target image set after coordinate transformation using the Kalman filtering algorithm to obtain the predicted sonar image at the current moment.

[0027] The present invention also provides an electronic device, including a memory and a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the above-mentioned underwater sonar image target detection method is implemented.

[0028] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned underwater sonar image target detection method is implemented.

[0029] The beneficial effects of the present invention are as follows: An underwater sonar image target detection method, device, electronic device, and storage medium provided by the present invention perform threshold segmentation on sonar images through OSTU to obtain the contours of target objects, and finally fuse the contours of target objects in historical sonar images to obtain the predicted sonar image at the current moment, improving the accuracy of sonar image detection and reducing the cost of sonar image detection at the same time. Description of the Drawings

[0030] Figure 1 It is a schematic flowchart of an embodiment of the underwater sonar image target detection method provided by the present invention;

[0031] Figure 2Schematic flowchart of an embodiment of the underwater sonar image processing process provided by the present invention;

[0032] Figure 3 Schematic structural diagram of an embodiment of the underwater sonar image target detection device provided by the present invention;

[0033] Figure 4 Schematic structural diagram of an embodiment of the electronic device provided by the present invention. Detailed implementation manners

[0034] The preferred embodiments of the present invention will be specifically described below with reference to the accompanying drawings. The accompanying drawings form a part of this application and are used together with the embodiments of the present invention to explain the principles of the present invention, rather than to limit the scope of the present invention.

[0035] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0036] In the description of the present invention, referring to "embodiment" means that the specific features, structures or characteristics described in connection with the embodiment may be included in at least one embodiment of the present invention. The phrase appears at various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the described embodiments may be combined with other embodiments.

[0037] Currently, most sonar image detection methods need to train image data through a convolutional neural network to obtain better results. However, this requires obtaining a large amount of image sample data through experiments and performing manual labeling, and only objects with sufficient valid samples and corresponding angles can produce better results, while objects and angles that have not been trained cannot be recognized. In addition, the existing methods do not consider the multipath noise and near-field noise that have a great impact on multi-beam sonar images. Therefore, the present invention proposes an information acquisition method for underwater acoustic images based on probability theory, which eliminates multipath noise and near-field noise through the Otsu method and connected component processing, and then uses Kalman filtering to obtain a stable contour boundary of the structure from multiple frames of images.

[0038] The following will separately elaborate on the specific embodiments in detail:

[0039] A specific embodiment of the present invention discloses an underwater sonar image target detection method, combined with Figure 1 seenFigure 1 This is a schematic flowchart of an embodiment of the underwater sonar image target detection method provided by the present invention, including step S101 and step S102, where:

[0040] In step S101, obtain multiple consecutive frames of sonar images before the current moment, perform threshold segmentation on each frame of the consecutive multiple frames of sonar images based on the maximum inter-class variance method, and perform dilation, connected component division, and connected component screening on each frame of the image after threshold segmentation to obtain a target image set;

[0041] In step S102, determine the target contour coordinates within each frame of the target image set, perform coordinate transformation on the target contour coordinates, and use the Kalman filtering algorithm to fuse each frame of the target image set after coordinate transformation to obtain the predicted sonar image at the current moment.

[0042] During implementation, first, multiple consecutive M frames of sonar images before the current moment can be obtained, and then each frame of the image can be subjected to threshold segmentation according to OSTU. The target object and the noise have different reflection characteristics, resulting in different intensities (gray levels) in the image from the background. The OTSU method can help determine a suitable threshold to separate the pixels of the target object from the background, making it easier to detect the target. After completing the threshold segmentation of each frame of the image, dilation, connected component division, and connected component screening can be performed on each frame of the image after threshold segmentation to obtain a target image set.

[0043] After obtaining the target image set, the target contour coordinates within each frame of the target image set can be determined, then the target contour coordinates can be subjected to coordinate transformation (such as transforming to the Cartesian coordinate system), and then the Kalman filtering algorithm is used to fuse each frame of the target image set after coordinate transformation. Finally, the predicted sonar image at the current moment can be obtained.

[0044] The underwater sonar image target detection method provided by the present invention can be applied to the field of sonar detection of underwater robots, and can also be applied to other fields that require sonar image detection. The present invention does not make specific limitations in this regard.

[0045] Compared with the prior art, the underwater sonar image target detection method provided in this embodiment performs threshold segmentation on the sonar image through OSTU to obtain the contour of the target object, and finally fuses the contours of the target objects in the historical sonar images to obtain the predicted sonar image at the current moment, improving the accuracy of sonar image detection while reducing the cost of sonar image detection.

[0046] Exemplarily, the performing threshold segmentation on each frame of the consecutive multiple frames of sonar images based on the maximum inter-class variance method includes:

[0047] Determine the segmentation threshold based on the Otsu method, and segment the pixels of the target object and the background pixels in each frame of the continuous multi-frame sonar images.

[0048] Specifically, when performing threshold segmentation on each frame of the continuous multi-frame sonar images according to Otsu, the pixels of the target object and the background pixels in each frame of the continuous multi-frame sonar images can be segmented according to Otsu.

[0049] Exemplarily, the method further includes:

[0050] After completing the threshold segmentation of each frame of the continuous multi-frame sonar images, adjust the brightness of the background to 0 and adjust the brightness of the target object to 255.

[0051] Specifically, after completing the threshold segmentation of each frame of the continuous multi-frame sonar images, image enhancement can also be performed on the threshold-segmented images, that is, adjust the brightness of the background to 0 and adjust the brightness of the target object to 255.

[0052] Exemplarily, the dilation, connected component division, and connected component screening of each frame of the threshold-segmented image include:

[0053] Perform dilation and connected component division processing on each frame of the threshold-segmented image;

[0054] Based on the total number of pixel points with brightness greater than 0 in the connected components divided from each frame of the threshold-segmented image, perform connected component screening.

[0055] Specifically, when performing dilation, connected component division, and connected component screening on each frame of the threshold-segmented image, first, dilation and connected component division processing can be performed on each frame of the threshold-segmented image, and then connected component screening is performed according to the total number of pixel points with brightness greater than 0 in the connected components divided from each frame of the threshold-segmented image.

[0056] Exemplarily, the connected component screening based on the total number of pixel points with brightness greater than 0 in the connected components divided from each frame of the threshold-segmented image includes:

[0057] Perform connected component screening based on the following formula:

[0058]

[0059] where I ccp represents the connected component selected from each frame of the threshold-segmented image, represents the i-th connected component in each frame of the threshold-segmented image, represents the total number of pixel points with brightness greater than 0 in each frame of the threshold-segmented image, Represents the total number of pixel points in each frame of the image after threshold segmentation, τ t Represents the threshold of pixel point ratio, μ t Represents the threshold of the number of effective pixel points Represents the first value of the threshold of pixel point ratio Represents the second value of the threshold of pixel point ratio, θ (i) Represents the aspect ratio of the i-th connected component in each frame of the image after threshold segmentation, θ t Represents the threshold of the aspect ratio of the connected component

[0060] Specifically, when screening the connected components according to the total number of pixel points with luminance greater than 0 in the connected components divided from each frame of the image after threshold segmentation, the connected components can be screened according to the above formula

[0061] Exemplarily, the determining the target contour coordinates in each frame of the target image set includes:

[0062] Obtaining the lower boundary of the effective pixels of the connected components in each frame of the target image set

[0063] Based on the lower boundary of the effective pixels of the connected components in each frame of the target image set, determining the target contour coordinates in each frame of the target image set

[0064] Specifically, when determining the target contour coordinates in each frame of the target image set, first, the lower boundary of the effective pixels of the connected components in each frame of the target image set can be obtained, and then, based on the lower boundary of the effective pixels of the connected components in each frame of the target image set, the target contour coordinates in each frame of the target image set can be determined

[0065] For example, the lower boundary of the effective pixels of the connected components can be determined according to the bright pixel points in the connected components of each frame of the image, and further, the target contour coordinates in each frame of the target image set can be determined

[0066] Exemplarily, the method further includes:

[0067] Before fusing each frame of the target image set after coordinate transformation using the Kalman filtering algorithm, performing multipath noise elimination on each frame of the target image set

[0068] Specifically, before fusing each frame of the target image set after coordinate transformation using the Kalman filtering algorithm, multipath noise elimination can also be performed on each frame of the target image set. For example, the pixel points of the effective lower boundary can be set as the lower bounds of the continuous same row and eliminated

[0069] The following combines a specific application scenario to better illustrate the technical solution of the present invention:

[0070] Combined Figure 2 seen Figure 2 is a schematic flowchart of an embodiment of the underwater sonar image processing process provided by the present invention. The core idea of the underwater sonar image target detection method provided by the present invention is as follows:

[0071] 1. Design a method for extracting the region of interest (ROI) of an image based on image morphology.

[0072] First, perform threshold segmentation on the sonar image using OTSU to retain the key information in the image. The target object and the noise have different reflection characteristics, resulting in different intensities (gray levels) in the image from the background. The OTSU method can help determine a suitable threshold to separate the pixels of the target object from the background, making it easier to detect the target. OTSU can automatically determine the optimal segmentation threshold according to the histogram of the image, making the sonar image processing more convenient and automated. For the sonar image processed by OTSU, all the background pixel values are changed to 0, and all the foreground pixel values are changed to brightness 255, so as to realize image enhancement and obtain object map I OT .

[0073] Perform dilation on I OT and divide the connected regions:

[0074]

[0075] where I cc is the image after processing I OT , f cc represents the operation of dilating and dividing the connected regions of the picture. Through the connected region processing, the target pixels of the image can be partitioned to obtain multiple sub-connected regions and each connected region is represented by a circumscribed rectangle for its circumscribed region.

[0076] Perform judgment and selection for all the connected regions in I cc , retain those that meet the requirements of the following formula, and form a new picture I . ccp .

[0077]

[0078] where is the total number of bright pixel points inside the rectangle, is the total number of all pixel points of the rectangle, τ t is the pixel ratio threshold, and the rectangle and the pixel points inside it that are less than this threshold are retained, μ tIt is the number threshold of effective pixel points, representing the minimum number of pixel points required within a valid ROI.

[0079] τ t It has two different values, which are respectively applied to the large aspect ratio area and the small aspect ratio area. That is, for the ROI with a relatively large aspect ratio, a smaller pixel occupancy threshold is adopted. Conversely, a larger threshold is adopted. This is because when the aspect ratio is too large, the possibility of the multi-path effect noise of the sonar in this area increases. By adopting a smaller pixel occupancy threshold, it can be used to eliminate the influence of multi-path effect noise. The value judgment formula of τ t is shown as follows:

[0080]

[0081] In the formula, is the aspect ratio of the i-th rectangular area, that is is the width of the i-th rectangular area, is the height of the i-th rectangular area.

[0082] 2. Design the extraction method of the key effective contour in the ROI.

[0083] On the basis of I ccp obtain the lower boundary of all effective pixels in each rectangular area to obtain the contour on the side where the obstacle is close to the sonar, that is:

[0084]

[0085] In the formula, f EB is the function to obtain the lower boundary in rectangle i, and its pseudo-code is:

[0086] Read the i-th rectangular area

[0087]

[0088] Read the row numbers of all bright pixel points in column j and take the maximum value of the row numbers, and save it in the j-th row of the two-dimensional array ;

[0089] All In the area, only The area of is the bright pixel point, which constitutes the lower boundary graph I eb .

[0090] In the above pseudo-code, represents the width of the i-th ROI, is a two-dimensional list, representing the maximum row number of all bright pixel points in the j-th column of the i-th ROI.

[0091] The present invention also designs an image optimization method for eliminating multipath noise. The specific process is as follows:

[0092]

[0093] f ET To judge The function method for the number of pixels in the set of pixel points of the effective lower boundary in a continuous same row, and its pseudo-code is as follows:

[0094]

[0095] Read in sequentially That is, the true row number of the effective lower bound point in the j-th column of the rectangular area ;

[0096] Then

[0097] Then the count of the k-th row

[0098] Then That is, delete this continuous lower bound.

[0099] In the above pseudo-code, represents the number of times that the row numbers of the effective lower bounds are continuous for k starting from the j-th column in the rectangular area , N t h is the threshold of the number of saved points. The s in

[0100] 3. Design a continuous frame image fusion processing method using Kalman filtering.

[0101] Convert the pixel points in all two-dimensional images from the polar coordinate system to the Cartesian coordinate system, convert the continuous M sonar images forward from the current moment to the coordinate system of the current moment image, and use the Kalman filtering algorithm to fuse these M frames of sonar pictures to obtain the predicted sonar image of the current frame.

[0102] During the continuous movement of the ROV, the formed sonar images have a continuous spatial relationship, and this spatial relationship is more intuitive in the Cartesian coordinate image. Therefore, as shown in the formula, the lower boundary map I et in the polar coordinate system is converted to the Cartesian coordinate system

[0103]

[0104] In the formula, is a Cartesian coordinate image, and P2C is the conversion relationship from polar coordinates to Cartesian coordinates, which is included in the OpenCV library.

[0105] Since the ROV is moving in real time, the coordinate position and orientation of the structure in the ocean relative to the sonar are changing in real time. Take M consecutive sonar images and convert them to the coordinate system at the current real-time moment through the following equation:

[0106]

[0107] In the formula, R i2n represents the rotation matrix relative to the current frame during the conversion of the i-th frame sonar image from the time before the current moment. It is assumed in the present invention that the ROV uses sonar for forward-looking operations, that is, the degree of freedom in the direction perpendicular to the water surface of the ROV is ignored. Therefore, R i2n only retains the two dimensions in the horizontal plane, and t i2n is the translation vector of the ROV relative to the current frame at the i-th previous frame, and only the two dimensions in the horizontal plane are considered. The above all come from the odometer information and are obtained by collecting through the Inertial Measurement Unit (IMU) on the ROV.

[0108] For the M-frame sonar images converted to the current coordinate system, Kalman filtering is used to obtain the fused image, and the image is excluded by a threshold to eliminate the influence of noise. The following will be abbreviated as z k The image predicted from the (k - 1)-th frame to the k-th frame is called The image obtained by fusing this image with z k is The specific process is as follows:

[0109]

[0110] For i = M - 1…0

[0111] k = M - i…

[0112]

[0113]

[0114]

[0115]

[0116] P k =(I - K k H k )P k∣k-1

[0117] In the above formula, F k is the state transition matrix, H k is the process noise covariance matrix, Q k is the observation matrix, R k is the measurement noise covariance matrix.

[0118] Finally, when i = 0, the prediction obtained is the predicted sonar image of the current frame obtained by fusing continuous M frames.

[0119] An embodiment of the present invention also provides an underwater sonar image target detection device. As shown in Figure 3 view, Figure 3 FIG. is a schematic structural diagram of an embodiment of the underwater sonar image target detection device provided by the present invention. The underwater sonar image target detection device 300 includes:

[0120] An acquisition module 301, configured to acquire a plurality of consecutive sonar images before the current moment, perform threshold segmentation on each of the plurality of consecutive sonar images based on the maximum inter-class variance method, and perform dilation, connected component division, and connected component screening on each of the images after threshold segmentation to obtain a target image set;

[0121] A determination module 302, configured to determine the target contour coordinates in each image of the target image set, perform coordinate transformation on the target contour coordinates, and fuse each image in the target image set after coordinate transformation using the Kalman filtering algorithm to obtain the predicted sonar image at the current moment.

[0122] For a more specific implementation manner of each module of the underwater sonar image target detection device, reference may be made to the description of the above-mentioned underwater sonar image target detection method, and it has similar beneficial effects, which will not be elaborated here.

[0123] It should be noted that the underwater sonar image target detection device may be set in the ROV or in other underwater sonar detection devices, and the present invention does not make specific limitations thereto.

[0124] An embodiment of the present invention also provides an electronic device. As shown in Figure 4 view, Figure 4 FIG. is a schematic structural diagram of an embodiment of the electronic device provided by the present invention. The electronic device 400 includes a processor 401, a memory 402, and a computer program stored on the memory 402 and executable on the processor 401. When the processor 401 executes the program, the above-mentioned underwater sonar image target detection method is implemented.

[0125] As a preferred embodiment, the above-mentioned electronic device 400 further includes a display 403, configured to display the processor 401 executing the above-mentioned underwater sonar image target detection method.

[0126] Exemplarily, a computer program can be divided into one or more modules / units. One or more modules / units are stored in the memory 402 and executed by the processor 401 to implement the present invention. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the electronic device 400. For example, the computer program can be divided into the acquisition module 301 and the determination module 302 in the above embodiments, and the specific functions of each module are as described above, which will not be elaborated here one by one.

[0127] The electronic device 400 can be a desktop computer, a notebook, a palm computer, a smart phone or other devices with an adjustable camera module.

[0128] Among them, the processor 401 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 401 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0129] Among them, the memory 402 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. Among them, the memory 402 is used to store a program, and after receiving an execution instruction, the processor 401 executes the program. The method defined by the process disclosed in any embodiment of the foregoing embodiments of the present invention can be applied to the processor 401 or implemented by the processor 401.

[0130] Among them, the display 403 can be an LCD display screen or an LED display screen. For example, the display screen on a mobile phone.

[0131] It can be understood that Figure 4 The structure shown is only a schematic structural diagram of the electronic device 400, and the electronic device 400 may further include more or fewer components than Figure 4 shown. Figure 4 Each component shown in can be implemented by hardware, software, or a combination thereof.

[0132] According to the computer-readable storage medium and the electronic device provided in the foregoing embodiments of the present invention, it can be implemented with reference to the content specifically described for implementing the underwater sonar image target detection method according to the present invention, and has beneficial effects similar to those of the underwater sonar image target detection method described above, which will not be elaborated here.

[0133] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the underwater sonar image target detection method described above is implemented.

[0134] Generally speaking, the computer instructions for implementing the method of the present invention can be carried by any combination of one or more computer-readable storage media. A non-transitory computer-readable storage medium may include any computer-readable medium except for the signal itself propagating temporarily.

[0135] A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0136] Computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. In particular, the Python language suitable for neural network computing and platform frameworks based on TensorFlow, PyTorch, etc. can be used. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0137] Those skilled in the art can understand that all or part of the processes for implementing the methods of the above embodiments can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium is a disk, an optical disk, a read-only memory, or a random access memory, etc.

[0138] The present invention discloses an underwater sonar image target detection method, device, electronic device, and storage medium. By performing threshold segmentation on the sonar image through OSTU to obtain the contour of the target object, and finally fusing the contours of the target object in the historical sonar images to obtain the predicted sonar image at the current moment, it improves the accuracy of sonar image detection while reducing the cost of sonar image detection.

[0139] The above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for underwater sonar image target detection, characterized in that: include: Acquire a plurality of continuous sonar image frames before the current moment, perform threshold segmentation on each frame of the plurality of continuous sonar image frames based on the maximum inter-class variance method, and perform expansion, connected domain division and connected domain screening on each frame of the image after the threshold segmentation to obtain a target image set; Determine the target contour coordinates in each frame of the target image set, perform coordinate transformation on the target contour coordinates, and use a Kalman filter algorithm to fuse each frame of the target image set after the coordinate transformation to obtain the predicted sonar image at the current moment; The step of dilating, dividing and screening connected domains of each frame image after the threshold segmentation includes: Performing dilation and connected domain division processing on each frame of the image after the threshold segmentation; Screening the connected domain based on the total number of pixels with brightness greater than 0 in the connected domain divided from each frame of the image after the threshold segmentation; The connected domain screening based on the total number of pixel points with brightness greater than 0 in the connected domain divided from each frame of the image after the threshold segmentation includes: Connected domain screening is performed based on the following formula: in, represents the connected domain selected from each frame of the image after threshold segmentation, Represents the first connected domains, Indicates the pixel ratio threshold, Indicates the threshold value of the number of valid pixels. Indicates the first value of the pixel ratio threshold, Indicates the second value of the pixel ratio threshold, Represents the first The aspect ratio of a connected domain, represents the connected domain aspect ratio threshold, for The total number of bright pixels inside the rectangle, for The total number of pixels in the rectangle.

2. The underwater sonar image target detection method according to claim 1, characterized in that: The method of performing threshold segmentation on each frame of the continuous multiple frames of sonar images based on the maximum inter-class variance method includes: The segmentation threshold is determined based on the maximum inter-class variance method, and the pixels of the target object and the background pixels in each frame of the continuous multiple frames of sonar images are segmented.

3. The underwater sonar image target detection method according to claim 2, characterized in that: The method further comprises: After completing the threshold segmentation of each frame of the continuous multi-frame sonar image, the brightness of the background is adjusted to 0, and the brightness of the target object is adjusted to 255.

4. The underwater sonar image target detection method according to claim 1, characterized in that: Determining the target contour coordinates in each frame of the target image set includes: Obtaining the lower boundary of valid pixels of the connected domain in each frame of the target image set; Based on the lower boundary of the valid pixels of the connected domain in each frame of the target image set, the coordinates of the target contour in each frame of the target image set are determined.

5. The underwater sonar image target detection method according to any one of claims 1 to 4, characterized in that: The method further comprises: Before using the Kalman filter algorithm to fuse each frame of the target image set after coordinate transformation, multipath noise elimination is performed on each frame of the target image set.

6. An underwater sonar image target detection device, characterized in that: include: An acquisition module is used to acquire a plurality of continuous sonar image frames before the current moment, perform threshold segmentation on each frame of the plurality of continuous sonar image frames based on the maximum inter-class variance method, and perform expansion, connected domain division and connected domain screening on each frame of the image after the threshold segmentation to obtain a target image set; A determination module is used to determine the target contour coordinates in each frame of the target image set, and to perform coordinate transformation on the target contour coordinates, and to fuse each frame of the target image set after the coordinate transformation using a Kalman filter algorithm to obtain the predicted sonar image at the current moment; The step of dilating, dividing and screening connected domains of each frame image after the threshold segmentation includes: Performing dilation and connected domain division processing on each frame of the image after the threshold segmentation; Screening the connected domain based on the total number of pixels with brightness greater than 0 in the connected domain divided from each frame of the image after the threshold segmentation; The connected domain screening based on the total number of pixel points with brightness greater than 0 in the connected domain divided from each frame of the image after the threshold segmentation includes: Connected domain screening is performed based on the following formula: in, represents the connected domain selected from each frame of the image after threshold segmentation, Represents the first connected domains, Indicates the pixel ratio threshold, Indicates the threshold value of the number of valid pixels. Indicates the first value of the pixel ratio threshold, Indicates the second value of the pixel ratio threshold, Represents the first The aspect ratio of a connected domain, represents the connected domain aspect ratio threshold, for The total number of bright pixels inside the rectangle, for The total number of pixels in the rectangle.

7. An electronic device, characterized in that: The invention comprises a memory and a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the underwater sonar image target detection method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the underwater sonar image target detection method according to any one of claims 1 to 5 is implemented.

Images