Sonar image visualization processing method, system and computing device
By enhancing, normalizing and pseudo-color mapping of the beam map of the sonar image, combined with the significance map weighted fusion, the complexity and inconvenience of visualization of the forward-view sonar image is solved, and the sonar is more easily observed. Span graph and small object detection.
Patent Information
- Application Number
- CN202411111415.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-14
AI Technical Summary
The existing forward-view sonar image visualization methods are complex and inconvenient to operation, require high professional knowledge and experience, and it takes a long time to read the original image data.
By obtaining the beammap of single-frame data of sonar, enhancement, normalization, pseudo-color mapping and fan transformation are carried out, combined with global and local significance graph weighted fusion, an easy-to-observe sonar fan diagram is generated to realize beammap-to-sound Visual transformation of nanospan graphs.
The visualization effect of sonar images is improved, making the image easier to observe on the display device, reducing the need for brightness adjustment, improving the visual processing speed, and real-time detection of small goals is achieved.
Smart Images

Figure CN118628423B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of underwater detection technology, and in particular to a sonar image visualization processing method, system and computing device. Background Art
[0002] Sonar (e.g., forward-looking sonar) is a sonar system installed on the front of a ship, submarine, or underwater vehicle to detect the front. It can provide real-time, high-resolution underwater images and information of the front area, which makes forward-looking sonar widely used in various vehicles. However, the raw image data obtained by forward-looking sonar is often complex and difficult to read directly. Not only does it require high professional knowledge and experience from the user, but reading the raw image data of forward-looking sonar often takes a long time and a lot of effort. Therefore, how to realize the visualization of sonar images of forward-looking sonar is a direction of continuous research in the industry. Existing methods for visualizing sonar images of forward-looking sonar mainly include fan chart conversion, brightness adjustment, and pseudo-color map mapping. When in use, it is necessary to adjust the visualization parameters such as brightness and contrast according to the observation needs. The user needs to have certain sonar control experience, and the operation is inconvenient. Summary of the invention
[0003] An object of the present invention is to provide a sonar image visualization processing method, system and computing device to solve the problems existing in the existing sonar image visualization method of sonar.
[0004] In order to achieve the above object, the technical solution adopted by the present invention is: a sonar image visualization processing method, comprising the following steps:
[0005] S1, obtain the beam pattern and corresponding sampling parameters of a single frame of sonar data;
[0006] S2, enhancing the beam pattern to obtain an enhanced image of the beam pattern;
[0007] S3, normalizing the enhanced image of the beam pattern to an 8-bit numerical space to obtain an enhanced image of the beam pattern of an 8-bit single channel;
[0008] S4, mapping the enhanced image of the 8-bit single-channel beam pattern into 8-bit three-channel image data through a pseudo color mapping map;
[0009] S5, calculating the pixel position of the fan diagram corresponding to the pixel position of the beam diagram according to the sampling parameters, and storing the memory offset position of the enhanced image of the 8-bit three-channel beam diagram using the fan diagram coordinates as an index to generate a sonar fan conversion map;
[0010] S6, reading data from the enhanced image of the 8-bit three-channel beam diagram through the memory offset in the sonar sector conversion map, and copying it to the corresponding position of the sector diagram to obtain the sonar sector diagram.
[0011] As a feasible example of the present invention, step S2 includes the following steps:
[0012] S2.1, calculate the global contrast level of the pixels in the image data of the beam map, and perform differential enhancement on the contrast of the pixels and the global level, which is recorded as a global saliency map;
[0013] S2.2, calculate the local contrast level in the image data of the beam map, and make a difference between the local mean and the global level to enhance the local overall contrast, which is recorded as a local saliency map;
[0014] S2.3, setting an object detection convolution kernel group, wherein the object detection convolution kernel group includes an object buffer part and a background estimation part, using the object detection convolution kernel group to perform a convolution operation on the image data of the beam map, and taking the maximum value between groups at each pixel position in the obtained convolution image group to obtain an object saliency map;
[0015] S2.4, normalizing the global saliency map, the local saliency map, and the object saliency map to unify the numerical domain;
[0016] S2.5, weighted fusion of the global saliency map, the local saliency map and the object saliency map is performed to obtain an enhanced image of the beam map.
[0017] As a feasible example of the present invention, the enhanced image of the beam pattern is subjected to brightness curve control, wherein the formula for controlling the brightness curve of the enhanced image of the beam pattern is: , where the parameters Represents the enhanced image after curve control, parameter The enhanced image of the beam pattern before curve control, parameter Represents the curve coefficient.
[0018] As a feasible example of the present invention, in step S3, the formula for normalizing the enhanced image of the beam pattern to an 8-bit numerical space is: , where the parameters Represents the enhanced image of the normalized 8-bit single-channel beam diagram, with parameters The enhanced image of the beam pattern before curve control, parameter and The maximum and minimum pixel values of the enhanced image representing the beam pattern.
[0019] According to another aspect of the present invention, the present invention further provides a sonar image visualization processing system, which comprises:
[0020] An acquisition unit, used to acquire a beam pattern of a single frame of sonar data and corresponding sampling parameters;
[0021] an enhancement unit, configured to enhance the beam pattern to obtain an enhanced image of the beam pattern;
[0022] A normalization unit, used for normalizing the enhanced image of the beam pattern to an 8-bit numerical space to obtain an enhanced image of the beam pattern of an 8-bit single channel;
[0023] A mapping unit, used for mapping an 8-bit single-channel beam pattern enhancement image into 8-bit three-channel image data through a pseudo-color mapping map;
[0024] A generating unit, used to calculate the pixel position of the fan diagram corresponding to the pixel position of the beam diagram according to the sampling parameters, store the memory offset position of the enhanced image of the 8-bit three-channel beam diagram with the fan diagram coordinates as the index, and generate a sonar fan conversion map;
[0025] The visualization unit is used to read data from the enhanced image of the 8-bit three-channel beam diagram through the memory offset in the sonar sector conversion map, and copy it to the corresponding position of the sector diagram to obtain the sonar sector diagram.
[0026] As a feasible example of the present invention, the enhancement unit further includes:
[0027] The global saliency module is used to calculate the global contrast level of pixels in the image data of the beam map, and to perform differential enhancement on the contrast of pixels with the global level to obtain a global saliency map;
[0028] The local saliency module is used to calculate the local contrast level in the image data of the beam map, and the local mean is differentiated from the global level to enhance the local overall contrast and obtain a local saliency map;
[0029] An object saliency module is used to set an object detection convolution kernel group, use the object detection convolution kernel group to perform a convolution operation on the image data of the beam map, and take the maximum value of the component at each pixel position in the obtained convolution image group to obtain an object saliency map;
[0030] A numerical domain unification module is used to normalize the global saliency map, the local saliency map, and the object saliency map to unify the numerical domain;
[0031] The weighted superposition module is used to perform weighted fusion of the global saliency map, the local saliency map and the object saliency map to obtain an enhanced image of the beam map.
[0032] As a feasible example of the present invention, the enhancement unit includes a brightness control module, which is used to perform brightness curve control on the enhanced image of the beam pattern, wherein the formula for performing brightness curve control on the enhanced image of the beam pattern is: , where the parameters Represents the enhanced image after curve control, parameter The enhanced image of the beam pattern before curve control, parameter Represents the curve coefficient.
[0033] As a feasible example of the present invention, the normalization unit normalizes the enhanced image of the beam pattern to an 8-bit numerical space by the formula: , where the parameters Represents the enhanced image of the normalized 8-bit single-channel beam diagram, with parameters The enhanced image of the beam pattern before curve control, parameter and The maximum and minimum pixel values of the enhanced image representing the beam pattern.
[0034] As a feasible example of the present invention, the visualization processing system includes a small target detection unit, which is used to detect small targets in the beam pattern in real time.
[0035] According to another aspect of the present invention, the present invention further provides a computing device, comprising a processor and a memory coupled to the processor, wherein the memory is used to store codes, and the processor is configured to execute the codes. When the codes are executed, the computing device executes a visualization processing method of the following steps, wherein the visualization processing method comprises the following steps:
[0036] S1, obtain the beam pattern and corresponding sampling parameters of a single frame of sonar data;
[0037] S2, enhancing the beam pattern to obtain an enhanced image of the beam pattern;
[0038] S3, normalizing the enhanced image of the beam pattern to an 8-bit numerical space to obtain an enhanced image of the beam pattern of an 8-bit single channel;
[0039] S4, mapping the enhanced image of the 8-bit single-channel beam pattern into 8-bit three-channel image data through a pseudo color mapping map;
[0040] S5, calculating the pixel position of the fan diagram corresponding to the pixel position of the beam diagram according to the sampling parameters, and storing the memory offset position of the enhanced image of the 8-bit three-channel beam diagram using the fan diagram coordinates as an index to generate a sonar fan conversion map;
[0041] S6, reading data from the enhanced image of the 8-bit three-channel beam diagram through the memory offset in the sonar sector conversion map, and copying it to the corresponding position of the sector diagram to obtain the sonar sector diagram.
[0042] Compared with the prior art, the present invention has at least the following beneficial effects:
[0043] First, the visualization processing method includes the step of enhancing the sonar beam pattern so that the sonar sector diagram (i.e., the sonar image finally presented) is easier to observe on the display device;
[0044] Secondly, the visualization processing method includes the step of normalizing the enhanced image of the beam pattern to an 8-bit numerical space to obtain an 8-bit single-channel enhanced image of the beam pattern, which enables effective image visualization output to be obtained without adjusting the brightness;
[0045] Third, the visualization processing method can realize the visualization conversion of beam pattern to sonar sector pattern. In each frame of data, it is only necessary to access the memory address according to the sonar sector conversion mapping map without coordinate calculation, which is conducive to improving the visualization processing speed of beam pattern to sonar image.
[0046] Other beneficial effects of the present invention will be further illustrated in the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 It is a flow chart of a sonar image visualization processing method according to a preferred embodiment of the present invention.
[0048] Figure 2 Schematic diagram of an object convolution kernel group according to the above preferred embodiment of the present invention.
[0049] Figure 3 It is a pseudo color mapping diagram according to the above preferred embodiment of the present invention.
[0050] Figure 4 It is a flow chart of a sonar image visualization processing method according to another preferred embodiment of the present invention.
[0051] Figure 5 It is a block diagram of a sonar image visualization processing system according to a preferred embodiment of the present invention.
[0052] Figure 6 It is a block diagram of a computing device according to a preferred embodiment of the present invention.
[0053] In the figure:
[0054] 10. Get unit;
[0055] 20. Enhancement unit; 21. Global saliency module; 22. Local saliency module; 23. Object saliency module; 24. Numerical domain unification module; 25. Weighted superposition module;
[0056] 30. Normalization unit;
[0057] 40. Mapping unit;
[0058] 50. Generate unit;
[0059] 60. Visualization unit;
[0060] 70. Small target detection unit;
[0061] 600, computing device; 610, processor; 620, memory; 630, transceiver. DETAILED DESCRIPTION
[0062] Before describing in detail any embodiment of the present invention, it should be understood that the present invention is not limited in its application to the construction and arrangement details of the components described below or illustrated in the following figures. The present invention can have other embodiments and can be practiced or carried out in various ways. In addition, it should be understood that the words and terms used here are for descriptive purposes and should not be considered restrictive. The use of "including" or "having" and its variations herein is intended to cover the items and their equivalents and additional items displayed below. Unless otherwise specified or limited, the terms "install", "connect", "support" and "couple" and their variations are widely used and cover direct installation and indirect installation, connection, support and connection. In addition, "connect" and "couple" are not limited to physical or mechanical connections or connections.
[0063] Furthermore, on the first aspect, in the disclosure of the present invention, the terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore the above terms cannot be understood as limitations on the present invention; on the second aspect, the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" cannot be understood as a limitation on the quantity.
[0064] Attached to the specification of the present invention Figures 1 to 3A sonar image visualization processing method according to a preferred embodiment of the present invention will be disclosed and explained in the following description. The present invention takes the visualization processing method for the sonar image of the forward-looking sonar as an example to disclose and explain the content of the visualization processing method, wherein the visualization processing method includes the following steps.
[0065] S1, raw data acquisition of forward-looking sonar: obtaining 16-bit stored sonar beam diagram and sampling parameters such as distance sampling number, beam sampling number, and sonar opening angle through the sonar communication protocol. In a specific example of the visualization processing method of the present invention, the visualization processing method obtains a set of continuous raw data through the sonar communication protocol, and the raw data of the forward-looking sonar includes the beam diagram of a single frame of forward-looking sonar data, the sampling number (i.e., the height of the raw image data), the beam number (i.e., the width of the raw image data), the sonar opening angle and other information, wherein the beam diagram of the single frame of data contained in the raw data is a single channel.
[0066] S2, image enhancement of the forward-looking sonar beam pattern: contrast enhancement, background clutter suppression, brightness curve control and other operations are performed on the forward-looking sonar beam pattern to make the sonar sector diagram easier to observe on the display device.
[0067] Specifically, the step of image enhancement of the beam pattern of the forward-looking sonar in step S2 includes: S2-1, calculating the contrast level of pixels in the image data of the beam pattern in the global world, performing differential enhancement of the pixel contrast between the pixel and the global level, and recording it as a global saliency map; S2-2, calculating the local contrast level of pixels in the image data of the beam pattern, performing differential enhancement of the local overall contrast between the local mean and the global level, and recording it as a local saliency map; S2-3, setting an object detection convolution kernel group, the object detection convolution kernel group includes an object buffer part and a background estimation part, using the object detection convolution kernel group to perform a convolution operation on the original image data of the sonar image, and taking the maximum value of the component at each pixel position in the obtained convolution image group to obtain an object saliency map; S2-4, normalizing the global saliency map, the local saliency map and the object saliency map to unify the numerical domain; S2-5, weighted fusion of the global saliency map, the local saliency map and the object saliency map to obtain an enhanced image of the beam pattern. Through the above steps, it is helpful to enhance the contrast between the real object echo and the background noise. For example, the real object can be the bottom of the water, obstacles, sunken ships, reefs, etc. For example, in a specific example of the present invention, it can be considered to first normalize the global saliency map, the local saliency map and the object saliency map to unify the numerical domain. For example, the range of the numerical domain is , and then set the weights of the global saliency map and the local saliency map to 0.2, and the weight of the object saliency map to 0.6. The global saliency map, the local saliency map, and the object saliency map are weighted and superimposed according to the above weights to obtain an enhanced image of the beam pattern, so as to enhance the contrast between the real object echo and the background noise in the enhanced image of the beam pattern.
[0068] Preferably, the step of enhancing the image of the beam pattern of the forward-looking sonar in step S2 further includes: S2-6, a step of controlling the brightness curve of the enhanced image of the beam pattern, the formula of which is: , where the parameters The enhanced image of the beam pattern after curve control, parameter The enhanced image of the beam pattern before curve control, parameter By means of step S2-6, the contrast of the high-intensity area can be reduced so that the brightness of the beam pattern on the screen is basically consistent. By controlling the curve coefficient (i.e., parameter ) can realize the control of the brightness of the beam pattern. In other words, the curve coefficient It can be selected by the visualization processing method according to different scenarios.
[0069] S3, sonar image pseudo-color mapping: normalize the enhanced image of the beam pattern to an 8-bit numerical space to obtain an 8-bit single-channel enhanced image of the beam pattern. Subsequently, the 8-bit single-channel enhanced image of the beam pattern is mapped to 8-bit three-channel image data through a pseudo-color mapping map, so that the color contrast of the beam pattern is more obvious, so that the sonar fan diagram is easier to observe on the display device. In a specific example of the visualization processing method of the present invention, the formula for normalizing the enhanced image of the beam pattern to an 8-bit numerical space is: , where the parameters Represents the normalized image, parameter represents the enhanced image before curve control, parameter and Represents the maximum and minimum pixel values of the enhanced image, and then the single-channel gray image is mapped to an RGB color image through a pseudo color map.
[0070] S4, sonar sector conversion map generation: the pixel position of the sector map corresponding to the pixel position of the beam map is calculated according to the sampling parameters of the forward-looking sonar, and the memory offset position of the enhanced image data of the 8-bit three-channel beam map is stored with the sector map coordinates as the index to generate the sonar sector conversion map. In a specific example of the visualization processing method of the present invention, the enhanced image data of the 8-bit three-channel beam map of the forward-looking sonar is sequentially stored in a continuous memory, the size of the converted sector map is defined as a radius of 500 pixels, and a mapping map containing a sector with a radius of 500 pixels is generated accordingly, the memory position of the corresponding pixel position is calculated according to the size of the beam map, and the memory position index is saved as a pixel value in the corresponding position of the mapping map. The mapping map is only generated when the first frame of echo data, the sonar range or opening angle changes, and the sector map pixel size changes. In the beam diagram, the horizontal axis represents the receiving angle of the signal, and the vertical axis represents the distance from the receiving array. At this time, the meaning of the pixel coordinates is close to the polar coordinates. The horizontal and vertical axes in the fan diagram represent the real position of the signal relative to the receiving array. At this time, the meaning of the pixel coordinates is close to the rectangular coordinate system, so the conversion formula from polar coordinates to rectangular coordinates can be applied, where the conversion formula is: ; ; Among them, the parameters is the distance corresponding to the ordinate in the beam pattern, and the parameter It is the angle corresponding to the horizontal axis in the beam diagram. The distance and angle need to refer to the sampling parameters. and It is the relative position of the signal relative to the receiving array. The pixel position on the fan diagram needs to be calculated in combination with the radius of the fan diagram.
[0071] S5, sonar fan map conversion: read data from the enhanced image of the 8-bit three-channel beam map through the memory offset in the sonar fan map conversion map, and copy it to the corresponding position of the fan map to obtain the sonar fan map. In a specific example of the visualization processing method of the present invention, a fan map image of the same size is generated according to the fan conversion map calculated in step S3, and the corresponding memory position in the sonar pseudo-color image is accessed according to the memory position corresponding to the pixel value of the map, and its pixel value is copied to the fan map image, so as to complete the visualization processing of the sonar image.
[0072] In addition, in step S1, after the raw data of the current-view sonar is acquired, the visualization processing method can also perform real-time detection of small targets in the beam map. For example, the visualization processing method can perform real-time detection of small targets with a size of less than 100 pixels in the beam map to facilitate subsequent sonar applications such as identification and obstacle avoidance, and can also further highlight small targets on the sonar image. In this way, the visualization processing method can not only visualize the sonar image, but also highlight small targets on the visualized image, making the small targets easy to observe.
[0073] The steps of the visualization processing method for real-time detection of small targets in the sonar image of the forward-looking sonar include: (a) image enhancement, segmentation, and acquisition of target position of the sonar image of the forward-looking sonar; (b) target shape recovery; (c) target multi-frame detection. The above steps make: first, the visualization processing method does not rely on large parallel computing devices when performing real-time detection of small targets in the sonar image of the forward-looking sonar, is easy to deploy, and has the characteristics of small calculation amount and fast running speed; second, the visualization processing method has few preset parameters and clear meanings when performing real-time detection of small targets in the sonar image of the forward-looking sonar, and the user can adjust them in time according to the use effect; third, the visualization processing method includes multi-frame detection when performing real-time detection of small targets in the sonar image of the forward-looking sonar, retains historical target records, and can retain the target position of the small target when the sonar image is temporarily disturbed, and has strong anti-interference ability.
[0074] Specifically, the step (a) further includes the steps of: (a.1), enhancing the contrast of the beam pattern of the forward-looking sonar and enhancing the pixel value intensity in the local extreme point area; (a.2), setting a dynamic threshold according to the pixel intensity range of the enhanced image of the beam pattern and segmenting the enhanced image of the beam pattern with the threshold, discarding the part with pixel intensity lower than the threshold and retaining the part with pixel intensity higher than the threshold, performing connected region analysis on the segmentation result, and taking the pixel with the highest pixel intensity in the connected region as the target position.
[0075] More specifically, the step (a.1) further includes the following steps: first, calculating the contrast level of pixels in the beam map globally, performing differential enhancement of the pixel contrast between the pixels and the global level, and recording it as a global saliency map; second, calculating the local contrast level of pixels in the original image data of the sonar image, performing differential enhancement of the local overall contrast between the local mean and the global level, and recording it as a local saliency map; third, setting an object detection convolution kernel group, wherein the object detection convolution kernel group includes an object buffer part and a background estimation part, performing a convolution operation on the image data of the beam map using the object detection convolution kernel group, and taking the maximum value of the component at each pixel position in the obtained convolution image group to obtain an object saliency map; fourth, normalizing the global saliency map, the local saliency map, and the object saliency map to unify the numerical domain; fifth, weightedly superimposing the global saliency map, the local saliency map, and the object saliency map to obtain an enhanced image of the beam map.
[0076] It is worth mentioning that the global saliency map and the local saliency map are used for global contrast enhancement, while the object saliency map is used to enhance the contrast of small connected regions. Therefore, in a specific example of the present invention, it is considered to set the weights of the global saliency map and the local saliency map to 0.2, and the weight of the object saliency map to 0.6. The global saliency map, the local saliency map and the object saliency map are weighted and superimposed according to the above weights to obtain an enhanced image of the beam map, so as to highlight small targets in the enhanced image of the beam map.
[0077] In step (a.2), the formula for dynamic threshold setting is , where the relation is the threshold, parameter is the threshold coefficient, parameter is the average pixel value of the enhanced image of the beam pattern, and the parameter is the maximum pixel value of the enhanced image of the beam map. For example, in a specific example of the present invention, a set of continuous beam maps obtained by forward-looking sonar detection is input, and the beam map contains small targets, river banks, underwater and other contents. The beam map is uploaded and recorded at 5Hz. Next, the beam map of one frame is first processed, and the global saliency map, local saliency map and object saliency map of the beam map are calculated. Then, the enhanced image of the beam map of this frame is obtained by weighted superposition, and the dynamic threshold coefficient is set to 0.3 again, and the enhanced image of the beam map is segmented. Then, a connected area analysis is performed in the segmentation result. After discarding non-target areas with too large an area, the pixel maximum position of the corresponding beam map in each remaining connected area is recorded respectively, recorded as the target position, and the pixel value of the target position is the pixel value of the small target.
[0078] In step (b), since the original shape of the small target may be destroyed to a certain extent during the process of enhancing the beam pattern, it is necessary to restore the target shape of the small target in the beam pattern according to the target position obtained in step (a).
[0079] Specifically, the step (b) further includes the steps of: (b.1) setting a threshold at the target position to divide the target boundary in the superpixel space, and (b.2) using linear interpolation to sample the target boundary position to the pixel space. The formula for setting the threshold at the target position is , where the parameters is the pixel value threshold at the target location, parameter is the pixel value of the small target (target position). For example, in a specific example of the present invention, the target position obtained in step (a) is traversed, the target boundary threshold is calculated according to the pixel intensity at the target position, the target boundary position in the superpixel space is obtained by linear calculation at the target boundary, and the target boundary position is sampled to the pixel space by linear interpolation, and the boundary of the small target is updated to achieve target shape recovery of the small target. For example, in a more specific example of the present invention, the calculated target boundary threshold is 1.8, the pixel value at the coordinate (1,1) is 1, and the pixel value at the coordinate (1,2) is 2, then the coordinates of the target boundary position in the divided superpixel space are (1,1.18), and then the coordinates (1,1.18) are linearly interpolated to obtain the target boundary coordinates in the pixel space as (1,2).
[0080] In step (c), since small targets detected in a single frame inevitably have false alarms, and the imaging performance of the forward-looking sonar will be affected when the aircraft suddenly accelerates or the forward-looking sonar is disturbed, the present invention uses multi-frame detection to improve the target performance.
[0081] Specifically, in some embodiments of the present invention, the step (c) further includes the steps of: (c.1), maintaining two target lists, namely a trusted target list and a target list to be observed, (c.2), comparing the small target of the current frame with the target in the trusted target list, and when the features of the small target of the current frame and the target in the trusted target list match, updating the target in the trusted target list, and for the current small target that does not appear in the trusted target list, comparing the small target of the current frame with the target in the target list to be observed, and when the features of the small target of the current frame and the target in the target list to be observed match, updating the target in the target list to be observed, A current small target that does not appear in the list of targets to be observed is taken as a new target. (c.3) If the new target appears at the edge of the beam pattern, the new target is added to the list of targets to be observed. Otherwise, the new target is regarded as a false alarm and discarded. (c.4) Check the list of targets to be observed, move the stably updated targets in the list of targets to be observed to the list of trusted targets, and regard the targets that cannot be updated for multiple consecutive frames as false alarms and discard them. (c.5) Check the list of trusted targets, and for the targets that have not been updated in the list of trusted targets, try to use the step (b) to find the target boundary in the beam pattern according to the target position. If no result is found, continue to retain the current target position and target boundary.
[0082] In a specific example of the present invention, in the step (c.4), the trust threshold can be set to 5, and the discard threshold can be set to 8, that is, when a target is successfully updated for 5 consecutive frames, it is considered to be stably appearing and is moved to the trusted target list, and a target that cannot be updated for 8 consecutive frames is considered to be a false alarm that appears flickeringly and is discarded.
[0083] Reference Figure 4 According to another aspect of the present invention, the present invention further provides a visualization processing method for sonar images, wherein the visualization processing method comprises the following steps:
[0084] Step 410, obtaining a beam pattern of a single frame of sonar data and corresponding sampling parameters;
[0085] Step 420, enhancing the beam pattern to obtain an enhanced image of the beam pattern;
[0086] Step 430, normalizing the enhanced image of the beam pattern to an 8-bit numerical space to obtain an enhanced image of the beam pattern of an 8-bit single channel;
[0087] Step 440, mapping the 8-bit single-channel beam pattern enhancement image into 8-bit three-channel image data through a pseudo color mapping map;
[0088] Step 450, calculating the pixel position of the fan diagram corresponding to the pixel position of the beam diagram according to the sampling parameters, and storing the memory offset position of the enhanced image of the 8-bit three-channel beam diagram using the fan diagram coordinates as an index to generate a sonar fan conversion map;
[0089] Step 460, reading data from the enhanced image of the 8-bit three-channel beam diagram through the memory offset in the sonar sector conversion map, and copying it to the corresponding position of the sector diagram to obtain the sonar sector diagram.
[0090] In the visualization processing method of the present invention, since the step of enhancing the beam pattern of the forward-looking sonar is included, the beam pattern of the forward-looking sonar can be displayed more clearly on the display device, making the beam pattern of the forward-looking sonar easier to observe on the display device. In addition, the visualization processing method of the present invention includes the step of normalizing the enhanced image of the beam pattern to an 8-bit numerical space. In this way, the visualization processing method can obtain effective image visualization output without adjusting the brightness. At the same time, the visualization processing method can realize the conversion of the beam pattern of the forward-looking sonar into a fan-shaped visualization. In each frame of the image, it is only necessary to access the memory address according to the mapping diagram without coordinate calculation, so as to improve the visualization processing speed of the beam pattern.
[0091] In a specific example of the visualization processing method of the present invention, the specific steps of enhancing the beam map of the forward-looking sonar may be: first, calculating the global contrast level of pixels in the image data of the beam map, and performing differential enhancement of the contrast of the pixels with the global level, which is recorded as a global saliency map; second, calculating the local contrast level in the image data of the beam map, and performing differential enhancement of the local overall contrast between the local mean and the global level, which is recorded as a local saliency map; third, setting an object detection convolution kernel group, wherein the object detection convolution kernel group includes an object buffer part and a background estimation part, and performing a convolution operation on the image data of the beam map using the object detection convolution kernel group, and taking the maximum value of the component at each pixel position in the obtained convolution image group to obtain an object saliency map; fourth, normalizing the global saliency map, the local saliency map and the object saliency map to unify the numerical domain; finally, weighted fusion of the global saliency map, the local saliency map and the object saliency map to obtain an enhanced image of the beam map, so that the visualization processing method can highlight the target in the sonar image and make the target more significant.
[0092] In a specific embodiment of the visualization processing method of the present invention, a specific method of weighted superposition of the global saliency map, the local saliency map and the object saliency map to obtain an enhanced image of the beam map may be: firstly normalizing the global saliency map, the local saliency map and the object saliency map to unify the numerical domain, for example, the range of the numerical domain is , and then set the weights of the global saliency map and the local saliency map to 0.2, and the weight of the object saliency map to 0.6. The global saliency map, the local saliency map, and the object saliency map are weighted and superimposed according to the above weights to obtain the enhanced image of the beam map, so as to highlight the small target in the enhanced image of the beam map.
[0093] In a more specific example of the visualization processing method of the present invention, the visualization processing method can further control the brightness curve of the enhanced image of the beam pattern to reduce the contrast of the high-intensity area so that the brightness on the screen of the beam pattern is basically consistent. The formula for controlling the brightness curve of the enhanced image can be: , where the parameters The enhanced image of the beam pattern after curve control, parameter The enhanced image of the beam pattern before curve control, parameter Represents the curve coefficient.
[0094] Reference Figure 5According to another aspect of the present invention, the present invention further provides a sonar image visualization processing system, which includes an acquisition unit 10, an enhancement unit 20, a normalization unit 30, a mapping unit 40, a generation unit 50 and a visualization unit 60. The acquisition unit 10 is used to acquire the beam pattern and corresponding sampling parameters of a single frame data of the forward-looking sonar; the enhancement unit 20 is used to enhance the beam pattern to obtain an enhanced image of the beam pattern; the normalization unit 30 is used to normalize the enhanced image of the beam pattern to an 8-bit numerical space to obtain an 8-bit single-channel enhanced image of the beam pattern; the mapping unit 40 is used to map the 8-bit single-channel enhanced image of the beam pattern to 8-bit three-channel image data through a pseudo-color mapping map; the generation unit 50 is used to calculate the pixel position of the fan diagram corresponding to the pixel position of the beam pattern according to the sampling parameters, store the memory offset position of the enhanced image of the 8-bit three-channel beam pattern with the fan diagram coordinates as the index, and generate a sonar fan conversion mapping map; the visualization unit 60 is used to read data from the enhanced image of the 8-bit three-channel beam pattern through the memory offset in the sonar fan conversion mapping map, and copy it to the corresponding position of the fan diagram to obtain a sonar fan diagram.
[0095] In the visualization processing system of the present invention, since the enhancement unit 20 can enhance the beam pattern of the sonar image, the beam pattern can be displayed more clearly on the display device, making the beam pattern easier to observe on the display device. Since the normalization unit 30 can normalize the enhanced image of the beam pattern to an 8-bit numerical space to obtain an 8-bit single-channel enhanced image of the beam pattern, the visualization processing system can obtain effective image visualization output without adjusting the brightness. At the same time, the visualization processing system can realize the transformation of the beam pattern to the fan-shaped visualization. In each frame of the image, it is only necessary to access the memory address according to the mapping diagram without coordinate calculation, so as to improve the visualization processing speed of the beam pattern.
[0096] Furthermore, the enhancement unit 20 includes a global saliency module 21 , a local saliency module 22 , an object saliency module 23 , a value domain unification module 24 and a weighted superposition module 25 . The global saliency module 21 is used to calculate the contrast level of pixels in the image data of the beam map in the global context, and the contrast of the pixels is enhanced by differentially enhancing the contrast of the pixels with the global level to obtain a global saliency map; the local saliency module 22 is used to calculate the local contrast level in the image data of the beam map, and the local mean is enhanced by differentially enhancing the local overall contrast with the global level to obtain a local saliency map; the object saliency module 23 is used to set an object detection convolution kernel group, and the image data of the beam map is convolved using the object detection convolution kernel group, and the object saliency map is obtained by taking the maximum value of the component at each pixel position in the obtained convolution image group; the numerical domain unification module 24 is used to normalize the global saliency map, the local saliency map and the object saliency map to unify the numerical domain; the weighted superposition module 25 is used to weightedly fuse the global saliency map, the local saliency map and the object saliency map to obtain an enhanced image of the beam map.
[0097] Continue to refer to the attached Figure 5 The visualization processing system also includes a small target detection unit 70, which is used to detect small targets in real time in the beam pattern so as to highlight the small targets on the sonar image. In this way, the visualization processing method can not only visualize the sonar image, but also highlight the small targets on the visualized image, making the small targets easy to observe.
[0098] Specifically, the process of the small target detection unit 70 detecting small targets in the sonar image of the forward-looking sonar in real time is: first, the sonar image of the forward-looking sonar is enhanced, segmented, and the target position is obtained; second, the target shape is restored; and third, the target is detected in multiple frames. The above steps make it possible for: first, the visualization processing method does not rely on large parallel computing devices when performing real-time detection of small targets in the sonar image of the forward-looking sonar, is easy to deploy, and has the characteristics of small calculation amount and fast running speed; second, the visualization processing method has few preset parameters and clear meanings when performing real-time detection of small targets in the sonar image of the forward-looking sonar, and the user can adjust them in time according to the use effect; third, the visualization processing method includes multi-frame detection when performing real-time detection of small targets in the sonar image of the forward-looking sonar, retains historical target records, and can retain the target position of the small target when the sonar image is temporarily disturbed, and has strong anti-interference ability.
[0099] Attached Figure 6A computing device 600 according to another preferred embodiment of the present invention is shown, wherein the computing device 600 includes a processor 610 and a memory 620 coupled to the processor 610, the memory 620 stores codes, the processor 610 is configured to execute the codes, and when the codes are executed, the computing device 600 performs the following steps: acquiring a beam pattern and corresponding sampling parameters of a single frame of forward-looking sonar data; enhancing the beam pattern to obtain an enhanced image of the beam pattern; normalizing the enhanced image of the beam pattern to an 8-bit numerical space to obtain a beam pattern; and An 8-bit enhanced image of a single-channel beam diagram is obtained; the 8-bit enhanced image of the single-channel beam diagram is mapped into 8-bit three-channel image data through a pseudo-color mapping diagram; the pixel position of the fan diagram corresponding to the pixel position of the beam diagram is calculated according to a sampling parameter, and the memory offset position of the enhanced image of the 8-bit three-channel beam diagram is stored with the fan diagram coordinates as an index to generate a sonar fan conversion mapping diagram; data is read from the enhanced image of the 8-bit three-channel beam diagram through the memory offset in the sonar fan conversion mapping diagram, and copied to the corresponding position of the fan diagram to obtain a sonar fan diagram.
[0100] In a possible implementation, the computing device 600 further includes a transceiver 630 , and the transceiver 630 communicates with the processor 610 and the memory 620 through an internal connection path.
[0101] It should be understood by those skilled in the art that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments, and the embodiments of the present invention may be deformed or modified in any way without departing from the principles.
Claims
1. A method for visualizing sonar images, characterized in that: The visualization processing method comprises the following steps: S1, obtaining the beam pattern and corresponding sampling parameters of a single frame of sonar data; S2, enhancing the beam pattern to obtain an enhanced image of the beam pattern; S3, normalizing the enhanced image of the beam pattern to an 8-bit numerical space to obtain an enhanced image of the beam pattern of an 8-bit single channel; S4, mapping the enhanced image of the 8-bit single-channel beam pattern into 8-bit three-channel image data through a pseudo color mapping map; S5, calculating the pixel position of the fan diagram corresponding to the pixel position of the beam diagram according to the sampling parameters, and storing the memory offset position of the enhanced image of the 8-bit three-channel beam diagram using the fan diagram coordinates as an index to generate a sonar fan conversion map; S6, reading data from the enhanced image of the 8-bit three-channel beam diagram through the memory offset in the sonar sector conversion map, and copying it to the corresponding position of the sector diagram to obtain the sonar sector diagram; The step S2 comprises the following steps: S2.1, calculate the global contrast level of the pixels in the image data of the beam map, and perform differential enhancement on the contrast of the pixels and the global level, which is recorded as a global saliency map; S2.2, calculate the local contrast level in the image data of the beam map, and make a difference between the local mean and the global level to enhance the local overall contrast, which is recorded as a local saliency map; S2.3, setting an object detection convolution kernel group, wherein the object detection convolution kernel group includes an object buffer part and a background estimation part, using the object detection convolution kernel group to perform a convolution operation on the image data of the beam map, and taking the maximum value between groups at each pixel position in the obtained convolution image group to obtain an object saliency map; S2.4, normalizing the global saliency map, the local saliency map, and the object saliency map to unify the numerical domain; S2.5, performing weighted fusion on the global saliency map, the local saliency map and the object saliency map to obtain an enhanced image of the beam map; The enhanced image of the beam pattern is subjected to brightness curve control, wherein the formula for controlling the brightness curve of the enhanced image of the beam pattern is: , where the parameters Represents the enhanced image after curve control, parameter The enhanced image of the beam pattern before curve control, parameter represents the curve coefficient; In step S3, the formula for normalizing the enhanced image of the beam pattern to an 8-bit numerical space is: , where the parameters Represents the enhanced image of the normalized 8-bit single-channel beam diagram, with parameters The enhanced image of the beam pattern before curve control, parameter and The maximum and minimum pixel values of the enhanced image representing the beam pattern.
2. The sonar image visualization processing method according to claim 1, characterized in that: In the step S1, a set of continuous raw data is acquired through the sonar communication protocol, the raw data including the beam diagram, sampling number, beam number, and sonar opening angle of a single channel of a single frame of forward-looking sonar data.
3. The sonar image visualization processing method according to claim 1, characterized in that: In step S2.4, the range of the value domain is In step S2.5, the weights of the global saliency map and the local saliency map are set to 0.2 respectively, and the weight of the object saliency map is set to 0.
6. The global saliency map, the local saliency map and the object saliency map are superimposed according to these weights to obtain an enhanced image of the beam map.
4. The sonar image visualization processing method according to any one of claims 1 to 3, characterized in that: When the visualization processing method is applied to a forward-looking sonar, the visualization processing method further comprises the step of: performing real-time detection of small targets in the sonar image of the forward-looking sonar, wherein the step of performing real-time detection of small targets comprises: (a) Enhance and segment the sonar image of the forward-looking sonar to obtain the target position; (b), target shape recovery; (c), Target multi-frame detection.
5. A sonar image visualization processing system, characterized in that: include: An acquisition unit, used to acquire a beam pattern of a single frame of sonar data and corresponding sampling parameters; an enhancement unit, configured to enhance the beam pattern to obtain an enhanced image of the beam pattern; A normalization unit, used for normalizing the enhanced image of the beam pattern to an 8-bit numerical space to obtain an enhanced image of the beam pattern of an 8-bit single channel; A mapping unit, used for mapping an 8-bit single-channel beam pattern enhancement image into 8-bit three-channel image data through a pseudo-color mapping map; A generating unit, used to calculate the pixel position of the fan diagram corresponding to the pixel position of the beam diagram according to the sampling parameters, store the memory offset position of the enhanced image of the 8-bit three-channel beam diagram with the fan diagram coordinates as the index, and generate a sonar fan conversion map; A visualization unit is used to read data from the enhanced image of the 8-bit three-channel beam diagram through the memory offset in the sonar sector conversion map, and copy it to the corresponding position of the sector diagram to obtain the sonar sector diagram; The enhancement unit further comprises: The global saliency module is used to calculate the global contrast level of pixels in the image data of the beam map, and to perform differential enhancement on the contrast of pixels with the global level to obtain a global saliency map; The local saliency module is used to calculate the local contrast level in the image data of the beam map, and the local mean is differentiated from the global level to enhance the local overall contrast and obtain a local saliency map; An object saliency module is used to set an object detection convolution kernel group, use the object detection convolution kernel group to perform a convolution operation on the image data of the beam map, and take the maximum value of the component at each pixel position in the obtained convolution image group to obtain an object saliency map; A numerical domain unification module is used to normalize the global saliency map, the local saliency map, and the object saliency map to unify the numerical domain; A weighted superposition module is used to perform weighted fusion of the global saliency map, the local saliency map and the object saliency map to obtain an enhanced image of the beam map; The enhancement unit includes a brightness control module for performing brightness curve control on the enhanced image of the beam pattern, wherein the formula for performing brightness curve control on the enhanced image of the beam pattern is: , where the parameters Represents the enhanced image after curve control, parameter The enhanced image of the beam pattern before curve control, parameter represents the curve coefficient; The normalization unit normalizes the enhanced image of the beam pattern to an 8-bit numerical space using the formula: , where the parameters Represents the enhanced image of the normalized 8-bit single-channel beam diagram, with parameters The enhanced image of the beam pattern before curve control, parameter and The maximum and minimum pixel values of the enhanced image representing the beam pattern.
6. The visualization processing system according to claim 5, characterized in that: The acquisition unit acquires a set of continuous raw data through a sonar communication protocol, wherein the raw data includes a single-channel beam diagram, a sampling number, a beam number, and a sonar opening angle of a single frame of forward-looking sonar data.
7. The visualization processing system according to claim 5, characterized in that: The range of the numerical domain unified by the numerical domain unification module is The weighted superposition module sets the weights of the global saliency map and the local saliency map to 0.2, and the weight of the object saliency map to 0.6, and superimposes the global saliency map, the local saliency map and the object saliency map according to these weights to obtain an enhanced image of the beam map.
8. The visualization processing system according to any one of claims 5 to 7, characterized in that: The visualization processing system includes a small target detection unit, which is used to detect small targets in the beam pattern in real time.
9. The visualization processing system according to claim 8, characterized in that: The process of the small target detection unit detecting small targets in real time in the sonar image of the forward-looking sonar is: first, the sonar image of the forward-looking sonar is enhanced, segmented, and the target position is obtained; second, the target shape is restored; and third, the target is detected in multiple frames.
10. A computing device, characterized in that The computing device comprises a processor and a memory coupled to the processor, wherein the memory is used to store codes, and the processor is configured to execute the codes. When the codes are executed, the computing device executes the visualization processing method according to any one of claims 1 to 4.
