Water surface target height calculation method, system and device and computer readable storage medium

Abstract

The invention discloses a water surface target height calculation method, system and device and a computer readable storage medium, and belongs to the technical field of target recognition and measurement. In order to solve the problem of large measurement error caused by high misjudgment rate and unstable water surface reference of traditional single-spectrum image recognition, a visible light image and a thermal imaging image are synchronously acquired through a double-spectrum imaging camera, color features and temperature features are extracted, a random forest classifier is adopted to perform multi-modal fusion to generate a binary water surface mask, and the image recognition accuracy is improved. And the water surface area is precisely limited to reduce background interference. And performing target detection in the mask range based on a YOLOv5 model, obtaining a target distance in combination with a laser range finder, constructing a geometric model, and converting a target pixel height into an actual physical height. According to the method, the influence of water surface fluctuation on measurement is effectively overcome, and the target recognition precision and the height measurement stability in a complex environment are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of target recognition and measurement technology, and in particular to a method, system, device and computer-readable storage medium for calculating the height of a water surface target. Background Art

[0002] In the field of surface target recognition and measurement, traditional target recognition algorithms usually perform target detection based on a single spectral image, lacking the effective fusion and utilization of different spectral information. Since visible light images are easily affected by factors such as lighting conditions and weather conditions, and cannot effectively distinguish the characteristic differences between the water surface and the shore, misjudgment is prone to occur when performing target recognition on surface scenes containing a large amount of background interference, and shore objects or floating objects are mistakenly identified as target ships, greatly reducing the accuracy of recognition.

[0003] On the other hand, target height measurement is commonly used in the field of vehicle identification, and typically uses the road surface as a reference for height line calibration. This method is not suitable for measuring target height on the water surface. Unlike the road surface, the water surface has large fluctuations in height, and the water level line changes frequently. This means that calibration based on the water surface as a reference is unstable, and the measurement results will be inaccurate due to water level fluctuations. Traditional measurement methods do not fully consider this special environmental factor of the water surface, and still use the traditional height calibration method suitable for the road surface, which cannot accurately identify the target height. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention proposes a method, system, device and computer-readable storage medium for calculating the height of a water surface target to solve the technical problems of insufficient accuracy in water surface target recognition and insufficient accuracy of traditional target height measurement methods in the existing technology.

[0005] The technical solution adopted by the present invention is a method for calculating the height of a water surface target.

[0006] In a first possible implementation method, the method includes: collecting visible light images and thermal imaging images of the area to be identified through a dual-spectral imaging acquisition camera; performing multimodal feature fusion processing on the visible light image and the thermal imaging image to generate a water surface mask that limits the water surface range; performing target recognition on the water surface mask according to a target detection algorithm, and marking the target through a target detection frame; obtaining the number of pixels in the height direction of the target detection frame, and calculating the height of the target based on the mapping relationship between the number of pixels and the physical size.

[0007] Furthermore, the multimodal feature fusion processing includes: extracting the HSV color features of the visible light image and the temperature features of the thermal imaging image; fusing the HSV color features and the temperature features through a random forest classifier to generate a binary water surface mask.

[0008] Furthermore, target recognition is performed on the water surface mask according to a target detection algorithm, and the target is marked using a target detection frame, including: extracting features from the water surface mask through the backbone network to generate a multi-scale feature map; further fusing the multi-scale feature map through a path aggregation network to generate an enhanced feature map; performing multi-scale feature optimization on the enhanced feature map and dynamic matching with an anchor frame to generate a target detection frame; processing the enhanced feature map through multi-scale feature optimization and dynamic matching with an anchor frame to generate a target detection frame for marking the target.

[0009] Furthermore, the enhanced feature map is processed by multi-scale feature optimization and dynamic matching of anchor frames to generate a target detection frame, including: using a multi-scale convolution kernel to perform layer-by-layer convolution on the enhanced feature map to generate a refined feature map; generating multiple sets of predefined anchor frames based on historical detection data to cover potential targets of different sizes and aspect ratios in the refined feature map; adjusting the position and size of the predefined anchor frames through a regression algorithm to obtain a target detection frame aligned with the real target; identifying the target category in each of the target detection frames; and outputting a target detection frame including the target category.

[0010] Furthermore, the number of pixels in the height direction of the target detection frame is obtained, and the height of the target is calculated based on the mapping relationship between the number of pixels and the physical size, including: obtaining the number of pixels in the height direction of the target within the target detection frame; obtaining the distance between the target and the detection device; obtaining the vertical field of view of the dual-spectrum imaging acquisition camera; and calculating the target height based on the number of pixels, distance, and vertical field of view using the method shown in the following target height calculation formula: Where H represents the target height, represents the vertical field of view of the device, d represents the distance between the target and the detection device, n represents the total vertical resolution of the image, and p represents the number of pixels of the target in the height direction within the target detection frame.

[0011] Furthermore, obtaining the distance between the target and the detection device includes: synchronously activating the ranging device and the dual-spectral imaging acquisition camera; using time gating technology to filter the water surface reflection noise and retain only the laser echo signal directly reflected by the target; and calculating the distance between the target and the detection device based on the laser echo signal.

[0012] Furthermore, the vertical field of view angle of the dual-spectral imaging acquisition camera is obtained, and the calculation formula is: in, Represents the vertical field of view of the device, s represents the sensor pixel pitch, f represents the focal length of the camera, n represents the total vertical resolution of the image, and m represents the total horizontal resolution of the image.

[0013] In combination with the first possible implementation, the second possible implementation includes a surface target height A computing system, the system comprising: an image acquisition module for acquiring visible light images and thermal imaging images of a to-be-identified area using a dual-spectral imaging acquisition camera; a water surface recognition module for performing multimodal feature fusion processing on the visible light images and thermal imaging images to generate a water surface mask that defines the water surface range; a target detection module for performing target recognition on the water surface mask according to a target detection algorithm and marking the target with a target detection frame; and a target height calculation module for obtaining the number of pixels in the height direction of the target detection frame and calculating the height of the target based on a mapping relationship between the number of pixels and the physical size.

[0014] In a third possible implementation, the present disclosure further proposes a device comprising a memory and a processor, wherein the memory stores computer instructions that can be run on the processor, and when the processor runs the computer instructions, it executes a method for calculating the height of a water surface target described in the first possible implementation.

[0015] In a fourth possible implementation, the present disclosure further proposes a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed, a method for calculating the height of a water surface target described in the first possible implementation can be implemented.

[0016] It can be seen from the above technical solution that the beneficial technical effects of the present invention are as follows: 1. Combining visible light images with thermal images, a binary water surface mask is generated through multimodal feature fusion to accurately define the water surface range, reduce misjudgments, and improve target recognition accuracy.

[0017] 2. The vertical field of view angle is calculated using camera intrinsic parameters and image resolution. Combined with the pixel height of the target detection frame and ranging data, a geometric model is established to calculate the actual height. This provides stable, high-precision height measurement results even in environments with fluctuating water levels and undulating water surfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly describes the drawings required for the specific embodiments or the description of the prior art. Similar elements or parts are generally identified by similar reference numerals throughout the drawings. Elements or parts in the drawings are not necessarily drawn to scale.

[0019] Figure 1 is a flow chart of an embodiment of the present invention; Figure 2 Examples of visible light images and thermal imaging images according to an embodiment of the present invention; Figure 3 This is an example diagram of marking a target using a target detection box according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following embodiments of the technical solution of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and are therefore only examples and are not intended to limit the scope of protection of the present invention.

[0021] It should be noted that, unless otherwise specified, the technical or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0022] This embodiment provides a method for calculating the height of a target on a water surface. The working principle of this embodiment is described in detail below: The flow chart of this embodiment is as follows Figure 1 As shown, it includes: collecting visible light images and thermal imaging images of the area to be identified through a dual-spectrum imaging acquisition camera, and examples of visible light images and thermal imaging images are shown in FIG. Figure 2 The visible light image and the thermal imaging image are subjected to multimodal feature fusion processing to generate a water surface mask that limits the water surface range.

[0023] The multimodal feature fusion process includes extracting HSV color features from the visible light image and temperature features from the thermal image; fusing these HSV color and temperature features using a random forest classifier to generate a binary water surface mask. In this embodiment, the visible light image is combined to provide texture and color information. The thermal image reflects temperature differences due to the different thermal radiation characteristics of the water surface and the shore. Multimodal feature fusion generates a binary water surface mask, which accurately defines the water surface and reduces misidentification. The water surface mask is a binary image (black and white), which improves target recognition accuracy.

[0024] Within the range of the water surface mask, the target is identified using a target detection algorithm and marked with a target detection frame. The target detection algorithm uses the YOLOv5 model as the basic framework, including a backbone network, a feature fusion layer, and a prediction layer. The specific steps include: First, a binary water surface mask is generated as the input image through multimodal feature fusion, and feature extraction is performed through the backbone network, which consists of convolutional layers, slice splicing layers, and composite layers: Convolutional layer: extracts low-level features, including edges, textures, and colors, and generates basic feature maps.

[0025] Slice splicing layer: Divide the feature map into multiple sub-regions, perform independent convolutions, and then splice them together to enhance the ability to capture local details.

[0026] Composite layer: Fusion of different-level features through cross-stage partial connections to learn target structured information, such as ship outlines.

[0027] Spatial Pyramid Pooling Layer: Expands the receptive field, integrates multi-scale features, improves adaptability to objects of different sizes, and outputs multi-scale feature maps that contain low-level details and high-level semantic information.

[0028] In the feature fusion layer, the output feature maps of the backbone network are further fused through the path aggregation network, including: Top-down path: transfer high-level semantic features (global location information) to enhance category discrimination capabilities.

[0029] Bottom-up path: transfer low-level detail features (edge ​​gradients) and preserve shape details.

[0030] Cascade fusion: The upper and lower layer features are integrated through upsampling and channel cascade to generate a multi-scale fusion feature map. The enhanced feature map that fuses multi-level features can improve the robustness of target detection.

[0031] At the prediction layer, the fused feature map enters the prediction layer to complete the final detection, including: Multi-scale convolution refinement: Optimize feature representation through 3×3 convolution to improve positioning accuracy.

[0032] Anchor box generation and regression: Generate multi-scale anchor boxes based on K-means clustering, and adjust the box position and size through bounding box regression.

[0033] Category and confidence prediction: The fully connected layer outputs the target category probability and confidence score (in this example, ≥0.7 is a valid detection).

[0034] Non-maximum suppression (NMS): Filters overlapping boxes and retains the best detection results, including location, category, and confidence. Output: The object detection box includes pixel coordinates, width in pixels, height in pixels, object category, and confidence, which is used for subsequent height calculation.

[0035] In this embodiment, a binary water surface mask is generated by multimodal feature fusion, and an example of marking the target by a target detection frame is shown in FIG. Figure 2 As shown, by excluding non-water surface areas and reducing interference from shore objects and floating objects, recognition accuracy can be improved. The YOLOv5 model used in this embodiment has a training dataset containing at least 10,000 images of ships, floating objects, and obstacles, covering sunny, rainy, foggy, and daytime and nighttime lighting conditions.

[0036] Calculate the height of the target according to the target height calculation formula, including: obtaining the number of pixels of the target in the height direction within the target detection frame; obtaining the distance between the target and the detection device; obtaining the vertical field of view angle of the dual-spectrum imaging acquisition camera, and calculating the target height according to the target height calculation formula.

[0037] The vertical field of view angle of the dual-spectral imaging acquisition camera is obtained using the following calculation formula: in, Represents the vertical field of view of the device, s represents the sensor pixel pitch, f represents the focal length of the camera, n represents the total vertical resolution of the image, and m represents the total horizontal resolution of the image.

[0038] The target calculation formula is: Where H represents the target height, represents the vertical field of view of the device, d represents the distance between the target and the detection device, n represents the total vertical resolution of the image, and p represents the number of pixels of the target in the height direction within the target detection frame.

[0039] The ranging device and the dual-spectral imaging camera are synchronously activated by hardware triggering (GPIO pulses in this embodiment). The trigger signal delay error is ≤1 millisecond, ensuring the spatiotemporal consistency of the ranging data and the target detection frame. The laser rangefinder transmits a pulse that reaches the target and is reflected back to the receiver. The distance is calculated by measuring the round-trip time of the pulse: Where d represents the distance between the target and the detection device, c represents the speed of light, and t represents the round-trip time of the pulse.

[0040] Time gating is used to filter out surface-reflected noise, retaining only the laser echo signal directly reflected from the target. Specifically, the signal within the time window from laser emission to a preset cutoff time is retained, while echoes beyond the cutoff time are filtered out. Furthermore, amplitude analysis of the received signal is performed to retain only pulses exceeding the noise floor (in this case, surface-scattered noise).

[0041] In this embodiment, the speed of light is optionally affected by the ambient temperature and humidity. Therefore, after obtaining the ambient temperature and humidity, the ranging error needs to be corrected using the formula: in, Indicates the distance after correction, d indicates the distance between the target and the detection device, that is, the distance before correction. represents the temperature coefficient, Indicates the temperature deviation, that is, the difference between the current ambient temperature and the reference temperature when the ranging device is calibrated. Indicates the humidity deviation, which is the difference between the current ambient humidity and the reference humidity when the ranging device is calibrated.

[0042] The distance between the target and the detection device is calculated based on the laser echo signal. In this embodiment, the laser rangefinder is coaxially mounted with the dual-spectral imaging acquisition camera, ensuring that the distance measurement direction is strictly aligned with the centerline of the camera's field of view. The laser rangefinder has a measurement accuracy of ±0.5 meters and a wavelength of 1550 nm to avoid interference from visible light.

[0043] The vertical field of view angle is calculated through the camera's intrinsic parameters and image resolution. Combined with the pixel height of the target detection frame and the ranging data, a geometric model is established to calculate the actual height. Even in environments with changing water levels and undulating water surfaces, it can still provide stable and high-precision height measurement results.

[0044] Based on the same inventive concept, the present disclosure also provides a water surface target height calculation system, including: an image acquisition module, used to collect visible light images and thermal imaging images of the area to be identified through a dual-spectral imaging acquisition camera; a water surface recognition module, used to perform multimodal feature fusion processing on the visible light image and thermal imaging image to generate a water surface mask that limits the water surface range; a target detection module, used to identify the target within the range of the water surface mask using a target detection algorithm, and mark the target with a target detection frame; a ranging module: used to obtain distance data through a radar or a laser rangefinder, and calculate the height of the target according to the target height calculation formula; an output module, used to output the target recognition result and the height measurement value.

[0045] Based on the same inventive concept, the present disclosure also proposes a device including a memory and a processor, wherein the memory stores computer instructions that can be run on the processor, and the processor executes the above-mentioned method for calculating the height of a water surface target when running the computer instructions.

[0046] Based on the same inventive concept, the present disclosure also proposes a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed, the above-mentioned method for calculating the height of a water surface target can be implemented.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

Claims

1. A method for calculating the height of a water surface target, characterized in that: include: The dual-spectral imaging camera is used to collect visible light images and thermal imaging images of the area to be identified; Performing multimodal feature fusion processing on the visible light image and the thermal imaging image to generate a water surface mask that limits the water surface range; Performing target recognition on the water surface mask according to a target detection algorithm, and marking the target with a target detection frame; The number of pixels in the height direction of the target detection frame is obtained, and the height of the target is calculated based on the mapping relationship between the number of pixels and the physical size.

2. A method for calculating the height of a water surface target according to claim 1, characterized in that: The multimodal feature fusion processing includes: Extracting HSV color features of the visible light image and temperature features of the thermal imaging image; The HSV color feature and the temperature feature are fused by a random forest classifier to generate a binary water surface mask.

3. A method for calculating the height of a water surface target according to claim 1, characterized in that: Performing target recognition on the water surface mask according to a target detection algorithm and marking the target with a target detection frame includes: Extracting features from the water surface mask using the backbone network to generate a multi-scale feature map; The multi-scale feature map is further fused through a path aggregation network to generate an enhanced feature map; Performing multi-scale feature optimization on the enhanced feature map and dynamically matching it with the anchor frame to generate a target detection frame; The enhanced feature map is processed by multi-scale feature optimization and anchor frame dynamic matching to generate an object detection frame for marking the object.

4. A method for calculating the height of a water surface target according to claim 3, characterized in that: The enhanced feature map is processed by multi-scale feature optimization and dynamic matching of anchor frames to generate a target detection frame, including: Using a multi-scale convolution kernel to perform layer-by-layer convolution on the enhanced feature map to generate a refined feature map; generating multiple sets of predefined anchor boxes based on historical detection data, covering potential targets of different sizes and aspect ratios in the refined feature map; Adjust the position and size of the predefined anchor frame through a regression algorithm to obtain a target detection frame aligned with the real target; Identifying the target category within each target detection frame; Output the target detection box including the target category.

5. A method for calculating the height of a water surface target according to claim 1, characterized in that: Obtaining the number of pixels in the height direction of the target detection frame and calculating the height of the target based on a mapping relationship between the number of pixels and the physical size includes: Obtaining the number of pixels of the target in the height direction within the target detection frame; Obtaining the distance between the target and the detection device; Obtaining the vertical field of view angle of the dual-spectrum imaging acquisition camera; The target height is calculated using the following target height calculation formula based on the number of pixels, distance, and vertical field of view: Where H represents the target height, represents the vertical field of view of the device, d represents the distance between the target and the detection device, n represents the total vertical resolution of the image, and p represents the number of pixels of the target in the height direction within the target detection frame.

6. A method for calculating the height of a water surface target according to claim 5, characterized in that: Obtaining the distance between the target and the detection device includes: Synchronously activating the ranging device and the dual-spectrum imaging acquisition camera; Time gating technology is used to filter the water surface reflection noise and only retain the laser echo signal directly reflected by the target; The distance between the target and the detection device is calculated according to the laser echo signal.

7. A method for calculating the height of a water surface target according to claim 5, characterized in that: The vertical field of view angle of the dual-spectral imaging acquisition camera is obtained using the following calculation formula: in, Represents the vertical field of view of the device, s represents the sensor pixel pitch, f represents the focal length of the camera, n represents the total vertical resolution of the image, and m represents the total horizontal resolution of the image.

8. A water surface target height calculation system, characterized in that: include: Image acquisition module, used to collect visible light images of the area to be identified through a dual-spectrum imaging camera imaging and thermal imaging; a water surface recognition module, configured to perform multimodal feature fusion processing on the visible light image and the thermal imaging image to generate a water surface mask that limits the water surface range; A target detection module is used to identify the target on the water surface mask according to a target detection algorithm and mark the target with a target detection frame; Target height calculation module: used to obtain the number of pixels in the height direction of the target detection frame, and calculate the height of the target based on the mapping relationship between the number of pixels and the physical size.

9. A device comprising a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, characterized in that: When the processor runs the computer instructions, it executes the method for calculating the height of a water surface target according to any one of claims 1 to 6.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed, the method for calculating the height of a water surface target as described in any one of claims 1 to 7 can be implemented.

Images