A method for acousto-optic fusion data dimensionality lifting and guiding light source delivery

Abstract

The application provides a method for sound-light fusion data dimension increasing and guiding light source delivery, and belongs to the technical field of underwater detection and imaging. The application acquires horizontal position information of a target object by using a forward-looking sonar, and on the basis, the target object is scanned by laser, combined with detection of laser reflection spots by an optical camera, pixel coordinates of the spots in camera imaging are extracted, so as to compensate for the elevation information of the target object. After obtaining the three-dimensional position information of the target object, the left and right deflection angles and the up and down pitch angles of the laser and the light source holder are calculated based on geometric relations, so as to realize accurate directional projection of the active light source. Compared with the prior art, the application can realize three-dimensional correspondence of sound-light data without a three-dimensional sonar device, can effectively improve the underwater optical imaging quality and sound-light fusion sensing capability, and is suitable for real-time detection and imaging tasks of underwater mobile platforms.

Description

Technical Field

[0001] This invention belongs to the field of underwater detection and imaging technology, and in particular relates to a method for upscaling acoustic-optical fusion data and guiding the delivery of light sources. Background Technology

[0002] Underwater environments suffer from insufficient lighting and complex propagation media, making it difficult for single imaging methods to meet the demands of high-precision detection. The need for underwater acoustic-optical fusion is growing, with more and more research attempting to register sonar and camera images to compensate for their respective shortcomings. Due to the high cost, complex data processing, and limited real-time performance of 3D sonar equipment, 2D forward-looking sonar is more commonly used in practical applications. However, forward-looking sonar faces the following challenges: sonar images lack elevation (third dimension) information of the target object, making it impossible to determine the target's upper and lower boundaries; and due to the lack of elevation information, active light sources (such as LEDs) cannot provide accurate directional illumination.

[0003] In existing technologies, some solutions attempt to supplement elevation information through fixed prior measurements or empirical estimations. However, their applicability is limited, only applicable to static, fixed-structure scenarios, and unsuitable for real-time imaging tasks in underwater mobile platform environments (such as underwater robots, i.e., ROVs mentioned below). Therefore, existing acoustic-optical fusion technologies still have the following shortcomings: they cannot achieve three-dimensional correspondence between acoustic and optical images; they cannot achieve precise pitch and deflection control of active light sources; and they cannot adapt to the spatial position of the target in real time during underwater missions. Summary of the Invention

[0004] To address the above problems, this invention proposes a method for upscaling acoustic-optical fusion data and guiding light source delivery, comprising the following steps:

[0005] Step 1: Control the laser to scan the target object, use an optical camera to acquire the image formed by laser reflection, and obtain the pixel coordinates of the laser spot in the optical image coordinate system through image preprocessing, region of interest definition, threshold segmentation, contour detection and center coordinate calculation.

[0006] Step 2: Based on the obtained laser spot pixel coordinates, combined with the optical camera imaging parameters, the relative installation positions of the optical camera and the laser, and the horizontal distance information of the target object obtained by the forward-looking sonar, establish a spatial geometric relationship model and calculate the pitch angle of the laser gimbal to compensate for the elevation information of the target object.

[0007] Step 3: Based on the horizontal position information of the target obtained by the forward-looking sonar and the pitch angle of the laser gimbal calculated in Step 2, calculate the left and right yaw angle and the up and down pitch angle of the LED light source gimbal, and control the LED light source to face the target position for directional illumination compensation.

[0008] Preferably, step 1 specifically includes:

[0009] A color image captured by an optical camera is acquired, and the color image is converted to grayscale according to a preset color space conversion rule, converting the multi-channel color image into a single-channel grayscale image; subsequently, the grayscale image is smoothed using a Gaussian filter, and high-frequency noise in the image is suppressed by setting the Gaussian filter kernel size and standard deviation parameters.

[0010] Based on the sonar images acquired by the forward-looking sonar, the target objects in the sonar images are detected, and the left and right boundary positions of the target objects in the sonar images are obtained. Based on the pre-established coordinate correspondence between the sonar images and the optical images, the left and right boundary positions of the target objects are mapped to the optical image coordinate system, and the region of interest corresponding to the target objects is determined in the optical image.

[0011] Within the defined region of interest, a brightness threshold is set for the preprocessed grayscale image, and a threshold segmentation operation is performed on the grayscale image to generate a binary image. Pixels with brightness values ​​higher than the threshold are marked as foreground pixels, and pixels with brightness values ​​lower than or equal to the threshold are marked as background pixels, in order to obtain a binarized image for laser spot detection.

[0012] Contour extraction is performed on the obtained binary image. Multiple candidate contours in the image are obtained by analyzing the connectivity between pixels. The area of ​​the region enclosed by each candidate contour is calculated, and the candidate contours are sorted according to the area. The contour with the largest area is selected as the target contour of the laser reflection spot.

[0013] Based on the obtained target contour of the laser reflection spot, a minimum bounding rectangle is constructed to enclose the target contour; by calculating the coordinates of the diagonal vertices of the minimum bounding rectangle, the center position of the rectangle in the optical image coordinate system is obtained, and the center position is used as the center pixel coordinate of the laser spot.

[0014] Preferably, step 2 involves calculating the laser gimbal pitch angle. The specific process is as follows:

[0015] Based on the vertical pixel offset between the camera's imaging field of view center and the target object center Combined with the number of pixels occupying half of the camera's field of view and half the vertical field of view height offset the pixel Mapped to the actual vertical height difference between the center of the target object and the center of the camera's imaging field of view. This allows us to obtain the elevation information of the target object in space; simultaneously, based on the known installation position relationship between the camera and the laser, we obtain the vertical height difference between the laser and the camera. ;

[0016] The vertical height difference between the target object and the camera Vertical height difference between the laser and the camera Depending on the different positional relationships between the target, camera, and laser, the equivalent vertical height difference between the target and the laser is obtained by superposition or difference calculation; this is then combined with the horizontal distance from the camera or laser to the target measured by forward-looking sonar. Construct a right-angled triangle geometric model with the laser as the emission point and the center of the target object as the pointing point;

[0017] Based on this geometric model, the equivalent vertical height difference between the target object and the laser is used. Horizontal distance from the target The tangent relationship between them is used to calculate the required pitch angle of the laser gimbal. This ensures that the laser's output optical axis can accurately point to the center of the target object.

[0018] Preferably, when the center of the target object is located below the camera and above the laser, the laser is set to illuminate the center of the target object, and the optical camera, laser, and forward-looking sonar are set to be on the same vertical line with no horizontal distance difference but a vertical distance difference; half of the camera's vertical field of view is set to... Half of the vertical field of view height is The horizontal distance from the camera to the target object is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is At this time, the laser gimbal's pitch angle The calculation process is as follows:

[0019]

[0020] in This indicates the inversion of the tangent value.

[0021] Preferably, when the center of the target object is located below the camera and below the laser, the laser is set to illuminate the center of the target object, and the optical camera, laser, and forward-looking sonar are set to be on the same vertical line with no horizontal distance difference but a vertical distance difference; half of the camera's vertical field of view is set to... Half of the vertical field of view height is The horizontal distance from the camera to the target object is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is At this time, the laser gimbal's pitch angle The calculation process is as follows:

[0022]

[0023] in This indicates the inversion of the tangent value.

[0024] Preferably, when the center of the target object is located above the camera and below the laser, the laser is set to illuminate the center of the target object, and the optical camera, laser, and forward-looking sonar are set to be on the same vertical line with no horizontal distance difference but a vertical distance difference; half of the camera's vertical field of view is set to... Half of the vertical field of view height is The horizontal distance from the camera to the target object is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is At this time, the tilt angle of the laser gimbal The calculation process is as follows:

[0025]

[0026] in This indicates the inversion of the tangent value.

[0027] Preferably, step 3 involves calculating the pitch angle of the light source gimbal. The specific process is as follows:

[0028] Combined with the tilt angle of the laser gimbal Based on the calculation results, a right-angled triangle geometric relationship model of the LED light source, laser, and target object in the vertical direction was established, and the known vertical installation height difference between the LED light source and the laser was obtained. The equation relating the vertical height of the target object relative to the laser and the vertical height of the target object relative to the LED light source;

[0029] Combined with the horizontal distance between the target object and the optical camera or LED light source With the tilt angle of the laser gimbal The tangent relationship is used to calculate the vertical height of the target object relative to the laser; combined with the horizontal distance... Vertical tilt angle of the LED light source pan-tilt head The tangent relationship, using The vertical height of the target object relative to the LED light source is shown; finally, based on the known vertical installation height difference between the LED light source and the laser... An equation is established to determine the vertical height of the target object relative to the laser, the vertical height of the target object relative to the LED light source, and the vertical pitch angle of the LED light source gimbal. .

[0030] Preferably, when the laser is above the light source and the target is horizontally above both, or when the laser is below the light source and the target is horizontally below both, the optical camera, laser, light source, and forward-looking sonar are all positioned on the same vertical line, with no horizontal distance difference but a vertical distance difference; based on the calculated laser gimbal pitch angle... The horizontal distance between the target and the optical camera is set to... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source toward the target object is Then the pitch angle of the light source pan-tilt unit The result can be obtained by calculating using the following formula:

[0031] =

[0032] in, This indicates the tangent of an angle.

[0033] Preferably, when the laser is above the light source and the target is horizontally below both, or when the laser is below the light source and the target is horizontally above both, the optical camera, laser, light source, and forward-looking sonar are all positioned on the same vertical line, with no horizontal distance difference but a vertical distance difference; based on the calculated laser gimbal pitch angle... The horizontal distance between the target and the optical camera is set to... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source toward the target object is Then the pitch angle of the light source pan-tilt unit The result can be obtained by calculating using the following formula:

[0034] =

[0035] in, This indicates the tangent of an angle.

[0036] Preferably, when the laser is above the light source and the target is horizontally positioned between them, or when the laser is below the light source and the target is horizontally positioned between them, the optical camera, laser, light source, and forward-looking sonar are all set to be on the same vertical line, with no horizontal distance difference but a vertical distance difference; based on the calculated laser gimbal pitch angle... The horizontal distance between the target and the optical camera is set to... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source toward the target object is Then the pitch angle of the light source pan-tilt unit satisfy:

[0037] =

[0038] in, This indicates the tangent of an angle.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] This invention overcomes the technical limitation of forward-looking sonar in acquiring elevation information: traditional two-dimensional sonar can only provide the horizontal distance and orientation of a target. It utilizes laser scanning to detect the upper and lower boundaries of the target, extending acoustic and optical data from two dimensions to three dimensions, significantly improving environmental understanding. It achieves near-three-dimensional detection capabilities without the need for expensive three-dimensional sonar equipment.

[0041] Achieving precise directional illumination from active light sources and improving optical imaging quality: The left and right deflection angles and up and down pitch angles of the LED light source can be adjusted in real time according to the target spatial coordinates to achieve fixed-point delivery.

[0042] Experimental results show that this method has low computational cost, is easy to implement in real time, and is suitable for underwater mobile platforms. The overall calculation of this method is based on geometric models and simple image processing, without relying on deep learning models or complex 3D reconstruction algorithms, and can run in real time on embedded platforms such as ROVs. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the overall process flow of the present invention.

[0044] Figure 2 Flowchart for obtaining laser spot coordinates.

[0045] Figure 3 This is a schematic diagram of laser scanning onto the metal casing in the embodiment.

[0046] Figure 4 This is a schematic diagram of the laser gimbal pitch angle calculation model.

[0047] Figure 5 This is a schematic diagram illustrating the acquisition of target information by sonar in an embodiment.

[0048] Figure 6 This is a schematic diagram of the LED light source gimbal pitch angle calculation model.

[0049] Figure 7 The image shows a comparison of the metal casing light source orientation compensation before (left) and after (right) in the embodiment.

[0050] Figure 8 The example shows a comparison of the disk light source before (left) and after (right) orientation compensation.

[0051] Figure 9 The example shows a comparison of the yellow ROV light source before (left) and after (right) directional compensation. Detailed Implementation

[0052] This invention proposes a method for upscaling acoustic-optical fusion data and guiding the delivery of light sources. By constructing a joint sensing system using forward-looking sonar, optical camera, laser, and LED light source controllable by a gimbal, the method acquires the horizontal position information of the target object and supplements the elevation information, completes the three-dimensional correspondence of the acoustic-optical image, and thereby achieves the precise directional delivery of active light sources.

[0053] This invention enables the determination of the upper and lower boundary positions of a target using laser scanning, even when conventional sonar cannot provide elevation information. This provides accurate data for controlling the elevation and deflection angles of the light source, thereby improving the imaging quality of underwater optical images and enhancing the applicability of underwater detection systems in low-visibility environments. The overall process is as follows: Figure 1 As shown.

[0054] Step 1, laser spot coordinate acquisition. The laser is controlled to scan the target object, and an optical camera is used to acquire the image formed by the laser reflection. Through image preprocessing, region of interest definition, threshold segmentation, contour detection, and center coordinate calculation, the pixel coordinates of the laser spot in the optical image coordinate system are obtained.

[0055] Step 2: Calculate the vertical pitch angle of the laser gimbal. Based on the laser spot pixel coordinates obtained in Step 1, combined with camera imaging parameters, the relative installation positions of the camera and laser, and the horizontal distance information of the target object obtained by forward-looking sonar, a spatial geometric relationship model is established to calculate the vertical pitch angle of the laser gimbal, which is used to compensate for the elevation information of the target object.

[0056] Step 3, LED light source gimbal angle calculation. Based on the horizontal position information of the target obtained by forward-looking sonar and the pitch angle of the laser gimbal calculated in Step 2, the left and right deflection angles and vertical pitch angles of the LED light source gimbal are calculated, and the LED light source is controlled to face the target position for directional illumination compensation.

[0057] The implementation process of the present invention will be further described below with reference to specific embodiments.

[0058] Example 1:

[0059] I. Acquisition of Laser Spot Coordinates

[0060] A color image captured by an optical camera is acquired, and the color image is converted to grayscale according to a preset color space conversion rule, converting the multi-channel color image into a single-channel grayscale image. Subsequently, the grayscale image is smoothed using a Gaussian filter. By setting the Gaussian filter kernel size and standard deviation parameters, high-frequency noise in the image is suppressed to obtain a pre-processed image with reduced noise interference and continuous laser spot edges.

[0061] Based on the sonar images acquired by the forward-looking sonar, the target objects in the sonar images are detected, and the left and right boundary positions of the target objects in the sonar images are obtained. Based on the pre-established coordinate correspondence between the sonar images and the optical images, the left and right boundary positions of the target objects are mapped to the optical image coordinate system, thereby determining the region of interest corresponding to the target objects in the optical images. Subsequent laser spot detection operations are performed in the region of interest.

[0062] Within the defined region of interest, a brightness threshold is set on the preprocessed grayscale image, and a threshold segmentation operation is performed on the grayscale image to generate a binary image. Pixels with brightness values ​​higher than the threshold are marked as foreground pixels, and pixels with brightness values ​​lower than or equal to the threshold are marked as background pixels, so as to obtain a binary image for laser spot detection.

[0063] Contour extraction is performed on the obtained binary image. Multiple candidate contours in the image are obtained by analyzing the connectivity between pixels. The area of ​​the region enclosed by each candidate contour is calculated, and the candidate contours are sorted according to their area. The contour with the largest area is selected as the target contour of the laser reflection spot for subsequent calculation of the center position of the spot.

[0064] Based on the target contour of the laser reflection spot obtained from the above steps, a minimum bounding rectangle is constructed to enclose the target contour; by calculating the coordinates of the diagonal vertices of the minimum bounding rectangle, the center position of the rectangle in the optical image coordinate system is obtained, and the center position is used as the center pixel coordinate of the laser spot.

[0065] II. Calculation of Laser Gimbal Pitch Angle

[0066] The laser is positioned so that it illuminates the center of the target object. The optical camera, laser, and forward-looking sonar are aligned vertically, with no horizontal distance difference but a vertical distance difference. The calculated pitch angle of the laser gimbal is then determined. Set half of the camera's vertical field of view (VFOV) to Half of the vertical field of view height is The horizontal distance from the camera to the target obtained by the forward-looking sonar is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is .

[0067] The image captured by the optical camera is obtained, and the laser spot coordinates, i.e., the pixel coordinates of the target object's center position, are obtained according to step 1. The height difference between the camera's imaging field of view center and the target object center is calculated based on these coordinates. Pixel distance in the corresponding optical image Set the number of pixels that occupy half of the camera's viewfinder to be [number]. .

[0068] The pitch angle of the laser gimbal The calculation results include:

[0069] Scenario 1:

[0070] Scenario 2:

[0071] Scenario 3:

[0072] Three situations, among which This indicates the inversion of the tangent value.

[0073] In case 1, the calculation method is as follows:

[0074] Set the laser to illuminate the center of the target object. Ensure the optical camera, laser, and forward-looking sonar are aligned vertically with no horizontal distance difference but a vertical distance difference. Position the target object's center below the camera and above the laser. Set half of the camera's vertical field of view to... Half of the vertical field of view height is The horizontal distance from the camera to the target object is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is The pitch angle of the laser is In this model, the pitch angle of the laser... satisfy:

[0075]

[0076] in, Indicates the pitch angle of the laser The tangent value.

[0077] Half the vertical field of view of the camera satisfy:

[0078]

[0079] set up The height of the image is the pixel height it occupies. The number of pixels occupying half of the camera's field of view is The conversion relationship between physical length and number of pixels is as follows:

[0080]

[0081] After sorting, we can obtain:

[0082]

[0083] Finally, we can obtain The calculation results are as follows:

[0084]

[0085] Substitution We can obtain:

[0086]

[0087] in This indicates the inversion of the tangent value.

[0088] In case 2, the calculation method is as follows:

[0089] Set the laser to illuminate the center of the target object. Ensure the optical camera, laser, and forward-looking sonar are aligned vertically with no horizontal distance difference but a vertical distance difference. Position the target object's center below the camera and below the laser. Set half of the camera's vertical field of view to... Half of the vertical field of view height is The horizontal distance from the camera to the target object is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is The pitch angle of the laser is In this model, the pitch angle of the laser... satisfy:

[0090]

[0091] in, Indicates the pitch angle of the laser The tangent value.

[0092] Half the vertical field of view of the camera satisfy:

[0093]

[0094] set up The height of the image is the pixel height it occupies. The number of pixels occupying half of the camera's field of view is The conversion relationship between physical length and number of pixels is as follows:

[0095]

[0096] After sorting, we can obtain:

[0097]

[0098] Finally, we can obtain The calculation results are as follows:

[0099]

[0100] Substitution We can obtain:

[0101]

[0102] in This indicates the inversion of the tangent value.

[0103] In case 3, the calculation method is as follows:

[0104] Set the laser to illuminate the center of the target object. Ensure the optical camera, laser, and forward-looking sonar are aligned vertically with no horizontal distance difference but a vertical distance difference. Position the target object's center above the camera and below the laser. Set half of the camera's vertical field of view to... Half of the vertical field of view height is The horizontal distance from the camera to the target object is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is The pitch angle of the laser is In this model, the pitch angle of the laser... satisfy:

[0105]

[0106] in, Indicates the pitch angle of the laser The tangent value.

[0107] Half the vertical field of view of the camera satisfy:

[0108]

[0109] set up The height of the image is the pixel height it occupies. The number of pixels occupying half of the camera's field of view is The conversion relationship between physical length and number of pixels is as follows:

[0110]

[0111] After sorting, we can obtain:

[0112]

[0113] Finally, we can obtain The calculation results are as follows:

[0114]

[0115] Substitution We can obtain:

[0116]

[0117] in This indicates the inversion of the tangent value.

[0118] III. LED Light Source Gimbal Angle Calculation

[0119] Set the origin position of the forward-looking sonar detection as The location of the target detected by the forward-looking sonar is , and The distance is , angle is The location of the target object The polar coordinates are Set the target location. The rectangular coordinates are According to the conversion formula between rectangular coordinates and polar coordinates, we can obtain The coordinates in the forward-looking sonar image are:

[0120]

[0121] in:

[0122]

[0123] Set the left and right tilt angle of the LED light source gimbal to be... The position of the LED light source in a Cartesian coordinate system with the previous sonar as the origin is: Then its orientation toward the target object position The angles of left and right deflection are:

[0124]

[0125] in This indicates the inverse of the tangent value, representing the position of the LED light source. Given quantities If the value is less than 0, the gimbal will tilt to the left; If the value is greater than 0, the gimbal will tilt to the right.

[0126] The laser gimbal pitch angle was calculated. The optical camera, laser, light source, and forward-looking sonar are all positioned on the same vertical line, with no horizontal distance difference but a vertical distance difference. The horizontal distance between the target and the optical camera is set to... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source towards the target object (i.e., the pitch angle of the light source pan-tilt unit) is: The pitch angle of the light source gimbal. The calculation results are as follows:

[0127] Scenario 1:

[0128] Scenario 2:

[0129] Scenario 3:

[0130] in, This indicates the inversion of the tangent value.

[0131] In case 1, the calculation method is as follows:

[0132] The laser is positioned above the light source, with the target horizontally positioned above both, or the laser is positioned below the light source, with the target horizontally positioned below both. The optical camera, laser, light source, and forward-looking sonar are all positioned on the same vertical line, with no horizontal distance difference but a vertical distance difference. Based on step 2, the laser gimbal pitch angle is calculated. Set the horizontal distance between the target object and the optical camera to be... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source towards the target object (i.e., the pitch angle of the light source pan-tilt unit) is: Then the elevation angle of the light source pan-tilt unit. satisfy:

[0133]

[0134] After sorting, we can obtain:

[0135] =

[0136] in, This indicates the tangent of an angle.

[0137] In case 2, the calculation method is as follows:

[0138] The laser is positioned above the light source, with the target horizontally positioned below either of them, or the laser is below the light source, with the target horizontally positioned above both. The optical camera, laser, light source, and forward-looking sonar are all positioned on the same vertical line, with no horizontal distance difference but a vertical distance difference. The laser gimbal pitch angle is calculated. Set the horizontal distance between the target object and the optical camera to be... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source towards the target object (i.e., the pitch angle of the light source pan-tilt unit) is: Then the elevation angle of the light source pan-tilt unit. satisfy:

[0139]

[0140] After sorting, we can obtain:

[0141] =

[0142] in, This indicates the tangent of an angle.

[0143] In case 3, the calculation method is as follows:

[0144] The laser is positioned above the light source, with the target horizontally positioned between the two, or the laser is below the light source, with the target horizontally positioned between the two. The optical camera, laser, light source, and forward-looking sonar are all positioned on the same vertical line, with no horizontal distance difference but a vertical distance difference. The laser gimbal pitch angle is calculated. Set the horizontal distance between the target object and the optical camera to be... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source towards the target object (i.e., the pitch angle of the light source pan-tilt unit) is: Then the elevation angle of the light source pan-tilt unit. satisfy:

[0145]

[0146] After sorting, we can obtain:

[0147] =

[0148] in, This indicates the tangent of an angle.

[0149] Example 2:

[0150] I. Equipment Used in the Example

[0151] This example uses an ROV equipped with forward-looking sonar, a monocular camera, a laser, and an LED light source as an experimental platform to verify the detection effect of underwater targets under the combined guidance of laser and sonar. This includes verifying the laser-assisted elevation information and the effect of directional illumination compensation for the target. Equipment parameters are shown in Table 1.

[0152] Table 1. Partial equipment parameters of the ROV used in the example.

[0153]

[0154] The target object was selected using a yellow metal casing, a checkerboard pattern, and a yellow ROV.

[0155] II. Laser Spot Acquisition

[0156] A color image captured by an optical camera is acquired, and the color image is converted to grayscale according to a preset color space conversion rule, converting the multi-channel color image into a single-channel grayscale image. Subsequently, the grayscale image is smoothed using a Gaussian filter. By setting the Gaussian filter kernel size and standard deviation parameters, high-frequency noise in the image is suppressed to obtain a pre-processed image with reduced noise interference and continuous laser spot edges.

[0157] Based on the sonar images acquired by the forward-looking sonar, the target objects in the sonar images are detected, and the left and right boundary positions of the target objects in the sonar images are obtained. Based on the pre-established coordinate correspondence between the sonar images and the optical images, the left and right boundary positions of the target objects are mapped to the optical image coordinate system, thereby determining the region of interest corresponding to the target objects in the optical images. Subsequent laser spot detection operations are performed in the region of interest.

[0158] Within the defined region of interest, a brightness threshold is set on the preprocessed grayscale image, and a threshold segmentation operation is performed on the grayscale image to generate a binary image. Pixels with brightness values ​​higher than the threshold are marked as foreground pixels, and pixels with brightness values ​​lower than or equal to the threshold are marked as background pixels, so as to obtain a binary image for laser spot detection.

[0159] Contour extraction is performed on the obtained binary image. Multiple candidate contours in the image are obtained by analyzing the connectivity between pixels. The area of ​​the region enclosed by each candidate contour is calculated, and the candidate contours are sorted according to their area. The contour with the largest area is selected as the target contour of the laser reflection spot for subsequent calculation of the center position of the spot.

[0160] Based on the target contour of the laser reflection spot obtained from the above steps, a minimum bounding rectangle is constructed to enclose the target contour; by calculating the coordinates of the diagonal vertices of the minimum bounding rectangle, the center position of the rectangle in the optical image coordinate system is obtained, and the center position is used as the center pixel coordinate of the laser spot.

[0161] Set the vertical coordinates of the upper and lower boundaries of the target object as follows: ,in The ordinate of the upper boundary. Let be the ordinate of the lower boundary. Set the ordinate of the target object's center as . The pixel coordinates of the obtained laser spot are shown in Table 2. A schematic diagram of the steps for obtaining the laser spot image coordinates is shown below. Figure 2 As shown in the diagram. A schematic diagram of the laser scanning of the target object in the example is shown below. Figure 3 As shown.

[0162] Table 2. Pixel ordinate information obtained from the laser spot

[0163]

[0164] III. Calculation of Laser Gimbal Pitch Angle

[0165] In this example, the detection points are all positioned below the camera and laser. Therefore, the laser gimbal elevation angle is calculated as follows:

[0166] Set the laser to illuminate the center of the target object. Ensure the optical camera, laser, and forward-looking sonar are aligned vertically, with no horizontal distance difference but a vertical distance difference. Set half of the camera's vertical field of view to... Half of the vertical field of view height is The horizontal distance from the camera to the target object is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is The pitch angle of the laser is In this model, the pitch angle of the laser... satisfy:

[0167]

[0168] in, Indicates the pitch angle of the laser The tangent value.

[0169] Half the vertical field of view of the camera satisfy:

[0170]

[0171] set up The height of the image is the pixel height it occupies. The number of pixels occupying half of the camera's field of view is The conversion relationship between physical length and number of pixels is as follows:

[0172]

[0173] After sorting, we can obtain:

[0174]

[0175] Finally, we can obtain The calculation results are as follows:

[0176]

[0177] in This means negating the tangent value, by substituting... We can obtain:

[0178]

[0179] In this example, the corresponding laser gimbal angle calculation model is as follows: Figure 4 As shown

[0180] IV. LED Light Source Gimbal Angle Calculation

[0181] Set the origin position of the forward-looking sonar detection as The location of the target detected by the forward-looking sonar is , and The distance is , angle is The location of the target object The polar coordinates are Set the target location. The rectangular coordinates are According to the conversion formula between rectangular coordinates and polar coordinates, we can obtain The coordinates in the forward-looking sonar image are:

[0182]

[0183] in:

[0184]

[0185] Set the left and right tilt angle of the LED light source gimbal to be... The position of the LED light source in a Cartesian coordinate system with the previous sonar as the origin is: Then its orientation toward the target object position The angles of left and right deflection are:

[0186]

[0187] in This indicates the inverse of the tangent value, representing the position of the LED light source. Given quantities If the value is less than 0, the gimbal will tilt to the left; If the value is greater than 0, the gimbal deflects to the right. A diagram illustrating sonar acquisition of target location information is shown below. Figure 5 As shown.

[0188] In this embodiment, the laser is positioned above the light source, and the target object is horizontally positioned below both. The optical camera, laser, light source, and forward-looking sonar are all positioned on the same vertical line, with no horizontal distance difference but a vertical distance difference. The laser gimbal pitch angle is calculated according to step 2. Set the horizontal distance between the target object and the optical camera to be... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source towards the target object (i.e., the pitch angle of the light source pan-tilt unit) is: Then the elevation angle of the light source pan-tilt unit. satisfy:

[0189]

[0190] in, This indicates the tangent of an angle.

[0191] After sorting, we can obtain: = .

[0192] The calculation results are shown in Table 3. The calculation model for the pitch angle of the LED light source gimbal in this example is as follows: Figure 6 As shown.

[0193] Table 3 Calculation of Directional Illumination Compensation Angle for LED Light Source

[0194]

[0195] In this example, the final result achieved is the directional delivery of the light source, as shown below. Figure 7 , Figure 8 , Figure 9 As shown in the figure, it can be seen that the light source can be accurately projected to the target location using this method.

[0196] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0197] While the specific embodiments of the present invention have been described above, they are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for upscaling acoustic-optical fusion data and guiding light source projection, characterized in that, Includes the following steps: Step 1: Control the laser to scan the target object, use an optical camera to acquire the image formed by laser reflection, and obtain the pixel coordinates of the laser spot in the optical image coordinate system through image preprocessing, region of interest definition, threshold segmentation, contour detection and center coordinate calculation. Step 2: Based on the acquired laser spot pixel coordinates, combined with the optical camera imaging parameters, the relative installation positions of the optical camera and the laser, and the horizontal distance information of the target object obtained by the forward-looking sonar, a spatial geometric relationship model is established to calculate the pitch angle of the laser gimbal, which is used to compensate for the elevation information of the target object; calculate the pitch angle of the laser gimbal. The specific process is as follows: Based on the vertical pixel offset between the camera's imaging field of view center and the target object center Combined with the number of pixels occupying half of the camera's field of view and half the vertical field of view height offset the pixel Mapped to the actual vertical height difference between the center of the target object and the center of the camera's imaging field of view. This allows us to obtain the elevation information of the target object in space; simultaneously, based on the known installation position relationship between the camera and the laser, we obtain the vertical height difference between the laser and the camera. ; The vertical height difference between the target object and the camera Vertical height difference between the laser and the camera Depending on the different positional relationships between the target, camera, and laser, the equivalent vertical height difference between the target and the laser is obtained by superposition or difference calculation; this is then combined with the horizontal distance from the camera or laser to the target measured by forward-looking sonar. Construct a right-angled triangle geometric model with the laser as the emission point and the center of the target object as the pointing point; Based on this geometric model, the equivalent vertical height difference between the target object and the laser is used. Horizontal distance from the target The tangent relationship between them is used to calculate the required pitch angle of the laser gimbal. This ensures that the laser's output optical axis can accurately point to the center of the target object; Step 3: Based on the horizontal position information of the target obtained by the forward-looking sonar and the pitch angle of the laser gimbal calculated in Step 2, calculate the left and right yaw angle and the up and down pitch angle of the LED light source gimbal, and control the LED light source to face the target position for directional illumination compensation.

2. The method for upscaling acoustic-optical fusion data and guiding light source projection as described in claim 1, characterized in that: Step 1 specifically includes: A color image captured by an optical camera is acquired, and the color image is converted to grayscale according to a preset color space conversion rule, converting the multi-channel color image into a single-channel grayscale image; subsequently, the grayscale image is smoothed using a Gaussian filter, and high-frequency noise in the image is suppressed by setting the Gaussian filter kernel size and standard deviation parameters. Based on the sonar images acquired by the forward-looking sonar, the target objects in the sonar images are detected, and the left and right boundary positions of the target objects in the sonar images are obtained. Based on the pre-established coordinate correspondence between the sonar images and the optical images, the left and right boundary positions of the target objects are mapped to the optical image coordinate system, and the region of interest corresponding to the target objects is determined in the optical image. Within the defined region of interest, a brightness threshold is set for the preprocessed grayscale image, and a threshold segmentation operation is performed on the grayscale image to generate a binary image. Pixels with brightness values ​​higher than the threshold are marked as foreground pixels, and pixels with brightness values ​​lower than or equal to the threshold are marked as background pixels, in order to obtain a binarized image for laser spot detection. Contour extraction is performed on the obtained binary image. Multiple candidate contours in the image are obtained by analyzing the connectivity between pixels. The area of ​​the region enclosed by each candidate contour is calculated, and the candidate contours are sorted according to the area. The contour with the largest area is selected as the target contour of the laser reflection spot. Based on the obtained target contour of the laser reflection spot, a minimum bounding rectangle is constructed to enclose the target contour; by calculating the coordinates of the diagonal vertices of the minimum bounding rectangle, the center position of the rectangle in the optical image coordinate system is obtained, and the center position is used as the center pixel coordinate of the laser spot.

3. The method for upscaling acoustic-optical fusion data and guiding light source projection as described in claim 1, characterized in that: When the center of the target is located below the camera and above the laser, the laser is set to illuminate the center of the target. The optical camera, laser, and forward-looking sonar are set to be on the same vertical line, with no horizontal distance difference but a vertical distance difference. Half of the camera's vertical field of view is set to... Half of the vertical field of view height is The horizontal distance from the camera to the target object is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is At this time, the laser gimbal's pitch angle The calculation process is as follows: in This indicates the inversion of the tangent value.

4. The method for upscaling acoustic-optical fusion data and guiding light source projection as described in claim 1, characterized in that: When the center of the target is located below the camera and below the laser, the laser is set to illuminate the center of the target. The optical camera, laser, and forward-looking sonar are set to be on the same vertical line, with no horizontal distance difference but a vertical distance difference. Half of the camera's vertical field of view is set to... Half of the vertical field of view height is The horizontal distance from the camera to the target object is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is At this time, the laser gimbal's pitch angle The calculation process is as follows: in This indicates the inversion of the tangent value.

5. The method for upscaling acoustic-optical fusion data and guiding light source projection as described in claim 1, characterized in that: When the center of the target is above the camera and below the laser, the laser is set to illuminate the center of the target. The optical camera, laser, and forward-looking sonar are set to be on the same vertical line, with no horizontal distance difference but a vertical distance difference. Half of the camera's vertical field of view is set to... Half of the vertical field of view height is The horizontal distance from the camera to the target object is The vertical distance between the camera and the laser is The height difference between the center of the camera's imaging field of view and the center of the target object is At this time, the tilt angle of the laser gimbal The calculation process is as follows: in This indicates the inversion of the tangent value.

6. The method for upscaling acoustic-optical fusion data and guiding light source projection as described in claim 1, characterized in that: Step 3 calculates the pitch angle of the light source gimbal. The specific process is as follows: Combined with the tilt angle of the laser gimbal Based on the calculation results, a right-angled triangle geometric relationship model of the LED light source, laser, and target object in the vertical direction was established, and the known vertical installation height difference between the LED light source and the laser was obtained. The equation relating the vertical height of the target object relative to the laser and the vertical height of the target object relative to the LED light source; Combined with the horizontal distance between the target object and the optical camera or LED light source With the tilt angle of the laser gimbal The tangent relationship is used to calculate the vertical height of the target object relative to the laser; combined with the horizontal distance... Vertical tilt angle of the LED light source pan-tilt head The tangent relationship, using The vertical height of the target object relative to the LED light source is shown; finally, based on the known vertical installation height difference between the LED light source and the laser... An equation is established to determine the vertical height of the target object relative to the laser, the vertical height of the target object relative to the LED light source, and the vertical pitch angle of the LED light source gimbal. .

7. The method for upscaling acoustic-optical fusion data and guiding light source projection as described in claim 6, characterized in that: When the laser is above the light source and the target is horizontally above both, or when the laser is below the light source and the target is horizontally below both, the optical camera, laser, light source, and forward-looking sonar are all positioned on the same vertical line, with no horizontal distance difference but a vertical distance difference; the laser gimbal pitch angle is calculated accordingly. The horizontal distance between the target and the optical camera is set to... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source toward the target object is Then the pitch angle of the light source pan-tilt unit The result can be obtained by calculating using the following formula: = in, This indicates the tangent of an angle.

8. The method for upscaling acoustic-optical fusion data and guiding light source projection as described in claim 6, characterized in that: When the laser is above the light source and the target is horizontally below both, or when the laser is below the light source and the target is horizontally above both, the optical camera, laser, light source, and forward-looking sonar are all positioned on the same vertical line, with no horizontal distance difference but a vertical distance difference; the laser gimbal pitch angle is calculated accordingly. The horizontal distance between the target and the optical camera is set to... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source toward the target object is Then the pitch angle of the light source pan-tilt unit The result can be obtained by calculating using the following formula: = in, This indicates the tangent of an angle.

9. The method for upscaling acoustic-optical fusion data and guiding light source projection as described in claim 6, characterized in that: When the laser is above the light source and the target is horizontally positioned between them, or when the laser is below the light source and the target is horizontally positioned between them, the optical camera, laser, light source, and forward-looking sonar are all set to be on the same vertical line, with no horizontal distance difference but a vertical distance difference; based on the calculated laser gimbal pitch angle... The horizontal distance between the target and the optical camera is set to... The vertical distance difference between the laser and the LED light source is The vertical pitch angle of the LED light source toward the target object is Then the pitch angle of the light source pan-tilt unit satisfy: = in, This indicates the tangent of an angle.

Images