An underwater scene distributed collaborative three-dimensional modeling method and system

Abstract

The application relates to the technical field of three-dimensional modeling, and particularly discloses a kind of underwater scene distributed collaborative three-dimensional modeling method and system, the method scans overlapping area by identifying platform and selects stable color feature point as color anchor point, adopts the multiscale feature description method of combining SIFT feature and Lab color space, carries out weighted evaluation based on color variance, spatial distribution uniformity and texture richness;Establish RGB three-channel depth attenuation compensation model, and carries out color attenuation correction to different depth images;Color anchor point information is used to calculate each platform dynamic white balance correction parameter, and eliminate the color temperature difference between platforms;Finally, a global color consistency optimization objective function is established, and the color of multiple platform images is unified through iterative optimization. The application is suitable for different water quality environments and depth ranges, and provides a high-quality three-dimensional modeling technical solution.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of three-dimensional modeling, and particularly relates to a distributed collaborative three-dimensional modeling method and system for underwater scenes. BACKGROUND

[0002] Underwater scene three-dimensional modeling technology has become an important means to obtain seabed topography, structure state and biological distribution information. The traditional single-platform underwater modeling method has the problems of limited coverage and low modeling efficiency, while distributed multi-platform collaborative modeling can greatly improve the operation efficiency and modeling accuracy, and has become the development trend of underwater three-dimensional modeling technology. However, the complexity of the underwater optical environment makes multi-platform collaborative modeling face unprecedented technical challenges.

[0003] The existing multi-platform collaborative three-dimensional modeling technology mainly faces the problem of color consistency. Due to the selective absorption and scattering of light in the underwater environment, there are serious color differences in images collected by different depths and different platforms, resulting in obvious color jumps and splicing artifacts in the spliced three-dimensional model. The current color correction method mainly uses simple linear transformation or correction strategy based on a single reference point, which cannot effectively handle the dynamic optical property changes of the underwater environment, and the correction effect is limited. At the same time, there is a lack of global optimization strategy for color consistency among multiple platforms, which cannot achieve true color uniformity, seriously affecting the visual quality and application value of the three-dimensional model.

[0004] Especially in key applications such as deep-sea oil and gas development, submarine cable maintenance, coral reef ecological monitoring, etc., the color authenticity and consistency of the three-dimensional model are extremely high, therefore, it is urgent to develop an underwater collaborative modeling technology that can effectively solve the color consistency problem of multi-platform images. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of the prior art, and to provide a distributed collaborative three-dimensional modeling method and system for underwater scenes, which establishes a multi-level color correction system based on color anchor points, effectively solves the color consistency problem of multi-platform collected images, realizes high-quality underwater three-dimensional model reconstruction, and meets the technical needs of precise three-dimensional modeling in the fields of marine engineering, underwater archaeology, ecological monitoring, etc.

[0006] In a first aspect, the present application provides a distributed collaborative three-dimensional modeling method for underwater scenes, characterized in that the method is used to solve the color consistency problem of multi-platform collected images in distributed collaborative three-dimensional modeling, and comprises the following steps:

[0007] Step S1, identifying at least two platform scanning overlapping areas, and selecting stable color feature points in the scanning overlapping areas as color anchor points;

[0008] Step S2, according to the scanning depth and scanning color value of each platform, a depth attenuation compensation model of RGB three channels is established, and color attenuation correction is performed on images collected at different depths;

[0009] Step S3, dynamic white balance correction parameters of each platform are calculated by using color anchor point information, and color temperature differences between different platforms are eliminated;

[0010] Step S4, a global color consistency optimization objective function is established, and color unification of multi-platform images is realized through iterative optimization.

[0011] Further, the color anchor point adopts a multi-scale feature description method, and combines SIFT features and Lab color space information to describe anchor point features;

[0012] The color anchor point selection standard is a weighted evaluation method based on color variance, spatial distribution uniformity and texture richness: ; wherein, is the i th candidate anchor point; is the candidate anchor point score, is the color variance of the anchor point between different platforms; is the spatial distribution uniformity of the candidate anchor point; is the texture richness of the candidate anchor point; The time sequence stability of the candidate anchor point is calculated:

[0013] ; wherein, is the time sequence stability of the anchor point ; is the time sequence standard deviation of the color of the anchor point ; is the time sequence mean of the color of the anchor point ; is the color value of the anchor point . Further, the K candidate anchor points with the highest stability value

[0014] are selected as the color anchor points, and K is not less than 2.

[0015] Further, a depth attenuation compensation model of RGB three channels is established: ; wherein: is the corrected RGB color vector; , , are the corrected red, green and blue channel values respectively; , , are the observed original red, green and blue channel values respectively;​ is the scanning depth; , , are the attenuation coefficients of red, green and blue channels respectively, the red light attenuation coefficient is 0.2-0.8 , the green light attenuation coefficient is 0.1-0.4 , and the blue light attenuation coefficient is 0.05-0.2 .

[0016] Further, based on the color anchor point information, the white balance correction matrix of each platform is calculated by minimizing the objective function: ; wherein, is the white balance correction matrix of the th platform; N is the total number of color anchor points; is the color vector of the th platform at the th anchor point; is the color vector of the reference platform at the th anchor point;

[0017] The form of the white balance correction matrix is: ; wherein, , , are the white balance correction coefficients of the red, green and blue channels of the th platform respectively, with a value range of 0.5-2.0;

[0018] The reference platform selection criterion is the platform that minimizes the overall color deviation:

[0019] ; wherein: is the number of the selected reference platform; is the average color of all platforms at the anchor point .

[0020] Further, a global color consistency optimization objective function is established: ; wherein, is the overall color inconsistency error; is the total number of platforms; is the overlapping pixel set of platforms and ; is the weight of pixel between platforms and ; and respectively are the final corrected color of the two platforms at the overlapping area and the final corrected color at the pixel .

[0021] Further, an evaluation index Q of consistency of the final corrected color is established to monitor the correction effect in real time;

[0022] wherein, The larger the value is, the better the consistency is; , respectively are the final corrected color of the two platforms at the overlapping area is the number of pixels in the overlapping area; factor 255² represents the maximum square difference of color value; when the quality index Q is lower than a preset threshold 0.85, the recorrection process is automatically triggered.

[0023] Based on the same inventive concept, in a second aspect, the present application provides a distributed collaborative three-dimensional modeling system for underwater scenes, which is used to execute the method of the first aspect; the system comprises: a plurality of underwater acquisition platforms, a platform communication unit, a color correction unit and a storage unit.

[0024] Further, the plurality of underwater acquisition platforms are integrated with a visible light camera, a near-infrared camera and a spectrometer, which are used to acquire multispectral image data and water optical parameters, the resolution of the visible light camera is not less than 1920×1080 pixels, and the working wavelength range of the spectrometer is 400-800 nm.

[0025] The communication unit transmits the data acquired by the acquisition platforms to the color correction unit, the communication unit supports a communication distance of not less than 500 m and a data transmission rate of not less than 10 Mbps.

[0026] The color correction unit comprises a color correction model, which is used to unify the colors of the multi-platform images.

[0027] The storage unit is used to store the original acquisition data and the corrected data.

[0028] The present application has the following beneficial effects:

[0029] ​The application realizes high color consistency of underwater multi-platform images by establishing a multi-level color correction system based on color anchor points, reduces the color difference of multi-platform images from 30% average deviation of traditional methods to below 5%, and improves the color consistency by 85%; effectively eliminates the color jump problem during image stitching, significantly improves the texture quality of the three-dimensional model; suitable for different water quality environments, depth ranges and lighting conditions, with strong environmental adaptability and robustness; the color correction time of a single frame image is less than 100 milliseconds, meeting the real-time processing requirements; still maintains good correction effect under the condition of 10% color anchor point loss, with high system stability, and provides a complete technical solution for underwater collaborative three-dimensional modeling. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A flow chart of a distributed collaborative three-dimensional modeling method for an underwater scene according to the application;

[0031] Figure 2 A schematic diagram of a distributed collaborative three-dimensional modeling system for an underwater scene according to the application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical scheme and advantages of the application clearer, the technical scheme in the application is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the application.

[0033] Embodiment 1

[0034] As shown in the flow chart of the distributed collaborative three-dimensional modeling method for an underwater scene according to the application, the method is used to solve the color consistency problem of multi-platform acquisition images in distributed collaborative three-dimensional modeling, and includes the following steps: Figure 1 Step S1, identifying at least two platform scanning overlapping areas, and selecting stable color feature points in the scanning overlapping areas as color anchor points.

[0035] The color anchor points adopt a multi-scale feature description method combining SIFT features and Lab color space information to describe anchor point features;

[0036] The color anchor point selection standard is a weighted evaluation method based on color variance, spatial distribution uniformity and texture richness:

[0037] ; wherein, is the i-th candidate anchor point; is the candidate anchor point score, ​​Color variance of anchor points between different platforms; Spatial distribution uniformity of candidate anchor points; Texture richness of candidate anchor points;

[0038] Temporal stability calculation of candidate anchor points: ; wherein, Temporal stability of anchor point ; Temporal sequence standard deviation of color of anchor point ; Temporal sequence mean of color of anchor point ; Color value of anchor point ;

[0039] Select the K candidate anchor points with the highest stability value as color anchor points, K is not less than 2.

[0040] When identifying the overlapping area of the platform scanning, first determine the relative position relationship of each platform by SLAM (simultaneous localization and mapping) technology, and calculate the spatial range of the overlapping area. The overlapping area should be not less than 20% of the scanning area of each platform, so as to ensure that there are enough common feature points for color correction.

[0041] For the calculation of spatial distribution uniformity , the nearest neighbor distance method is adopted:

[0042] ; wherein, Distance between anchor point and the jth nearest neighbor anchor point, Adjustment parameter, Desired minimum anchor point spacing (usually set to 1 / 20 of the image width), K is the number of nearest neighbors considered (usually take 5).

[0043] Texture richness is calculated by local binary pattern (LBP) and gradient magnitude:

[0044] ; wherein, LBP variance of anchor point P_i neighborhood, Normalized gradient magnitude.

[0045] For example, in the application of coral reef modeling, the stable color area of the coral surface is selected as the anchor point, and the dynamic target area such as swimming fish and suspended particles is avoided.

[0046] Step S2, according to the scanning depth and scanning color value of each platform, a depth attenuation compensation model of RGB three channels is established, and color attenuation correction is performed on images collected at different depths.

[0047] A depth attenuation compensation model of RGB three channels is established: ; wherein: is the corrected RGB color vector; , , are the corrected red, green and blue channel values respectively; , , are the observed original red, green and blue channel values respectively; is the scanning depth; , , are the attenuation coefficients of red, green and blue channels respectively, the red light attenuation coefficient is 0.2-0.8 , the green light attenuation coefficient is 0.1-0.4 , and the blue light attenuation coefficient is 0.05-0.2 .

[0048] The parameter calibration of the depth attenuation compensation model is completed by field measurement. A standard color card is used to take pictures at different depths, and the attenuation coefficients are obtained by least square fitting. For example, in clear seawater (visibility > 15m), the typical parameters are ; in turbid water (visibility < 5m), the parameters are adjusted to .

[0049] In color correction, for the red channel at a depth of d = 10m, the correction formula is: , which realizes the attenuation compensation of red light.

[0050] Step S3, using color anchor point information, the dynamic white balance correction parameters of each platform are calculated to eliminate the color temperature difference between different platforms.

[0051] Based on the color anchor point information, the white balance correction matrix of each platform is calculated by minimizing the objective function: ; wherein, is the white balance correction matrix of the i th platform; N is the total number of color anchor points; is the color value of the i th platform at the j color vector at the anchor point; for the reference platform in the first color vector at the anchor point;

[0052] The form of the white balance correction matrix is: ; wherein, , , are the white balance correction coefficients of the red, green and blue channels of the first platform, respectively, and the value range is 0.5-2.0;

[0053] The reference platform selection criterion is the platform that minimizes the overall color deviation:

[0054] ; wherein: is the number of the selected reference platform; is the average color of all platforms at the anchor point .

[0055] The solution of the white balance correction matrix adopts the least square method. After setting the objective function, the solution is obtained by the pseudo-inverse matrix:

[0056] ; wherein, is the anchor color matrix of the reference platform, is the pseudo-inverse of the anchor color matrix of the jth platform.

[0057] The selection of the reference platform is achieved by traversing all platforms, calculating the deviation of each platform from the average color, and selecting the platform with the smallest deviation. For example, in a three-platform system, if the total deviation of platform 1 is 1.2, the total deviation of platform 2 is 0.8, and the total deviation of platform 3 is 1.5, then platform 2 is selected as the reference platform.

[0058] Step S4, establish a global color consistency optimization objective function, and realize color unification of multi-platform images through iterative optimization.

[0059] The global color consistency optimization objective function is established as: ; wherein, is the overall color inconsistency error; is the total number of platforms; is the overlapping pixel set of platform and ; is the weight of pixel between platform and ; and are the platforms and the final corrected color at the pixel .

[0060] The global optimization adopts the gradient descent method, the learning rate is set to 0.05, and the maximum iteration number is 50 times. ; Wherein, is a noise variance parameter, and respectively, the intensity value of the platform and at the pixel .

[0061] During the optimization process, when the error change of three consecutive iterations is less than 0.001, it is considered to be converged and the iteration is stopped.

[0062] The consistency evaluation index Q of the final corrected color is established to monitor the correction effect in real time;

[0063] , wherein, The larger the value is, the better the consistency is; , respectively, the RGB value of the final corrected color of the two platforms in the overlapping area; is the number of pixels in the overlapping area; the factor 255 is the maximum square difference of the color value; when the quality index is lower than the preset threshold 0.85, the re-correction process is automatically triggered.

[0064] In a certain coral reef protection area, three AUVs carrying the system of the application are used for collaborative modeling. The operation area is about 2000 square meters, the water depth is 8-25 meters, and the visibility is 12 meters. The water temperature is 26 DEG C, the salinity is 34.5 ‰, and the turbidity is 1.8 NTU; the equipment is configured with a 24 million pixel camera, a lens focal length of 35mm, and an attenuation coefficient: .

[0065] When the traditional method is used, the average color difference between the three platforms is 28.5%; after the application of the application, the color difference is reduced to 3.8%, and the consistency evaluation index Q reaches 0.96.

[0066] The application eliminates obvious color jump, the model texture transitions naturally, and the visual quality is significantly improved. The color correction time of an average frame image (24 million pixels) is 85ms, which meets the real-time processing requirements. In the case that 15% of the anchor points are invalid due to water flow disturbance, the system can still maintain the correction effect of Q=0.88; the generated three-dimensional model is successfully applied to the coral reef health state evaluation, and the recognition accuracy reaches more than 95%.

[0067] Through quantitative comparative analysis, the application has significant improvement in color consistency, processing efficiency and system stability compared with the prior art, and provides reliable technical support for underwater ecological monitoring.

[0068] Embodiment 2

[0069] As Figure 2 shown, it is a composition schematic diagram of an underwater scene distributed collaborative three-dimensional modeling system of the application, the system comprising: a plurality of underwater acquisition platforms, a platform communication unit, a color correction unit and a storage unit.

[0070] The plurality of underwater acquisition platforms are used for acquiring multispectral image data and water optical parameters by integrating a visible light camera, a near-infrared camera and a spectrometer, the resolution of the visible light camera is not less than 1920*1080 pixels, and the working wavelength range of the spectrometer is 400-800 nm.

[0071] The underwater acquisition platform adopts modular design, each platform weighs about 45 kg, and the maximum operating depth is 100 m. The visible light camera adopts a Sony IMX477 sensor and is equipped with a lens system with a waterproof level of IP68; the spectrometer adopts a CMOS linear array sensor in the range of 400-800 nm, and the spectral resolution is 2 nm.

[0072] The platform communication unit transmits the data collected by the acquisition platform to the color correction unit, the communication distance supported by the communication unit is not less than 500 m, and the data transmission rate is not less than 10 Mbps. The communication unit adopts underwater acoustic communication technology, the carrier frequency is 12 kHz, and stable 10 Mbps data transmission can be realized within a distance of 500 m. A data compression algorithm is provided, which can compress the image data to 30% of the original size, ensuring real-time transmission.

[0073] The color correction unit comprises a color correction model for unifying the colors of multi-platform images. The color correction unit integrates a dedicated image processing chip, is equipped with 8GB DDR4 memory and 256GB SSD storage, and supports parallel processing of multiple image data.

[0074] The storage unit is used for storing original acquisition data and corrected data, the storage unit adopts a distributed storage architecture, each platform is equipped with local storage, and a centralized storage server is arranged on the mother ship, so as to ensure data security and access efficiency.

[0075] The above specific embodiments further illustrate the purpose, technical solutions and advantages of the application. It should be understood that the above description is only a specific embodiment of the application and is not used to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A distributed collaborative 3D modeling method for underwater scenes, characterized in that, The method described above is used to solve the color consistency problem of multi-platform acquired images in distributed collaborative 3D modeling, and includes the following steps: Step S1: Identify at least two overlapping scan areas of the platform and select stable color feature points from the overlapping scan areas as color anchor points; The color anchor points are described using a multi-scale feature description method, which combines SIFT features and Lab color space information to describe anchor point features. The color anchor selection criteria are based on a weighted evaluation method of color variance, spatial distribution uniformity, and texture richness. ;in, For the first One candidate anchor point; Score the candidate anchor points. The color variance of the anchor point across different platforms; For the spatial uniformity of candidate anchor points; Texture richness of candidate anchor points; Perform time-series stability calculations on candidate anchor points: ;in, anchor point The timing stability; anchor point Standard deviation of color over time; anchor point Time series mean of color; anchor point The color value; Step S2: Based on the scanning depth and scanning color values ​​of each platform, establish an RGB three-channel depth attenuation compensation model to correct color attenuation in images acquired at different depths. Step S3: Using color anchor point information, calculate the dynamic white balance correction parameters for each platform to eliminate color temperature differences between different platforms; Based on color anchor point information, the white balance correction matrix for each platform is calculated by minimizing the objective function: ;in, For the first White balance correction matrix for each platform; N is the total number of color anchor points; For the first The platform in the Color vectors at each anchor point; For reference platform in the first Color vectors at each anchor point; The form of the white balance correction matrix is: ;in, , , The first The white balance correction coefficients for the red, green, and blue channels of each platform range from 0.5 to 2.

0. The reference platform selection criterion is to select the platform that minimizes the overall color deviation. ;in: The selected reference platform number; For all platforms at the anchor point The average color at that location; Step S4: Establish a global color consistency optimization objective function, and achieve color uniformity of images across multiple platforms through iterative optimization.

2. The method according to claim 1, characterized in that, Select stability value The K highest candidate anchor points are used as color anchor points, where K is not less than 2.

3. The method according to claim 2, characterized in that, Establish a depth attenuation compensation model for the RGB three channels: ;in: The corrected RGB color vector; , , These are the corrected values ​​for the red, green, and blue channels, respectively. , , These are the observed original red, green, and blue channel values, respectively. This refers to the scan depth. , , These are the attenuation coefficients for the red, green, and blue channels, respectively, and the red light attenuation coefficient. for Green light attenuation coefficient for Blue light attenuation coefficient for .

4. The method according to claim 3, characterized in that, Establish a global color consistency optimization objective function: ;in, This is due to overall color inconsistency error; Total number of platforms; For the platform and The set of pixels in the overlapping region; For pixels On the platform and Weights between them; and Platforms and In pixels The final color correction at that location.

5. The method according to claim 4, characterized in that, Establish a consistency evaluation index Q for the final corrected color and monitor the correction effect in real time; ,in, A higher value indicates better consistency; , These are the RGB values ​​of the final corrected colors for the two platforms in the overlapping area, respectively. The number of pixels in the overlapping region; the factor 255² represents the maximum squared difference of color values; when the quality index When the value falls below the preset threshold of 0.85, a recalibration process is automatically triggered.

6. A distributed collaborative 3D modeling system for underwater scenes, used to perform the method according to any one of claims 1-5, characterized in that, The system includes: multiple underwater acquisition platforms, a platform communication unit, a color correction unit, and a storage unit; The multiple underwater acquisition platforms integrate visible light cameras, near-infrared cameras, and spectrometers to acquire multispectral image data and water optical parameters. The resolution of the visible light camera is no less than 1920×1080 pixels, and the working wavelength range of the spectrometer is 400-800nm. The platform communication unit transmits the data collected by the acquisition platform to the color correction unit. The communication unit supports a communication distance of not less than 500m and a data transmission rate of not less than 10Mbps. The color correction unit includes a color correction model for unifying the colors of images across multiple platforms. The storage unit is used to store the original acquired data and the corrected data.

Images