An underwater miniature multi-mode imaging system

Abstract

The application discloses an underwater micro multi-mode imaging system, belongs to the technical field of underwater optical imaging, and is used for solving the problems that the active illumination enhancement effect is difficult to accurately evaluate in the underwater imaging process, and a single imaging mode cannot adapt to complex underwater environments. The system comprises an active illumination double-increment response construction module, a marginal effective imaging yield turning point judgment module and a multi-mode marginal yield recovery adaptation module. By continuously increasing the active illumination intensity according to the same detection increment, the target structure response and the scattering reference response under different illumination states are obtained, the target effective information increment and the scattering response increment are determined, and whether the current imaging mode enters the state of insufficient effective imaging yield is judged according to the marginal effective imaging yield change. Further, the target imaging mode is determined according to the target effective information response result of the candidate imaging mode and the marginal effective imaging yield, and the adaptive switching of the imaging mode is realized.

Description

Technical Field

[0001] This invention belongs to the field of underwater optical imaging technology, specifically relating to an underwater miniature multi-mode imaging system. Background Technology

[0002] Existing underwater imaging equipment typically employs active illumination to increase the light intensity of the target area, thereby improving target visibility in low-light underwater environments. However, as the intensity of active illumination continues to increase, the brightness diffusion and scattering interference caused by water scattering also intensify, gradually reducing the improvement in target structure information brought about by the increased illumination. In some cases, further increasing the illumination intensity may not yield effective imaging benefits.

[0003] Existing technologies typically adjust active illumination parameters based on changes in image brightness or single-shot imaging quality indicators. This makes it difficult to accurately analyze the relationship between the enhancement of effective target information and the enhancement of scattering interference during active illumination enhancement. It also makes it impossible to determine whether further increasing active illumination within the current illumination intensity range still has effective imaging value. Furthermore, once the current imaging mode enters a state of insufficient effective imaging gains, there is a lack of a mechanism for recovery and adaptation based on the marginal effective imaging gains of different imaging modes. This makes it difficult for the device to select a more suitable imaging mode according to the current target imaging requirements.

[0004] Therefore, how to determine the effective imaging gain turning point in the illumination enhancement process through target response and scattering response analysis during active illumination changes, and how to achieve multi-mode imaging adaptation when the current mode gain is insufficient, thereby improving the utilization efficiency of active illumination in underwater target imaging, is a technical problem that urgently needs to be solved.

[0005] Therefore, we propose an underwater miniature multi-mode imaging system. Summary of the Invention

[0006] The purpose of this invention is to provide an underwater miniature multi-mode imaging system to solve the problems of existing underwater imaging processes that cannot accurately determine the changes in effective imaging gains after active illumination enhancement, and that cannot adaptively adapt to the marginal effective imaging gains of different imaging modes after the current imaging mode enters a state of insufficient effective imaging gains.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an underwater miniature multi-mode imaging system, comprising: The active illumination dual-incremental response construction module is used to acquire the reference illumination intensity and reference underwater image under the current imaging mode, determine the single detection increment according to the illumination adjustment range, and perform two consecutive equal-increment illuminations. Based on the target structure response and scattering reference response of the three consecutive illumination states, the effective target information increment and scattering response increment of the two equal-increment illuminations are determined. The marginal effective imaging benefit turning point determination module is used to determine the first marginal effective imaging benefit and the second marginal effective imaging benefit based on the target effective information increment and scattering response increment of two equal-amplification events, and to determine whether the current imaging mode has entered a state of insufficient effective imaging benefit based on the second marginal effective imaging benefit and the changes in the two marginal effective imaging benefits. The multi-mode marginal benefit recovery adaptation module is used to determine candidate imaging modes and perform dual-incremental active illumination detection when the current imaging mode enters a state of insufficient effective imaging benefits or a state of insufficient dual-incremental detection conditions. It determines effective candidate modes based on the target effective information response results and marginal effective imaging benefit status, determines the target imaging mode based on the second marginal effective imaging benefit of each effective candidate mode, and controls the underwater miniature multi-mode imaging device to switch to the target imaging mode.

[0008] Preferably, the specific process for determining the single detection increment in the active illumination dual incremental response construction module is as follows: The difference between the maximum permissible illumination intensity corresponding to the current imaging mode and the reference illumination intensity is determined as the current remaining illumination adjustment amount; When the current remaining lighting adjustment is not less than twice the minimum stable adjustment range, the reference lighting intensity is multiplied by the preset detection ratio to obtain the candidate detection increment; when the candidate detection increment is not less than the minimum stable adjustment range, the candidate detection increment is determined as the initial detection increment; otherwise, the minimum stable adjustment range is determined as the initial detection increment. If twice the initial detection increment does not exceed the current remaining illumination adjustment amount, the initial detection increment is determined as the single detection increment; otherwise, the current remaining illumination adjustment amount is evenly distributed to the two illumination enhancement processes, and the illumination increment corresponding to each process is determined as the single detection increment. When the current remaining illumination adjustment is less than twice the minimum stable adjustment range, it is determined that the current imaging mode is in a state of insufficient dual-incremental detection conditions.

[0009] Preferably, the specific process of dual-incremental detection in the active illumination dual-incremental response construction module is as follows: During dual incremental detection, the current imaging mode, exposure time, gain, and focus state remain unchanged; A single detection increment is added to the reference illumination intensity to obtain the first detection illumination intensity and acquire the first underwater image; the same single detection increment is added to the first detection illumination intensity to obtain the second detection illumination intensity and acquire the second underwater image, so that the reference illumination intensity, the first detection illumination intensity and the second detection illumination intensity form three continuous illumination states, and the same single detection increment is used between two adjacent illumination states.

[0010] Preferably, the specific process for determining the target effective information increment and scattering response increment in the active illumination dual incremental response construction module is as follows: The target region is determined based on the target reference position in the reference underwater image, and a scattering reference region is determined in the vicinity of the target region; the spatial transformation relationship between the reference underwater image and the first and second underwater detection images is established to form the corresponding target regions and scattering reference regions in the three underwater images; The target structural response is determined based on the local structural contrast response, edge response, local structural contrast scale reference, and edge response scale reference; the scattering reference response is determined based on the non-structural brightness diffusion response, brightness diffusion scale reference, spatial fluctuation scale reference, and effective spatial fluctuation amount. The difference between the target structure responses of adjacent illumination states is used to obtain the target effective information increment corresponding to two equal-intensity light enhancements. The difference between the scattering reference responses of adjacent illumination states is used to obtain the scattering response increment corresponding to two equal-intensity light enhancements.

[0011] Preferably, the specific process for determining the marginal effective imaging revenue and the decrease in marginal revenue in the marginal effective imaging revenue inflection point determination module is as follows: For the first and second equal-increase luminance, the corresponding scattering response increments are compared with the preset minimum scattering response increment benchmark. When the scattering response increment is not lower than the preset minimum scattering response increment benchmark, the marginal effective imaging gain is determined based on the ratio between the target effective information increment and the corresponding scattering response increment; when the scattering response increment is lower than the preset minimum scattering response increment benchmark, the marginal effective imaging gain is determined based on the ratio between the target effective information increment and the preset minimum scattering response increment benchmark. When the marginal effective imaging benefit of the first imaging is higher than that of the second imaging, the difference between the two is defined as the decrease in marginal benefit; otherwise, the decrease in marginal benefit is defined as zero.

[0012] Preferably, the specific process for determining the insufficient effective imaging benefit state in the marginal effective imaging benefit transition determination module is as follows: When the second marginal effective imaging gain is less than or equal to zero, the current imaging mode is determined to have entered a state of insufficient effective imaging gain. When the second marginal effective imaging gain is greater than zero, the marginal gain decrease is compared with the preset effective gain decrease threshold; when the marginal gain decrease is greater than the preset effective gain decrease threshold, it is determined that the current imaging mode has entered the state of insufficient effective imaging gain; otherwise, it is determined that the current imaging mode has not entered the state of insufficient effective imaging gain. When the current imaging mode has not entered the state of insufficient effective imaging benefits, the second detection illumination intensity will be used as the current working illumination intensity to continue the current underwater target imaging; when the current imaging mode enters the state of insufficient effective imaging benefits, it will switch to the multi-mode marginal benefit recovery and adaptation module.

[0013] Preferably, the specific process for determining candidate imaging modes in the multi-mode marginal benefit recovery adaptation module is as follows: Acquire the mode parameter information corresponding to each imaging mode supported by the underwater miniature multi-mode imaging device, as well as the target type and target imaging requirements corresponding to the current target imaging task; The mode parameter information includes the active illumination channel that can be called by the corresponding imaging mode, the active illumination adjustment range, the minimum stable adjustment amplitude, the basic illumination intensity, the main imaging parameters, and the applicable target type. The current target type is matched with the applicable target type of each imaging mode, and it is determined whether the corresponding imaging mode has active illumination adjustment capability. The imaging mode that matches the current target type, has active illumination adjustment capability, and is different from the current imaging mode is determined as the candidate imaging mode.

[0014] Preferably, the specific process of dual-incremental active illumination detection for candidate imaging modes is as follows: For each candidate imaging mode, if the corresponding active illumination channel has an actual output illumination intensity, the actual output illumination intensity is determined as the detection start illumination intensity; otherwise, the corresponding base illumination intensity is determined as the detection start illumination intensity. The difference between the maximum allowable illumination intensity of the candidate imaging mode and the initial illumination intensity of the detection is determined as the candidate remaining illumination adjustment amount; when the candidate remaining illumination adjustment amount is not less than twice the minimum stable adjustment amplitude, the single detection increment is formed according to the single detection increment determination method in the active illumination dual incremental response construction module; otherwise, the candidate imaging mode is excluded. The candidate imaging mode with dual-incremental detection conditions is invoked, the main imaging parameters are loaded and the active illumination channel is enabled. The initial illumination intensity is used as the reference illumination intensity to obtain a reference underwater image. The active illumination intensity is increased twice consecutively according to the same single detection increment to obtain the first and second underwater images.

[0015] Preferably, the specific process for determining the effective candidate modes in the multi-mode marginal revenue recovery adaptation module is as follows: The target effective information increment and scattering response increment corresponding to the candidate imaging mode are determined in the same way as in the active illumination dual incremental response construction module, and the marginal effective imaging gain status is determined in the same way as in the marginal effective imaging gain turning point determination module. The target structure response, local structure contrast response, and edge response corresponding to the second underwater image of the candidate imaging mode are acquired to form the effective target information response result corresponding to the candidate imaging mode. When the local structure contrast response reaches the minimum local structure contrast response requirement of the corresponding candidate imaging mode and the current target type, and the edge response reaches the minimum edge response requirement of the corresponding candidate imaging mode and the current target type, the effective target information response result is determined to meet the current target imaging requirements. Candidate imaging modes whose effective information response results meet the current target imaging requirements and have not entered the state of insufficient effective imaging benefits are identified as effective candidate modes.

[0016] Preferably, the specific process of target imaging mode determination and mode switching in the multi-mode marginal benefit recovery adaptation module is as follows: When only one valid candidate mode exists, that valid candidate mode is determined as the target imaging mode. When there are multiple effective candidate modes, compare the second marginal effective imaging gains corresponding to each effective candidate mode, and determine the effective candidate mode with the highest second marginal effective imaging gains as the target imaging mode; when there are two or more effective candidate modes and the second marginal effective imaging gains are both the highest values, determine the target imaging mode from among them according to the preset mode priority. When performing dual-incremental detection in the target imaging mode, the second detection illumination intensity is determined as the target working illumination intensity. The underwater miniature multi-mode imaging device is controlled to switch to the target imaging mode, the main imaging parameters corresponding to the target imaging mode are loaded, and the corresponding active illumination unit is controlled according to the target working illumination intensity to continue performing the current underwater target imaging.

[0017] Compared with the prior art, the beneficial effects of the present invention are: (1) The underwater micro multi-mode imaging system continuously increases the active illumination intensity according to the same detection increment, and obtains the target structure response and scattering reference response under different illumination conditions to determine the effective information increment and scattering response increment of the target. This enables a comprehensive evaluation of the improvement of effective imaging information and the change of scattering response during the active illumination enhancement process, avoiding evaluation bias caused by judging the illumination effect solely based on changes in image brightness.

[0018] (2) The underwater micro multi-mode imaging system determines whether the current imaging mode has entered a state of insufficient effective imaging gain by judging the marginal effective imaging gain change relationship corresponding to the two illumination enhancement processes. This enables the determination of the effectiveness of the current active illumination to be further enhanced, and avoids the increase in scattering response and reduction in imaging quality caused by continuing to increase the illumination intensity when the effective imaging gain is insufficient.

[0019] (3) The underwater miniature multi-mode imaging system evaluates the target effective information response results and marginal effective imaging benefits of the candidate imaging modes when the current imaging mode enters a state of insufficient effective imaging benefits, and selects the target imaging mode for switching, thereby realizing the adaptive adjustment of imaging modes under different target imaging requirements and underwater environmental conditions, and improving the stability of underwater target imaging. Attached Figure Description

[0020] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1; Please see Figure 1 This invention provides an underwater miniature multi-mode imaging system, comprising: Active illumination dual-incremental response construction module: This module acquires the reference illumination intensity and reference underwater image under the current imaging mode. It determines the single detection increment based on the illumination adjustment range corresponding to the current illumination state, and continuously increases the active illumination intensity twice with the same single detection increment to acquire the first and second underwater images. Based on the target structure response corresponding to the target area and the scattering reference response corresponding to the scattering reference area under the three consecutive illumination states, it determines the target effective information increment and scattering response increment corresponding to the two equal-increment illuminations. The specific process is as follows: When an underwater miniature multi-mode imaging device continuously images the current target, it acquires the current imaging mode, the current illumination intensity of the active illumination unit, and the underwater image acquired under the current illumination intensity. The current illumination intensity is determined as the reference illumination intensity, and the corresponding underwater image is determined as the reference underwater image.

[0023] Read the exposure time, gain, and focus state corresponding to the reference underwater image, and keep the current imaging mode and the above imaging parameters unchanged during the current dual-increment detection process, so that the reference underwater image, the first detection underwater image, and the second detection underwater image maintain the same main imaging conditions except for the active illumination intensity.

[0024] Obtain the maximum permissible illumination intensity corresponding to the current imaging mode and the minimum stable adjustment range corresponding to the active illumination unit, and determine the difference between the maximum permissible illumination intensity and the reference illumination intensity as the current remaining illumination adjustment amount.

[0025] When the current remaining lighting adjustment is not less than twice the minimum stable adjustment range, the reference lighting intensity is multiplied by the preset detection ratio to obtain the candidate detection increment, and the candidate detection increment is compared with the minimum stable adjustment range.

[0026] When the candidate detection increment is not less than the minimum stable adjustment range, the candidate detection increment is determined as the initial detection increment; when the candidate detection increment is less than the minimum stable adjustment range, the minimum stable adjustment range is determined as the initial detection increment.

[0027] Furthermore, twice the initial detection increment is compared with the current remaining illumination adjustment: when twice the initial detection increment does not exceed the current remaining illumination adjustment, the initial detection increment is determined as the single detection increment; when twice the initial detection increment exceeds the current remaining illumination adjustment, the current remaining illumination adjustment is evenly distributed to the two illumination enhancement processes, and the illumination increment corresponding to each process is determined as the single detection increment.

[0028] When the current remaining illumination adjustment is less than twice the minimum stable adjustment range, it is determined that the current imaging mode is in a state of insufficient dual-incremental detection conditions, and the process is switched to the multi-mode marginal benefit recovery and adaptation module.

[0029] After determining the single detection increment, a single detection increment is added to the reference illumination intensity to obtain the first detection illumination intensity. After the active illumination output stabilizes, the first underwater detection image is acquired. Then, the same single detection increment is added to the first detection illumination intensity to obtain the second detection illumination intensity. After the active illumination output stabilizes, the corresponding second underwater detection image is acquired.

[0030] It should be noted that the maximum permissible illumination intensity is pre-calibrated based on the permissible output range of the active illumination unit and the illumination range corresponding to the effective response of the imaging unit; the minimum stable adjustment amplitude is determined based on the minimum illumination adjustment amount corresponding to the requirement that changes in active illumination intensity can produce a stable and recognizable image response; the preset detection ratio is pre-set based on the active illumination adjustment range, illumination adjustment accuracy, and the effective response range of the imaging unit. The reference illumination intensity, the first detection illumination intensity, and the second detection illumination intensity form three continuous illumination states, where the same single detection increment is used between two adjacent illumination states; different imaging modes determine their own single detection increments within their own illumination adjustment ranges, and the same absolute illumination increment is not required.

[0031] Obtain the target reference position corresponding to the current imaging task, and determine the target search area in the reference underwater image based on the target reference position.

[0032] The target search region is converted to the luminance channel and median filtering is applied. Candidate segmentation thresholds are set sequentially within the pixel luminance range of the filtered target search region. For each candidate segmentation threshold, the percentage and average luminance of pixels with luminance values ​​not higher than the candidate segmentation threshold, as well as the percentage and average luminance of pixels with luminance values ​​higher than the candidate segmentation threshold, are statistically analyzed. The pixel percentages of the two luminance categories are multiplied, and the result is multiplied by the square of the difference between the average luminance values ​​of the two luminance categories to obtain the inter-class difference of the corresponding candidate segmentation threshold. The candidate segmentation threshold with the largest inter-class difference is determined as the target segmentation threshold.

[0033] The pixels within the target search area are classified according to the target segmentation threshold. Pixels with brightness values ​​not higher than the target segmentation threshold are classified into the first brightness category, and pixels with brightness values ​​higher than the target segmentation threshold are classified into the second brightness category. The brightness category to which the pixel corresponding to the target reference position belongs is determined, and this brightness category is determined as the target candidate category.

[0034] The target candidate category is first dilated to connect adjacent or partially disconnected target candidate pixel regions. Then, erosion is performed to shrink the expanded region boundary and retain the already connected target candidate pixel regions, thus completing the morphological closing operation.

[0035] Based on pixel connectivity, the target candidate pixels after morphological closing operation are divided into connected regions. The connected regions containing the target reference position are determined as the target regions, and a target mask corresponding to the target regions is formed.

[0036] It should be noted that median filtering is used to suppress isolated brightness noise within the target search area; morphological closing operation is mainly used to connect local breaks in the target candidate area and fill small gaps to improve the continuity of the target area.

[0037] Using a reference underwater image as a spatial reference, spatial transformation relationships are established between the reference underwater image and the first detected underwater image, and between the reference underwater image and the second detected underwater image, specifically as follows: Feature points and their binary feature descriptors are obtained from the baseline underwater image and the corresponding exploration underwater image, respectively, using ORB feature extraction. The Hamming distance between the feature descriptors in the two underwater images is calculated, and corresponding feature points are selected based on the Hamming distance to form candidate matching point pairs.

[0038] For candidate matching point pairs, the RANSAC method is used for screening. Matching points that meet the requirements of solving the two-dimensional affine transformation are selected from the candidate matching point pairs, and the candidate two-dimensional affine transformation matrix is ​​calculated. The position of each candidate matching point in the reference underwater image is transformed according to each candidate two-dimensional affine transformation matrix. The reprojection error between the transformed position and the actual position of the corresponding matching point in the probe underwater image is calculated, and the matching points whose reprojection error does not exceed the preset registration error threshold are determined as valid matching points.

[0039] Compare the number of valid matching points corresponding to each candidate 2D affine transformation matrix; when there is only one candidate 2D affine transformation matrix with the most valid matching points, determine the candidate 2D affine transformation matrix as the spatial transformation relationship; when there are multiple candidate 2D affine transformation matrices with the same number of valid matching points, calculate the average reprojection error of their valid matching points respectively, and determine the candidate 2D affine transformation matrix with the lowest average reprojection error as the spatial transformation relationship.

[0040] Among them, ORB is a directional fast corner point and rotation binary descriptor feature extraction method, RANSAC is a random sampling consistency method; the preset registration error threshold is determined in advance based on the normal image registration error range when the relative position of the underwater miniature multi-mode imaging device and the target remains stable.

[0041] According to the spatial transformation relationship, the target mask in the reference underwater image is mapped to the first and second underwater detection images respectively, so that the three underwater images form corresponding target areas.

[0042] Starting from the outer boundary of the target mask in the reference underwater image, a ring-shaped candidate region is formed by expanding outward. The local gradient magnitude and local brightness variance of each pixel in the ring-shaped candidate region are calculated respectively. The inter-class difference determination method is the same as that used for the target segmentation threshold. The corresponding segmentation threshold is determined within the range of local gradient magnitude and local brightness variance. The segmentation threshold corresponding to the maximum inter-class difference is determined as the gradient segmentation threshold and the variance segmentation threshold respectively.

[0043] Pixels with local gradient magnitudes not exceeding the gradient segmentation threshold and local brightness variances not exceeding the variance segmentation threshold are identified as candidate water pixels. Morphological opening is first performed on these candidate water pixels, i.e., erosion is performed to remove isolated pixels and small discrete regions, followed by dilation to restore the main area of ​​the remaining region. Subsequently, morphological closing is performed, i.e., dilation is performed to connect adjacent but partially disconnected candidate water regions, followed by erosion to restore region boundaries and fill small local gaps.

[0044] The candidate pixels of the water body are divided into connected regions according to the pixel connectivity relationship. Connected regions with an area not less than the preset minimum effective water body area are retained, and the retained connected water body regions are collectively determined as the scattering reference region.

[0045] Among them, morphological opening operation is used to reduce the influence of isolated noise and scattered regions on the water body region determination results, and morphological closing operation is used to improve the continuity and integrity of water body candidate regions; the preset minimum effective water body area is predetermined based on the effective water body pixel range required for subsequent scattering response calculation.

[0046] According to the established spatial transformation relationship, the scattering reference area in the reference underwater image is mapped to the first and second underwater detection images respectively, so that the three underwater images form a corresponding spatial range of adjacent water bodies.

[0047] It should be noted that the target reference position is determined based on the initial target position given by the current imaging task during the initial target imaging, and is updated based on the previously determined target position during continuous imaging. The scattering reference region is located near the target region and within the current active illumination coverage area. It excludes non-aquatic structures with obvious edges or local texture changes by combining local gradient magnitude and local brightness variance, and is used to characterize the scattering response of nearby water bodies caused by changes in active illumination.

[0048] The target regions in the three underwater images were processed using the same Gaussian smoothing method, and the brightness gradients in the horizontal and vertical directions were calculated to obtain the gradient magnitude of each underwater image.

[0049] In the reference underwater image, the target structure edge is formed by non-maximum suppression and double-threshold edge connection based on the corresponding brightness gradient. The position of the target structure edge is determined as the structure analysis position, and the normal direction corresponding to each structure analysis position is determined according to the gradient direction of the target structure edge. The structure analysis positions are mapped to the first and second underwater images according to the spatial transformation relationship, and the normal direction corresponding to each mapped structure analysis position is determined by the linear transformation part in the corresponding two-dimensional affine transformation matrix.

[0050] For each corresponding structural analysis location in the three underwater images, the same local analysis range is set on both sides of the edge according to the normal direction corresponding to the structural analysis location. The average brightness of the local analysis range on both sides is calculated, and the absolute difference between the two average brightness is determined as the local structural contrast value of the corresponding structural analysis location. The local structural contrast value corresponding to each structural analysis location is averaged to obtain the local structural contrast response corresponding to each underwater image.

[0051] The gradient magnitudes at corresponding structural analysis locations in the three underwater images are calculated, and the average gradient magnitudes at each structural analysis location are taken to obtain the edge responses corresponding to the three underwater images.

[0052] The local structure contrast response corresponding to the reference underwater image is compared with the minimum local structure contrast benchmark. When the local structure contrast response corresponding to the reference underwater image is not lower than the minimum local structure contrast benchmark, the local structure contrast response is determined as the local structure contrast scale benchmark; when the local structure contrast response corresponding to the reference underwater image is lower than the minimum local structure contrast benchmark, the minimum local structure contrast benchmark is determined as the local structure contrast scale benchmark.

[0053] The edge response corresponding to the reference underwater image is compared with the minimum edge response benchmark. When the edge response corresponding to the reference underwater image is not lower than the minimum edge response benchmark, the edge response is determined as the edge response scale benchmark; when the edge response corresponding to the reference underwater image is lower than the minimum edge response benchmark, the minimum edge response benchmark is determined as the edge response scale benchmark.

[0054] Based on the local structural contrast response, edge response, local structural contrast scale benchmark, and edge response scale benchmark corresponding to each continuous illumination state, the target structural response corresponding to each continuous illumination state is determined according to the following formula: ; Where m is the continuous illumination state number, and the reference illumination state, the first detection illumination state, and the second detection illumination state correspond to zero, one, and two, respectively; This represents the target structure response corresponding to the m-th consecutive lighting state; The corresponding local structural contrast response; This corresponds to the edge response; As a benchmark for local structural comparison; This serves as a benchmark for edge response scales.

[0055] The difference between the target structure response corresponding to the first underwater image and the target structure response corresponding to the reference underwater image is used to obtain the target effective information increment corresponding to the first equal-amount brightening; the difference between the target structure response corresponding to the second underwater image and the target structure response corresponding to the first underwater image is used to obtain the target effective information increment corresponding to the second equal-amount brightening.

[0056] Brightness channels were extracted from the scattering reference regions in the three underwater images, and low-pass filtering was performed using the same filtering scale to obtain the corresponding low-frequency water body response.

[0057] The low-frequency water body response in each scattering reference area is averaged to obtain the unstructured brightness diffusion response corresponding to each lighting state; the absolute difference in brightness between adjacent pixels in the low-frequency water body response is calculated and averaged to obtain the corresponding low-frequency spatial fluctuation.

[0058] The unstructured brightness diffusion response corresponding to the baseline underwater image is compared with the minimum brightness diffusion baseline: When the unstructured brightness diffusion response corresponding to the reference underwater image is not lower than the minimum brightness diffusion reference, the unstructured brightness diffusion response is determined as the brightness diffusion scale reference; when the unstructured brightness diffusion response corresponding to the reference underwater image is lower than the minimum brightness diffusion reference, the minimum brightness diffusion reference is determined as the brightness diffusion scale reference.

[0059] Compare the low-frequency spatial fluctuations corresponding to the baseline underwater image with the minimum spatial fluctuation baseline: When the low-frequency spatial fluctuation amount corresponding to the reference underwater image is not lower than the minimum spatial fluctuation reference, the low-frequency spatial fluctuation amount is determined as the spatial fluctuation scale reference; when the low-frequency spatial fluctuation amount corresponding to the reference underwater image is lower than the minimum spatial fluctuation reference, the minimum spatial fluctuation reference is determined as the spatial fluctuation scale reference.

[0060] For each continuous lighting state, the corresponding low-frequency spatial fluctuation is compared with the minimum spatial fluctuation benchmark: When the low-frequency spatial fluctuation is not lower than the minimum spatial fluctuation benchmark, the low-frequency spatial fluctuation is determined as the effective spatial fluctuation under the corresponding lighting condition; when the low-frequency spatial fluctuation is lower than the minimum spatial fluctuation benchmark, the minimum spatial fluctuation benchmark is determined as the effective spatial fluctuation under the corresponding lighting condition.

[0061] Based on the unstructured luminance diffusion response, luminance diffusion scale reference, spatial fluctuation scale reference, and effective spatial fluctuation amount corresponding to each continuous illumination state, the scattering reference response corresponding to each continuous illumination state is determined according to the following formula: ; in, This is the scattering reference response corresponding to the m-th consecutive lighting state; This corresponds to the unstructured brightness diffusion response; As a benchmark for brightness diffusion scale; As a benchmark for spatial fluctuation scale; This represents the effective spatial fluctuation amount corresponding to the lighting state.

[0062] The scattering reference response corresponding to the first underwater image is subtracted from the scattering reference response corresponding to the reference underwater image to obtain the scattering response increment corresponding to the first equal-amount enhancement; the scattering reference response corresponding to the second underwater image is subtracted from the scattering reference response corresponding to the first underwater image to obtain the scattering response increment corresponding to the second equal-amount enhancement.

[0063] It should be noted that the local structural contrast response and edge response characterize the degree of local brightness distinction and edge preservation of the target structure, respectively. After in-mode scaling, both constitute the target structural response. The unstructured brightness diffusion response characterizes the overall enhancement of low-frequency brightness in adjacent water bodies, and the ratio between the spatial fluctuation scale benchmark and the effective spatial fluctuation amount characterizes the degree to which low-frequency water body brightness tends to diffuse continuously and smoothly. Both constitute the scattering reference response. The minimum local structural contrast benchmark, minimum edge response benchmark, minimum brightness diffusion benchmark, and minimum spatial fluctuation benchmark are predetermined based on the minimum effective response level that can be stably distinguished under normal stable imaging conditions. Three underwater images use corresponding target spatial ranges, scattering reference regions, and structural analysis locations, and employ the same filtering scale, ensuring that the increase in effective target information and scattering response corresponding to two equal-amount brightening operations has a consistent analytical basis.

[0064] The marginal effective imaging gain transition determination module is used to determine the first marginal effective imaging gain based on the first target effective information increment and the first scattering response increment determined by the active illumination dual incremental response construction module, to determine the second marginal effective imaging gain based on the second target effective information increment and the second scattering response increment, and to determine whether the current imaging mode has entered a state of insufficient effective imaging gain based on the second marginal effective imaging gain and the change between the two marginal effective imaging gains. The specific process is as follows: Obtain the target effective information increment and scattering response increment corresponding to the first and second equal-amplification.

[0065] Based on the proportional relationship between the increase in effective target information and the increase in scattering response corresponding to each equal increase in brightness, the corresponding marginal effective imaging gain is determined according to the following formula: ; Where i = 1, 2; The marginal effective imaging gain corresponding to the i-th equal-amount brightening; The increment of effective target information corresponding to the i-th equal-amount enhancement; This represents the scattering response increment corresponding to the i-th equal-amount enhancement; This serves as the preset minimum scattering response benchmark.

[0066] Based on the difference between the first and second marginal effective imaging gains, the decrease in marginal revenue is determined according to the following formula: ; in, This represents the decrease in marginal revenue. For the first marginal effective imaging benefit; This represents the benefit of the second marginal effective imaging.

[0067] When the second marginal effective imaging gain is less than or equal to zero, the current imaging mode is determined to have entered a state of insufficient effective imaging gain.

[0068] When the second marginal effective imaging gain is greater than zero, the marginal gain decrease is compared with the preset effective gain decrease threshold. When the marginal gain decrease is not greater than the preset effective gain decrease threshold, it is determined that the current imaging mode has not entered the state of insufficient effective imaging gain. When the marginal gain decrease is greater than the preset effective gain decrease threshold, it is determined that the current imaging mode has entered the state of insufficient effective imaging gain.

[0069] When the current imaging mode does not reach the state of insufficient effective imaging gains, the current imaging mode is maintained, and the second detection illumination intensity is used as the current working illumination intensity to continue to perform the current underwater target imaging.

[0070] When the current imaging mode enters a state of insufficient effective imaging gains, the active illumination intensity in the current imaging mode is stopped from being increased, and the process switches to the multi-mode marginal gain recovery adaptation module.

[0071] It should be noted that the preset minimum scattering response benchmark is determined based on the normal fluctuation range of the scattering reference response change under stable imaging conditions, in order to avoid the marginal effective imaging gain being abnormally amplified due to the scattering response increment being too small; the preset effective gain reduction threshold is determined by repeatedly performing double incremental detection when the target position, water state and main imaging parameters remain stable and the active illumination is within the normal calibration illumination range, based on the normal fluctuation upper limit of the marginal gain reduction.

[0072] It should be noted that marginal effective imaging gain is used to characterize the effective imaging gain obtained by the improvement of target effective information relative to the synchronously generated scattering response enhancement during a single equal-amount illumination process; the state of insufficient effective imaging gain indicates that continuing to enhance active illumination in the current mode can no longer maintain the original marginal imaging value, and does not mean that the current underwater image itself cannot complete basic imaging.

[0073] It should be noted that when the current imaging mode has not entered the state of insufficient effective imaging benefits and it is still necessary to continue to increase the active illumination intensity, the current working illumination intensity can be used as the new reference illumination intensity, and the active illumination dual incremental response construction and marginal effective imaging benefit inflection determination can be re-executed to continue to determine the effective imaging benefits corresponding to subsequent light enhancement.

[0074] The multi-mode marginal benefit recovery and adaptation module is used to determine candidate imaging modes based on the current target imaging requirements when the current imaging mode enters a state of insufficient effective imaging benefits or insufficient dual-incremental detection conditions, and to perform dual-incremental active illumination detection on each candidate imaging mode; it determines the effective candidate mode based on the target effective information response results and marginal effective imaging benefit state corresponding to each candidate imaging mode, and determines the target imaging mode based on the second marginal effective imaging benefit corresponding to each effective candidate mode, controlling the underwater miniature multi-mode imaging device to switch to the target imaging mode to continue underwater target imaging. The specific process is as follows: Acquire the mode parameter information corresponding to each imaging mode supported by the underwater miniature multi-mode imaging device, as well as the target type and target imaging requirements corresponding to the current target imaging task.

[0075] The mode parameter information includes the active illumination channel that can be called in the corresponding imaging mode, the active illumination adjustment range, the minimum stable adjustment range, the basic illumination intensity, the main imaging parameters, and the applicable target types.

[0076] The current target type is matched with the applicable target type of each imaging mode, and it is determined whether the corresponding imaging mode has active illumination adjustment capability. The imaging mode that matches the current target type, has active illumination adjustment capability, and is different from the current imaging mode is determined as the candidate imaging mode.

[0077] For each candidate imaging mode, the current output state of the active illumination channel corresponding to the candidate imaging mode is obtained; when the corresponding active illumination channel currently has an actual output illumination intensity, the actual output illumination intensity is determined as the detection start illumination intensity; when the corresponding active illumination channel currently does not have an actual output illumination intensity, the basic illumination intensity corresponding to the candidate imaging mode is determined as the detection start illumination intensity.

[0078] The difference between the maximum permissible illumination intensity of the candidate imaging mode and the initial illumination intensity of the detection is determined as the candidate remaining illumination adjustment amount, and the candidate remaining illumination adjustment amount is compared with twice the minimum stable adjustment amplitude corresponding to the candidate imaging mode. When the candidate remaining illumination adjustment amount is not less than twice the minimum stable adjustment amplitude, the candidate imaging mode is determined to have the dual-incremental detection condition, and the corresponding single detection increment is formed according to the single detection increment determination method in the active illumination dual-incremental response construction module; when the candidate remaining illumination adjustment amount is less than twice the minimum stable adjustment amplitude, the candidate imaging mode is excluded.

[0079] For a candidate imaging mode with dual-incremental detection conditions, the candidate imaging mode is invoked, the corresponding main imaging parameters are loaded and the corresponding active illumination channel is enabled, the initial illumination intensity of the detection is used as the reference illumination intensity of the candidate imaging mode, and the corresponding reference underwater image is acquired; based on the reference illumination intensity, the active illumination intensity is increased twice consecutively according to the same single detection increment, and the first underwater image and the second underwater image are acquired respectively.

[0080] For the three underwater images corresponding to the candidate imaging modes, the target area and scattering reference area are determined in the same way as in the active illumination dual incremental response construction module, and the target structure response and scattering reference response corresponding to the three consecutive illumination states are formed respectively. Then, the target effective information increment and scattering response increment corresponding to the first equal-intensity illumination and the second equal-intensity illumination are determined respectively.

[0081] Following the same method as in the marginal effective imaging benefit turning point determination module, determine the first marginal effective imaging benefit, the second marginal effective imaging benefit, and the marginal benefit decrease corresponding to the candidate imaging mode, and determine whether the candidate imaging mode has entered the state of insufficient effective imaging benefit.

[0082] The target structure response corresponding to the second underwater image of the candidate imaging mode is obtained, as well as the local structure contrast response and edge response corresponding to the formation of the target structure response. The target structure response, local structure contrast response and edge response are combined to form the effective target information response result corresponding to the candidate imaging mode.

[0083] For each imaging mode and corresponding target type, underwater images are pre-acquired and calibrated using standard targets with known structural information. From the calibrated underwater images that can identify the target structure required for the current target imaging task, the distribution of local structural contrast response and edge response is statistically analyzed, and the minimum local structural contrast response requirement and minimum edge response requirement for the corresponding imaging mode and target type are determined respectively.

[0084] The local structure contrast response and edge response corresponding to the second underwater image of the candidate imaging mode are compared with the corresponding minimum response requirements. When both meet the corresponding minimum response requirements, the effective target information response result of the candidate imaging mode is determined to meet the current target imaging requirements; otherwise, the effective target information response result of the candidate imaging mode is determined not to meet the current target imaging requirements.

[0085] For candidate imaging modes whose effective information response results meet the current target imaging requirements and have not entered the state of insufficient effective imaging benefits, the candidate imaging mode is determined as an effective candidate mode.

[0086] When there is only one valid candidate mode, the valid candidate mode is determined as the target imaging mode; when there are multiple valid candidate modes, the second marginal effective imaging gains corresponding to each valid candidate mode are compared, and the valid candidate mode with the highest second marginal effective imaging gains is determined as the target imaging mode; when there are two or more valid candidate modes and the second marginal effective imaging gains are both the highest values, the target imaging mode is determined from them according to the preset mode priority.

[0087] The second detection illumination intensity corresponding to the dual incremental detection when the target imaging mode is executed is obtained, and the second detection illumination intensity is determined as the target working illumination intensity; the underwater miniature multi-mode imaging device is controlled to switch to the target imaging mode, the main imaging parameters corresponding to the target imaging mode are loaded, and the corresponding active illumination unit is controlled according to the target working illumination intensity to continue to execute the current underwater target imaging.

[0088] It should be noted that the minimum local structure contrast response requirement and the minimum edge response requirement are determined based on calibrated underwater images that meet the target structure identification requirements under the corresponding imaging mode and target type, respectively. This ensures that the basic target imaging capability of the candidate imaging mode has a judgment criterion independent of the marginal benefit evaluation. The target effective information response result of the candidate imaging mode is used to determine whether the imaging mode can meet the basic structure imaging requirements of the current target imaging task. The marginal effective imaging benefit is used to further determine whether continuing to increase active illumination under this mode still has effective imaging value. Both determine the effective candidate modes from the aspects of basic imaging capability and illumination enhancement effectiveness, respectively. Different candidate imaging modes may have different active illumination adjustment ranges and single detection increments, but they all use the same target structure response, scattering reference response, and marginal effective imaging benefit formation logic for intra-mode evaluation. Among multiple effective candidate modes, the second marginal effective imaging benefit is used as the mode comparison criterion. The second detection illumination intensity is used as the target working illumination intensity to ensure that the actual working state after mode switching is consistent with the illumination state corresponding to the candidate imaging mode when completing the target imaging requirement verification and marginal effective imaging benefit evaluation.

[0089] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An underwater miniature multi-mode imaging system, characterized in that, include: The active illumination dual-incremental response construction module is used to acquire the reference illumination intensity and reference underwater image under the current imaging mode, determine the single detection increment according to the illumination adjustment range, and perform two consecutive equal-increment illuminations. Based on the target structure response and scattering reference response of the three consecutive illumination states, the effective target information increment and scattering response increment of the two equal-increment illuminations are determined. The marginal effective imaging benefit turning point determination module is used to determine the first marginal effective imaging benefit and the second marginal effective imaging benefit based on the target effective information increment and scattering response increment of the two equal-intensity light enhancements. For the first equal-intensity light enhancement and the second equal-intensity light enhancement, the corresponding scattering response increment is compared with the preset minimum scattering response increment benchmark. When the scattering response increment is not lower than the preset minimum scattering response increment benchmark, the marginal effective imaging gain is determined based on the ratio between the target effective information increment and the corresponding scattering response increment. When the scattering response increment is lower than the preset minimum scattering response increment benchmark, the marginal effective imaging gain is determined based on the ratio between the target effective information increment and the preset minimum scattering response increment benchmark; when the first marginal effective imaging gain is higher than the second marginal effective imaging gain, the difference between the two is determined as the marginal gain decrease; otherwise, the marginal gain decrease is determined to be zero, and the current imaging mode is judged to have entered the state of insufficient effective imaging gain based on the second marginal effective imaging gain and the change between the two marginal effective imaging gains. The multi-mode marginal benefit recovery adaptation module is used to determine candidate imaging modes and perform dual-incremental active illumination detection when the current imaging mode enters a state of insufficient effective imaging benefits or a state of insufficient dual-incremental detection conditions. It determines effective candidate modes based on the target effective information response results and marginal effective imaging benefit status, determines the target imaging mode based on the second marginal effective imaging benefit of each effective candidate mode, and controls the underwater miniature multi-mode imaging device to switch to the target imaging mode.

2. The underwater miniature multi-mode imaging system according to claim 1, characterized in that, The specific process for determining the single detection increment in the active illumination dual incremental response construction module is as follows: The difference between the maximum permissible illumination intensity corresponding to the current imaging mode and the reference illumination intensity is determined as the current remaining illumination adjustment amount; When the current remaining lighting adjustment is not less than twice the minimum stable adjustment range, the candidate detection increment is obtained by multiplying the reference lighting intensity by the preset detection ratio. If the candidate detection increment is not less than the minimum stable adjustment range, the candidate detection increment is determined as the initial detection increment; otherwise, the minimum stable adjustment range is determined as the initial detection increment. When twice the initial detection increment does not exceed the current remaining lighting adjustment, the initial detection increment is determined as the single detection increment; Otherwise, the remaining lighting adjustment amount will be evenly distributed across the two lighting enhancement processes, and the corresponding lighting increment for each process will be determined as the single detection increment. When the current remaining illumination adjustment is less than twice the minimum stable adjustment range, it is determined that the current imaging mode is in a state of insufficient dual-incremental detection conditions.

3. The underwater miniature multi-mode imaging system according to claim 2, characterized in that, The specific process of dual-incremental detection in the active illumination dual-incremental response construction module is as follows: During dual incremental detection, the current imaging mode, exposure time, gain, and focus state remain unchanged; By adding a single detection increment to the reference illumination intensity, the first detection illumination intensity is obtained and the first underwater image is acquired. Based on the first detection illumination intensity, the same single detection increment is added to obtain the second detection illumination intensity and acquire the second underwater detection image, so that the reference illumination intensity, the first detection illumination intensity and the second detection illumination intensity form three continuous illumination states, and the same single detection increment is used between two adjacent illumination states.

4. The underwater miniature multi-mode imaging system according to claim 3, characterized in that, The specific process for determining the target effective information increment and scattering response increment in the active illumination dual incremental response construction module is as follows: The target region is determined based on the target reference position in the reference underwater image, and a scattering reference region is determined in the vicinity of the target region; the spatial transformation relationship between the reference underwater image and the first and second underwater detection images is established to form the corresponding target regions and scattering reference regions in the three underwater images; The target structural response is determined based on the local structural contrast response, edge response, local structural contrast scale reference, and edge response scale reference; the scattering reference response is determined based on the non-structural brightness diffusion response, brightness diffusion scale reference, spatial fluctuation scale reference, and effective spatial fluctuation amount. The difference between the target structure responses of adjacent illumination states is used to obtain the target effective information increment corresponding to two equal-intensity light enhancements. The difference between the scattering reference responses of adjacent illumination states is used to obtain the scattering response increment corresponding to two equal-intensity light enhancements.

5. The underwater miniature multi-mode imaging system according to claim 1, characterized in that, The specific process for determining the insufficient effective imaging benefit state in the marginal effective imaging benefit inversion determination module is as follows: When the second marginal effective imaging gain is less than or equal to zero, the current imaging mode is determined to have entered a state of insufficient effective imaging gain. When the second marginal effective imaging gain is greater than zero, the marginal gain decrease is compared with the preset effective gain decrease threshold. When the marginal revenue decrease is greater than the preset effective revenue decrease threshold, the current imaging mode is determined to have entered the state of insufficient effective imaging revenue; otherwise, the current imaging mode is determined not to have entered the state of insufficient effective imaging revenue. When the current imaging mode has not entered the state of insufficient effective imaging benefits, the second detection illumination intensity will be used as the current working illumination intensity to continue the current underwater target imaging; when the current imaging mode enters the state of insufficient effective imaging benefits, it will switch to the multi-mode marginal benefit recovery and adaptation module.

6. The underwater miniature multi-mode imaging system according to claim 1, characterized in that, The specific process for determining candidate imaging modes in the multi-mode marginal benefit recovery adaptation module is as follows: Acquire the mode parameter information corresponding to each imaging mode supported by the underwater miniature multi-mode imaging device, as well as the target type and target imaging requirements corresponding to the current target imaging task; The mode parameter information includes the active illumination channel that can be called by the corresponding imaging mode, the active illumination adjustment range, the minimum stable adjustment amplitude, the basic illumination intensity, the main imaging parameters, and the applicable target type. The current target type is matched with the applicable target type of each imaging mode, and it is determined whether the corresponding imaging mode has active illumination adjustment capability. The imaging mode that matches the current target type, has active illumination adjustment capability, and is different from the current imaging mode is determined as the candidate imaging mode.

7. The underwater miniature multi-mode imaging system according to claim 6, characterized in that, The specific process of dual-incremental active illumination detection for candidate imaging modes is as follows: For each candidate imaging mode, when the corresponding active illumination channel has an actual output illumination intensity, the actual output illumination intensity is determined as the detection start illumination intensity; Otherwise, the corresponding base lighting intensity will be determined as the initial lighting intensity for detection; The difference between the maximum permissible illumination intensity of the candidate imaging mode and the initial illumination intensity of the detection is determined as the candidate remaining illumination adjustment amount; When the candidate remaining illumination adjustment amount is not less than twice the minimum stable adjustment amplitude, the single detection increment is formed according to the single detection increment determination method in the active illumination dual incremental response construction module; otherwise, the candidate imaging mode is excluded. The candidate imaging mode with dual-incremental detection conditions is invoked, the main imaging parameters are loaded and the active illumination channel is enabled. The initial illumination intensity is used as the reference illumination intensity to obtain a reference underwater image. The active illumination intensity is increased twice consecutively according to the same single detection increment to obtain the first and second underwater images.

8. The underwater miniature multi-mode imaging system according to claim 7, characterized in that, The specific process for determining effective candidate modes in the multi-mode marginal revenue recovery adaptation module is as follows: The target effective information increment and scattering response increment corresponding to the candidate imaging mode are determined in the same way as in the active illumination dual incremental response construction module, and the marginal effective imaging gain status is determined in the same way as in the marginal effective imaging gain turning point determination module. The target structure response, local structure contrast response and edge response corresponding to the second underwater image of the candidate imaging mode are obtained to form the effective information response result of the target corresponding to the candidate imaging mode. When the local structure contrast response reaches the minimum local structure contrast response requirement of the corresponding candidate imaging mode and the current target type, and the edge response reaches the minimum edge response requirement of the corresponding candidate imaging mode and the current target type, the target effective information response result is determined to meet the current target imaging requirements. Candidate imaging modes whose effective information response results meet the current target imaging requirements and have not entered the state of insufficient effective imaging benefits are identified as effective candidate modes.

9. The underwater miniature multi-mode imaging system according to claim 8, characterized in that, The specific process of target imaging mode determination and mode switching in the multi-mode marginal benefit recovery adaptation module is as follows: When only one valid candidate mode exists, that valid candidate mode is determined as the target imaging mode. When there are multiple effective candidate modes, compare the second marginal effective imaging gains corresponding to each effective candidate mode, and determine the effective candidate mode with the highest second marginal effective imaging gains as the target imaging mode; when there are two or more effective candidate modes and the second marginal effective imaging gains are both the highest values, determine the target imaging mode from among them according to the preset mode priority. When performing dual-incremental detection in the target imaging mode, the second detection illumination intensity is determined as the target working illumination intensity. The underwater miniature multi-mode imaging device is controlled to switch to the target imaging mode, the main imaging parameters corresponding to the target imaging mode are loaded, and the corresponding active illumination unit is controlled according to the target working illumination intensity to continue performing the current underwater target imaging.

Images