An unmanned ship dynamic light correction method and system based on a residual diffusion model

Abstract

The application discloses an unmanned ship dynamic light correction method and system based on a residual diffusion model, which comprises the following steps: first, an unmanned ship dynamic light correction method based on a residual diffusion model is provided, which comprises the following steps: S1, acquiring unmanned ship navigation images and light condition data thereof; S2, extracting light characteristics by using a light perception module; S3, performing light correction based on a residual diffusion model, specifically comprising the following two processing steps: a forward diffusion process: simulating dynamic distribution of light interference; a reverse diffusion process: gradually removing light interference to generate a corrected image; and S4, outputting the corrected image for target identification and path planning of the unmanned ship; second, an unmanned ship dynamic light correction system based on a residual diffusion model is provided, which comprises the following modules: a data acquisition module, a light perception module, a residual diffusion light correction module and an output and application module; and the application has the advantages that the light perception module and the residual diffusion correction model are introduced, and the definition and detail visibility are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the image processing technical field of unmanned ships, and in particular to an unmanned ship dynamic light correction method and system based on a residual diffusion model. BACKGROUND

[0002] Intelligent Shipping Systems (ISS) is a comprehensive system that utilizes advanced sensor technology, artificial intelligence (AI) algorithms, communication networks, and automation technology to achieve intelligent monitoring, navigation, and management of unmanned ship navigation states. It is widely used in marine transportation, offshore patrol, environmental monitoring, and search and rescue tasks, etc. Through intelligent means, it improves the efficiency and safety of navigation.

[0003] Computer Vision (CV) is an important branch of artificial intelligence, aiming to endow computers with the ability to process, analyze, and understand images or videos. It combines image processing, pattern recognition, deep learning, and multi-sensor fusion technologies, and has become a key support means for unmanned ship intelligent systems, providing important technical support for target detection, path planning, and dynamic obstacle avoidance, etc.

[0004] In the vision system of unmanned ships, due to the complexity of the navigation environment, the dramatic changes in lighting conditions have a significant impact on image quality. Strong light reflection, shadow areas, and insufficient lighting in the environment can cause the quality of images captured by the camera to decline. Especially under dynamic lighting conditions, these problems will be further exacerbated, leading to reduced image contrast, color distortion, and loss of target information. This situation not only significantly reduces the accuracy of target detection and path planning, but also threatens the navigation safety of unmanned ships.

[0005] Existing image correction methods are mostly based on traditional image enhancement techniques or simple denoising algorithms, which are difficult to fully address dynamic lighting problems. In view of the above problems, how to design a robust lighting correction method that can not only improve image quality under strong reflection and shadow conditions, but also meet the real-time processing requirements of unmanned ships, has become a hot and difficult point of current research. SUMMARY

[0006] The purpose of the present application is to overcome the above-mentioned defects in the prior art, and to provide an unmanned ship dynamic light correction method and system based on a residual diffusion model, which can effectively deal with strong reflection, shadow coverage, and uneven lighting conditions by introducing a light perception module and a residual diffusion correction model, significantly improving the clarity and detail visibility of unmanned ship images.

[0007] To achieve the above-mentioned purpose, the present application is realized through the following two aspects:

[0008] In a first aspect, the present application provides a dynamic light correction method for unmanned ships based on residual diffusion model, which comprises the following steps:

[0009] S1: Obtain the image and its light condition data of the unmanned ship sailing;

[0010] S2: Extract the light characteristics by using the light perception module, including light intensity, shadow distribution and reflection area;

[0011] S3: Perform light correction based on the residual diffusion model, which specifically comprises the following two processing steps:

[0012] Forward diffusion process: simulate the dynamic distribution of light interference;

[0013] Reverse diffusion process: gradually remove the light interference to generate a corrected image;

[0014] S4: Output the corrected image for target recognition and path planning of the unmanned ship.

[0015] As preferred, the S2 comprises the following sub-steps:

[0016] S21. Construction and training of multi-task deep learning model: construct a multi-task deep learning model containing multiple task branches of light intensity extraction, shadow area detection and reflection area identification;

[0017] S22. Light intensity extraction: extract the intensity distribution of light in the image, and generate a brightness map to reflect the overall light condition;

[0018] S23. Shadow area detection: use the shadow detection task branch of the multi-task deep learning model; identify the shadow area in the image, and extract the boundary, shape and coverage of the shadow;

[0019] S24. Reflection area identification: through the reflection area identification task branch of the multi-task deep learning model, locate the high reflection area in the image, and distinguish the reflection area from the non-reflection area;

[0020] S25. Integrate the light intensity, shadow distribution and reflection area characteristics output by the multi-task deep learning model to generate a light characteristic description, which provides input for subsequent light correction.

[0021] As preferred, in step S3, the residual diffusion model adds light conditions to model and correct light interference, wherein the specific processing steps of the forward diffusion process are: when adding noise to the image, input the light distribution characteristics as a condition, combine the dynamic light characteristics to affect the distribution of noise, and the noise generation formula is:

[0022]

[0023] where x t : target image or image at current time; x t-1 : image at previous time, as the basis of the current image; l: lighting condition, adjusting the lighting effect of the image; x res : residual, the difference between the target image and the degraded image; β t : diffusion coefficient, controlling the intensity of noise and residual; I: identity matrix, representing the covariance of noise; N(μ,σ 2 ): Gaussian distribution, used to model the probability distribution of the current image, where the mean variance σ 2 = β t ·I.

[0024] As preferred, in step S3, the residual diffusion model models and corrects the lighting interference by adding the lighting condition, wherein the inverse diffusion process specifically handles the steps: inverse diffusion restores the target image by gradually removing noise, residual is used to guide the denoising direction in this process, accelerate and optimize image restoration, and the inverse diffusion formula is:

[0025] x t-1 = x t - ∈θ(x t ,t,l) + λ·X res

[0026] where x t : image state at current time step t; x t-1 : image state at time step t-1;

[0027] ∈θ(x t ,t,l): conditional denoising network, predicting noise; λ: weight, controlling the intensity of residual correction; X res : residual, the difference between the target image and the degraded image.

[0028] As preferred, the network structure of the residual diffusion model adds a lighting condition embedding module and a residual diffusion fusion module, which includes the following contents:

[0029] Lighting condition embedding module: build global and local embedding mechanism, and embed lighting distribution feature l into diffusion model network; global embedding: use multi-layer perception (MLP) to globally encode lighting characteristics to generate lighting condition vector; local embedding: locally encode lighting characteristics through convolution network to generate lighting condition feature map with the same size as input image;

[0030] Residual diffusion fusion module: adjust the encoder and decoder parts of the diffusion model, introduce the dynamic guidance of residual information Xres, in the encoder, the illumination condition features and residual information are fused with the image features through feature splicing or additive fusion; in the decoder, a residual feature module is added to dynamically adjust the denoising direction and enhance the image correction effect.

[0031] As preferred, the loss function of the residual diffusion model includes the following parts:

[0032] Denoising reconstruction loss: minimize the error between the predicted noise of the denoising network and the real noise to optimize the denoising accuracy, and the formula is: L denoise =||∈θ(x t ,t,l)-∈|| 2

[0033] Where ∈θ(x t ,t,l) is the output noise prediction of the denoising network, and ∈ is the real noise;

[0034] Illumination correction loss: optimize the corrected image according to the illumination distribution characteristics, so that the illumination characteristics are consistent with the target, and the formula is: L light =||f(x output )-f(x input || 2

[0035] Where f(x) is an illumination feature extraction function (illumination mean), x output and x input are the corrected and original images respectively;

[0036] Detail enhancement loss: enhance the detail recovery effect of the corrected image through gradient or edge constraint, and the formula is:

[0037] Where, represents the gradient operator, is the real target image;

[0038] Total loss function: combine the above three loss functions to balance the influence of different losses with weight parameters λ1, λ2, λ3, and define as: L = λ1L denoise + λ2L light + λ3L detail .

[0039] In the second aspect, the application provides an unmanned ship dynamic illumination correction system based on a residual diffusion model, comprising:

[0040] A data acquisition module is used to acquire image data of the unmanned ship under different illumination conditions;

[0041] An illumination perception module is configured to acquire illumination characteristics of the input image.

[0042] A residual diffusion illumination correction module is configured to dynamically correct the illumination based on a residual diffusion model and generate a corrected image.

[0043] An output and application module is configured to output the corrected image to support target recognition and path planning of the unmanned ship.

[0044] The above modules are in a progressive relationship to complete the logical link.

[0045] Preferably, the illumination perception module further comprises a preprocessing unit configured to perform preliminary processing on the received image data.

[0046] Preferably, the residual diffusion illumination correction module further comprises a post-processing unit configured to perform post-processing on the corrected image and output the processed image to the output and application module.

[0047] Compared with the prior art, the present application has the following advantages:

[0048] The present application can accurately simulate the dynamic distribution of illumination interference and effectively remove the illumination interference by introducing the illumination perception module and the residual diffusion model, and can effectively deal with complex illumination conditions such as strong reflection, shadow coverage and uneven illumination, thereby significantly improving the illumination uniformity and detail clarity of the image.

[0049] Further, the illumination characteristics are extracted by the multi-task deep learning model, which enhances the adaptability of the correction process to complex illumination scenes; meanwhile, the residual diffusion mechanism is used to optimize the efficiency and effect of the correction, and the corrected image shows higher accuracy and robustness in subsequent unmanned ship target recognition and path planning tasks, and is suitable for various complex navigation environments. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0051] Figure 1 is a flow principle diagram of an unmanned ship dynamic illumination correction method based on a residual diffusion model provided by embodiment one of the present application;

[0052] Figure 2 is a framework principle diagram of an unmanned ship dynamic illumination correction system based on a residual diffusion model provided by embodiment two of the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are one of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0054] Embodiment one:

[0055] Please refer to Figure 1 Embodiment one of the present application provides a dynamic light correction method for unmanned ship based on residual diffusion model, which comprises the following steps:

[0056] S1: obtaining the image of the unmanned ship sailing and its light condition data;

[0057] Among them, through the camera equipment, the real-time image of the unmanned ship under different light conditions (strong light, shadow, high reflection, etc.) is obtained, and the related parameters of the light environment are recorded;

[0058] S2: using the light perception module to extract the light characteristics of the image of the unmanned ship sailing, generating the light description, the light characteristics including the light intensity, the shadow distribution and the reflection area and other characteristics;

[0059] Among them, the S2 comprises the following sub-steps:

[0060] S21. Construction and training of multi-task deep learning model: constructing a multi-task deep learning model, which contains light intensity extraction, shadow area detection and reflection area identification and other task branches;

[0061] S22. Light intensity extraction: extracting the brightness intensity distribution of the light in the image, generating a brightness map to reflect the overall light condition;

[0062] S23. Shadow area detection: using the shadow detection task branch of the multi-task deep learning model; identifying the shadow area in the image, extracting the boundary, shape and coverage of the shadow;

[0063] S24. Reflection area identification: through the reflection area identification task branch of the multi-task deep learning model, positioning the high reflection area in the image, and distinguishing the reflection area from the non-reflection area;

[0064] S25. Integrating the light intensity, shadow distribution and reflection area characteristics output by the multi-task deep learning model to generate a light characteristic description vector, providing input for subsequent light correction;

[0065] Further, a multi-task deep learning model: by sharing the feature extraction layer, jointly training the illumination intensity, shadow distribution and reflection region detection tasks, enhancing the robustness of illumination perception.

[0066] S3: illumination correction based on residual diffusion model, specifically including the following two processing steps:

[0067] Forward diffusion process: simulate the dynamic distribution of illumination interference;

[0068] Reverse diffusion process: gradually remove the illumination interference to generate a corrected image;

[0069] In step S3,

[0070] The residual diffusion model models and corrects the illumination interference by adding the illumination condition,

[0071] Wherein, the specific processing steps of the forward diffusion process are: when adding noise to the image, input the illumination distribution feature as a condition, combine the dynamic illumination characteristic to affect the distribution of noise, and the noise generation formula is:

[0072]

[0073] Wherein, x t : target image or image at current time; x t-1 : image at previous time, as the basis of the current image; l: illumination condition, adjusting the illumination effect of the image; X res : residual, the difference between the target image and the degraded image; β t : diffusion coefficient, controlling the intensity of noise and residual; I: unit matrix, indicating the covariance of noise; N(μ,σ 2 ): Gaussian distribution, used to model the probability distribution of the current image, where the mean Variance σ 2 = β t ·I;

[0074] Wherein, the specific processing steps of the reverse diffusion process are: the reverse diffusion restores the target image by gradually removing the noise, the residual is used to guide the denoising direction in this process, accelerate and optimize image recovery, and the reverse diffusion formula is:

[0075] x t-1 = x t - ∈θ(x t , t, l) + λ·X res

[0076] Wherein, x t : image state at current time step t; x t-1 : image state at time step t-1; ∈θ(x tLdeno(x, t, l) : conditional denoising network, predict noise; λ: weight, control the strength of residual correction; X res : residual, the difference between the target image and the degraded image.

[0077] And the network structure of the residual diffusion model adds a light condition embedding module and a residual diffusion fusion module, which includes the following contents:

[0078] Light condition embedding module: build global and local embedding mechanism, integrate light distribution feature l into diffusion model network; Global embedding: use multi-layer perception (MLP) to globally encode light characteristics to generate light condition vector; Local embedding: encode light characteristics locally through convolution network to generate light condition feature map with the same size as the input image;

[0079] Residual diffusion fusion module: adjust the encoder and decoder part of the diffusion model, introduce dynamic guidance of residual information Xres, in the encoder, through feature concatenation (Concatenation) or addition fusion (Addition) method, the light condition feature and the residual information are fused with the image feature; In the decoder, add a residual feature module to dynamically adjust the denoising direction and enhance the image correction effect.

[0080] Further, the loss function of the residual diffusion model includes the following parts:

[0081] Denoising reconstruction loss: minimize the error between the predicted noise of the conditional denoising network and the real noise to optimize the denoising accuracy, the formula is: L denoise = ||∈θ(x t ,t,l)-∈|| 2

[0082] Where ∈θ(x t ,t,l) is the output noise prediction of the denoising network, ∈ is the real noise;

[0083] Light correction loss: optimize the corrected image according to the light distribution characteristics, so that its light characteristics are consistent with the target, the formula is: L light = ||f(x output )-f(x input || 2

[0084] Where f(x) is the light feature extraction function (light mean), x output and x input are the corrected and original images respectively;

[0085] Detail enhancement loss: enhance the detail recovery effect of the corrected image through gradient or edge constraint, the formula is:

[0086] wherein, denotes the gradient operator, is a real target image;

[0087] Total loss function: The above three loss functions are combined to balance the influence of different losses with weight parameters λ1, λ2, λ3, defined as: L = λ1L denoise + λ2L light + λ3L detail .

[0088] S4: output the corrected image, perform post-processing on the corrected image, including contrast adjustment and detail increase, etc. post-processing, and finally output a high-quality image. The high-quality image is used for target recognition and path planning of unmanned ships and other applications.

[0089] Wherein, the unmanned ship navigation image is acquired in a dynamic environment, and may encounter sea surface fluctuations, different light conditions (such as day and night, and changeable weather), and other natural environmental disturbances. Finally, outputting a high-quality image is to solve these problems. Target detection algorithms based on deep learning (such as YOLO, Faster R-CNN, etc.) can be used to identify target objects in the image. Input the high-quality picture and identify the surrounding objects (such as buoys, marine obstacles, other ships, etc.) through the target detection algorithm. High-quality images can help these algorithms more accurately classify and locate, thereby avoiding collisions and navigation errors. Based on the target recognition results of the high-quality image, path planning algorithms (such as A*, Dijkstra algorithm, RRT, etc.) can calculate the optimal navigation path in real time. The unmanned ship can continuously adjust its path to avoid dynamic obstacles such as other ships or sea waves.

[0090] The unmanned ship dynamic light correction method based on the residual diffusion model of the embodiment one has the advantages that: by introducing the light perception module and the residual diffusion model, the technical solution can accurately simulate the dynamic distribution of light interference and effectively remove the light interference, can effectively deal with complex light conditions such as strong reflection, shadow coverage and uneven light, and significantly improve the light uniformity and detail clarity of the image; further, the light characteristics are extracted through the multi-task deep learning model, which enhances the adaptability of the correction process to complex light scenes; at the same time, the residual diffusion mechanism is used to optimize the efficiency and effect of the correction. After the corrected image is processed to generate a high-quality image, the high-quality image shows higher accuracy and robustness in the tasks of target recognition and path planning of the unmanned ship, and is suitable for various complex navigation environments.

[0091] Embodiment two:

[0092] Please refer to Figure 2Embodiment two of the present application provides an unmanned ship dynamic light correction system based on a residual diffusion model, comprising:

[0093] A data acquisition module is configured to acquire image data of the unmanned ship under different light conditions and record environmental light characteristics.

[0094] A light perception module is configured to acquire light characteristics of the input image, wherein the light characteristics include light intensity, shadow distribution and reflection region identification, and provide necessary light distribution data information for the subsequent residual diffusion model.

[0095] A residual diffusion light correction module is configured to dynamically correct the light based on the residual diffusion model and generate a corrected image.

[0096] An output and application module is configured to output the corrected image to support target recognition and path planning applications.

[0097] The above modules are in a progressive relationship to complete the logical link.

[0098] The data acquisition module further comprises a preprocessing unit configured to preliminarily process the received image data.

[0099] The residual diffusion light correction module further comprises a post-processing unit configured to post-process the corrected image, including contrast adjustment and detail enhancement, so that the image quality of the corrected image is enhanced and optimized, and then the corrected image is output to the output and application module.

[0100] The light perception module uses a deep learning model to extract light intensity, shadow distribution and reflection region characteristics, and the output and application module supports real-time correction and dynamic updating.

[0101] The unmanned ship dynamic light correction system based on the residual diffusion model of embodiment two of the present application can effectively deal with strong reflection, shadow coverage and uneven light conditions by introducing the light perception module and the residual diffusion light correction module, and significantly improve the clarity and detail visibility of the unmanned ship image.

[0102] The above is only a preferred embodiment of the present application and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A dynamic light correction method for unmanned ships based on residual diffusion model, characterized in that, It comprises the following steps: S1: obtaining unmanned ship navigation image and its illumination condition data; S2: extracting illumination characteristics using an illumination perception module; S3: performing illumination correction based on a residual diffusion model, specifically including the following two processing steps: Forward diffusion process: simulating the dynamic distribution of illumination interference; Reverse diffusion process: gradually removing illumination interference to generate a corrected image; S4: outputting the corrected image for target recognition and path planning of the unmanned ship; The S2 includes the following sub-steps: S21. Construction and training of a multi-task deep learning model: a multi-task deep learning model is constructed, which contains multiple task branches of illumination intensity extraction, shadow area detection and reflection area identification; S22. Illumination intensity extraction: extract the intensity distribution of illumination in the image, generate a brightness map to reflect the overall illumination condition; S23. Shadow area detection: using the shadow detection task branch of the multi-task deep learning model; Identify the shadow area in the image, extract the boundary, shape and coverage of the shadow; S24. Reflection area identification: through the reflection area identification task branch of the multi-task deep learning model, locate the high reflection area in the image, and distinguish the reflection area from the non-reflection area; S25. Integrate the illumination intensity, shadow distribution and reflection area characteristics output by the multi-task deep learning model to generate illumination characteristic description and provide input for subsequent illumination correction; In step S3, the residual diffusion model models and corrects the illumination interference by adding the illumination condition, wherein the specific processing steps of the forward diffusion process are: when adding noise to the image, the illumination condition is input as a condition, and the dynamic illumination condition is combined to affect the distribution of noise, and the noise generation formula is: where x t : target image or image at current time; x t-1 : image at previous time, as the basis of current image; l: lighting condition, adjusting the lighting effect of image; x res : residual, difference between target image and degraded image; β t : diffusion coefficient, controlling the intensity of noise and residual; I: identity matrix, representing the covariance of noise; N(μ,σ 2 ): Gaussian distribution, used to model the probability distribution of current image, where mean variance σ 2 = β t ·I; In step S3, the residual diffusion model models and corrects the illumination interference by adding the illumination condition, wherein the specific processing steps of the reverse diffusion process are: reverse diffusion restores the target image by gradually removing noise, and the residual is used to guide the denoising direction, accelerate and optimize image restoration, and the reverse diffusion formula is: x t-1 = x t - ∈θ(x t , t, l) + λ · X res where x t : image state at current time step t; x t-1 : image state at time step t-1; ∈θ(x t , t, l): conditional denoising network, predicting noise; λ: weight, controlling the strength of residual correction; X res : residual, the difference between target image and degraded image.

2. The dynamic light correction method for unmanned ship based on residual diffusion model according to claim 1, characterized in that, The network structure of the residual diffusion model adds a light condition embedding module and a residual diffusion fusion module, which includes the following contents: Light condition embedding module: build global and local embedding mechanism, integrate light condition l into diffusion model network; Global embedding: use multi-layer perception to globally encode the illumination characteristics to generate the light condition vector; Local embedding: encode the illumination characteristics locally through convolution network to generate the light condition feature map with the same size as the input image; Residual diffusion fusion module: adjust the encoder and decoder part of the diffusion model, introduce the dynamic guidance of residual information Xres, in the encoder, the light condition characteristics and residual information are fused with the image features through feature splicing or additive fusion; In the decoder, add a residual feature module to dynamically adjust the denoising direction and enhance the effect of image correction.

3. The dynamic light correction method for unmanned ship based on residual diffusion model according to claim 1, characterized in that, The loss function of the residual diffusion model includes the following parts: De-noising reconstruction loss: By minimizing the error between the predicted noise by the conditional de-noising network and the true noise, the de-noising accuracy is optimized, and the formula is: L denoise = ||∈θ(x t ,t,l)-∈|| 2 where ∈θ(x t t, l) is the output noise prediction of the denoising network, and ∈ is the real noise. Illumination correction loss: the corrected image is optimized according to the illumination distribution characteristics, so that the illumination characteristics are consistent with the target, and the formula is: L light =||f(x output )-f(x input || 2 where f(x) is an illumination feature extraction function, x output and x input are the corrected and original images, respectively. Detail enhancement loss: through gradient or edge constraint, the detail recovery effect of the corrected image is enhanced, and the formula is: wherein denotes the gradient operator, is the real target image; Total loss function: The above three loss functions are combined to balance the influence of different losses with weight parameters λ1, λ2, λ3, defined as: L = λ1L denoise + λ2L light + λ3L detail .

4. An unmanned ship dynamic light correction system based on residual diffusion model, characterized in that, It is used to execute the unmanned ship dynamic illumination correction method based on the residual diffusion model as claimed in any one of claims 1-3, comprising: The data acquisition module is configured to acquire image data of the unmanned ship under different illumination conditions. The illumination perception module is configured to acquire illumination characteristics of the input image. The residual diffusion illumination correction module is configured to dynamically correct the illumination based on a residual diffusion model to generate a corrected image. The output and application module is configured to output the corrected image to support target recognition and path planning of the unmanned ship. The modules are in a progressive relationship to complete logical links.

5. The dynamic light correction system for unmanned ship based on residual diffusion model according to claim 4, characterized in that, The illumination perception module further includes a preprocessing unit configured to perform preliminary processing on the received image data.

6. The dynamic light correction system for unmanned ship based on residual diffusion model according to claim 4, characterized in that, The residual diffusion illumination correction module further includes a post-processing unit configured to perform post-processing on the corrected image before outputting the corrected image to the output and application module.

Images