An image enhancement method and system for a cross-medium robot

Abstract

The application discloses an image enhancement method and system for a cross-medium robot, and the method comprises the following steps: acquiring a degraded visual image in a cross-medium complex environment and performing cross-medium degradation characterization to obtain a unified degradation description; a nonlinear biomimetic vision model based on a stochastic resonance mechanism is constructed to convert the enhancement problem into a physical model solving problem; a physical information neural network and an operator learning paradigm are used for inference solving to learn a function space mapping from a degraded image to an enhanced image under zero reference conditions; a high-level visual task optimization goal is introduced to make the enhanced result give consideration to visual quality and task performance; multi-objective collaborative optimization is performed on biomimetic vision constraints, physical information residuals, zero reference quality constraints and high-level visual task constraints to output an enhanced image. The application can uniformly process multiple problems such as defogging, non-uniform light correction and underwater optical degradation, reduces the dependence on paired reference samples, enhances the model generalization capability, and improves the reliability of cross-medium robot visual perception.

Description

Technical Field

[0001] This invention relates to image processing, computer vision, robot perception, physical information neural networks, and marine intelligent equipment technology, and in particular to an image enhancement method and system for cross-media robots. Background Technology

[0002] The development of marine resources, marine environmental protection, marine engineering operations, and the application of cross-domain unmanned systems have created an urgent need for the autonomous perception capabilities of cross-medium robots. When cross-medium robots perform navigation, detection, identification, measurement, and operational tasks in complex environments such as the sea surface, nearshore, shallow water, underwater, and water-land transition zones, visual images have advantages such as intuitive information, low equipment cost, small payload, and easy integration with high-level intelligent tasks, making them an important source of information for robots to acquire about the external environment.

[0003] However, the visual imaging mechanisms differ significantly across complex environments. In air or water vapor environments, suspended particles, haze, waves, and salt spray can lead to enhanced image scattering, decreased contrast, and loss of distant details. In water-land transition or non-uniform lighting environments, solar reflection, shadow occlusion, artificial light source attenuation, and localized strong reflections can cause uneven brightness distribution, localized overexposure, or underexposure. In underwater environments, light is affected by absorption and scattering by water and suspended media, with inconsistent attenuation across different wavelengths, easily resulting in optical degradation such as color cast, low illumination, low contrast, texture blurring, and noise amplification.

[0004] Existing image enhancement methods mainly include imaging model-based methods, prior rule-based methods, and end-to-end methods based on deep neural networks. Imaging model-based methods typically rely on simplified assumptions about medium parameters, illumination distribution, and scattering characteristics; prior rule-based methods are effective for specific environments but lack generalization ability when facing cross-medium scene changes; deep learning methods based on reference samples require a large number of paired degraded and sharp images, but in real-world cross-medium robot operation scenarios, samples are difficult to collect sufficiently, and sharp reference images are naturally lacking.

[0005] Therefore, a cross-media image enhancement technique is needed that can uniformly address issues such as dehazing, non-uniform light correction, and underwater optical degradation enhancement even under conditions of insufficient or zero labeled samples. This technique should be able to incorporate interpretable physical and biomimetic visual priors, utilize neural networks for efficient reasoning and solving of complex models, and further construct optimized targets for high-level visual tasks such as target detection, segmentation, recognition, and tracking. This will ensure that the enhancement results not only meet the visual quality requirements of the human eye but also serve the autonomous perception and decision-making of cross-media robots.

[0006] The main shortcomings of existing technologies are: First, existing methods are not adaptable to various media scenarios. The degradation mechanisms of air haze, non-uniform lighting, and underwater absorption and scattering are different, making it difficult for enhancement models trained or tuned in a single scenario to generalize to scene switching during continuous operation of robots across media.

[0007] Second, existing deep learning methods are heavily reliant on reference samples. In complex environments across media, it is difficult to obtain strictly paired degraded and clear images, and pseudo-labels can easily introduce algorithmic biases, leading to problems such as color distortion, excessive texture enhancement, or insufficient detail recovery in real-world tasks.

[0008] Third, traditional physical models are complex to solve and inefficient. Models describing the combined effects of nonlinear visual perception, light propagation, scattering, and noise often involve complex partial differential equations or nonlinear operators, and direct numerical solutions are computationally expensive, which is not conducive to online or quasi-online applications of robots.

[0009] Fourth, current visual quality optimizations mostly target the pixel level or human visual perception, lacking direct constraints on the performance of higher-level visual tasks. While image enhancement may improve brightness, contrast, or color metrics, it may not necessarily improve the stability and accuracy of robotic tasks such as object detection, semantic segmentation, and object tracking.

[0010] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0011] The main objective of this invention is to overcome the deficiencies in the aforementioned background technology and provide an image enhancement method and system for cross-media robots.

[0012] To achieve the above objectives, the present invention adopts the following technical solution: An image enhancement method for cross-media robots includes the following steps: S1. Acquire degraded visual images in complex cross-media environments; S2. Perform cross-media degradation characterization on the degraded visual image to obtain a unified degradation description; S3. Construct a nonlinear bionic vision mechanism model to transform the image enhancement problem into a nonlinear model solution problem with physical meaning; S4. Use a physical information neural network to reason and solve the nonlinear bionic vision mechanism model, and learn the function space mapping from the degraded image to the enhanced image under zero reference or weak reference conditions. S5. Introduce optimization objectives for high-level vision tasks to ensure that the enhancement results balance visual quality and robot task performance. S6. Perform multi-objective collaborative optimization on constraints of biomimetic vision model, physical information residual, zero-reference image quality constraint, and high-level vision task constraint, and output an enhanced image.

[0013] An image enhancement system for cross-media robots, for implementing the method, includes: The image acquisition module is used to acquire degraded visual images in complex environments across media; The cross-media degradation characterization module is used to perform cross-media degradation characterization on the input image; The nonlinear bionic vision modeling module is used to establish visual perception models based on random resonance or nonlinear threshold characteristics. The physical information neural network inference module is used to infer image enhancement results under physical constraints; The zero-reference visual quality constraint module is used to guide network training or inference in the absence of paired, clear reference images. The high-level vision task guidance module is used to guide the image enhancement process through task output, task loss, or intermediate feature feedback. The multi-objective weight optimization module is used to comprehensively weigh various optimization objectives. The enhanced image output module is used to output enhanced images and provide feedback on task performance to adjust and optimize target weights or enhancement strategies.

[0014] The present invention has the following beneficial effects: This invention proposes an image enhancement method and system for cross-medium robots. For visual images acquired by cross-medium robots in environments with fog, land-water transitions, non-uniform lighting, and complex underwater optical degradation, it achieves unified enhancement of image dehazing, non-uniform light correction, underwater color shift and scattering degradation suppression, texture detail restoration, and high-level visual task adaptation, even in the absence of paired reference samples. This invention uses degraded visual images from complex cross-medium environments as input, constructs a visual perception model based on nonlinear biomimetic visual mechanisms such as stochastic resonance, transforming the image enhancement problem into a physically meaningful nonlinear model solving problem. It utilizes a physical information neural network and operator learning paradigm to fit the function space mapping of this model, achieving inference solutions under zero-reference or weak-reference conditions. Simultaneously, it introduces optimization objectives and dynamic weight allocation mechanisms for high-level visual tasks, ensuring that the output enhanced image balances visual quality and robot task performance.

[0015] This invention provides a unified solution to various image degradation problems faced by robots across different media. The visual imaging mechanisms differ significantly in complex environments across different media. In air or water vapor environments, suspended particles, haze, waves, and salt spray can lead to enhanced image scattering, decreased contrast, and loss of distant details. In water-land transition or non-uniform lighting environments, solar reflection, shadow occlusion, attenuation of artificial light sources, and local strong reflections can cause uneven brightness distribution, local overexposure, or underexposure. In underwater environments, light is affected by absorption and scattering by water and suspended media, with inconsistent attenuation across different wavelengths, easily resulting in optical degradation such as color cast, low illumination, low contrast, texture blurring, and noise amplification. By introducing a nonlinear biomimetic vision mechanism and cross-media degradation representation, this invention can be applied to tasks such as dehazing, non-uniform light correction, underwater color shift correction, underwater scattering suppression, and low-contrast texture enhancement. It effectively overcomes the shortcomings of existing methods in terms of adaptability to media scenes. Existing imaging model-based methods usually rely on simplified assumptions about media parameters, illumination distribution, and scattering characteristics. Methods based on prior rules are only effective for specific environments and lack generalization when facing cross-media scene changes. Enhancement models trained or tuned in a single scene are difficult to generalize to scene switching in the continuous operation of cross-media robots. This invention fundamentally reduces the complexity of designing models for a single scene.

[0016] This invention reduces the reliance on paired reference samples and artificial labels. Existing deep learning methods based on reference samples require a large number of paired degraded images and clear images. However, in real-world cross-media robot operating scenarios, it is difficult to collect sufficient samples, and clear reference images are naturally lacking. False labels can easily introduce algorithmic biases, leading to problems such as color distortion, excessive texture enhancement, or insufficient detail recovery in real-world tasks. This invention uses zero-reference visual quality constraints, physical model residuals, and degradation consistency constraints to embed biomimetic visual models, optical degradation laws, and image quality priors as differentiable constraints into neural network training. The enhanced model can be trained or adaptively optimized using unlabeled degraded images in real-world cross-media scenarios, making it particularly suitable for applications in marine and land-sea transition zones where samples are scarce.

[0017] This invention improves the interpretability and generalization ability of the model. The combination of a nonlinear biomimetic vision model and a physical information neural network constrains the network output to the visual perception mechanism and optical degradation laws. The network approximates the solution of complex nonlinear models by learning operator mappings from input functions to output functions. The stochastic resonance mechanism can explain the synergistic effect between nonlinear systems, noise, and weak signals. When system parameters match the degraded image signal and noise perturbation, some noise energy can be converted into effective signal energy, enhancing low-contrast structures, weak edges, and dark textures beyond the perception threshold. Compared to traditional end-to-end black-box models, this invention effectively mitigates the failure risk under unknown water conditions, unknown lighting, and unknown scattering conditions.

[0018] This invention reduces the solution cost of complex nonlinear models. Traditional physical model solving often involves complex partial differential equations or nonlinear operators, describing the combined effects of nonlinear visual perception, light propagation, scattering, and noise. Direct numerical solutions are computationally expensive, hindering online or quasi-online robot applications. By using operator learning and physical information neural network fitting of function space mapping, this invention avoids the extensive scene-by-scene iterations required in traditional partial differential equation solving, improving the inference efficiency of the model on robot endpoints or edge computing platforms.

[0019] The enhancement results of this invention are more suitable for high-level visual tasks in robots. Existing visual quality optimizations often target the pixel level or human perception. While image enhancement may improve brightness, contrast, or color metrics, it may not necessarily improve the stability and accuracy of robotic tasks such as object detection, semantic segmentation, and object tracking. This invention establishes a collaborative relationship between the enhancement model and task models for detection, segmentation, recognition, and tracking. The intermediate features of the enhancement network can interact with semantic features, spatial attention, or task loss extracted by the high-level visual network, ensuring that the enhancement results retain edges, textures, target regions, and semantic structures useful for robotic tasks. Through multi-objective weight allocation, a balance can be achieved between visual quality, human perception, physical consistency, and task performance based on the current medium scene, image degradation level, and task requirements. The enhancement process prioritizes the protection of target boundaries, semantic structures, and task-related textures, thereby improving the reliability of cross-medium robots in recognition, localization, navigation, and operation in complex environments.

[0020] This invention can be deployed as a cross-media robot vision preprocessing module, an edge computing enhancement module, or a system combining cloud training and edge inference, and has good engineering applicability and deployment flexibility.

[0021] Other beneficial effects of the embodiments of the present invention will be further described below. Attached Figure Description

[0022] Figure 1 This is a flowchart of an image enhancement method for cross-media robots according to an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of the cross-media visual quality enhancement technology architecture based on biomimetic mechanisms in an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of the nonlinear threshold characteristics of visual neurons in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of a visual quality optimization framework for high-level visual task guidance according to an embodiment of the present invention.

[0026] Figure 5This is a comparison of the enhancement effects in typical degradation scenarios (foggy weather, non-uniform lighting, underwater optical degradation) according to embodiments of the present invention. Detailed Implementation

[0027] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0028] The complex environment of the ocean-land transition zone poses severe challenges to robot information acquisition, environmental perception, and motion adaptation. During cross-medium detection, the mechanisms of visual image degradation vary significantly across different scenarios, and sample data is scarce. Existing imaging model-based methods typically rely on simplified assumptions about medium parameters, illumination distribution, and scattering characteristics. Methods based on prior rules are only effective for specific environments and struggle to generalize to cross-medium scene transformations. End-to-end network-based methods require a large number of paired degraded and clear images, but sufficient samples are difficult to collect in real-world cross-medium robot operating scenarios. Therefore, how to scientifically explore the visual quality characterization in cross-medium scenes and construct visual quality enhancement optimization targets oriented towards heterogeneous degradation features to improve the cross-medium generalization ability of visual quality enhancement is a key scientific problem that this invention aims to solve.

[0029] This invention addresses visual images acquired by cross-medium robots in environments characterized by fog, water-land transitions, non-uniform lighting, and complex underwater optical degradation. It aims to provide a cross-medium image enhancement method and system that achieves unified enhancement of images, including dehazing, non-uniform light correction, underwater color shift and scattering degradation suppression, texture detail restoration, and adaptation to high-level visual tasks, even in the absence of paired reference samples. The method takes degraded visual images from complex cross-medium environments as input and constructs a visual perception model based on nonlinear biomimetic visual mechanisms such as stochastic resonance, transforming the image enhancement problem into a physically meaningful nonlinear model solving problem. It utilizes a physical information neural network and operator learning paradigm to fit the function space mapping of this model, achieving inference solutions under zero-reference or weak-reference conditions. Simultaneously, it introduces optimization objectives and dynamic weight allocation mechanisms oriented towards high-level visual tasks, ensuring that the output enhanced image balances visual quality and robot task performance.

[0030] Furthermore, this invention aims to establish a nonlinear biomimetic vision mechanism model that can generalize to represent heterogeneous degradation features. It utilizes a physical information neural network to perform efficient reasoning and solution on the model, reducing the dependence on labeled data and traditional numerical iteration. It also introduces high-level vision task objectives into the image enhancement and optimization process, making the enhanced images more suitable for cross-media robots to perform tasks such as detection, segmentation, recognition, localization, and navigation.

[0031] See Figure 1 This invention provides an image enhancement method for cross-media robots, comprising the following steps: Step S1: Acquire degraded visual images in complex cross-media environments.

[0032] In some embodiments, the degraded visual images in step S1 may include air haze images, salt spray or water vapor scattering images, non-uniform lighting images of the water-land transition zone, underwater images of shallow or deep water scenes, and continuous visual images collected when a cross-medium robot enters a water body from the air, exits the water body, or moves near the water surface.

[0033] Step S2: Perform cross-media degradation characterization on the degraded visual image to obtain a unified degradation description.

[0034] In some embodiments, the cross-medium degradation characterization in step S2 includes: integrating brightness distribution, color channel attenuation, local contrast, edge sharpness, background scattering intensity, dark area to highlight area ratio, and spatiotemporal continuity index to form a unified degradation description suitable for dehazing, non-uniform light correction, and underwater optical degradation enhancement tasks. The degradation description serves as the input condition for subsequent biomimetic visual models and physical information neural networks.

[0035] Step S3: Construct a nonlinear bionic vision mechanism model to transform the image enhancement problem into a nonlinear model solution problem with physical meaning.

[0036] In some embodiments, the construction of the nonlinear bionic visual mechanism model in step S3 includes: simulating the process by which the human visual system extracts effective signals in a noisy, redundant environment through a nonlinear threshold and a stochastic resonance mechanism; constructing a visual perception model based on the synergistic effect of a nonlinear system, degraded image signals, and noise perturbations; wherein the stochastic resonance mechanism describes the synergistic effect between the nonlinear system, noise, and excitation signals; and when the system parameters match the degraded image signals and noise perturbations, some noise energy is converted into effective signal energy, enabling weak structural signals to break through the perception threshold and be enhanced. Figure 3 The nonlinear thresholding characteristics of visual neurons, as shown, represent the nonlinear thresholding characteristics exhibited by the visual system during the perception and processing of light signals, enabling it to extract effective signals from constantly changing signals containing noise redundancy. The visual perception model is described using a bistable nonlinear system or a DaVin oscillator. The dynamic equations of the DaVin oscillator form include inertial, damping, elastic force, and nonlinear terms, with the driving force and noise terms on the right-hand side. Threshold nonlinearity is achieved through a potential function. The visual perception model is extended from a one-dimensional form to a two-dimensional image space and multiple channels of color images. For each color channel of a color image, a visual perception model in the form of partial differential equations about pixel coordinates is established, making the image enhancement process manifest as the suppression of degraded features, the enhancement of weak structural signals, and the restoration of color dynamic range.

[0037] In some embodiments, the key parameter setting method for the bistable nonlinear system or DaVin oscillator in step S3 includes: decomposing the input image into three color channels: red, green, and blue; performing discrete wavelet transform on the image data of the currently processed color channel to obtain low-frequency sub-bands and high-frequency detail sub-bands in the horizontal, vertical, and diagonal directions; rearranging the image data of the current color channel into a one-dimensional vector; calculating the variance of the one-dimensional vector to characterize the overall dispersion of the pixel values ​​of the channel; calculating the difference between the maximum and minimum pixel values ​​of the current color channel as the dynamic range of the channel; and based on the variance and the dynamic range... The reciprocal product is used to calculate the intermediate coefficient, which comprehensively reflects the grayscale fluctuation and pixel span of the image channel; the low-frequency sub-band is rearranged into a one-dimensional vector, and the variance of this one-dimensional vector is calculated to characterize the energy distribution or smoothing structure change of the wavelet low-frequency part; the system response speed parameter of the current color channel is determined by twice the product of the intermediate coefficient and the variance of the low-frequency sub-band, and the corresponding system response speed parameters of the three color channels are calculated independently; the system barrier parameter is set as an adaptive parameter learned through backpropagation during network training, which is used to learn adaptive barrier height adjustment; the scale factor is set to a predetermined hyperparameter value.

[0038] Step S4: Use a physical information neural network to reason and solve the nonlinear bionic vision mechanism model, and learn the function space mapping from the degraded image to the enhanced image under zero reference or weak reference conditions.

[0039] See Figure 2 As shown, in some embodiments, the reasoning and solution using a physical information neural network in step S4 includes: constructing a neural network for fitting a nonlinear operator based on an operator learning paradigm. The neural network includes a branch network and a backbone network. The branch network encodes the sampled values ​​of the input degraded image at a finite number of sensor locations, and the backbone network encodes the coordinates of any point in the domain of the output function. The output value of the nonlinear operator at the coordinates is approximated by the dot product of the branch network output and the backbone network output, thereby realizing the mapping learning from the input function space to the output function space. The residuals of the biomimetic visual model, image structure preservation constraints, and color consistency constraints are written into the loss function, enabling the network to learn the mapping from the degraded function space to the enhanced function space under zero reference conditions. The function space mapping is directly modeled through operator learning, combining data with physical knowledge, reducing the computational complexity of solving partial differential equations, and improving the model reasoning efficiency.

[0040] In some embodiments, the method for constructing the loss function in step S4 includes: constructing a bionic vision model residual loss based on the partial differential equation form of the nonlinear bionic vision mechanism model. The specific form of the bionic vision model residual loss is as follows: for each color channel of the color image, multiply the second-order mixing partial derivative of the enhanced image with respect to pixel coordinates by the damping ratio and the sum of the first-order partial derivatives of the enhanced image with respect to the horizontal and vertical directions, subtract the product of the system response speed parameter and the enhanced image, subtract the product of the barrier parameter and the cube of the enhanced image, and subtract the input degraded image. The result is used as a residual to constrain the enhancement result. To satisfy the physical laws of the visual perception model, a barrier smoothing loss is constructed. For the barrier parameters of each color channel, gradients are calculated in the horizontal and vertical directions of the image, and the sum of the squares of the gradient values ​​in the two directions is used as the barrier smoothing loss component of that channel. The components of each color channel are summed to smooth the details of the enhanced image. A color consistency constraint loss is constructed. The sum of the squares of the pairwise differences between the means of each color channel in the enhanced image is calculated, specifically including the sum of the squares of the differences between the means of the red and green channels, the means of the red and blue channels, and the means of the green and blue channels, to maintain the color consistency of the enhanced image.

[0041] Step S5: Introduce optimization objectives for high-level vision tasks to ensure that the enhancement results balance visual quality and robot task performance.

[0042] In some embodiments, the method for constructing the optimization objective for high-level vision tasks in step S5 includes: connecting a high-level vision network for object detection, semantic segmentation, keypoint localization, object tracking, or visual navigation to an enhancement model; evaluating the impact of the enhancement results on the high-level vision task at the output level, enabling the high-level vision task to provide feedback on object boundaries, semantic regions, texture structures, and motion consistency, and suppressing over-enhancement or color distortion caused by simply pursuing visual perception; and at the feature level, using a low-to-high-level visual feature fusion mechanism to enable high-level visual features to participate in the learning of the enhancement network, with the intermediate features of the enhancement network interacting with the semantic features, spatial attention, or task loss extracted by the high-level vision network. Figure 4The high-level vision task-guided visual quality optimization structure, based on a biomimetic visual perception model, simultaneously supervises the optimization process of underwater visual images from both the output and feature levels, forming a methodological framework that caters to both human and robotic applications. The specific implementation of the low-to-high-level visual feature fusion mechanism includes: selecting features extracted by the high-level vision network and features from the original image; taking multiple feature layers with different spatial resolutions; transforming both to the same dimension through convolutional layers; using a transposed attention mechanism to calculate an attention map representing the relationship between the two features; the attention map being a matrix of channel number × channel number, representing the relationship between the high-level features and the original image features; constructing a fused feature map using the attention map; and using it as the input to the corresponding layer of the visual quality optimization network decoder after processing by a feedforward network.

[0043] In some embodiments, the collaborative relationship between the physical information neural network inference and the high-level visual task guidance in step S5 includes: the enhanced image obtained by the physical information neural network inference is used as the input of the high-level visual task guidance module; the high-level visual task guidance module feeds back the physical information neural network and the multi-objective collaborative optimization module through task output, task loss, or intermediate features; the intermediate features of the enhancement network and the semantic features extracted by the high-level visual network interact pixel-by-pixel through a low-to-high-level visual feature fusion mechanism; the pixel-by-pixel interaction includes selecting multiple feature layers with different spatial resolutions, performing feature transformation, transposed attention calculation, and feedforward network processing on each feature layer, and using the resulting fused feature map as the input of the next layer of the visual quality optimization network decoder, so that the high-level visual features deeply participate in the learning and training process of the image enhancement network.

[0044] Step S6: Perform multi-objective collaborative optimization on the constraints of the bionic vision model, physical information residuals, zero-reference image quality constraints, and high-level vision task constraints, and output the enhanced image.

[0045] In some embodiments, the multi-objective collaborative optimization in step S6 includes: combining the constraints of the biomimetic visual model, physical information residuals, zero-reference image quality constraints, and high-level visual task constraints into optimization objectives; integrating the constraints through a multi-objective collaborative optimization module to perform a collaborative trade-off between the optimization objectives; coordinating the contributions of each objective through a fixed weight, adaptive weight, or deep reinforcement learning weight allocation strategy, wherein the weight allocation strategy adaptively adjusts the weights of each optimization objective according to the medium scene, degradation degree, task type, or reinforcement learning strategy; after enhancing the image output, returning the performance feedback of the enhancement result on the high-level visual task to the multi-objective collaborative optimization module for further adjusting the weights of the optimization objectives or the enhancement strategy, thereby improving the adaptability and reliability of cross-medium robot visual perception tasks.

[0046] In one embodiment, the present invention also provides an image enhancement system for cross-media robots, for implementing the above-described method, see reference. Figures 1 to 4 The system includes: an image acquisition module for acquiring degraded visual images in complex cross-media environments; a cross-media degradation characterization module for characterizing input images across media; a nonlinear biomimetic visual modeling module for establishing a visual perception model based on stochastic resonance or nonlinear threshold characteristics; a physical information neural network inference module for inferring image enhancement results under physical constraints; a zero-reference visual quality constraint module for guiding network training or inference in the absence of paired clear reference images; a high-level visual task guidance module for guiding the image enhancement process through task output, task loss, or intermediate feature feedback; a multi-objective weight optimization module for synergistically balancing various optimization objectives; and an enhanced image output module for outputting enhanced images and providing feedback on task performance to adjust the optimization objective weights or enhancement strategies.

[0047] The following further describes specific embodiments of the present invention and examples of its algorithm implementation.

[0048] A method and system for image enhancement in cross-media robots are disclosed. This method takes degraded visual images in complex cross-media environments as input, constructs a visual perception model based on nonlinear biomimetic visual mechanisms such as stochastic resonance, and transforms the image enhancement problem into a nonlinear model solving problem with physical meaning. It then uses a physical information neural network and operator learning paradigm to fit the function space mapping of this model, achieving inference solutions under zero-reference or weak-reference conditions. Simultaneously, it introduces optimization objectives and dynamic weight allocation mechanisms for high-level visual tasks, ensuring that the output enhanced image balances visual quality and robot task performance. At the system level, this invention can be deployed as a cross-media robot visual preprocessing module, an edge computing enhancement module, or a system combining cloud training and edge inference. The system includes at least an image acquisition module, a cross-media degradation representation module, a nonlinear biomimetic visual modeling module, a physical information neural network inference module, a zero-reference visual quality constraint module, a high-level visual task guidance module, a multi-objective weight optimization module, and an enhanced image output module.

[0049] like Figure 1 The diagram illustrates the process of a cross-media image enhancement solution provided by an embodiment of the present invention.

[0050] The degraded visual image is used as the input of this invention. The degraded visual image may include the image to be dehazed, the image under non-uniform illumination, and the underwater optical degradation image, etc. The input image first enters the cross-medium degradation characterization module, which uniformly characterizes the brightness distribution, color attenuation, scattering degradation, local contrast, and edge sharpness under different media environments.

[0051] The cross-media degradation characterization results are sequentially input into the nonlinear bionic vision mechanism modeling module, the physical constraint problem-solving construction module, and the physical information neural network inference and solution module. Specifically, the nonlinear bionic vision mechanism modeling module is used to establish a visual perception model based on stochastic resonance or nonlinear threshold characteristics; the physical constraint problem-solving construction module is used to transform the visual perception model into a constraint problem with physical meaning; and the physical information neural network inference and solution module is used to infer the image enhancement results under the physical constraints.

[0052] The zero-reference visual quality constraint module is connected to the physical information neural network inference and solution module. It is used to guide network training or inference by using constraints such as image structure preservation, color consistency, degradation consistency and visual quality evaluation when there are no paired clear reference images.

[0053] The high-level vision task guidance module is connected to the multi-objective collaborative optimization module and can include high-level vision tasks such as detection, segmentation, recognition, and tracking. This module guides the image enhancement process to preserve target boundaries, semantic regions, and task-related textures through task output, task loss, or intermediate feature feedback.

[0054] The multi-objective collaborative optimization module integrates constraints from the biomimetic vision model, physical information, zero-reference visual quality, and high-level vision task constraints. It collaboratively weighs various optimization objectives and outputs an enhanced image. After the enhanced image is output, its task performance feedback can be returned to the multi-objective collaborative optimization module for further adjustment of the optimization objective weights or enhancement strategies, thereby improving the adaptability and reliability of cross-media robot visual perception tasks.

[0055] Based on the above system architecture, the image enhancement method provided in this embodiment of the invention mainly includes the following steps: Step S1: Acquire degraded visual images in complex cross-medium environments. The degraded visual images include, but are not limited to, air haze images, salt spray or water vapor scattering images, non-uniform lighting images in the water-land transition zone, underwater images of shallow or deep water scenes, and continuous visual images acquired by the cross-medium robot when it enters the water body from the air, exits the water body, or moves near the water surface.

[0056] Step S2 involves performing cross-media degradation characterization on the input image. Based on features such as image brightness distribution, color channel attenuation, local contrast, edge sharpness, background scattering intensity, dark to highlight ratio, and spatiotemporal continuity, a degradation description applicable to dehazing, non-uniform light correction, and underwater optical degradation enhancement tasks is formed. This degradation description can serve as input conditions for subsequent biomimetic vision models and physical information neural networks.

[0057] Step S3: Establish a nonlinear bionic visual mechanism model. Simulate the process by which the human visual system extracts effective signals in a noisy, redundant environment through nonlinear thresholding and stochastic resonance mechanisms. Construct a visual perception model based on the synergistic effects of a nonlinear system, degraded image signals, and noise perturbations. This model can be described using a bistable nonlinear system or a DaVin oscillator, and can be extended to two-dimensional image space and multi-channel color images, enabling image enhancement to manifest as suppression of degraded features, enhancement of weak structural signals, and restoration of color dynamic range.

[0058] Step S4 transforms the biomimetic vision model into a physically constrained problem. Constraints in the form of partial differential equations, nonlinear operators, or energy functionals are established based on the nonlinear visual perception model to ensure that the enhanced image satisfies both physical and biomimetic vision priors in terms of brightness, contrast, color, structure, and degradation consistency. Through this step, cross-media image enhancement no longer relies solely on manual labels but searches for enhancement results under interpretable model constraints.

[0059] Step S5 involves using a physical information neural network for inference and solution. A neural network is constructed to fit the nonlinear operator. This neural network may include branch networks and a backbone network, which respectively encode the input degraded image, medium-related conditions, model parameters, and pixel or spatial coordinates, and output the enhancement result at the corresponding position or channel. During training, the biomimetic vision model residuals, boundary conditions, image structure preservation constraints, color consistency constraints, and degradation reconstruction constraints are written into the loss function, enabling the network to learn the mapping from the degradation function space to the enhancement function space under zero-reference conditions.

[0060] Step S6: Construct an optimization objective for high-level vision. Integrate high-level vision networks such as object detection, semantic segmentation, keypoint localization, object tracking, or visual navigation into the enhancement model. At the output level, evaluate the impact of the enhancement results on the high-level vision task. At the feature level, a low-to-high-level visual feature fusion mechanism allows high-level visual features to participate in the learning of the enhancement network. High-level vision tasks can provide feedback on object boundaries, semantic regions, texture structure, and motion consistency, thereby suppressing over-enhancement or color distortion caused by simply pursuing visual appeal.

[0061] Step S7: Perform multi-objective collaborative optimization and output the enhanced image. The optimization objective is composed of constraints from the biomimetic visual model, physical information residuals, zero-reference image quality, and high-level visual task. The contributions of each objective are coordinated using fixed weights, adaptive weights, or deep reinforcement learning weight allocation strategies. The final output is an enhanced image suitable for cross-media robot perception tasks, and, if necessary, simultaneously outputs degradation parameters, enhancement confidence, or task adaptation evaluation results.

[0062] The following provides a further explanation of the core technical implementation principles and key technical aspects of this invention.

[0063] like Figure 2 As shown, this invention constructs a biomimetic visual perception model based on a stochastic resonance mechanism. Based on the model's representation and quantification of visual quality, a zero-reference objective function constrained physical information neural network is designed to infer and solve the biomimetic visual model. Combined with optimization objectives for higher-level visual tasks, this achieves unsupervised cross-media visual quality enhancement across various media scenarios. Underwater visual images are severely degraded due to absorption and scattering, and these degradation characteristics are time-varying, space-varying, and often accompanied by extreme conditions. Constructing a biomimetic visual perception mechanism model can introduce additional prior knowledge, helping to narrow the exploration space for visual quality optimization possibilities. This is crucial for achieving underwater visual quality optimization under limited sample conditions. High-quality underwater enhanced images should ultimately serve higher-level downstream visual tasks. Therefore, establishing a zero-reference solution space and supplementing it with performance representations of higher-level visual tasks as optimization objectives constitutes the core implementation idea of ​​this invention. Specifically, this study models a nonlinear biomimetic visual perception mechanism, utilizing the nonlinear threshold characteristics exhibited by the visual system during the perception and processing of light signals. This constructs a visual perception model capable of removing degenerate features such as underwater absorption and scattering, and explores the relationship between the biomimetic model parameters and the model output. This reveals the optimization representation of visual quality by the biomimetic visual mechanism, providing a flexible and generalized solution space for underwater visual optimization under zero-reference conditions. Furthermore, it employs physical information neural network (PIN) inference for solution, fitting the function space of nonlinear operators to construct a physical model inference learning paradigm based on operator learning. This improves model generalization and data utilization efficiency, reduces dependence on labeled data, and avoids the computational complexity and cost of traditional numerical solutions. Finally, it optimizes visual quality for higher-level vision tasks through multi-objective optimization, ensuring the performance of higher-level visual tasks and enabling deep participation of these tasks in the learning and training of visual quality optimization. A multi-objective optimization method based on deep reinforcement learning is used to allocate weights between the visual quality optimization objective and the objectives of each higher-level visual perception task, achieving multi-task balanced optimization and enabling the method to be applied in practice. This invention starts from the perception mechanism of the biological visual system, establishes a generalized physical model of visual perception, and combines a zero-reference objective function and a physical information learning paradigm to achieve unsupervised visual quality enhancement in various media scenarios.

[0064] The specific implementation details of each key technical aspect are described below.

[0065] (1) Modeling method for nonlinear visual perception mechanism

[0066] like Figure 3As shown, the Human Visual System (HVS) possesses unparalleled capabilities in perceiving and processing optical information. Faced with constantly changing signals and noise redundancy, the visual system can very quickly select effective data signals. In neuroscience and physical psychology, this ability is described as a visual perception mechanism called stochastic resonance. The stochastic resonance mechanism originates from the nonlinear threshold-like characteristics of visual neurons. It is essentially a statistical phenomenon describing a synergistic effect between a nonlinear system, noise, and the excitation signal. When these three are externally added or altered through certain inherent processes to achieve a certain matching state, the system can convert internal noise energy into signal energy, thereby enabling weak signals to break through the threshold and be amplified.

[0067] Nonlinear systems can be represented using a Davin oscillator with a bistable potential well, and the dynamic equations can be understood from the derivation of the Brownian motion equations. Consider a mass of [mass value missing] immersed in a liquid. The Brownian particle. It is subject to a driving force. and damping force The synergistic effect of liquid molecules is described, in which Defined as the displacement of the Brownian particle. Let be the damping coefficient. Therefore, the macroscopic equation of Brownian motion can be described as: (1) When both sides are divided by When we get the equation: (2) in, and Let represent the damping ratio and the molecular collision force per unit mass, respectively. , Indicates noise intensity. This represents white Gaussian noise with a mean of 0 and a variance of 1. If an external potential field exists, equation (1) can be further written as: (3) in The potential function describes the external potential field. The potential function... Substitute into formula (2) and add the external driving force. Then formula (2) will be written as: (4) The above equation represents a bistable nonlinear system described by a Davin oscillator. The left side of the equation is... It is the inertial term. It is a damping term. It is an elastic force term. It is a nonlinear term. The right-hand side of the equation... and These are the driving force term and the noise term, respectively. Thus, formula (4) describes a system consisting of a nonlinear system, a signal, and noise, which achieves threshold nonlinearity through a potential function.

[0068] To optimize underwater color visual images, the one-dimensional Daphne oscillator can be generalized to a two-dimensional Daphne oscillator. Therefore, equation (4) can be transformed into: (5) in, It is the solution to formula (4), that is, the optimized image. These are the three channels of a color image. These are the pixel coordinates of the image.

[0069] Regarding the parameters in formulas (4) and (5) Configure adaptive value retrieval: parameter To calculate the system response speed parameters corresponding to random resonance in the color channels, the channel-by-channel calculation method is as follows: The input image is decomposed into three color channels: red, green, and blue. A single-channel image is denoted as Img(:,:,d) (d=1,2,3). A discrete wavelet transform is performed on the current channel image to obtain the low-frequency subband LL and the horizontal, vertical, and diagonal high-frequency subbands HL, LH, and HH. The current channel image is then rearranged into a one-dimensional vector A. i And calculate the vector variance Characterizes the overall dispersion of channel pixels; calculates the channel dynamic range dr=max(Img(:,:,d))-min(Img(:,:,d)); constructs intermediate coefficients. Rearrange the low-frequency subband LL into a one-dimensional vector B. i Calculate the low-frequency variance Characterizes the low-frequency energy distribution; ultimately, the current channel adaptive parameters The three channels are solved independently to obtain their respective solutions. .

[0070] Parameter β c The system barrier parameter γ does not require manual pre-setting; it is adaptively learned through backpropagation during network training and used to dynamically adjust the system barrier height. The parameter γ is a scale hyperparameter, which can be uniformly preset to a value of 1.

[0071] In the training phase of the physical information neural network of this invention, boundary conditions and degradation reconstruction constraints are discarded, and three types of loss functions are jointly used to construct the overall optimization objective. The specific forms of each loss function are as follows: The residual loss Ls of the biomimetic visual model corresponds to the residual of the partial differential equation of the two-dimensional DaVin oscillator:

[0072] The residual is used to constrain and enhance the image to meet the laws of the biomimetic visual physics model.

[0073] Barrier smoothing loss Lp:

[0074] x , y These are the gradient operators for the horizontal and vertical directions of the image, respectively. This loss is used to smooth and enhance image details.

[0075] Color consistency constraint loss:

[0076] In the formula, u is the input image, ε = {(R, G), (R, B), (G, B)}, q and p represent one channel of the enhanced image, and μ( This represents calculating the mean of the image channels, and constraining the color consistency of the output image by the sum of the squared differences between the three channel means.

[0077] Thus, the underwater vision quality optimization problem is transformed into solving a partial differential equation describing the visual perception mechanism, which can then be used as the objective function to constrain the neural network.

[0078] (2) Solving the physical model based on operator learning

[0079] The biomimetic visual perception model proposed above is described by complex dynamic partial differential equations. By utilizing the operator learning paradigm, it is hoped that the potential properties of the biomimetic visual perception system can be discovered and learned from the data, thus promoting the efficient solution of partial differential equations. It is an input function The operator, That is the corresponding output function. For any point within the domain Output Since is a real number, the indefinite integral operator can be learned by training the backbone and branch networks, with the network input consisting of _____. and It consists of two parts, and the output is Therefore, we have the generalized universal approximation theorem for operators: (3) in express The dot product in [the context]. Furthermore... functions and Different types of neural networks can be selected, all satisfying the classic universal approximation theorem for functions. Operator learning, by directly modeling the function space mapping, constructs a solution framework for biomimetic visual models, promoting the organic integration of data and physical knowledge. This not only improves the physical interpretability and generalization of neural networks but also reduces the solution complexity of partial differential equations, thereby enhancing the computational efficiency of the model.

[0080] (3) Method for constructing underwater vision quality optimization targets for high-level vision

[0081] To enhance the ability of visual quality optimization to address practical tasks, this invention proposes using multiple high-level visual tasks as optimization targets for the visual quality optimization method. For example... Figure 4 As shown, based on the biomimetic visual perception model, the optimization process of underwater visual images is supervised simultaneously from the output level and the feature level, forming a method framework that takes into account both human eye and robot applications.

[0082] In one specific embodiment, high-level visual networks such as object detection, semantic segmentation, key point localization, object tracking, or visual navigation each use their own pre-trained networks, and each high-level visual network itself does not need to redesign its network structure; the network features in each task network are fused with the features of the image enhancement network through a low-to-high-level visual feature fusion mechanism, and participate in the training of the image enhancement network, thereby guiding the image enhancement process to adapt to the needs of high-level visual tasks.

[0083] At the feature level, a high-level visual guidance module is designed, utilizing features extracted from multiple high-level visual networks to adjust the selection of visual quality optimization features. Specifically, the selection... Each spatial resolution is and High-level visual / original image features ( , ),in and These are the height and width of the input image, respectively. The high-level visual guidance module executes... and The pixel-by-pixel interactions between them are analyzed, and the final fused feature map is provided. During the interaction, a convolutional layer is first used to... and The image is transformed to the same dimension, and then the attention map is computed using a low-computational transposed attention mechanism. The attention map for high-level visual perception can be described as follows: (6) in and It is a convolutional layer. As feature channels, thus obtaining the attention map bitmap for high-level visual perception. It can characterize and The interrelationships between them. Finally, using... The image features are constructed as follows: (7) in, It is a feedforward network. For the first The fused feature maps of each module become the decoder of the visual quality optimization network. The high-level vision guidance module acts as a bridge between the high-level vision network and the vision quality optimization network, establishing feature-level connections and performing collaborative optimization for the two heterogeneous tasks.

[0084] Experimental verification

[0085] To verify the applicability and enhancement effect of the cross-media image enhancement method described in this invention under different degraded media scenarios, three typical cross-media degradation scenarios—foggy images, non-uniform illumination images, and underwater optically degraded images—were selected for experimental verification. Specifically, the foggy images used the O-HAZE dataset (45 images); the non-uniform illumination images used the LOL dataset (450 images); and the underwater optically degraded images used the UIEBD dataset (890 images). These datasets correspond to different degradation mechanisms, such as air haze scattering, complex non-uniform illumination, and underwater absorption scattering, and can be used to verify the generalization enhancement capability of the method of this invention for cross-media heterogeneous degraded images.

[0086] In this embodiment, three types of test images are input into the cross-media image enhancement model of the present invention. The model first performs cross-media degradation characterization on the input images, extracting degradation features such as brightness distribution, color attenuation, local contrast, scattering degradation, and structural sharpness; then, it establishes a visual quality enhancement model using a nonlinear biomimetic vision mechanism, and solves the enhancement model through a physical information neural network; finally, it performs multi-objective collaborative optimization on the enhancement results by combining zero-reference visual quality constraints and high-level visual task-related constraints, and outputs the enhanced image.

[0087] Experimental results show that, for foggy images, the method of this invention can effectively reduce the gray-white occlusion caused by fog scattering and improve the contrast and edge clarity of distant scenes; for non-uniformly lit images, the method of this invention can increase the brightness of dark areas, enhance local details, and suppress overexposure and color distortion; for underwater optically degraded images, the method of this invention can improve the color cast, low contrast and texture blur of underwater images, making the underwater scene structure, coral texture and background areas clearer. Figure 5The images show a comparison of the enhancement effects of the present invention in typical degradation scenarios (foggy weather, non-uniform lighting, and underwater optical degradation). Figure 5 As can be seen from the before-and-after comparison, the method of the present invention can improve the visual quality of images in three typical degradation scenarios, and has good cross-media adaptability.

[0088] The Natural Image Quality Evaluation (NIQE) metric was further used to objectively evaluate the enhancement results. Test results show that the average NIQE score of the enhancement results was 4.7830 on the O-HAZE foggy image test set; 5.9924 on the LOL non-uniform illumination image test set; and 4.4872 on the UIEBD underwater optical degradation image test set. Since the NIQE metric is commonly used to measure image naturalness, a lower score indicates better image naturalness and perceived quality. Therefore, the above results demonstrate that the method of this invention can achieve relatively stable image enhancement effects in various cross-media scenarios, including foggy weather, non-uniform illumination, and underwater optical degradation.

[0089] Table 1. NIQE scores for various cross-media image enhancement results

[0090] This verifies that the method of the present invention can perform unified enhancement processing on heterogeneous degraded images under different media environments. It can not only improve the subjective visual effect of the human eye, but also achieve good enhancement results on the no-reference image quality evaluation index, indicating that the present invention has good cross-scene generalization ability and engineering application feasibility.

[0091] In summary, this invention proposes an image enhancement method and system for cross-media robots. The core technical principle of this invention lies in transforming the cross-media image enhancement problem from a simple pixel mapping problem into a "physical information reasoning and solving problem under the constraints of a biomimetic visual perception model".

[0092] At the level of biomimetic visual modeling, the stochastic resonance mechanism can explain the synergistic effect between nonlinear systems, noise, and weak signals: when the system parameters match the degraded image signal and noise perturbation, some noise energy can be converted into effective signal energy, enabling low-contrast structures, weak edges, and dark textures to break through the perception threshold and be enhanced.

[0093] At the level of physical information neural networks, this invention does not require the establishment of a large number of paired training samples for every medium environment. Instead, it embeds biomimetic visual models, optical degradation laws, and image quality priors as differentiable constraints into the neural network training. The network learns operator mappings from input functions to output functions to approximate solutions to complex nonlinear models, thereby reducing the iterative costs of traditional numerical solutions and improving the generalization ability to unknown scenarios.

[0094] At the high-level visual optimization level, this invention establishes a collaborative relationship between the augmentation model and task models such as detection, segmentation, recognition, and tracking. The intermediate features of the augmentation network can interact with semantic features, spatial attention, or task loss extracted by the high-level visual network, ensuring that the augmentation results retain edges, textures, target regions, and semantic structures useful for the robotic task. Through multi-objective weight allocation, a balance can be achieved between visual quality, human perception, physical consistency, and task performance, based on the current media scene, image degradation level, and task requirements.

[0095] Compared with the prior art, the significant technical advantages of the present invention are reflected in the following aspects: First, this invention can uniformly address various image degradation problems faced by cross-media robots. By introducing nonlinear biomimetic vision mechanisms and cross-media degradation representations, it can be applied simultaneously to tasks such as dehazing, non-uniform light correction, underwater color shift correction, underwater scattering suppression, and low-contrast texture enhancement, reducing the complexity of designing separate models for each scene.

[0096] Second, this invention reduces the reliance on paired reference samples and manual labels. Through zero-reference visual quality constraints, physical model residuals, and degradation consistency constraints, the augmented model can be trained or adaptively optimized using unlabeled degraded images in real cross-media scenarios, making it suitable for applications in marine and land-sea transition zones where samples are scarce.

[0097] Third, this invention improves the interpretability and generalization ability of the model. The combination of nonlinear biomimetic vision model and physical information neural network makes the network output subject to the visual perception mechanism and optical degradation law, which can alleviate the failure risk of end-to-end black box model under unknown water, unknown lighting and unknown scattering conditions.

[0098] Fourth, this invention reduces the solution cost of complex nonlinear models. By using operator learning and physical information neural network fitting of function space mapping, it avoids the large number of scene-by-scene iterations required in traditional partial differential equation solving, thus improving the inference efficiency of the model on robot end-users or edge computing platforms.

[0099] Fifth, the enhancement results of this invention are more suitable for high-level vision tasks in robots. By introducing task objectives such as detection, segmentation, and tracking, the enhancement process can prioritize the protection of target boundaries, semantic structures, and task-related textures, thereby improving the reliability of cross-media robots in recognition, localization, navigation, and operation in complex environments.

[0100] This invention also provides a storage medium for storing a computer program, which, when executed, performs at least the methods described above.

[0101] This invention also provides a control device, including a processor and a storage medium for storing a computer program; wherein the processor executes the computer program by performing at least the method described above.

[0102] This invention also provides a processor that executes a computer program, at least performing the methods described above.

[0103] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc or CD-ROM; magnetic surface memory can be disk storage or magnetic tape storage. The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0104] In the several embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0105] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0106] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0107] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0109] The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0110] The features disclosed in the several product embodiments provided by this invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0111] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0112] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or application, should be considered within the scope of protection of the present invention.

Claims

1. An image enhancement method for cross-media robots, characterized in that, Includes the following steps: S1. Acquire degraded visual images in complex cross-media environments; S2. Perform cross-media degradation characterization on the degraded visual image to obtain a unified degradation description; S3. Construct a nonlinear bionic vision mechanism model to transform the image enhancement problem into a nonlinear model solution problem with physical meaning; S4. Use a physical information neural network to reason and solve the nonlinear bionic vision mechanism model, and learn the function space mapping from the degraded image to the enhanced image under zero reference or weak reference conditions. S5. Introduce optimization objectives for high-level vision tasks to ensure that the enhancement results balance visual quality and robot task performance. S6. Perform multi-objective collaborative optimization on constraints of biomimetic vision model, physical information residual, zero-reference image quality constraint, and high-level vision task constraint, and output an enhanced image.

2. The method according to claim 1, characterized in that, The cross-medium degradation characterization described in step S2 includes: By integrating brightness distribution, color channel attenuation, local contrast, edge sharpness, background scattering intensity, ratio of dark to bright areas, and spatiotemporal continuity indicators, a unified degradation description suitable for tasks such as dehazing, non-uniform light correction, and underwater optical degradation enhancement is formed. This degradation description serves as the input condition for subsequent biomimetic visual models and physical information neural networks.

3. The method according to claim 1, characterized in that, The construction of the nonlinear biomimetic visual mechanism model in step S3 includes: The process of extracting effective signals by the human visual system in a noisy and redundant environment through nonlinear threshold and stochastic resonance mechanism is simulated. A visual perception model is constructed by the synergistic effect of nonlinear system, degraded image signal and noise perturbation. The stochastic resonance mechanism describes the synergistic effect between nonlinear system, noise and excitation signal. When the system parameters are matched with the degraded image signal and noise perturbation, part of the noise energy is converted into effective signal energy, which makes the weak structure signal break through the perception threshold and be enhanced. The visual perception model is described using a bistable nonlinear system or a Davin oscillator. The dynamic equations in the Davin oscillator form include inertial terms, damping terms, elastic force terms, and nonlinear terms. The right side of the equations consists of driving force terms and noise terms. Threshold nonlinearity is achieved through a potential function. The visual perception model is extended from one-dimensional form to two-dimensional image space and multiple channels of color images. For each color channel of a color image, a visual perception model in the form of a partial differential equation about pixel coordinates is established, so that the image enhancement process manifests as the suppression of degradation features, the enhancement of weak structural signals, and the restoration of color dynamic range.

4. The method according to claim 3, characterized in that, The key parameter setting method for the bistable nonlinear system or Davin oscillator mentioned in step S3 includes: The input image is decomposed into three color channels: red, green, and blue. Discrete wavelet transform is performed on the image data of the current color channel to obtain low-frequency sub-bands and high-frequency detail sub-bands in the horizontal, vertical, and diagonal directions. The image data of the current color channel is rearranged into a one-dimensional vector, and the variance of the one-dimensional vector is calculated to characterize the overall dispersion of the pixel values ​​of the channel. The difference between the maximum and minimum pixel values ​​of the current color channel is calculated as the dynamic range of the channel. An intermediate coefficient is calculated based on the product of the variance and the reciprocal of the dynamic range to comprehensively reflect the grayscale fluctuation and pixel span of the image channel. The low-frequency subbands are rearranged into a one-dimensional vector, and the variance of the one-dimensional vector of the low-frequency subbands is calculated to characterize the energy distribution or smoothing structure changes of the low-frequency part of the wavelet. The system response speed parameter of the current color channel is determined by twice the product of the intermediate coefficient and the low-frequency sub-band variance. The corresponding system response speed parameters of the three color channels are calculated independently. The system barrier parameters are set to adaptive parameters learned through backpropagation during network training, which are used to learn adaptive barrier height adjustment. Set the scaling factor to a predetermined hyperparameter value.

5. The method according to claim 1, characterized in that, Step S4, which involves using a physical information neural network for reasoning and solving, includes: A neural network for fitting nonlinear operators is constructed based on the operator learning paradigm. The neural network includes a branch network and a backbone network. The branch network encodes the sampled values ​​of the input degraded image at a finite number of sensor locations, and the backbone network encodes the coordinates of any point in the domain of the output function. The output value of the nonlinear operator at the coordinates is approximated by the dot product of the output of the branch network and the output of the backbone network, thereby realizing the mapping learning from the input function space to the output function space. By incorporating the residuals of the biomimetic visual model, image structure preservation constraints, and color consistency constraints into the loss function, the network learns the mapping from the degenerate function space to the augmented function space under zero-reference conditions. By directly modeling function space mappings through operator learning, data can be combined with physical knowledge, reducing the computational complexity of solving partial differential equations and improving the efficiency of model inference.

6. The method according to claim 5, characterized in that, The method for constructing the loss function in step S4 includes: The residual loss of the bionic vision model is constructed based on the partial differential equation form of the nonlinear bionic vision mechanism model. The specific form of the residual loss of the bionic vision model is as follows: For each color channel of the color image, the second-order mixing partial derivative of the enhanced image with respect to the pixel coordinates is multiplied by the damping ratio and the sum of the first-order partial derivatives of the enhanced image with respect to the horizontal and vertical directions. The product of the system response speed parameter and the enhanced image is then subtracted, followed by the product of the barrier parameter and the cube of the enhanced image, and finally the input degraded image is subtracted. The result is used as the residual to constrain the enhancement result to meet the physical laws of the visual perception model. To construct a barrier smoothing loss, for the barrier parameters of each color channel, gradients are calculated in the horizontal and vertical directions of the image, and the sum of the squares of the gradient values ​​in the two directions is used as the barrier smoothing loss component of that channel. The components of each color channel are summed to smooth the details of the enhanced image. A color consistency constraint loss is constructed, which calculates the sum of squares of the pairwise differences between the means of each color channel in the enhanced image. Specifically, this includes the sum of squares of the differences between the means of the red and green channels, the red and blue channels, and the green and blue channels, in order to maintain the color consistency of the enhanced image.

7. The method according to claim 1, characterized in that, The method for constructing the optimization objective for high-level vision tasks in step S5 includes: Integrate high-level visual networks for target detection, semantic segmentation, key point localization, target tracking, or visual navigation into the augmentation model. Evaluate the impact of the augmentation results on high-level visual tasks at the output level, enabling high-level visual tasks to provide feedback on target boundaries, semantic regions, texture structures, and motion consistency, and suppressing over-enhancement or color distortion caused by simply pursuing visual perception. At the feature level, a low-to-high-level visual feature fusion mechanism is used to enable high-level visual features to participate in the learning of the augmentation network. The intermediate features of the augmentation network interact with the semantic features, spatial attention, or task loss extracted by the high-level visual network. The specific implementation of the low-to-high-level visual feature fusion mechanism includes: selecting features extracted by the high-level visual network and features of the original image; taking multiple feature layers with different spatial resolutions; transforming both to the same dimension through convolutional layers; using a transposed attention mechanism to calculate an attention map representing the relationship between the two; the attention map is a matrix of channel number × channel number, representing the relationship between the high-level features and the original image features; using the attention map to construct a fusion feature map; and using it as the input of the corresponding layer of the visual quality optimization network decoder after processing by a feedforward network.

8. The method according to claim 1, characterized in that, The collaborative relationship between the physical information neural network inference solution and the high-level vision task guidance mentioned in step S5 includes: The enhanced image obtained by the physical information neural network inference solution is used as the input of the high-level vision task guidance module. The high-level vision task guidance module feeds back the physical information neural network and the multi-objective collaborative optimization module through task output, task loss or intermediate features. The intermediate features of the augmentation network and the semantic features extracted by the high-level visual network interact pixel-by-pixel through a low-to-high-level visual feature fusion mechanism. The pixel-by-pixel interaction includes selecting multiple feature layers with different spatial resolutions, performing feature transformation, transpose attention calculation, and feedforward network processing on each feature layer, and using the resulting fused feature map as the input to the next layer of the visual quality optimization network decoder, so that the high-level visual features are deeply involved in the learning and training process of the image augmentation network.

9. The method according to claim 1, characterized in that, The multi-objective cooperative optimization mentioned in step S6 includes: The constraints of the biomimetic vision model, physical information residual, zero-reference image quality, and high-level vision task are combined to form the optimization objective. The multi-objective collaborative optimization module integrates the various constraints and makes a collaborative trade-off between the optimization objectives. The contributions of each objective are coordinated through a fixed weight, adaptive weight, or deep reinforcement learning weight allocation strategy. The weight allocation strategy adaptively adjusts the weights of each optimization objective based on the medium scenario, degradation degree, task type, or reinforcement learning strategy. After enhancing the image output, the performance feedback of the enhancement results on high-level vision tasks is returned to the multi-objective collaborative optimization module for further adjustment and optimization of target weights or enhancement strategies, thereby improving the adaptability and reliability of cross-media robot visual perception tasks.

10. An image enhancement system for cross-media robots, used to implement the method according to any one of claims 1 to 9, characterized in that, include: The image acquisition module is used to acquire degraded visual images in complex environments across media; The cross-media degradation characterization module is used to perform cross-media degradation characterization on the input image; The nonlinear bionic vision modeling module is used to establish visual perception models based on random resonance or nonlinear threshold characteristics. The physical information neural network inference module is used to infer image enhancement results under physical constraints; The zero-reference visual quality constraint module is used to guide network training or inference in the absence of paired, clear reference images. The high-level vision task guidance module is used to guide the image enhancement process through task output, task loss, or intermediate feature feedback. The multi-objective weight optimization module is used to comprehensively weigh various optimization objectives. The enhanced image output module is used to output enhanced images and provide feedback on task performance to adjust and optimize target weights or enhancement strategies.

Images