Forest field near-infrared image generation method and system based on image case guidance

Abstract

The application provides a guided forest field near-infrared image generation method and system based on image cases, and belongs to the technical field of image generation. The method comprises the following steps: acquiring registered visible light case images and near-infrared case images under the same forest field scene as a case image pair; jointly encoding the first case image pair to extract case condition features representing the mapping relationship from the visible light domain to the near-infrared domain; encoding the visible light case image in the second case image pair to extract scene geometric layout information as first semantic structure features; fusing the case condition features and the first semantic structure features to obtain control features; based on the control features, using an implicit diffusion model to gradually denoise an initial tensor in an implicit space, and in the denoising process, the control features are injected into multiple levels of the trunk branch through the bypass branch of the diffusion model to guide the denoising direction; and decoding to generate a forest field near-infrared image. The image generated by the application is consistent in spatial structure and conforms to the near-infrared imaging rule.

Description

Technical Field

[0001] This application belongs to the field of image generation technology, and in particular relates to a method and system for generating near-infrared images of forest farms based on image case pair guidance. Background Technology

[0002] With the rapid development of smart forestry and multimodal image analysis technologies, forest image data has become an important basis for forest resource surveys, vegetation growth monitoring, fire early warning, and ecological environmental protection. Compared with visible light images, near-infrared images have significant advantages in reflecting the cell structure of plant leaves, water content, and tree reflectivity, playing a crucial role in analyzing vegetation status and forest conditions. Therefore, acquiring high-quality near-infrared images of forests that match visible light images is of great significance for improving forest scene understanding, vegetation status analysis, and the reliability of intelligent forestry monitoring systems. However, compared with visible light cameras, near-infrared imaging equipment suffers from high deployment costs and low data acquisition frequency. Furthermore, the acquired near-infrared images often exhibit perspective shifts, exposure differences, and time mismatches with visible light images, making it difficult to acquire large-scale pairs of visible light-near-infrared forest data. This data insufficiency not only limits the construction of multimodal forest datasets but also restricts the training of vegetation analysis, target detection, and environmental monitoring algorithms. Near-infrared image generation technology aims to utilize existing visible light image information to generate near-infrared images that conform to imaging rules and are consistent with visible light scenes, thereby compensating for the limited near-infrared image data.

[0003] Near-infrared image generation in forest areas is a subtask of cross-modal image generation, aiming to address the challenges of scarce near-infrared data in forest areas and the unrealistic results of traditional generation techniques. Existing solutions utilize convolutional neural networks to learn the mapping relationship between visible light and near-infrared images to achieve cross-modal image generation. While these methods have achieved some success in general scenarios, they still face significant challenges in forest scenes. Generative adversarial networks, due to their tendency to sacrifice diversity in generated results for local visual realism, exhibit limitations compared to emerging diffusion models in specific complex tasks. The development of diffusion models has provided new solutions for cross-modal image generation and has led to unprecedented improvements in condition-driven high-quality image generation techniques.

[0004] However, while existing diffusion-based cross-modal image generation methods demonstrate strong generation capabilities in general scenarios, they face significant challenges in the specific domain of near-infrared image generation in forest farms. Near-infrared imaging is not simply color mapping or style transfer; near-infrared images are closely related to vegetation leaf structure, water content, tree canopy status, and reflectivity of shadow areas. These key features are difficult to accurately depict using only a single visible light image or simple textual information. For example, in the same forest farm scene, near-infrared images may differ significantly under different seasons, humidity levels, and lighting conditions. Relying solely on visible light guidance is insufficient to provide the model with the necessary information for reconstruction, such as vegetation reflectivity patterns and brightness levels. The resulting near-infrared images often suffer from vegetation reflectivity distortion, missing shadow areas, blurred local boundaries, and inconsistencies with the real scene, failing to meet the high-precision, high-fidelity data requirements of forest farm monitoring and vegetation analysis systems. Summary of the Invention

[0005] This application provides a method and system for generating near-infrared images of forest farms based on image case pair guidance. By introducing case image pairs containing registered visible-near-infrared images as prior knowledge into the near-infrared image generation process, and combining it with precise spatial structure control of the visible light image to be converted, the method effectively solves the problems of vegetation reflection distortion, detail blurring, and inconsistency with the real scene in the generation of near-infrared images in complex forest farm scenes, and significantly improves the physical realism, structural consistency, and scene adaptability of the generated near-infrared images.

[0006] Firstly, this application provides a method for generating near-infrared images of forest farms based on image case pair guidance, including: Acquire multiple case image pairs under the same forest farm scene; the case image pairs include registered visible light case images and near-infrared case images; The first case image pair is jointly encoded to extract case condition features that characterize the mapping relationship from the visible light domain to the near-infrared domain; The visible light case image in the second case image pair is used as the visible light image to be converted and encoded to extract the first semantic structure feature used to characterize the scene geometric layout information in the visible light image to be converted; The case condition features and the first semantic structure features are fused to obtain control features; Based on the control features, a pre-trained implicit diffusion model is used to progressively denoise the initial tensor in the implicit space to generate the implicit space tensor of the target near-infrared image. The denoising process includes injecting the control features into multiple different levels of the main branch of the implicit diffusion model through the bypass branch of the implicit diffusion model to guide the denoising direction. The implicit spatial tensor of the target near-infrared image is decoded to generate a forest near-infrared image that is consistent with the spatial structure of the visible light image to be converted and matches the mapping relationship expressed by the first image case pair.

[0007] Optionally, the joint encoding of the first case image pair to extract case condition features characterizing the mapping relationship from the visible light domain to the near-infrared domain includes: The first coding network is used to extract features from the visible light case image and the near-infrared case image in the first case image, respectively, to obtain the semantic structure features of the visible light case image and the semantic structure features of the near-infrared case image; The semantic structural features of the visible light case image and the semantic structural features of the near-infrared case image are concatenated along the channel dimension to obtain the case condition features.

[0008] Optionally, the step of encoding the visible light case image in the second case image pair as the visible light image to be converted, and extracting a first semantic structural feature to characterize the scene geometric layout information in the visible light image to be converted, includes: The visible light image to be converted is encoded using a second encoding network with the same structure as the first encoding network but with independent parameters, in order to extract a first semantic structure feature that characterizes the scene geometric layout information in the visible light image to be converted.

[0009] Optionally, fusing the case condition features and the first semantic structure features to obtain control features includes: The control feature is obtained by adding the case condition feature and the first semantic structure feature.

[0010] Optionally, the first aspect also includes: The visible light image to be converted is mapped to an implicit space to obtain the first implicit tensor; Initialize a noise graph and map the noise graph to an implicit space to obtain a second implicit tensor; The first implicit tensor and the second implicit tensor are concatenated along the channel dimension to obtain the initial tensor.

[0011] Optionally, the implicit diffusion model is a conditional diffusion model based on the UNet architecture; the bypass branch of the implicit diffusion model is a ControlNet network; the step of injecting the control features into multiple different levels of the main branch of the implicit diffusion model through the bypass branch to guide the denoising direction includes: The feature tensor output by the main branch in the i-th feature block of the implicit diffusion model is calculated using the following formula: ; in, It is the feature tensor output by the main branch in the i-th feature block; This represents the calculation process for the main branch corresponding to the i-th feature block; w is the feature tensor output by the main branch in the (i-1)th feature block; i Let be the weight of the i-th feature block; This represents the zero-initialized convolutional layer on the output side of the i-th feature block; The feature tensor output by the bypass branch in the i-th feature block; This represents the control calculation process corresponding to the i-th feature block of the bypass branch; This represents the zero-initialized convolutional layer on the input side of the i-th feature block; c represents the control feature. express The network weights of the branch model.

[0012] Optionally, the first aspect also includes: The implicit diffusion model is trained using multiple case image pairs as a training set; the loss function of the implicit diffusion model is the mean square error loss between the predicted noise and the actual noise.

[0013] Secondly, this application provides a near-infrared image generation system for forest farms based on image case pair guidance, comprising: The acquisition module is used to acquire multiple case image pairs under the same forest farm scene; the case image pairs include registered visible light case images and near-infrared case images; The first encoding module is used to jointly encode the first case image pair and extract case condition features to characterize the mapping relationship from the visible light domain to the near-infrared domain. The second encoding module is used to encode the visible light case image in the second case image pair as the visible light image to be converted, so as to extract the first semantic structure feature for characterizing the scene geometric layout information in the visible light image to be converted; The feature fusion module is used to fuse the case condition features and the first semantic structure features to obtain control features; The tensor generation module is used to generate the implicit space tensor of the target near-infrared image by progressively denoising the initial tensor in the implicit space based on the control features and using a pre-trained implicit diffusion model. The denoising process includes injecting the control features into multiple different levels of the main branch of the implicit diffusion model through the bypass branch of the implicit diffusion model to guide the denoising direction. The decoding module is used to decode the implicit spatial tensor of the target near-infrared image to generate a forest near-infrared image that is consistent with the spatial structure of the visible light image to be converted and matches the mapping relationship expressed by the first image case pair.

[0014] Thirdly, this application provides a computer device including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the steps of the image case pair-guided forest near-infrared image generation method described in the first aspect.

[0015] Fourthly, this application provides a computer-readable storage medium for storing a computer program; when executed by a processor, the computer program implements the steps of the image case pair-guided forest farm near-infrared image generation method described in the first aspect.

[0016] Beneficial effects: By jointly encoding case images, the mapping relationship from the visible light domain to the near-infrared domain is extracted as case condition features. This allows the model to no longer rely on vegetation reflection patterns that are difficult to infer accurately from a single visible light image, but instead directly draws on real near-infrared priors. This significantly improves the physical realism of the near-infrared generated images in terms of vegetation reflection intensity, brightness levels, and shadow representation, and avoids reflection distortion.

[0017] By fusing the case condition features with the first semantic structure features (scene geometric layout information) of the visible light image to be converted, the control features simultaneously include spatial structure constraints and cross-modal mapping priors. This ensures that the near-infrared generated image maintains a high degree of consistency with the input visible light image in terms of spatial layout such as roads and tree canopy boundaries, while the grayscale distribution conforms to the near-infrared imaging rules, thus achieving a balance between structural fidelity and modal conversion.

[0018] The fused control features are injected into multiple different levels of the main branch through the bypass branch of the diffusion model, realizing multi-scale and multi-level fine-grained condition guidance. This enables the denoising process to accurately follow the control information in both macro-layout and micro-texture, enhancing generation stability and detail expression, and improving boundary clarity and scene adaptability in complex forest scenes.

[0019] Initial tensor splicing and iterative denoising are performed in the implicit space. The pre-trained implicit diffusion model is used to map the visible light image to be converted to a low-dimensional implicit space, which greatly reduces the computational complexity and memory consumption. At the same time, the information of the visible light image to be converted is used as the prior input to provide a clear starting point for denoising, reduce the learning difficulty of the implicit diffusion model, accelerate the convergence speed, and improve the generalization ability and generation stability under limited training data. Attached Figure Description

[0020] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a method for generating near-infrared images of forest farms based on image case pair guidance, provided in an embodiment of this application; Figure 2 Example diagrams of collected case image pairs provided for embodiments of this application; Figure 3 A schematic diagram of the structure of the near-infrared image generation method for forest farms based on image case pair guidance provided in the embodiments of this application; Figure 4 Comparison of near-infrared image generation results using different case image pairs provided in the embodiments of this application; Figure 5 A comparison diagram of near-infrared image generation results of this application and other methods provided for embodiments of this application; Figure 6 This is a schematic diagram of the structure of a forest farm near-infrared image generation system based on image case pair guidance, provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] Example 1 like Figure 1 and Figure 3 As shown, this embodiment provides a method for generating near-infrared images of forest farms based on image case pair guidance, including: Step 101: Obtain multiple case image pairs under the same forest farm scene; the case image pairs include registered visible light case images and near-infrared case images.

[0024] In this step, visible light and near-infrared images are collected under different seasons, weather conditions, and lighting conditions. Feature point matching is performed on the acquired images to reduce the viewing angle offset between the visible light and near-infrared images. Figure 2As shown. For example, 6500 sets of corresponding visible light and near-infrared images of forest scenes were used. The images underwent feature point matching, spatial registration, center cropping, random cropping, rotation, flipping, and normalization to unify the image size to 512×512. For each set of training samples, one visible light-near-infrared image pair (i.e., the first case image pair) was selected as the sample to be converted, where the visible light image to be converted is denoted as x. Qv The near-infrared image of the target is denoted as x. Qn Then retrieve another pair with x from the case library. Qv Visible-near-infrared image pairs with similar scenes serve as reference case pairs (i.e., the first case image pair) for the visible light image to be converted, thereby enabling the reference case pairs to provide near-infrared response priors that better match the current visible light scene to be converted. The reference case pairs provide cross-modal mapping information from the visible light domain to the near-infrared domain, while the visible light image to be converted provides spatial structure information during the generation of the target near-infrared image.

[0025] The visible light domain refers to imaging within the electromagnetic spectrum (wavelengths approximately 380nm to 750nm) visible to the human eye. Pixel values ​​in the image reflect the color, texture, and visible light reflectance of an object's surface. The near-infrared domain refers to imaging in the near-infrared band (wavelengths approximately 750nm to 1400nm). Pixel values ​​in the image reflect physical information invisible to the naked eye, such as vegetation cell structure, water content, and chlorophyll density.

[0026] The visible light case image in the reference case is denoted as x. Cv Near-infrared case images are denoted as x. Cn This constructs a set of information pairs: visible light image to be converted - target near-infrared image - visible light case image - near-infrared case image. The first 5000 data pairs are used as the training set, and the last 1500 data pairs are used as the test set.

[0027] Step 102: Jointly encode the first case image pair and extract case condition features to characterize the mapping relationship from the visible light domain to the near-infrared domain.

[0028] In this step, the first encoding network selects a convolutional coding network as the case encoding module E. C The visible light case image x Cv Near-infrared case image x Cn The data is input into the case encoding module to obtain the semantic structural features of the visible light case images. Semantic structural features of near-infrared case images The two are then concatenated along the channel dimension to form the case condition features. The case inputs are used as case conditions in the near-infrared image generation process of the forest farm; among them, the visible light case image is used to provide spatial information and visible light brightness information of the forest farm scene, such as the color, texture, and visible light reflectance of object surfaces; the near-infrared case image is used to provide near-infrared reflectance information under the same scene, such as vegetation cell structure, water content, and chlorophyll density; the case condition feature c C Used to describe the cross-modal mapping relationship between the two.

[0029] Step 103: The visible light case image in the second case image pair is used as the visible light image to be converted and encoded to extract the first semantic structure feature used to characterize the scene geometric layout information in the visible light image to be converted.

[0030] The second encoding network selects an image encoding module with the same structure as that in step 102 but with independent parameters for the visible light image x to be converted. Qv Encode to extract the first semantic structure feature c Q =E Q (x Qv ).

[0031] Step 104: The case condition features and the first semantic structure features are fused to obtain control features.

[0032] For example, the case condition features and the first semantic structure features are added together to form the control features; the control features serve as the control information for the bypass branch of the implicit diffusion model, so that the implicit diffusion model considers both the spatial structure constraints of the current image and the mapping relationship between the visible light case image and the near-infrared case image in the first case image pair during the generation of near-infrared images of the forest farm.

[0033] Step 105: Based on the control features, the initial tensor in the implicit space is progressively denoised using a pre-trained implicit diffusion model to generate the implicit space tensor of the target near-infrared image; wherein, the denoising process includes injecting the control features into multiple different levels of the main branch of the implicit diffusion model through the bypass branch of the implicit diffusion model to guide the denoising direction.

[0034] Before this step, a latent space encoder (such as a pre-trained VAE encoder) is needed to map the visible light image to be converted into a latent space, obtaining the latent space tensor z of the visible light image to be converted. Qv , which serves as the first implicit tensor.

[0035] Initialize a noise map The initialized noise map is then mapped to the implicit space using the same VAE encoder, yielding the implicit space tensor z of the Gaussian noise map. N, to serve as the second implicit tensor; where N(0,1) represents the standard Gaussian distribution.

[0036] The first implicit tensor and the second implicit tensor are concatenated along the channel dimension (i.e., dim=1) to obtain the initial tensor.

[0037] In this step, the diffusion model employs Stable Diffusion 2.1 with a ControlNet bypass branch. The main branch of this model is a deep neural network based on a UNet architecture. The diffusion model comprises an encoder, intermediate layers, and a decoder, which consist of residual blocks, spatial attention modules, upsampling modules, downsampling modules, and skip connections. The diffusion model is initialized using the StableDiffusion 2.1 network parameters, and the encoder input format is modified to accommodate the concatenated implicit spatial tensor. The ControlNet bypass branch injects control information into the corresponding feature blocks of the main branch through zero-initialized convolutional layers, ensuring that the diffusion model maintains the structural consistency of the visible light image to be converted while adhering to the near-infrared imaging patterns provided by the first case image pair during denoising.

[0038] The formula for importing the bypass branch control signal is as follows: .

[0039] The update formula for the corresponding feature blocks in the main branch is: .

[0040] in, The feature tensor output by the (Stable Diffusion) backbone branch in the i-th feature block; This represents the computation process corresponding to the i-th feature block of the main branch; the computation process includes residual calculation, attention calculation, upsampling calculation, downsampling calculation, etc., of the feature tensor at the corresponding level of the main branch; w is the feature tensor output by the main branch in the (i-1)th feature block; i Let be the weight of the i-th feature block; This represents the zero-initialized convolutional layer on the output side of the i-th feature block; The feature tensor output by the (ControlNet) bypass branch in the i-th feature block; This represents the control calculation process corresponding to the i-th feature block of the bypass branch; the control calculation process includes residual calculation of the copy block corresponding to the trunk branch, attention calculation, upsampling calculation, downsampling calculation, etc. This represents the zero-initialized convolutional layer on the input side of the i-th feature block; c represents the control feature. express The network weights of the branch model.

[0041] In this step, the training process of the pre-trained implicit diffusion model includes training the implicit diffusion model using multiple case image pairs as the training set; the loss function of the implicit diffusion model is the mean square error loss between the predicted noise and the real noise.

[0042] Specifically, the hidden space encoder is first used to generate the real near-infrared target image x Qn Mapping to an implicit space tensor z0, and then adding Gaussian noise to z0 at random time step t to obtain z t The implicit diffusion model uses z t The time step t and control feature c are used as input prediction noise, and the following loss function L is used for training: .

[0043] Where ε represents the actual noise; Represents the L2 norm; Let be the expectation operator, representing the average of the loss term corresponding to the random time step t, the target implicit space tensor z0, and the noise ε following the standard Gaussian distribution N(0,1).

[0044] Using a trained implicit diffusion model to extract from the noisy image z t The noise-free near-infrared image z0 is estimated. The implicit diffusion model, loaded with trained parameters, is iteratively denoised in the latent space. The noise added in the previous step is calculated based on the noise in the current step. After t time steps of iteration, a clean implicit space tensor is finally obtained. The sampling formula is as follows: .

[0045] in, The cumulative product of the variance scheduling parameters at time step t-1; The cumulative product of the variance scheduling parameters at time step t; This represents the noise predicted by the trained implicit diffusion model. To control the variance of sampling randomness; Let be the Gaussian noise sampled from the standard normal distribution at the t-th time step.

[0046] Step 106: Decode the implicit spatial tensor of the target near-infrared image to generate a forest near-infrared image that is consistent with the spatial structure of the visible light image to be converted and matches the mapping relationship expressed by the image case pair.

[0047] The noise-free implicit space tensor obtained in step 105 is mapped back to pixel space through a latent space decoder (VAE decoder) to obtain the final generated near-infrared image of the forest farm; such as Figure 4 As shown, Figure 4Images (a), (b), and (c) in the figure show a comparison of the near-infrared image generation results using different case image pairs. The final generated near-infrared image of the forest farm is consistent with the visible light image to be converted in terms of spatial layout such as roads, water bodies, and tree canopy boundaries, while being similar to the near-infrared case images in terms of vegetation reflectance intensity, brightness distribution, and shadow effects.

[0048] This embodiment further provides a comparison diagram of the infrared image generation results of this application and other methods, such as... Figure 5 As shown, Figure 5 (a) is a schematic diagram of a real near-infrared image; Figure 5 (b) is a schematic diagram of the result of the method in this application; Figure 5 (c) is a schematic diagram of the results from UniControl; Figure 5 (d) is a schematic diagram of the results from AnyControl.

[0049] In summary, this application provides a method for generating near-infrared images of forest farms based on image case pair guidance. By acquiring multiple case image pairs from the same forest farm scene and designing a specialized joint encoding mechanism (i.e., using a first encoding network to extract features from the visible light and near-infrared images in the case pair and then stitching them together in the channel dimension), case condition features are extracted to characterize the mapping relationship from the visible light domain to the near-infrared domain. This allows the implicit diffusion model to no longer rely on near-infrared reflectance patterns closely related to vegetation leaf structure, water content, and canopy state, which are difficult to accurately infer from a single visible light image. Instead, it directly borrows known and real near-infrared response priors from reference cases. This provides physical constraints for subsequent near-infrared image generation, enabling the generated near-infrared images to match the real near-infrared imaging patterns in terms of vegetation reflectance intensity, brightness levels, and shadow effects. This effectively avoids the reflection distortion problem commonly encountered when relying solely on visible light guidance, fundamentally improving the physical realism of the generated near-infrared images.

[0050] A second encoding network, with the same structure as the first encoding network but independent parameters, is used to encode the visible light image to be converted. Its scene geometric layout information is extracted as the first semantic structural feature, which is then fused (e.g., added) with the aforementioned case condition features to obtain the final control feature. By decoupling, extracting, and recombining the two core information sources of "spatial structure" and "modal mapping," the final control feature carries both the immutable geometric constraints of the visible light image to be converted, such as roads and tree canopy boundaries, and the variable near-infrared reflectance patterns provided by the case image pairs. Using this control feature to guide the diffusion model ensures that the spatial layout of the generated near-infrared image is highly consistent with the input visible light image, while its pixel grayscale distribution strictly follows the mapping relationship expressed by the case image pairs, thus achieving a good balance between structural fidelity and modal conversion accuracy.

[0051] Image generation is achieved using a pre-trained implicit diffusion model (such as Stable Diffusion with a ControlNet bypass branch). The fused control features are creatively injected into multiple different levels of the main branch via the bypass branch. Unlike simple conditional concatenation or single-level injection, this multi-level injection mechanism utilizes zero-initialized convolutional layers and layer weights to deeply integrate control information into different feature blocks of the encoder, intermediate layers, and decoder within the UNet diffusion model structure. This allows the model to be guided by control features at different scales and abstraction levels during the progressive denoising process, ensuring precise adherence to control signals in both macroscopic layout and microscopic texture details. This significantly enhances the stability and controllability of the generation process, resulting in near-infrared images that outperform existing solutions in terms of local boundary sharpness and adaptability to complex scenes.

[0052] The first implicit tensor obtained by mapping the visible light image to be converted and the second implicit tensor obtained by mapping random noise are concatenated along the channel dimension to form an initial tensor. Denoising and sampling of the diffusion model are then performed in the low-dimensional implicit space. The pre-trained implicit diffusion model compresses the high-dimensional pixel space into an efficient latent space, significantly reducing the computational complexity and memory consumption of the subsequent diffusion model. Simultaneously, by directly concatenating the visible light image information to be converted as prior information into the input, a clear starting point is provided for the denoising process. This allows the diffusion model to focus its learning ability on learning near-infrared information supplemented by control features, rather than reconstructing the basic structure from pure noise. This reduces the learning difficulty of the diffusion model, accelerates convergence, and improves generalization ability and the stability of the generated results with limited training data.

[0053] Example 2 Based on the same inventive concept as Embodiment 1, this embodiment provides a forest farm near-infrared image generation system guided by image case pairs. Since the principle of this system in solving the problem is similar to the forest farm near-infrared image generation method guided by image case pairs described in Embodiment 1, the implementation of this system can refer to the implementation of the forest farm near-infrared image generation method guided by image case pairs.

[0054] like Figure 6 As shown, this embodiment provides a near-infrared image generation system for forest farms based on image case pair guidance, including: The acquisition module 10 is used to acquire multiple case image pairs under the same forest farm scene; the case image pairs include registered visible light case images and near-infrared case images.

[0055] The first encoding module 20 is used to jointly encode the first case image pair and extract case condition features to characterize the mapping relationship from the visible light domain to the near-infrared domain.

[0056] The second encoding module 30 is used to encode the visible light case image in the second case image pair as the visible light image to be converted, so as to extract the first semantic structure feature for characterizing the scene geometric layout information in the visible light image to be converted.

[0057] The feature fusion module 40 is used to fuse the case condition features and the first semantic structure features to obtain control features.

[0058] Tensor generation module 50 is used to generate an implicit space tensor of the target near-infrared image by progressively denoising the initial tensor in the implicit space based on the control features and using a pre-trained implicit diffusion model; wherein, the denoising process includes injecting the control features into multiple different levels of the main branch of the implicit diffusion model through the bypass branch of the implicit diffusion model to guide the denoising direction.

[0059] The decoding module 60 is used to decode the implicit spatial tensor of the target near-infrared image to generate a forest near-infrared image that is consistent with the spatial structure of the visible light image to be converted and matches the mapping relationship expressed by the first image case pair.

[0060] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0061] Example 3 This embodiment provides a computer device, including a processor and a memory; wherein, when the processor executes the computer program stored in the memory, it implements the steps of the forest farm near-infrared image generation method based on image case pair guidance described in Embodiment 1.

[0062] For a more detailed explanation of the above method, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0063] Example 4 This embodiment provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, it implements the steps of the image case pair-guided forest farm near-infrared image generation method described in Embodiment 1.

[0064] For a more detailed explanation of the above method, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0065] Example 5 This embodiment provides a computer program product, including computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, they implement the steps of the forest farm near-infrared image generation method based on image case pair guidance described in Embodiment 1.

[0066] For a more detailed explanation of the above method, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0067] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems, devices, storage media, and computer program products disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0068] Those skilled in the art will clearly understand that the techniques in the embodiments of this application can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application or some parts of the embodiments.

[0069] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0070] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0071] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0072] The present application has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present application. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and implementation methods of the present application without departing from the spirit and scope of the present application, and all such modifications and improvements fall within the scope of the present application. The scope of protection of the present application is determined by the appended claims.

Claims

1. A method for generating near-infrared images of forest farms based on image case pair guidance, characterized in that, include: Acquire multiple case image pairs under the same forest farm scene; the case image pairs include registered visible light case images and near-infrared case images; The first case image pair is jointly encoded to extract case condition features that characterize the mapping relationship from the visible light domain to the near-infrared domain; The visible light case image in the second case image pair is used as the visible light image to be converted and encoded to extract the first semantic structure feature used to characterize the scene geometric layout information in the visible light image to be converted; The case condition features and the first semantic structure features are fused to obtain control features; Based on the control features, a pre-trained implicit diffusion model is used to progressively denoise the initial tensor in the implicit space to generate the implicit space tensor of the target near-infrared image. The denoising process includes injecting the control features into multiple different levels of the main branch of the implicit diffusion model through the bypass branch of the implicit diffusion model to guide the denoising direction. The implicit spatial tensor of the target near-infrared image is decoded to generate a forest near-infrared image that is consistent with the spatial structure of the visible light image to be converted and matches the mapping relationship expressed by the first image case pair.

2. The method for generating near-infrared images of forest farms according to claim 1, characterized in that, The joint encoding of the first case image pair to extract case condition features characterizing the mapping relationship from the visible light domain to the near-infrared domain includes: The first coding network is used to extract features from the visible light case image and the near-infrared case image in the first case image, respectively, to obtain the semantic structure features of the visible light case image and the semantic structure features of the near-infrared case image; The semantic structural features of the visible light case image and the semantic structural features of the near-infrared case image are concatenated along the channel dimension to obtain the case condition features.

3. The method for generating near-infrared images of forest farms according to claim 2, characterized in that, The step of encoding the visible light case image in the second case image pair as the visible light image to be converted, and extracting a first semantic structural feature to characterize the scene geometric layout information in the visible light image to be converted, includes: The visible light image to be converted is encoded using a second encoding network with the same structure as the first encoding network but with independent parameters, in order to extract a first semantic structure feature that characterizes the scene geometric layout information in the visible light image to be converted.

4. The method for generating near-infrared images of forest farms according to claim 1, characterized in that, The process of fusing the case condition features and the first semantic structure features to obtain control features includes: The control feature is obtained by adding the case condition feature and the first semantic structure feature.

5. The method for generating near-infrared images of forest farms according to claim 1, characterized in that, Also includes: The visible light image to be converted is mapped to an implicit space to obtain the first implicit tensor; Initialize a noise graph and map the noise graph to an implicit space to obtain a second implicit tensor; The first implicit tensor and the second implicit tensor are concatenated along the channel dimension to obtain the initial tensor.

6. The method for generating near-infrared images of forest farms according to claim 1, characterized in that, The implicit diffusion model is a conditional diffusion model based on the UNet architecture; the bypass branch of the implicit diffusion model is a ControlNet network; the step of injecting the control features into multiple different levels of the main branch of the implicit diffusion model through the bypass branch to guide the denoising direction includes: The feature tensor output by the main branch in the i-th feature block of the implicit diffusion model is calculated using the following formula: ； in, It is the feature tensor output by the main branch in the i-th feature block; This represents the calculation process for the main branch corresponding to the i-th feature block; w is the feature tensor output by the main branch in the (i-1)th feature block; i Let be the weight of the i-th feature block; This represents the zero-initialized convolutional layer on the output side of the i-th feature block; The feature tensor output by the bypass branch in the i-th feature block; This represents the control calculation process corresponding to the i-th feature block of the bypass branch; This represents the zero-initialized convolutional layer on the input side of the i-th feature block; c represents the control feature. express The network weights of the branch model.

7. The method for generating near-infrared images of forest farms according to claim 1, characterized in that, Also includes: The implicit diffusion model is trained using multiple case image pairs as a training set; The loss function of the implicit diffusion model is the mean square error loss between the predicted noise and the actual noise.

8. A near-infrared image generation system for forest farms based on image case pair guidance, characterized in that, include: The acquisition module is used to acquire multiple case image pairs under the same forest farm scene; the case image pairs include registered visible light case images and near-infrared case images; The first encoding module is used to jointly encode the first case image pair and extract case condition features to characterize the mapping relationship from the visible light domain to the near-infrared domain. The second encoding module is used to encode the visible light case image in the second case image pair as the visible light image to be converted, so as to extract the first semantic structure feature for characterizing the scene geometric layout information in the visible light image to be converted; The feature fusion module is used to fuse the case condition features and the first semantic structure features to obtain control features; The tensor generation module is used to generate the implicit space tensor of the target near-infrared image by progressively denoising the initial tensor in the implicit space based on the control features and using a pre-trained implicit diffusion model. The denoising process includes injecting the control features into multiple different levels of the main branch of the implicit diffusion model through the bypass branch of the implicit diffusion model to guide the denoising direction. The decoding module is used to decode the implicit spatial tensor of the target near-infrared image to generate a forest near-infrared image that is consistent with the spatial structure of the visible light image to be converted and matches the mapping relationship expressed by the first image case pair.

9. A computer device, characterized in that, It includes a processor and a memory; wherein, when the processor executes the computer program stored in the memory, it implements the steps of the image case pair-guided forest near-infrared image generation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; when executed by a processor, the computer programs implement the steps of the image case pair-guided forest farm near-infrared image generation method as described in any one of claims 1-7.

Images