Infrared image enhancement method and device, equipment and storage medium

Abstract

The invention relates to the technical field of image data processing, and discloses an infrared image enhancement method, device and equipment and a storage medium, which are used for the generalization ability of infrared image enhancement and the authenticity of a generated image. The infrared image enhancement method comprises the following steps: acquiring a data pair set formed by point cloud data generated by simulation and corresponding infrared images and a to-be-enhanced target image, selecting a target object data pair from the set, extracting an image area and point cloud projection information of the target object data pair in a simulation image, and projecting point cloud to a target image coordinate system according to the information, and accurately positioning a pasting position and a bounding box, carrying out background removal, color adaptation, texture blurring and edge feathering processing on an original image region, generating a fused and optimized object image, and pasting the fused and optimized object image to a specified position of a target image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and in particular to an infrared image enhancement method, device, equipment and storage medium. BACKGROUND

[0002] The infrared image enhancement technology refers to a kind of technical means for improving the visual effect and information extraction capability of infrared image through digital image processing method.

[0003] The prior art provides an infrared image enhancement method based on a generative adversarial network, which mainly uses a deep network model containing a generator and a discriminator as the technical means. The generator usually adopts an encoding-decoding structure, and is responsible for converting the input low-quality infrared image into an enhanced image. The discriminator discriminates the authenticity of the image through a convolutional network structure, and carries out minimax game with the generator through joint training of combined adversarial loss, content loss and perceptual loss, so as to drive the generator to output infrared images with high signal-to-noise ratio and enhanced details. However, the performance of this method is highly dependent on large-scale, high-quality and accurately paired training data sets. The acquisition of real infrared data is costly and cumbersome, and is prone to registration errors. In addition, the model training process is unstable, and is prone to pattern collapse and over-smoothing effect, resulting in distorted details or unnatural textures of the generated images, and poor generalization ability. SUMMARY

[0004] The present application provides an infrared image enhancement method, device, equipment and storage medium to solve the problem of serious dependence on real data sets and poor generalization ability and image distortion in the prior art.

[0005] The first aspect of the present application provides an infrared image enhancement method, comprising: obtaining simulated point cloud data of a simulated object and corresponding simulated infrared image data to form a data pair set; obtaining a target infrared image to be enhanced; selecting data pairs of a plurality of target objects from the data pair set; extracting an original image region of each target object in the simulated infrared image and corresponding simulated point cloud projection information; projecting the simulated point cloud data of each target object into the image coordinate system of the target infrared image according to the simulated point cloud projection information to determine the target pasting position and the bounding box of the object in the image; based on the target pasting position and the bounding box, performing background removal, color adaptation, texture blurring and edge feathering processing on the original image region to obtain a fused and optimized object image; and pasting the fused and optimized object image to the corresponding position of the target infrared image. In an embodiment, the simulation point cloud data and the corresponding simulation infrared image data of the generated object are obtained, and a data pair set is formed, including: performing point cloud sampling on a three-dimensional mesh model of a preset object to generate simulation point cloud data of the corresponding object; placing the three-dimensional mesh model at a fixed position in a virtual scene under multiple working conditions of simulated vehicle movement, and generating simulation infrared images of corresponding view angles of the simulation point cloud data by simulating infrared camera imaging; and storing the simulation point cloud data and the simulation infrared images generated at the same time and the same view angle in association to form the data pair set. In an embodiment, the simulation infrared images of corresponding view angles of the simulation point cloud data are generated by simulating infrared camera imaging under multiple working conditions of simulated vehicle movement, including: constructing a virtual driving scene containing roads, terrains and environmental elements, and setting the three-dimensional mesh model as a static obstacle in the virtual driving scene; simulating the process of a vehicle passing through the position of the static obstacle under multiple working conditions according to a predefined vehicle movement trajectory; and collecting rendered pictures of infrared cameras at fixed time intervals during the passing process of the vehicle, and each frame of picture is a simulation infrared image of a corresponding view angle of the simulation point cloud data at the current time.

[0006] In an embodiment, the original image region of each target object in the simulation infrared image and the corresponding simulation point cloud projection information are extracted, including: obtaining the simulation infrared image in the selected target object data pair and the corresponding camera parameters when the image is generated; projecting the simulation point cloud data in the target object data pair into a pixel coordinate system of the simulation infrared image according to the camera parameters to obtain a two-dimensional pixel coordinate set of the target object point cloud; calculating a minimum circumscribed rectangle containing all the two-dimensional pixel coordinates, and determining an image region surrounded by the minimum circumscribed rectangle as the original image region; and taking the camera parameters, the two-dimensional pixel coordinate set and the information of the minimum circumscribed rectangle as the simulation point cloud projection information of the corresponding target object.

[0007] In an embodiment, the projecting the simulation point cloud data of each target object into the image coordinate system of the target infrared image based on the simulation point cloud projection information to determine the target pasting position and the bounding box of the target object in the image comprises: extracting first image features from a raw image region of a simulation infrared image corresponding to the simulation point cloud projection information; extracting second image features in a predetermined search region in the target infrared image; matching the first image features and the second image features, and calculating an estimated pose of the target object in a world coordinate system of the target infrared image based on a matching result; transforming the simulation point cloud data from a raw world coordinate system thereof to the world coordinate system of the target infrared image according to the estimated pose; projecting the transformed simulation point cloud data into a pixel coordinate system of the target infrared image according to camera parameters of the target infrared image to obtain a target two-dimensional pixel coordinate; and calculating the target pasting position and the bounding box of the target object in the target infrared image based on the target two-dimensional pixel coordinate.

[0008] In an embodiment, the background removal, color adaptation, texture blurring, and edge feathering processing of the raw image region based on the target pasting position and the bounding box to obtain a fused and optimized object image comprises: identifying and segmenting an object foreground in the raw image region, setting an Alpha value of a pixel in a background region to be completely transparent, and obtaining a foreground object image with a transparent background; extracting an image block at the target pasting position from the target infrared image based on a region defined by the bounding box, and calculating an average color of the image block; adjusting overall hue and brightness of the foreground object image with the transparent background based on a difference between the average color and an average color of the foreground object image with the transparent background; filtering the RGB color channels of the foreground object image after color adaptation through a Gaussian filter; and performing feathering processing on an Alpha channel of an edge region of the filtered foreground object image to generate a gradient mask with a gradually decreasing transparency from inside to outside, thereby obtaining the fused and optimized object image.

[0009] In an embodiment, the filtering the RGB color channels of the foreground object image after color adaptation through a Gaussian filter comprises: performing Gaussian filtering on the RGB color channels of the foreground object image after color adaptation to generate a basic blurred image; identifying a contour region of the foreground object image after color adaptation through an edge detection algorithm, and generating an edge region reinforcement mask; and performing weighted fusion of the basic blurred image and the foreground object image after color adaptation according to the edge region reinforcement mask, so that the blurring degree of the contour region is higher than that of an internal region.

[0010] The second aspect of the present application provides an infrared image enhancement device, comprising: a first acquisition module for acquiring simulation point cloud data of a simulation generated object and corresponding simulation infrared image data, forming a data pair set; a second acquisition module for acquiring a target infrared image to be enhanced; an extraction module for selecting data pairs of multiple target objects from the data pair set; extracting the original image area of each target object in the simulation infrared image and the corresponding simulation point cloud projection information; a projection module for projecting the simulation point cloud data of each target object into the image coordinate system of the target infrared image according to the simulation point cloud projection information, determining the target pasting position and the bounding box thereof in the image; a processing module for performing background removal, color adaptation, texture blurring and edge feathering processing on the original image area based on the target pasting position and the bounding box, to obtain a fusion optimized object image; a pasting module for pasting the fusion optimized object image to the corresponding position of the target infrared image. In a feasible implementation, the first acquisition module comprises: a first generation unit for performing point cloud sampling on a three-dimensional mesh model of a preset object to generate simulation point cloud data of the corresponding object; a second generation unit for placing the three-dimensional mesh model at a fixed position in a virtual scene under multiple working conditions of simulating vehicle movement, and generating simulation infrared images of corresponding view angles of the simulation point cloud data by simulating infrared camera imaging; and an association unit for associating and storing the simulation point cloud data and the simulation infrared images generated at the same time and the same view angle to form a data pair set.

[0011] In a feasible implementation, the second generation unit is specifically configured to: construct a virtual driving scene containing roads, terrains and environmental elements, and set the three-dimensional mesh model as a static obstacle in the virtual driving scene; simulate the process of a vehicle passing through the position of the static obstacle under multiple working conditions according to a predefined vehicle movement trajectory; and collect rendered pictures of infrared cameras at fixed time intervals during the passing process of the vehicle, and each frame of picture is a simulation infrared image of a corresponding view angle of the simulation point cloud data at the current time.

[0012] In a feasible implementation, the extraction module is specifically configured to: acquire the simulation infrared image in the selected target object data pair and the corresponding camera parameters when the image is generated; project the simulation point cloud data in the target object data pair into the pixel coordinate system of the simulation infrared image according to the camera parameters, to obtain a two-dimensional pixel coordinate set of the corresponding target object point cloud; calculate a minimum circumscribed rectangle frame containing all the two-dimensional pixel coordinates, and determine the image area surrounded by the minimum circumscribed rectangle frame as the original image area; and take the information of the camera parameters, the two-dimensional pixel coordinate set and the minimum circumscribed rectangle frame as the simulation point cloud projection information of the corresponding target object.

[0013] In one feasible implementation, the projection module is specifically used for: extracting a first image feature from the original image region of the simulated infrared image corresponding to the simulated point cloud projection information; extracting a second image feature from a predetermined search region in the target infrared image; matching the first image feature and the second image feature, and calculating the estimated pose of the target object in the world coordinate system of the target infrared image based on the matching result; transforming the simulated point cloud data from its original world coordinate system to the world coordinate system of the target infrared image according to the estimated pose; projecting the transformed simulated point cloud data into its pixel coordinate system according to the camera parameters of the target infrared image to obtain the target two-dimensional pixel coordinates; and calculating the target pasting position and bounding box in the target infrared image according to the target two-dimensional pixel coordinates.

[0014] In one feasible implementation, the processing module includes: a processing unit, configured to identify and segment the object foreground in the original image region, set the pixel alpha value of the background region to be completely transparent, and obtain a foreground object image with a transparent background; a color adjustment unit, configured to extract an image patch at the target pasting position from the target infrared image based on the region defined by the bounding box, calculate the average color of the image patch, and adjust the overall hue and brightness of the foreground object image with a transparent background according to the difference between the average color and the average color of the foreground object image with a transparent background; a filtering unit, configured to filter the RGB color channels of the color-adapted foreground object image using a Gaussian filter; and a feathering unit, configured to feather the alpha channel of the edge region of the filtered foreground object image to generate a gradient mask with gradually decreasing transparency from the inside out, thereby obtaining a fused and optimized object image.

[0015] In one feasible implementation, the filtering unit is specifically used to: perform a Gaussian filtering operation on the RGB color channels of the color-adapted foreground object image to generate a basic blurred image; identify the contour region of the color-adapted foreground object image through an edge detection algorithm and generate an edge region enhancement mask; and perform weighted fusion of the basic blurred image and the color-adapted foreground object image according to the edge region enhancement mask, so that the blur degree of the contour region is higher than that of the inner region.

[0016] A third aspect of the present invention provides an electronic device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the electronic device to perform the infrared image enhancement method described above.

[0017] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the infrared image enhancement method described above.

[0018] In the technical solution provided by this invention, simulated point cloud data of a simulated object and corresponding simulated infrared image data are obtained to form a data pair set; the target infrared image to be enhanced is obtained; multiple target object data pairs are selected from the data pair set; the original image region of each target object in the simulated infrared image and the corresponding simulated point cloud projection information are extracted; according to the simulated point cloud projection information, the simulated point cloud data of each target object is projected onto the image coordinate system of the target infrared image to determine its target pasting position and bounding box in the image; based on the target pasting position and the bounding box, the original image region is subjected to background removal, color adaptation, texture blurring, and edge feathering processing to obtain a fused and optimized object image; the fused and optimized object image is pasted to the corresponding position of the target infrared image. In this embodiment of the invention, by constructing a training set using simulated object point cloud and infrared image data, the dependence on massive amounts of real infrared image data is effectively avoided, significantly reducing the cost and difficulty of data acquisition. Point cloud projection information is used to guide the target object's pasting position and bounding box in the image. Combined with background removal, color adaptation, texture blurring, and edge feathering, high realism and adaptive fusion of the generated object and target infrared image are achieved, significantly improving the naturalness and detail realism of the enhanced image. This overcomes the problems of texture distortion and poor generalization caused by data loss or registration errors in traditional GAN ​​methods. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of one embodiment of the infrared image enhancement method in this invention; Figure 2 This is a schematic diagram of another embodiment of the infrared image enhancement method in this invention; Figure 3 This is a schematic diagram of one embodiment of the infrared image enhancement device in this invention; Figure 4 This is a schematic diagram of another embodiment of the infrared image enhancement device in this invention; Figure 5 This is a schematic diagram of one embodiment of the electronic device in this invention. Detailed Implementation

[0020] This invention provides an infrared image enhancement method, apparatus, device, and storage medium. By using simulated data to replace real data and achieving accurate fusion through point cloud projection, it significantly improves the generalization ability and image realism of image enhancement.

[0021] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] It is understood that the executing entity of this invention can be an infrared image enhancement device, a terminal, or a server; no specific limitation is made here. This embodiment of the invention will be described using a server as an example.

[0023] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the infrared image enhancement method in this invention includes: 101. Obtain the simulated point cloud data of the simulated object and the corresponding simulated infrared image data to form a data pair set; A high-fidelity physical simulation engine generates simulated point cloud data of objects that are highly consistent with real-world scenes, along with their corresponding simulated infrared image data. These data strictly adhere to the physical characteristics of sensors and ambient lighting models, ensuring spatiotemporal alignment and physical consistency among multimodal data. Specifically, the simulated point cloud data is generated through a lidar sensor model and includes information such as the object's three-dimensional coordinates and reflection intensity. The simulated infrared image is rendered based on a thermal radiation model and material thermal properties, reflecting the infrared characteristics of the object under different temperature and environmental conditions. Each set of point cloud data and infrared image constitutes a data pair, and an automated annotation process generates ground truth information such as object category, location, and bounding box, ultimately forming a large-scale, multi-category, and multi-scene data pair set.

[0024] 102. Acquire the infrared image of the target to be enhanced; Select the target infrared image to be enhanced from the actual acquired infrared image sequence or existing dataset. This image is usually captured by infrared sensors in the real world and may have problems such as sparse targets, simple scenes or insufficient samples. In order to improve the effectiveness of enhancement, the target image needs to be preprocessed, including image denoising, resolution standardization, dynamic range adjustment, etc., to ensure that it is compatible with the simulation data in terms of performance style and physical characteristics.

[0025] 103. Select multiple data pairs of target objects from the data pair set; This process intelligently selects multiple target object pairs from a dataset. The selection process comprehensively considers factors such as object category, scale, pose, frequency of occurrence, and thermal feature matching degree to ensure that the selected objects are semantically and physically compatible with the target image scene. For example, if the target image is an urban road scene, data pairs of categories such as vehicles, pedestrians, and traffic signs are prioritized; if it is a wilderness environment, natural objects such as animals, vegetation, or rocks can be selected. This step is typically implemented using content-based retrieval algorithms or deep learning matching models to improve selection efficiency and the reasonableness of the enhancement effect.

[0026] 104. Extract the original image region of each target object in the simulated infrared image and the corresponding simulated point cloud projection information; For each selected target object data pair, the system accurately extracts the original image region of the object from its corresponding simulated infrared image, typically represented as a pixel-level segmentation mask or bounding box. Simultaneously, it acquires the simulated point cloud projection information associated with the object, including the point cloud's projection matrix in the image coordinate system, camera intrinsic and extrinsic parameters, and geometric attributes such as depth distribution.

[0027] 105. Based on the simulated point cloud projection information, project the simulated point cloud data of each target object onto the image coordinate system of the target infrared image to determine its target pasting position and bounding box in the image. The simulated point cloud data is reprojected onto the image coordinate system of the target infrared image. Through coordinate transformation and perspective projection calculations, the specific target pasting position and bounding box of the object in the target image are determined. This process needs to consider the relative pose differences, scale variations, and possible distortion effects between the target image and the simulated image. Geometric correction is typically performed through affine or perspective transformations to ensure that the projected object has the correct size, orientation, and positional relationship in the target image, thereby maintaining the spatial consistency and physical plausibility of the scene.

[0028] 106. Based on the target pasting position and bounding box, the original image area is processed by background removal, color adaptation, texture blurring and edge feathering to obtain the fused and optimized object image; A series of fusion and optimization processes are performed on the original object image region extracted from the simulated infrared image. First, background removal is performed by using a segmentation mask to remove pixels that do not belong to the object. Next, color and brightness adaptation is performed by adjusting the grayscale value and contrast of the object according to the overall tone and thermal distribution characteristics of the target image. Then, texture blurring is applied to simulate the thermal diffusion and noise characteristics commonly found in real infrared images. Finally, edge feathering is performed to make the object boundary transition naturally with the background and avoid sharp or unrealistic edge effects.

[0029] 107. Paste the fused and optimized object image into the corresponding position in the target infrared image. An alpha mask is generated based on the target pasting location to determine the fusion weights of the object's pixels. A Poisson equation is constructed using the background gradient of the target image as a constraint, ensuring that the Laplacian gradient of the pasted object's edge region remains continuous with the background. This gradient domain fusion technique effectively eliminates brightness differences and edge artifacts, preserving the object's thermal radiation characteristics while ensuring natural texture transitions. Finally, a multi-band mixing strategy is employed to fuse low-frequency thermal radiation features and high-frequency texture details at different scales, achieving a physically consistent and visually realistic composite effect.

[0030] In this embodiment of the invention, by constructing a training set using simulated object point cloud and infrared image data, the dependence on massive amounts of real infrared image data is effectively avoided, significantly reducing the cost and difficulty of data acquisition. Point cloud projection information is used to guide the target object's pasting position and bounding box in the image. Combined with background removal, color adaptation, texture blurring, and edge feathering, high realism and adaptive fusion of the generated object and target infrared image are achieved, significantly improving the naturalness and detail realism of the enhanced image. This overcomes the problems of texture distortion and poor generalization caused by data loss or registration errors in traditional GAN ​​methods.

[0031] Please see Figure 2 Another embodiment of the infrared image enhancement method in this invention includes: 201. Perform point cloud sampling on the 3D mesh model of the preset object to generate the corresponding simulated point cloud data of the object; A 3D mesh model is acquired and placed in a virtual scanning environment. By simulating the scanning mechanism of a lidar system, rays are emitted onto the model surface from multiple perspectives. When a ray intersects the model surface, the 3D coordinates, normal vector, and reflection intensity of the intersection point are recorded. Simultaneously, the radar cross-section value is calculated based on material properties. To simulate the characteristics of a real sensor, a distance attenuation model and noise perturbation are introduced. The point cloud density is dynamically adjusted based on the ray propagation distance, and Gaussian noise is added to simulate measurement errors. The final generated point cloud data not only retains the geometric features of the object but also possesses the statistical distribution characteristics of a real radar point cloud.

[0032] 202. Under various working conditions simulating vehicle motion, a three-dimensional mesh model is placed at a fixed position in the virtual scene, and a simulated infrared image corresponding to the viewpoint of the simulated point cloud data is generated by simulating infrared camera imaging. A virtual driving scenario is constructed, including road, terrain, and environmental elements, and a 3D mesh model is set as a static obstacle in the virtual driving scenario. According to the predefined vehicle movement trajectory, the process of the vehicle passing through the location of the static obstacle under various working conditions is simulated. The rendered images of the infrared camera during the vehicle's passage are collected at fixed time intervals, and each frame is a simulated infrared image of the viewpoint corresponding to the simulated point cloud data at the current moment. A complete road environment, including asphalt pavement, traffic signs, green vegetation, and building structures, is created using professional scene modeling tools. A 3D mesh model is imported and placed as static obstacles at specified coordinates within the scene. Based on the vehicle dynamics model, various typical driving trajectories are preset, including uniform straight-line speed, acceleration for overtaking, emergency braking, and cornering, ensuring coverage of all possibilities in real-world driving scenarios. During simulation, the relative positions of the virtual vehicle and obstacles are calculated in real time. A physical rendering engine simulates the imaging mechanism of infrared sensors: based on material emissivity, ambient temperature, air transmittance, and thermal radiation transmission laws, physically realistic thermal images are dynamically generated. Infrared camera rendering output is synchronously acquired at a fixed frequency, with each frame accompanied by a precise timestamp, camera internal and external parameters, and vehicle pose data. Simultaneously, a point cloud sampling thread is initiated to extract simulated point cloud data from the corresponding viewpoint within the same spatiotemporal coordinate system. A hardware time synchronization mechanism ensures millisecond-level alignment accuracy between the two modalities. The resulting simulated infrared image not only possesses realistic thermodynamic characteristics but also maintains strict spatial consistency with the point cloud data.

[0033] 203. Associate and store the simulated point cloud data generated at the same time and from the same viewpoint with the simulated infrared image to form a data pair set; A timestamp synchronization mechanism is used to pair simulated point cloud data and simulated infrared images generated at the same time, while simultaneously recording metadata such as sensor parameters, vehicle pose, and environmental status at that moment. A hierarchical storage structure is adopted, with simulated point cloud data stored in PCD format, simulated infrared images saved in 16-bit grayscale format, and metadata labeled in JSON format. All data pairs are indexed by scene type, object category, and time series to generate a set of data pairs.

[0034] 204. Acquire the infrared image of the target to be enhanced; 205. Select multiple data pairs of target objects from the data pair set; The scene semantic features of the target infrared image are analyzed, including environment type, temporal information, and object composition. Then, data pairs with similar semantic contexts are retrieved from the data pair set. Object instances that are compatible with the target scene in terms of scale, orientation, and thermal features are prioritized. By calculating the similarity score between the simulation data and the target image in the feature space, the most suitable target object data pairs for enhancement are selected.

[0035] 206. Extract the original image region of each target object in the simulated infrared image and the corresponding simulated point cloud projection information; Obtain the simulated infrared image from the selected target object data pair and the corresponding camera parameters used to generate the image; project the simulated point cloud data from the target object data pair onto the pixel coordinate system of the simulated infrared image according to the camera parameters to obtain the two-dimensional pixel coordinate set of the corresponding target object point cloud; calculate the minimum bounding rectangle containing all two-dimensional pixel coordinates, and determine the image area enclosed by the minimum bounding rectangle as the original image area; use the camera parameters, the two-dimensional pixel coordinate set, and the minimum bounding rectangle information as the simulated point cloud projection information of the corresponding target object.

[0036] The process involves acquiring a simulated infrared image of the target object and the complete camera parameters used to generate it. These parameters include the camera intrinsic matrix, distortion coefficients, extrinsic rotation and translation vectors, and the transformation matrix from the world coordinate system to the camera coordinate system. Simulated point cloud data acquired simultaneously with the simulated infrared image is converted from the world coordinate system to the camera coordinate system. Each 3D point cloud is projected onto a 2D pixel plane using perspective projection. Strict radial and tangential distortion corrections for the camera lens are considered during projection to ensure the accuracy of the projected coordinates, thus obtaining the set of 2D pixel coordinates corresponding to the target object's point cloud. A convex hull algorithm is used to calculate the minimum bounding rectangle of these 2D pixels. The smallest area rectangle that can enclose all points is determined using a rotating caliper method, and the boundary tolerance between two pixels is extended to avoid edge clipping. The image region defined by this minimum bounding rectangle is extracted from the original infrared image as the original image region, and the coordinates of the top-left corner, width, height, and rotation angle of the rectangle are recorded. Finally, the camera parameters, the set of 2D pixel coordinates, the geometric parameters of the minimum bounding rectangle, and the coordinate transformation matrix are packaged and stored together to form the complete simulated point cloud projection information of the target object.

[0037] 207. Based on the simulated point cloud projection information, project the simulated point cloud data of each target object onto the image coordinate system of the target infrared image to determine its target pasting position and bounding box in the image. First image features are extracted from the original image region of the simulated infrared image corresponding to the simulated point cloud projection information; second image features are extracted from the predetermined search region in the target infrared image; the first image features and the second image features are matched, and the estimated pose of the target object in the world coordinate system of the target infrared image is calculated based on the matching result; according to the estimated pose, the simulated point cloud data is transformed from its original world coordinate system to the world coordinate system of the target infrared image; according to the camera parameters of the target infrared image, the transformed simulated point cloud data is projected into its pixel coordinate system to obtain the target's two-dimensional pixel coordinates; according to the target's two-dimensional pixel coordinates, the target pasting position and bounding box in the target infrared image are calculated.

[0038] The original image region in the simulated infrared image is preprocessed. Adaptive histogram equalization is used to enhance thermal contrast, and anisotropic diffusion filtering is used to suppress noise while preserving thermal radiation edges. A Gaussian difference scale space is constructed, and thermal radiation extreme points are detected at different resolution levels as candidate feature point locations. The orientation gradient histogram of each feature point is calculated, and the thermal gradient direction distribution characteristics of its surrounding area are statistically analyzed to form an initial descriptor. Local binary pattern texture features in the neighborhood of the feature point are extracted, and its microscopic thermal distribution pattern is quantified. The thermal gradient features and texture features are weighted and fused to generate a composite feature description vector with thermal radiation invariance and scale adaptability. These feature vectors together constitute the first image feature that can uniquely characterize the structural characteristics of the target object in the infrared domain.

[0039] The search area is defined based on the estimated size and category of the target object. Contrast-limited adaptive histogram equalization is applied to the search area to enhance the thermal feature representation. A Gaussian-scale spatial pyramid is constructed. At different resolution levels, an accelerated robust feature algorithm is used to detect the location of key points and determine the main feature direction. The directional gradient histogram descriptor of the neighborhood of each key point is calculated. At the same time, local temperature distribution statistics are extracted as a thermal feature supplement. The influence of illumination changes is eliminated by descriptor normalization. The descriptor vectors of all feature points are organized and stored according to a spatial grid. Finally, a second image feature that can comprehensively represent the visual content and thermodynamic properties of the search area is obtained.

[0040] The fast approximate nearest neighbor algorithm is used to perform bidirectional cross-validation matching on the first and second image feature descriptors, retaining high-quality matching pairs that are the nearest neighbors and whose distance ratio is less than a threshold. The random sampling consensus algorithm is applied to iteratively estimate the fundamental matrix, and mismatched point pairs that do not conform to the epipolar geometric constraints are eliminated. Based on the inlier set, the perspective n-point algorithm is used to solve the initial pose value of the target object relative to the target infrared camera coordinate system. The reprojection error is minimized by the Levenburg-Marquardt optimization algorithm to obtain the six-degree-of-freedom pose parameters of the target object in the world coordinate system of the target infrared image, including the three-dimensional rotation matrix and translation vector.

[0041] Based on the six-degree-of-freedom predicted pose calculated by feature matching, a 4x4 homogeneous transformation matrix is ​​constructed. This matrix contains a 3x3 rotation matrix and a 3x1 translation vector. The three-dimensional coordinates of the simulated point cloud data in its original world coordinate system are read. Each point cloud is represented by homogeneous coordinates and left-multiplied by the transformation matrix to complete a rigid transformation. During the transformation, the right-hand coordinate system rule is strictly followed to keep the relative geometric relationship between the point clouds unchanged. The transformed homogeneous coordinates are restored to three-dimensional Euclidean coordinates, thereby obtaining the three-dimensional position of each point cloud in the world coordinate system of the target infrared image.

[0042] The intrinsic parameter matrix of the target infrared camera is read, including the focal length and principal point coordinates, as well as the radial and tangential distortion coefficients. The transformed 3D point cloud coordinates are converted from the world coordinate system to the camera coordinate system. Then, the normalized coordinates on the ideal imaging plane are calculated through the perspective projection model. The distortion model is applied to perform nonlinear correction on the normalized coordinates to eliminate the influence of lens distortion. The corrected coordinates are multiplied by the intrinsic parameter matrix to obtain the final target 2D pixel coordinates. The projection points that fall within the effective range of the image are then selected, thus completing the mapping from 3D space to 2D pixels.

[0043] Convex hull analysis is performed on the target 2D pixel coordinate set obtained by projection to calculate the smallest convex polygon that can contain all points. The rotating caliper algorithm is used to solve for the minimum area bounding rectangle of the convex polygon to obtain the bounding box with the best orientation. The center point of the bounding box is calculated as the target pasting position based on the vertex coordinates. At the same time, the geometric parameters such as the width, height and rotation angle of the box are recorded. The edge of the bounding box is expanded to add a margin of two pixels to avoid the clipping effect. Finally, the target pasting area containing position coordinates and complete size information is output.

[0044] 208. Identify and segment the foreground objects in the original image region, set the alpha value of the background pixels to be completely transparent, and obtain an image of the foreground objects with a transparent background; The original image region is processed using a pre-trained U-Net semantic segmentation network. This U-Net semantic segmentation network is specifically optimized for infrared image features and can accurately identify the boundary between the foreground and background of an object and generate a binary segmentation mask. A conditional random field model is used to refine the binary segmentation mask. Isolated noise points are eliminated and edge burrs are repaired through spatial smoothing constraints. The optimized segmentation mask is converted into an alpha channel, in which the pixels in the foreground region remain completely opaque, and the transparency of the pixels in the background region is set to zero. The original image region and the alpha channel are synthesized to generate a PNG image of the foreground object with a transparent background, ensuring that the object edges are smooth and natural and the background region is completely transparent.

[0045] 209. Based on the region defined by the bounding box, extract the image patch at the target pasting location from the target infrared image, calculate the average color of the image patch, and adjust the overall hue and brightness of the foreground object image with a transparent background according to the difference between the average color and the average color of the foreground object image with a transparent background. Based on the bounding box coordinates, image patches corresponding to the target infrared image are extracted. After converting the image patch to the HSV color space, fully saturated and invalid pixels are excluded. The weighted average values ​​of its hue and luminance channels are calculated. The same operation is performed on the foreground object image with a transparent background. The pixel color values ​​in the effective area of ​​the alpha channel are counted. The difference vectors of the two image patches in the hue and luminance dimensions are calculated. Based on this difference, a color transformation matrix is ​​constructed. The hue offset and luminance gain values ​​of the foreground object image are gradually adjusted using a linear interpolation method while keeping its internal contrast relationship unchanged. The adjusted foreground object image is converted back to the original color space to ensure that its overall color characteristics are statistically consistent with the ambient lighting conditions of the target pasting location.

[0046] 210. Apply a Gaussian filter to the RGB color channels of the foreground object image after color adaptation; Gaussian filtering is applied to the RGB color channels of the color-adapted foreground object image to generate a basic blurred image. The contour region of the color-adapted foreground object image is identified by an edge detection algorithm, and an edge region enhancement mask is generated. The basic blurred image and the color-adapted foreground object image are then weighted and fused according to the edge region enhancement mask, so that the blur degree of the contour region is higher than that of the interior region.

[0047] The foreground object image after color adaptation is separated into RGB three channels. Gaussian filter kernel functions with different parameters are applied independently to each color channel. Based on the characteristics of infrared images, a larger standard deviation is set for the red channel with obvious thermal features to enhance the thermal diffusion effect, while a smaller standard deviation is set for the green and blue channels to maintain detail balance. A basic blurred image is generated through convolution operation. The Alpha channel of the foreground object image is processed by the Canny edge detection algorithm. The object contour boundary is accurately identified by double threshold hysteresis processing and non-maximum suppression. The detected edge regions are morphologically dilated to form an edge region enhancement mask. This mask generates a feathered region with gradually varying width around the contour line. Weighted fusion calculation is performed based on the mask. A higher weight is used in the edge enhancement mask region to favor the basic blurred image, while a higher weight is used in the internal region to preserve the original image details. A smooth gradient in the transition region is achieved through linear interpolation. Finally, a fused image with a moderately blurred contour region and relatively clear internal region is generated. This processing effectively simulates the thermal diffusion phenomenon and optical vignetting effect of object edges in real infrared imaging.

[0048] 211. Feather the Alpha channel of the edge region of the filtered foreground object image to generate a gradient mask with gradually decreasing transparency from the inside to the outside, thereby obtaining the fused and optimized object image. The binary alpha mask of the filtered foreground object image is extracted. The Euclidean distance from each foreground pixel to the nearest background pixel in the mask is calculated through distance transformation to generate a distance map. The distance map is normalized, mapping the maximum distance value to a completely opaque state and the minimum distance value to a completely transparent state. A Gaussian smoothing filter is applied to the normalized distance map for convolution, generating a smooth transparency gradient. The filter kernel size is adaptively adjusted according to the image resolution to ensure that the feathering width is proportional to the object size. Finally, the processed gradient mask is fused with the original alpha channel at the pixel level, preserving the complete opacity of the central region while making the edge regions exhibit an exponentially decaying transparency change, ultimately generating a fused and optimized object image with a smooth transition boundary.

[0049] 212. Paste the fused and optimized object image into the corresponding position in the target infrared image.

[0050] The fusion region is determined based on the target pasting location. The optimized fused object image is decomposed into RGB color channels and an Alpha transparency channel. For each color channel, a Poisson equation is constructed and a sparse linear system is solved using the gradient field of the corresponding region of the target infrared image as a constraint. This ensures that the color changes within the pasted object retain their own characteristics while naturally blending with the gradient field of the boundary region and the background image. Simultaneously, the gradient transparency of the Alpha channel is used as a fusion weight, and a weighted average calculation is performed in the boundary region. Finally, a multi-band mixing technique is used to fuse the synthesized result with the target image at different scales, preserving low-frequency illumination consistency while ensuring a natural transition of high-frequency details, ultimately generating a physically realistic enhanced infrared image.

[0051] In this embodiment of the invention, by generating data pairs of simulated point clouds and infrared images, accurately locating point cloud projections, and optimizing the entire process of background removal, color adaptation, texture blurring, and edge feathering, the realism, adaptability, and scene coordination of adding target objects to infrared images can be significantly improved without the need for real datasets. This effectively overcomes the problems of poor generalization ability and fusion distortion caused by the dependence on real data in traditional methods.

[0052] The infrared image enhancement method in the embodiments of the present invention has been described above. The infrared image enhancement device in the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 3 One embodiment of the infrared image enhancement device in this invention includes: The first acquisition module 301 is used to acquire the simulated point cloud data of the simulated object and the corresponding simulated infrared image data to form a data pair set; The second acquisition module 302 is used to acquire the infrared image of the target to be enhanced; The selection module 303 is used to select data pairs of multiple target objects from the data pair set; Extraction module 304 is used to extract the original image region of each target object in the simulated infrared image and the corresponding simulated point cloud projection information; The projection module 305 is used to project the simulated point cloud data of each target object onto the image coordinate system of the target infrared image based on the simulated point cloud projection information, and to determine the target pasting position and bounding box in the image. Processing module 306 is used to perform background removal, color adaptation, texture blurring and edge feathering on the original image area based on the target pasting position and bounding box to obtain a fused and optimized object image; The pasting module 307 is used to paste the fused and optimized object image to the corresponding position in the target infrared image. In this embodiment of the invention, by constructing a training set using simulated object point cloud and infrared image data, the dependence on massive amounts of real infrared image data is effectively avoided, significantly reducing the cost and difficulty of data acquisition. Point cloud projection information is used to guide the target object's pasting position and bounding box in the image. Combined with background removal, color adaptation, texture blurring, and edge feathering, high realism and adaptive fusion of the generated object and target infrared image are achieved, significantly improving the naturalness and detail realism of the enhanced image. This overcomes the problems of texture distortion and poor generalization caused by data loss or registration errors in traditional GAN ​​methods.

[0053] Please see Figure 4 Another embodiment of the infrared image enhancement device in this invention includes: The first acquisition module 301 is used to acquire the simulated point cloud data of the simulated object and the corresponding simulated infrared image data to form a data pair set; The second acquisition module 302 is used to acquire the infrared image of the target to be enhanced; The selection module 303 is used to select data pairs of multiple target objects from the data pair set; Extraction module 304 is used to extract the original image region of each target object in the simulated infrared image and the corresponding simulated point cloud projection information; The projection module 305 is used to project the simulated point cloud data of each target object onto the image coordinate system of the target infrared image based on the simulated point cloud projection information, and to determine the target pasting position and bounding box in the image. Processing module 306 is used to perform background removal, color adaptation, texture blurring and edge feathering on the original image area based on the target pasting position and bounding box to obtain a fused and optimized object image; The pasting module 307 is used to paste the fused and optimized object image to the corresponding position in the target infrared image. Optionally, the first acquisition module 301 can be specifically used for: The first generation unit 3011 is used to sample the point cloud of the three-dimensional mesh model of the preset object and generate the simulation point cloud data of the corresponding object. The second generation unit 3012 is used to place the three-dimensional mesh model at a fixed position in the virtual scene under various working conditions of simulated vehicle motion, and generate a simulated infrared image with a viewpoint corresponding to the corresponding simulated point cloud data by simulating infrared camera imaging. The association unit 3013 is used to associate and store the simulated point cloud data and simulated infrared image generated at the same time and from the same viewpoint to form a data pair set. Optionally, the second generating unit 3013 can be specifically used for: A virtual driving scenario is constructed, including road, terrain, and environmental elements, and a 3D mesh model is set as a static obstacle in the virtual driving scenario. According to the predefined vehicle movement trajectory, the process of the vehicle passing through the location of the static obstacle under various working conditions is simulated. The rendered images of the infrared camera during the vehicle's passage are collected at fixed time intervals, and each frame is a simulated infrared image of the viewpoint corresponding to the simulated point cloud data at the current moment.

[0054] Optionally, the extraction module 304 is specifically used for: Obtain the simulated infrared image from the selected target object data pair and the corresponding camera parameters used to generate the image; project the simulated point cloud data from the target object data pair onto the pixel coordinate system of the simulated infrared image according to the camera parameters to obtain the two-dimensional pixel coordinate set of the corresponding target object point cloud; calculate the minimum bounding rectangle containing all two-dimensional pixel coordinates, and determine the image area enclosed by the minimum bounding rectangle as the original image area; use the camera parameters, the two-dimensional pixel coordinate set, and the minimum bounding rectangle information as the simulated point cloud projection information of the corresponding target object.

[0055] Optionally, the projection module 305 can be specifically used for: First image features are extracted from the original image region of the simulated infrared image corresponding to the simulated point cloud projection information; second image features are extracted from the predetermined search region in the target infrared image; the first image features and the second image features are matched, and the estimated pose of the target object in the world coordinate system of the target infrared image is calculated based on the matching result; according to the estimated pose, the simulated point cloud data is transformed from its original world coordinate system to the world coordinate system of the target infrared image; according to the camera parameters of the target infrared image, the transformed simulated point cloud data is projected into its pixel coordinate system to obtain the target's two-dimensional pixel coordinates; according to the target's two-dimensional pixel coordinates, the target pasting position and bounding box in the target infrared image are calculated.

[0056] Optional, processing module 306: Processing unit 3061 is used to identify and segment the object foreground in the original image region, set the pixel alpha value of the background region to be completely transparent, and obtain an image of the foreground object with a transparent background. Color adjustment unit 3062 is used to extract image patches at the target pasting location from the target infrared image based on the region defined by the bounding box, calculate the average color of the image patches, and adjust the overall hue and brightness of the foreground object image with a transparent background based on the difference between the average color and the average color of the foreground object image with a transparent background. The filtering unit 3063 is used to filter the RGB color channels of the foreground object image after color adaptation using a Gaussian filter. Feathering unit 3064 is used to feather the Alpha channel of the edge region of the filtered foreground object image to generate a gradient mask with gradually decreasing transparency from the inside to the outside, thereby obtaining the fused and optimized object image.

[0057] Optionally, the filter unit 3063 can be specifically used for: Gaussian filtering is applied to the RGB color channels of the color-adapted foreground object image to generate a basic blurred image. The contour region of the color-adapted foreground object image is identified by an edge detection algorithm, and an edge region enhancement mask is generated. The basic blurred image and the color-adapted foreground object image are then weighted and fused according to the edge region enhancement mask, so that the blur degree of the contour region is higher than that of the interior region.

[0058] In this embodiment of the invention, by generating data pairs between simulated point clouds and infrared images, accurately locating point cloud projections, and optimizing the entire process of background removal, color adaptation, texture blurring, and edge feathering, the realism, adaptability, and scene coordination of adding target objects to infrared images can be significantly improved without the need for real datasets. This effectively overcomes the problems of poor generalization and fusion distortion caused by the dependence on real data in traditional methods. aboveFigure 3 and Figure 4 The infrared image enhancement device in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The electronic device in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0059] See Figure 5 As shown, the electronic device includes a processor 500 and a memory 501. The memory 501 stores machine-executable instructions that can be executed by the processor 500. The processor 500 executes the machine-executable instructions to implement the infrared image enhancement method described above.

[0060] Furthermore, Figure 5 The electronic device shown also includes a bus 502 and a communication interface 503. The processor 500, the communication interface 503 and the memory 501 are connected via the bus 502.

[0061] The memory 501 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 503 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 502 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0062] The processor 500 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 500 or by instructions in software form. The processor 500 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 501. The processor 500 reads the information in memory 501 and, in conjunction with its hardware, completes the method steps of the aforementioned embodiment.

[0063] The present invention also provides an electronic device, the computer device including a memory and a processor, the memory storing computer-readable instructions, which, when executed by the processor, cause the processor to perform the steps of the infrared image enhancement method described in the above embodiments.

[0064] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the infrared image enhancement method.

[0065] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0066] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An infrared image enhancement method, characterized in that, The infrared image enhancement method includes: Acquire the simulated point cloud data of the generated object and the corresponding simulated infrared image data to form a set of data pairs; Acquire infrared images of the target to be enhanced; Select multiple data pairs of target objects from the set of data pairs; Extract the original image region of each target object in the simulated infrared image and the corresponding simulated point cloud projection information; Based on the simulated point cloud projection information, the simulated point cloud data of each target object is projected onto the image coordinate system of the target infrared image to determine its target pasting position and bounding box in the image. Based on the target pasting position and the bounding box, the original image region is subjected to background removal, color adaptation, texture blurring and edge feathering to obtain a fused and optimized object image. The fused and optimized object image is pasted onto the corresponding position in the target infrared image.

2. The infrared image enhancement method according to claim 1, characterized in that, The acquisition of simulated point cloud data of the simulated object and corresponding simulated infrared image data constitutes a data pair set, including: Point cloud sampling is performed on the 3D mesh model of the preset object to generate the corresponding simulated point cloud data of the object; Under various conditions simulating vehicle motion, the three-dimensional mesh model is placed at a fixed position in the virtual scene, and a simulated infrared image corresponding to the viewpoint of the simulated point cloud data is generated by simulating infrared camera imaging. Simulated point cloud data generated at the same time and from the same perspective are associated and stored with simulated infrared images to form a set of data pairs.

3. The infrared image enhancement method according to claim 2, characterized in that, The step of placing the 3D mesh model at a fixed position in a virtual scene under various simulated vehicle motion conditions, and generating a simulated infrared image with a viewpoint corresponding to the simulated point cloud data through simulated infrared camera imaging, includes: A virtual driving scene containing road, terrain, and environmental elements is constructed, and the three-dimensional mesh model is set as a static obstacle in the virtual driving scene; Based on a predefined vehicle trajectory, the process of a vehicle passing through the location of the static obstacle under various working conditions is simulated. The infrared camera captures rendered images of vehicles passing by at fixed time intervals. Each frame is a simulated infrared image of the viewpoint corresponding to the simulated point cloud data at the current moment.

4. The infrared image enhancement method according to claim 1, characterized in that, The extraction of the original image region and corresponding simulated point cloud projection information of each target object in the simulated infrared image includes: Acquire the simulated infrared image from the selected target object data pair and the corresponding camera parameters when generating the image; The simulated point cloud data in the target object data pair is projected onto the pixel coordinate system of the simulated infrared image according to the camera parameters to obtain a two-dimensional pixel coordinate set of the corresponding target object point cloud. Calculate the minimum bounding rectangle containing all the two-dimensional pixel coordinates, and determine the image region enclosed by the minimum bounding rectangle as the original image region; The camera parameters, the set of two-dimensional pixel coordinates, and the information of the minimum bounding rectangle are used as the simulated point cloud projection information of the corresponding target object.

5. The infrared image enhancement method according to claim 1, characterized in that, The step of projecting the simulated point cloud data of each target object onto the image coordinate system of the target infrared image based on the simulated point cloud projection information, and determining its target pasting position and bounding box in the image, includes: Extract the first image features from the original image region of the simulated infrared image corresponding to the simulated point cloud projection information; Extract second image features within a predetermined search area in the target infrared image; The first image feature and the second image feature are matched, and the estimated pose of the target object in the world coordinate system of the target infrared image is calculated based on the matching result; Based on the estimated pose, the simulated point cloud data is transformed from its original world coordinate system to the world coordinate system of the target infrared image; Based on the camera parameters of the target infrared image, the transformed simulated point cloud data is projected into its pixel coordinate system to obtain the target's two-dimensional pixel coordinates; Based on the target's two-dimensional pixel coordinates, calculate the target pasting position and bounding box in the target infrared image.

6. The infrared image enhancement method according to claim 1, characterized in that, The process of performing background removal, color adaptation, texture blurring, and edge feathering on the original image region based on the target pasting location and the bounding box to obtain a fused and optimized object image includes: Identify and segment the foreground objects in the original image region, and set the pixel alpha value of the background region to be completely transparent to obtain an image of the foreground object with a transparent background. Based on the area defined by the bounding box, an image patch at the target pasting location is extracted from the target infrared image, and the average color of the image patch is calculated. Based on the difference between the average color and the average color of the foreground object image with a transparent background, the overall hue and brightness of the foreground object image with a transparent background are adjusted. The RGB color channels of the foreground object image after color adaptation are filtered using a Gaussian filter. Feathering is applied to the alpha channel of the edge region of the filtered foreground object image to generate a gradient mask with gradually decreasing transparency from the inside out, thus obtaining the fused and optimized object image.

7. The infrared image enhancement method according to claim 6, characterized in that, The step of filtering the RGB color channels of the color-adapted foreground object image using a Gaussian filter includes: Gaussian filtering is applied to the RGB color channels of the foreground object image after color adaptation to generate a basic blurred image. The contour region of the foreground object image after color adaptation is identified by an edge detection algorithm, and an edge region enhancement mask is generated. The basic blurred image and the color-adapted foreground object image are weighted and fused according to the edge region enhancement mask, so that the blur degree of the contour region is higher than that of the inner region.

8. An infrared image enhancement device, characterized in that, The infrared image enhancement device includes: The first acquisition module is used to acquire the simulated point cloud data of the simulated object and the corresponding simulated infrared image data to form a data pair set; The second acquisition module is used to acquire infrared images of the target to be enhanced. A selection module is used to select multiple data pairs of target objects from the data pair set; The extraction module is used to extract the original image region of each target object in the simulated infrared image and the corresponding simulated point cloud projection information. The projection module is used to project the simulated point cloud data of each target object onto the image coordinate system of the target infrared image according to the simulated point cloud projection information, and determine its target pasting position and bounding box in the image. The processing module is used to perform background removal, color adaptation, texture blurring, and edge feathering on the original image region based on the target pasting position and the bounding box to obtain a fused and optimized object image; The pasting module is used to paste the fused and optimized object image to the corresponding position of the target infrared image.

9. An electronic device, characterized in that, The electronic device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the electronic device to perform the infrared image enhancement method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the infrared image enhancement method as described in any one of claims 1-7.

Images