Image generation and editing method and system based on large model

Abstract

The application provides a large model-based image generation and editing method and system, wherein the method comprises the following steps: collecting an original image of a target object and dynamic temperature environment information, wherein the original image comprises thermal radiation data of a material in an infrared wave band; searching for reflection data corresponding to the thermal radiation data from a preset mapping relationship table, forming a thermal radiation texture map according to the reflection data and the dynamic temperature environment information; analyzing the thermal radiation texture map through a material migration method to identify a material characteristic region of the target object, and extracting reflection attribute parameters from the material characteristic region; and performing physical property editing on the original image based on the reflection attribute parameters and the dynamic temperature environment information to obtain a target edited image. The application improves the physical authenticity and light and shadow consistency when replacing the material in the dynamic temperature environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and image processing, and particularly relates to a large model-based image generation and editing method and system. BACKGROUND

[0002] In the field of film and television production, virtual reality, etc., it is necessary to intelligently replace and edit the material of the object in the image, such as replacing a cloth sofa with a leather sofa. Such editing not only needs to change the appearance of the material, but also needs to maintain the light and shadow effects and physical properties of the original scene, especially the material performance under different environmental light conditions. The prior art needs an image editing method that can automatically analyze the material characteristics and maintain physical reality.

[0003] The current mainstream scheme adopts a deep learning-based image generation technology, which learns the visual features of different materials by training a neural network model to directly generate an image region of a new material. This scheme uses a generative adversarial network to analyze the texture features of the original image, predicts the color and texture distribution of the new material, and embeds the generated result into the original image through image fusion technology.

[0004] This method mainly relies on the surface texture features of the visible light image, and ignores the optical and physical properties of the material, resulting in distorted material performance under different light conditions after editing. The material replacement result lacks compliance with the basic physical laws, and is difficult to adapt to the needs of complex light scenes, especially unnatural reflection effects when the environmental light changes. SUMMARY

[0005] The present application provides a large model-based image generation and editing method and system to solve the problems of poor physical reality and low light consistency in material replacement under dynamic temperature environment in the prior art.

[0006] In a first aspect, the present application provides a large model-based image generation and editing method, comprising:

[0007] Collecting an original image of a target object and dynamic temperature environment information, the original image including thermal radiation data of the material in the infrared band;

[0008] Finding the reflection data corresponding to the thermal radiation data from a pre-set mapping relationship table, and forming a thermal radiation texture map according to the reflection data and the dynamic temperature environment information;

[0009] Analyzing the thermal radiation texture map by a material migration method to identify a material feature region of the target object, and extracting reflection attribute parameters from the material feature region;

[0010] Based on the reflection attribute parameter and the dynamic temperature environment information, a physical attribute editing is performed on the original image to obtain a target edited image.

[0011] Optionally, the thermal radiation texture image is parsed by the material migration method to identify a material feature region of the target object, and a reflection attribute parameter is extracted from the material feature region, including:

[0012] The texture unit in the thermal radiation texture image is matched with the mapping relationship table by the material migration method, and a material feature region is generated according to a matching result;

[0013] A material boundary is identified from the material feature region;

[0014] Region data within the material boundary is converted into a reflection attribute parameter.

[0015] Optionally, the texture unit in the thermal radiation texture image is matched with the mapping relationship table by the material migration method, and a material feature region is generated according to a matching result, including:

[0016] A reflection feature value is extracted from the texture unit;

[0017] The reflection feature value is matched with a reference value range in the mapping relationship table by the material migration method;

[0018] According to a matching result, a material characteristic corresponding to each texture unit is determined, and a material characteristic label is assigned to each texture unit according to the material characteristic;

[0019] Texture units with the same material characteristic label are aggregated to form a material feature region.

[0020] Optionally, the material boundary is identified from the material feature region, including:

[0021] A difference value between adjacent texture units in the material feature region is calculated;

[0022] When the difference value exceeds a preset difference threshold, a boundary point corresponding to the difference value is marked;

[0023] The boundary points are connected in a spatial order to form a material boundary.

[0024] Optionally, the thermal radiation texture image is formed according to the reflection data and the dynamic temperature environment information, including:

[0025] The dynamic temperature environment information is converted into temperature distribution data;

[0026] Based on the reflection data and the temperature distribution data, multiple texture units are generated;

[0027] The texture units are arranged and combined according to the spatial coordinate order of the original image to form a thermal radiation texture map.

[0028] Optionally, generating multiple texture units based on the reflection data and the temperature distribution data includes:

[0029] Establish a spatial mapping relationship between the reflection data and the temperature distribution data;

[0030] Based on the spatial location mapping relationship, the values ​​of the reflection data located in the same spatial location are bound to the values ​​of the temperature distribution data to generate multiple texture feature pairs;

[0031] Based on the coordinate information and data content of each texture feature pair, a corresponding texture unit is constructed.

[0032] Optionally, the step of physically editing the original image based on the reflection attribute parameters and the dynamic temperature environment information to obtain the target edited image includes:

[0033] The reflection attribute parameters and the dynamic temperature environment information are combined according to their physical correspondence to generate an optical parameter set;

[0034] Convert the set of optical parameters into target optical property data;

[0035] Using the target optical attribute data, the optical attribute data of the original image is edited to form an initial edited image;

[0036] Based on the physical constraints of the reflection attribute parameters, the initial edited image is subjected to spatial consistency optimization processing to output the target edited image.

[0037] Secondly, this application provides an image generation and editing system based on a large model, comprising:

[0038] The acquisition module is used to acquire the original image and dynamic temperature environment information of the target object. The original image includes the thermal radiation data of the material in the infrared band.

[0039] The forming module is used to look up the reflection data corresponding to the thermal radiation data from a preset mapping table, and form a thermal radiation texture map based on the reflection data and the dynamic temperature environment information;

[0040] The extraction module is used to analyze the thermal radiation texture map using a material transfer method to identify the material feature regions of the target object and extract reflection attribute parameters from the material feature regions.

[0041] The editing module is used to edit the physical properties of the original image based on the reflection attribute parameters and the dynamic temperature environment information to obtain the target edited image.

[0042] Thirdly, this application provides a computing device including a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform an image generation and editing method based on a large model as described in any of the first aspects.

[0043] Fourthly, this application provides a computer storage medium storing computer program instructions, which, when executed by a processor, implement the image generation and editing method based on a large model as described in any one of the first aspects.

[0044] This application provides an image generation and editing method based on a large model. The method includes: acquiring an original image of a target object and dynamic temperature environment information, wherein the original image includes thermal radiation data of the material in the infrared band; searching for reflection data corresponding to the thermal radiation data from a preset mapping table; forming a thermal radiation texture map based on the reflection data and the dynamic temperature environment information; parsing the thermal radiation texture map using a material transfer method to identify the material feature regions of the target object and extracting reflection attribute parameters from the material feature regions; and editing the physical properties of the original image based on the reflection attribute parameters and the dynamic temperature environment information to obtain a target edited image.

[0045] The technical solution provided in this application has the following beneficial effects:

[0046] This application acquires image data containing the infrared properties of an object's material and information on environmental temperature changes, providing multimodal input for subsequent material analysis. It establishes a physical relationship between thermal radiation and optical reflection properties, enabling the conversion of infrared data to optical parameters. It generates a texture representation that integrates the object's surface temperature distribution and reflection properties, providing structured data for material analysis. It achieves intelligent recognition and region segmentation of the object's material features. It accurately acquires the optical property parameters of different material regions. It ensures that the image after material replacement maintains the same light and shadow physical properties as the original scene. It outputs physically realistic material replacement results.

[0047] Furthermore, this application also uses a material transfer algorithm to match the texture units in the thermal radiation texture map with the mapping table to generate accurate material feature regions; then it identifies the boundary positions between materials; and finally converts the data within the boundary range into reflection attribute parameters that can be used for rendering.

[0048] Furthermore, this application achieves accurate identification of complex material boundaries and effective extraction of reflection property parameters, providing an accurate physical parameter basis for subsequent material replacement, ensuring that the transition of the edited material is natural and conforms to physical laws.

[0049] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating an image generation and editing method based on a large model, provided in this application embodiment;

[0052] Figure 2 A schematic diagram of the structure of an image generation and editing system based on a large model provided in this application embodiment;

[0053] Figure 3 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0054] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0055] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0056] In the field of image generation and editing technology, existing deep learning-based material replacement methods suffer from a key drawback: they rely solely on the surface texture features of visible light images for material generation, neglecting the optical and physical properties of the materials. This leads to distortion of the edited materials under different environmental conditions. In particular, when the ambient temperature changes, the thermal radiation and optical reflection properties of the materials do not change in tandem, causing the replaced materials to exhibit lighting and shadow effects that do not conform to physical laws in dynamic environments. This problem stems from the one-sided processing of the physical properties of materials in existing methods, necessitating a material editing method that integrates multiple physical quantity features.

[0057] To address the aforementioned issues, this application proposes an image generation and editing method based on a large model. Its innovation lies in simultaneously acquiring the original image of the target object and dynamic temperature environment information. The original image includes the material's thermal radiation data in the infrared band. By searching for reflection data corresponding to the thermal radiation data from a pre-defined mapping table, and combining this with the dynamic temperature environment information, a thermal radiation texture map is formed. A material transfer method is then used to analyze the texture map to identify material feature regions and extract reflection attribute parameters. Finally, physical property editing is completed based on the reflection attribute parameters and the dynamic temperature environment information. This method establishes a dynamic correlation between the material's optical and thermodynamic properties through the collaborative processing of thermal radiation data and temperature information. This ensures that the image after material replacement maintains physically realistic lighting effects even with temperature changes, fundamentally solving the material distortion problem caused by neglecting the coupling of multiple physical quantities in existing technologies, and significantly improving the realism of material editing in complex environments.

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] Figure 1 A flowchart of an image generation and editing method based on a large model provided in this application embodiment is shown below. Figure 1 As shown, the method includes:

[0060] Step 101: Acquire the original image and dynamic temperature environment information of the target object. The original image includes the thermal radiation data of the material in the infrared band.

[0061] In step 101, the original image represents composite image data containing visible and infrared bands, where the infrared band data reflects the thermal radiation characteristics of the material surface. Dynamic temperature environment information represents real-time temperature change data of the target object's environment, including spatial distribution and temporal series characteristics. Thermal radiation data represents the radiation intensity values ​​of the material surface recorded in the infrared band image, which are related to the material type and temperature.

[0062] In this embodiment, a multispectral imaging device is used to simultaneously acquire visible light and infrared images of the target object, wherein the infrared image directly contains the thermal radiation data of the material; at the same time, a distributed temperature sensor network is used to monitor the temperature changes of the environment around the object in real time, and the temperature monitoring data is aligned with the image acquisition timestamp to form spatiotemporally synchronized dynamic temperature environment information; finally, the visible light image, infrared thermal radiation data and temperature environment information are packaged into raw input data.

[0063] For example, in the scenario of replacing the material of a ceramic cup, a type A multispectral camera is used to capture visible light and infrared images of the cup. The infrared images record the thermal radiation values ​​of various points on the surface of the cup. At the same time, a type B temperature sensor array is arranged around the cup to record the ambient temperature distribution in real time. The thermal radiation data of the infrared images and the temperature data collected by the sensors are time-aligned to ensure that each frame of the image is associated with the corresponding ambient temperature information.

[0064] Step 102: Find the reflection data corresponding to the thermal radiation data from the preset mapping table, and form a thermal radiation texture map based on the reflection data and the dynamic temperature environment information.

[0065] In step 102, the mapping table represents a pre-established two-dimensional lookup table, with the horizontal axis representing thermal radiation intensity values ​​and the vertical axis representing temperature values. The data in the table represents the corresponding reflectivity. The reflectivity data represents a set of parameters reflecting the visible light reflection characteristics of the material surface. The thermal radiation texture map is a multidimensional data set organized by texture units according to the image spatial structure, containing a multidimensional texture data structure of spatial coordinates, reflectivity data, and temperature data.

[0066] In this embodiment, interpolation is performed from a preset two-dimensional lookup table with thermal radiation data as the horizontal axis and ambient temperature as the vertical axis to obtain the corresponding reflection data value; the obtained reflection data and temperature data are spatially aligned and recombined according to the pixel coordinates of the original image; a texture unit containing position coordinates, reflection value and temperature value is constructed for each pixel; all texture units are arranged in the image space to form a thermal radiation texture map.

[0067] For example, in the case of a ceramic cup, the reflection data Z is obtained by looking up the thermal radiation value X and the ambient temperature Y at a certain point on the surface of the cup in a preset table. This query is performed on all points on the surface of the cup, and the obtained reflection data and the corresponding temperature value are recombined according to the pixel position. In the final generated thermal radiation texture map, each texture unit contains coordinates (i,j), reflection value Z and temperature value Y.

[0068] Step 103: Analyze the thermal radiation texture map using the material transfer method to identify the material feature regions of the target object and extract the reflection attribute parameters from the material feature regions.

[0069] In step 103, the material feature region represents a set of consecutive pixels with the same physical properties, determined by matching thermal radiation characteristics with a preset standard. The reflection attribute parameters represent a set of parameters describing the optical properties of the material, including diffuse reflection coefficient, specular reflection intensity, etc.

[0070] In this embodiment, for each texture unit in the thermal radiation texture map, its reflection feature value is extracted and compared with the reference range in the mapping relationship table to determine its material type and mark it; adjacent texture units with the same mark are clustered into material feature regions; the difference in reflection value of texture units in adjacent regions is calculated, and when the difference exceeds a threshold, it is marked as a boundary point; the boundary points are connected to form a closed material boundary; finally, the reflection data of the region within the boundary is converted into standardized reflection attribute parameters.

[0071] For example, in the case of a ceramic cup, the analysis of the thermal radiation texture map identifies two material regions: the cup body (ceramic) and the tabletop (wood). The abrupt change in reflectance at the boundary between the two regions is detected and connected to form the outline boundary of the cup bottom. The reflectance data within the cup body region is converted into optical parameters such as the diffuse reflectance coefficient and high light intensity of the ceramic material.

[0072] Step 104: Based on the reflection attribute parameters and the dynamic temperature environment information, the physical attributes of the original image are edited to obtain the target edited image.

[0073] In step 104, the target edited image refers to the final output image that has been completed after material replacement and conforms to physical reality.

[0074] In this embodiment, the extracted reflection attribute parameters and temperature data are weighted and fused to generate a set of optical parameters for a new material; the parameter set is converted into optical attribute data that can be directly applied to the image using a physical rendering algorithm; the optical characteristics of the corresponding area of ​​the original image are replaced with the new attribute data; finally, the edge area of ​​the material is gradient-processed to ensure a natural transition between different materials.

[0075] For example, by combining the reflective properties of a ceramic cup with temperature data, optical parameters for a metal material can be generated; the image data of the original cup body area can be replaced with the metal parameters; and a blur gradient can be applied to the edge areas such as the rim and handle to make the transition between the metal cup and the wooden tabletop more natural.

[0076] This method, by fusing infrared thermal radiation and temperature environment data, achieves the preservation of physical properties during material replacement. Under dynamic temperature conditions, it accurately reproduces the material's light and shadow changes, ensuring the physical authenticity of the editing results. Compared to traditional methods, it solves the problem of light and shadow distortion caused by temperature factors, making the material editing effect more in line with natural laws.

[0077] To address the issue of preserving physical properties during material replacement, in some embodiments, step 103 involves parsing the thermal radiation texture map using a material migration method to identify the material feature regions of the target object and extracting reflection attribute parameters from these regions.

[0078] Step 201: Using the material transfer method, match the texture units in the thermal radiation texture map with the mapping relationship table, and generate material feature regions based on the matching results.

[0079] In step 201, the texture unit is the smallest processing unit in the thermal radiation texture map. Each unit data structure contains spatial coordinate information, reflection data value and temperature data value. This structure binds the location information with physical properties, providing a basic data unit for material analysis.

[0080] In this embodiment, the reflection feature data of each texture unit is first extracted from the thermal radiation texture map and compared with the reference value range in the mapping table. Based on the comparison result, the material category to which the texture unit belongs is determined, and a unified identifier is assigned to adjacent texture units of the same material. Finally, all texture units with the same identifier are clustered to form a complete material feature region.

[0081] Step 202: Identify the material boundary from the material feature region.

[0082] In step 202, the material boundary represents the transition zone between different material feature regions. It is identified by analyzing the characteristic differences between adjacent texture units. The closed curve formed by connecting the boundary points divides the different material feature regions.

[0083] In this embodiment, the difference in reflectance values ​​of texture units between adjacent material feature regions is calculated. When the difference exceeds a preset threshold, the location is marked as a boundary point. These boundary points are then connected in an orderly manner according to spatial adjacency to form a closed material boundary line. The smoothness of the boundary line is optimized using an interpolation algorithm to ensure a natural transition.

[0084] Step 203: Convert the region data within the material boundary into reflection attribute parameters.

[0085] In step 203, the region data refers to the reflection characteristic data contained in all texture units within the material boundary, which belongs to physical optical property data rather than simple color or texture data. Specifically, it includes optical parameters such as reflection intensity and reflection curve characteristics at each location point. These data are obtained by standardizing the reflection data values ​​in the texture units and are used to describe the optical response characteristics of the material.

[0086] In this embodiment, all texture unit data within the material boundary are first normalized to eliminate fluctuations caused by measurement errors. Then, a standard reflection parameter template is obtained from the mapping table according to the material type, and the actual measurement data is fitted to the template to finally output a standardized set of reflection attribute parameters.

[0087] Here is a specific example:

[0088] In a specific embodiment of ceramic cup material replacement, a multispectral camera is first used to capture a thermal radiation texture map of the ceramic cup. Each texture unit in the map contains coordinate information, reflectance value, and temperature value. Using a material transfer method, the reflectance value of each texture unit is compared with a reference value range in a mapping table. The reference value range for ceramic material is 0.7 to 0.9, and for wooden tabletops, it is 0.3 to 0.5. When the reflectance value of a texture unit falls within the range of 0.7 to 0.9, it is identified as ceramic material and marked as a material feature area. The difference in reflectance values ​​between adjacent texture units is calculated. When the difference in reflectance values ​​between adjacent units exceeds 0.4, it is identified as a material boundary point. These boundary points are connected to form a complete cup outline boundary line. Statistical analysis is performed on the reflectance value data of all texture units within the ceramic material region within the boundary line. The average reflectance value is calculated to be 0.8, and the standard deviation is 0.05. Based on these data, standard reflectance attribute parameters for ceramic material are converted, where the diffuse reflectance coefficient is taken as the average reflectance value of 0.8, and the specular intensity is set to 0.3 based on the reflectance value fluctuation range.

[0089] In the embodiments of this application, the method ensures that the optical properties of the object surface after material replacement are consistent with the real physical laws by accurately identifying the material area and extracting the reflection attribute parameters. In particular, it can maintain the correct light and shadow performance under dynamic temperature environment, which solves the material distortion problem caused by ignoring temperature factors in traditional methods.

[0090] To address the issue of accurate identification of material feature regions, in some embodiments, step 201: matching the texture units in the thermal radiation texture map with the mapping table using a material migration method, and generating material feature regions based on the matching results, includes:

[0091] Step 301: Extract reflection feature values ​​from the texture unit.

[0092] In step 301, the reflection feature value refers to the key parameters representing the material's reflection characteristics extracted from the texture unit, including the main reflection intensity and reflection curve characteristics.

[0093] In this embodiment, each texture unit in the thermal radiation texture map is first analyzed to extract characteristic parameters from its reflection data, including the main reflection peaks and the morphological features of the reflection curve. These characteristic parameters can effectively distinguish the reflection characteristics of different materials.

[0094] Step 302: Using the material transfer method, match the reflection feature value with the reference value range in the mapping table.

[0095] In step 302, the reference value range is a pre-defined standard range of reflection characteristics for various materials in the mapping table, used to determine the material category to which the texture unit belongs. The matching process is the process of comparing the measured features with the standard range.

[0096] In this embodiment, the extracted reflection feature values ​​are compared one by one with the reference value range in the mapping table to calculate the similarity between the measured features and the standard features of various materials. The material category to which each texture unit most likely belongs is determined by the similarity comparison.

[0097] Step 303: Based on the matching results, determine the material characteristics corresponding to each texture unit, and assign a material characteristic tag to each texture unit according to the material characteristics.

[0098] In step 303, material properties include the physical properties of different materials such as metals, ceramics, and plastics. Material property tags are label data used to identify the material category to which a texture unit belongs; the same tag indicates the same material.

[0099] In this embodiment, after determining the material category corresponding to each texture unit based on the matching results, a corresponding material characteristic tag is assigned to each unit. The tag data includes a material type code and confidence information for subsequent processing.

[0100] Step 304: Aggregate texture units with the same material property marker to form a material feature region.

[0101] In step 304, the aggregation process refers to the process of merging spatially adjacent texture units with the same characteristic markers into a continuous region.

[0102] In this embodiment, all texture units with the same material label are first identified. Then, region growing is performed based on the spatial relationship of these units, merging adjacent units with the same label into a continuous region. The resulting material feature region will be used for subsequent boundary recognition and attribute extraction.

[0103] Here is a specific example:

[0104] In a specific embodiment of ceramic cup material replacement, reflection feature values ​​are first extracted from the texture units of the thermal radiation texture map. These values ​​represent the reflection data Z recorded for each texture unit. Using a material transfer method, these reflection feature values ​​are matched with reference value ranges in a mapping table. The reference value range for ceramic material is set to 0.7 to 0.9, and for wooden tabletops, it is set to 0.3 to 0.5. These reference value ranges are typical material reflection characteristic ranges obtained through extensive experimental measurements. When the reflection feature value of a texture unit falls between 0.7 and 0.9, the unit is determined to be ceramic material and assigned a ceramic material label 1; when the reflection feature value falls between 0.3 and 0.5, it is determined to be wooden material and assigned a label 2. All adjacent texture units labeled 1 are aggregated to form the material feature region of the ceramic cup body, while units labeled 2 are aggregated to form the tabletop material feature region.

[0105] In this embodiment of the application, the method achieves accurate identification of material features and region division through multi-level feature extraction and matching, providing a reliable foundation for subsequent material boundary identification and attribute extraction, and ensuring the accurate preservation of physical properties during material replacement.

[0106] To address the issue of accurate material boundary identification, in some embodiments, step 202: identifying the material boundary from the material feature region includes:

[0107] Step 401: Calculate the difference value between adjacent texture units in the material feature region.

[0108] In step 401, adjacent texture units refer to texture units that are directly connected in spatial position in the thermal radiation texture map. Adjacency is determined by the adjacency relationship of the unit coordinates. Specifically, in a two-dimensional image grid, if the coordinates of two texture units are adjacent in the horizontal, vertical, or diagonal directions (i.e., an eight-neighbor relationship), they are considered adjacent units. This adjacency relationship ensures that all possible directions of change can be covered during material boundary detection. The difference value is a quantitative indicator of the degree of difference in reflectivity between adjacent texture units, reflecting the drastic degree of material change.

[0109] In this embodiment, the spatial adjacency relationship of all texture units within the material feature region is first determined, and then the degree of difference in reflection features for each pair of adjacent units is calculated. The difference calculation comprehensively considers multiple feature dimensions such as reflection intensity and reflection curve morphology to obtain a comprehensive difference assessment result.

[0110] Step 402: When the difference value exceeds the preset difference threshold, mark the boundary position corresponding to the difference value as a boundary point.

[0111] In step 402, the preset difference threshold is a boundary judgment standard pre-set based on material characteristics, used to distinguish between natural variations within the material and true boundaries. Its value is pre-set according to the material type and temperature sensitivity. The preset method is as follows: by experimentally measuring the range of differences in reflectance characteristics of different materials at typical temperatures, the lower limit of this range is taken as the threshold to ensure that true material changes can be identified while ignoring natural fluctuations within the material. The boundary position refers to the spatial coordinate point corresponding to the difference in reflectance characteristics between adjacent texture units exceeding the threshold, i.e., the actual dividing point between the two materials. This position records the precise image coordinates of the abrupt change in difference and is the reference position for constructing the material boundary, usually located at the common edge or vertex of adjacent units. The boundary point refers to the specific location point at the material boundary that is confirmed as the true boundary.

[0112] In this embodiment, the calculated difference value is compared with a preset threshold. When the difference exceeds the threshold, it is determined that a material boundary exists at that location. The unit boundary positions corresponding to all difference values ​​exceeding the threshold are recorded to form a preliminary set of boundary points. Simultaneously, noise filtering is performed to eliminate isolated anomalies.

[0113] Step 403: Connect the boundary points in spatial order to form a material boundary.

[0114] In step 403, spatial order refers to the positional arrangement of boundary points on the image plane.

[0115] In this embodiment, the boundary point set is first spatially sorted, and then the points are connected sequentially according to the nearest neighbor principle. A smoothing algorithm is used during the connection process to optimize the boundary direction, ensuring the boundary curve is natural and continuous. This ultimately forms a complete and closed material boundary line.

[0116] Here is a specific example:

[0117] In a specific embodiment of ceramic cup material replacement, boundary recognition processing is first performed on the marked ceramic material feature areas and wooden tabletop material feature areas. For adjacent ceramic texture units and wooden texture units, the absolute difference in their reflectance values ​​is calculated as the difference value, where the average reflectance value of ceramic units is 0.8 and the average reflectance value of wooden units is 0.4, and the difference value between the two is 0.4. According to a preset material boundary judgment standard, when the difference value exceeds a threshold of 0.35, it is judged as a valid boundary point. This threshold is obtained by analyzing the statistical values ​​of reflectance differences at the junctions of various common materials. After marking all these junction points exceeding the threshold as boundary points, they are connected according to their spatial distribution order in the image. First, adjacent points are connected horizontally to form line segments, and then they are connected vertically to finally form a closed cup body outline boundary line.

[0118] In this embodiment, the method achieves accurate identification of material boundaries through difference calculation and threshold determination. Combined with spatial connection and curve optimization, it ensures the continuity and accuracy of the boundary lines, providing a reliable basis for subsequent material attribute extraction and replacement.

[0119] To address the issue of high-precision construction of thermal radiation texture maps, in some embodiments, step 102: forming a thermal radiation texture map based on the reflection data and the dynamic temperature environment information includes:

[0120] Step 501: Convert the dynamic temperature environment information into temperature distribution data.

[0121] In step 501, temperature distribution data refers to structured data formed by reorganizing the dynamic temperature environment information of the object surface according to spatial location, with each data point containing location coordinates and temperature value.

[0122] In this embodiment, the collected dynamic temperature data is first spatiotemporally aligned to ensure that each temperature sampling point matches the image acquisition time. Then, based on the spatial relationship of the sampling points, the temperature values ​​are interpolated to the grid positions corresponding to the image pixels to form a temperature distribution map consistent with the image resolution.

[0123] Step 502: Generate multiple texture units based on the reflection data and the temperature distribution data.

[0124] In this embodiment, the reflection data and temperature distribution data are first mapped one-to-one according to pixel position. Then, a texture unit containing three data items is created for each location point: the point coordinates, the reflection value, and the temperature value. This association method ensures that each texture unit completely records the physical characteristics of the point.

[0125] Step 503: Arrange and combine the texture units according to the spatial coordinate order of the original image to form a thermal radiation texture map.

[0126] In step 503, the spatial coordinate order refers to the arrangement rule of pixels in the original image.

[0127] In this embodiment, all texture units are arranged into a two-dimensional array of the same size as the original image, following the row and column order of the original image. Each array element corresponds to one texture unit, ultimately forming a thermal radiation texture map containing complete spatial information and physical properties.

[0128] Here is a specific example:

[0129] In a specific embodiment where a ceramic cup is replaced with a metal cup, a multispectral camera is first used to capture visible light and infrared images of the ceramic cup. The infrared image records the thermal radiation value of each pixel on the cup's surface. Simultaneously, a temperature sensor array is used to collect ambient temperature data around the cup in real time, and the temperature data is synchronized with the infrared image. Based on the thermal radiation value X of a point on the cup's surface in the infrared image, and the corresponding ambient temperature Y, the reflection data Z of that point is obtained by querying a preset thermal radiation-reflectivity mapping table. The X value comes from the pixel value of the infrared image, the Y value comes from the synchronized temperature sensor data, and the Z value is obtained by looking up the table. This query process is repeated for all pixels on the cup's surface. The obtained reflection data Z and the corresponding temperature value Y are matched and combined according to pixel coordinates to generate texture units containing coordinate information, reflection value, and temperature value. All texture units are reassembled according to the pixel arrangement order of the original image to form a complete thermal radiation texture map. Each texture unit in the map contains coordinates i,j, reflection value Z, and temperature value Y.

[0130] In this embodiment of the application, the method constructs a thermal radiation texture map containing complete physical properties by accurately associating reflection data with temperature information, providing a reliable data foundation for subsequent material analysis and editing, and ensuring the accurate preservation of thermodynamic properties during material processing.

[0131] To address the issue of high-precision texture unit generation, in some embodiments, step 502: generating multiple texture units based on the reflection data and the temperature distribution data includes:

[0132] Step 601: Establish the spatial location mapping relationship between the reflection data and the temperature distribution data.

[0133] In step 601, the spatial location mapping relationship refers to the positional correspondence rule between reflection data points and temperature data points in the image space, ensuring that the physical characteristics of the same object are accurately associated at the same location.

[0134] In this embodiment, the spatial coordinate transformation relationship between the reflection data acquisition points and the temperature measurement points is first established, and all data are unified to the same coordinate system through coordinate transformation. Then, for each reflection data point, the corresponding temperature data point is found, and a one-to-one location mapping table is established.

[0135] Step 602: Based on the spatial location mapping relationship, bind the values ​​of the reflection data and the temperature distribution data that are in the same spatial location to generate multiple texture feature pairs.

[0136] In step 602, a texture feature pair refers to a data combination formed by binding the reflection value and temperature value at the same spatial location, which is the basic element for constructing a texture unit. The binding process is to associate and encapsulate the reflection data value and temperature distribution data value at the same coordinate point one-to-one through a spatial location mapping relationship. Specifically, the operation is as follows: first, the coordinate position of the reflection data point is found according to the mapping table, then the temperature distribution data value is extracted at the same coordinate position, and then these two values ​​are packaged with the coordinate information of the point to form a complete texture feature pair. Each feature pair contains three elements: {coordinates (x, y), reflection value, and temperature value}, ensuring that the physical characteristics of each spatial point are completely recorded.

[0137] In this embodiment, a data pair containing reflection and temperature values ​​is created for each valid data point based on the established location mapping relationship. The generation of the data pair requires ensuring that the reflection and temperature values ​​originate from the same physical location and are synchronized in time to maintain consistency in physical characteristics.

[0138] Step 603: Construct the corresponding texture unit based on the coordinate information and data content of each texture feature pair.

[0139] In step 603, the coordinate information refers to the two-dimensional position (x, y) of the texture unit in the original image, which is directly inherited from the coordinate data of the texture feature pair. The data content includes the reflection value (from the query result of the mapping table) and the temperature value (from the dynamic temperature environment information conversion) of the location point.

[0140] In this embodiment, a complete texture unit data structure is constructed by supplementing the coordinate information and other derived features of a point based on texture feature pairs. Each texture unit contains complete information such as location identifier, reflection characteristics, and temperature characteristics, providing comprehensive data support for subsequent processing.

[0141] Here is a specific example:

[0142] In a specific embodiment where a ceramic cup is replaced with a metal cup, the thermal radiation data captured by a multispectral camera is first spatially aligned with the ambient temperature data collected by a temperature sensor to establish a one-to-one correspondence between reflection data points and temperature data points. The position coordinates of each data point are calculated using camera imaging parameters and sensor layout parameters. For a specific location on the cup surface, the reflection data value is the Z value obtained by querying a thermal radiation-reflectivity mapping table, and the corresponding temperature data value is the measured Y value at that location. These two values ​​are bound to the coordinate information i and j of that point to generate a texture feature pair containing three data items. This binding process is repeated for all valid locations on the cup surface to obtain a complete set of texture feature pairs. Then, based on the coordinate information i and j in each texture feature pair, the position of that point in the image is determined. Combining the included reflection value Z and temperature value Y, a corresponding texture unit data structure is constructed. This structure completely records the spatial information and physical characteristics of that location point.

[0143] In this embodiment, the method constructs a texture unit containing multiple physical properties through precise spatial mapping and feature binding, ensuring the accurate correlation between reflection and temperature properties during material analysis, and providing reliable data support for subsequent material identification and editing.

[0144] To address the issue of physical realism in material editing, in some embodiments, step 104: editing the physical properties of the original image based on the reflection attribute parameters and the dynamic temperature environment information to obtain the target edited image, includes:

[0145] Step 701: Combine the reflection attribute parameters with the dynamic temperature environment information according to the physical correspondence to generate an optical parameter set.

[0146] In step 701, the physical correspondence refers to the inherent physical correlation between reflection attribute parameters (such as diffuse reflectance and specular reflection intensity) and temperature data, which originates from experimental data on the thermo-optical properties of the material. By measuring the reflectance variation curves of various materials at different temperatures, a quantitative model of the reflection parameters changing with temperature is established, forming a mathematical expression of the reflection-temperature coupling relationship, ensuring that the generated set of optical parameters conforms to real physical laws. The set of optical parameters refers to a comprehensive parameter group generated by combining reflection attribute parameters and temperature information according to physical laws, including a parameter group containing material optical properties and thermal radiation properties such as reflectance and roughness.

[0147] In this embodiment, the correlation between each parameter in the reflection property parameters and temperature change is first analyzed. Then, a weighted combination model of reflection parameters and temperature data is established based on the physical properties of the material. Finally, an optical parameter set that includes both optical properties and temperature effects is generated.

[0148] Step 702: Convert the set of optical parameters into target optical property data.

[0149] In step 702, the target optical attribute data refers to material property data that can be directly applied to image editing, obtained by physically-based rendering transformation of the optical parameter set. The transformation process is a crucial step in converting theoretical parameters into practically usable image attributes. Specifically, the optical parameter set is first input into the ray tracing engine. Based on the material type, a corresponding shading model is selected (e.g., the Cook-Torrance model for metal). The interaction between light and the material surface is simulated in conjunction with scene lighting conditions. The final color value, specular intensity, and surface roughness of each pixel are calculated, generating target optical attribute data that can be directly used for image editing. This process strictly adheres to optical physics equations to ensure the physical accuracy of the numerical transformation.

[0150] In this embodiment, the optical parameter set is input into the physical rendering model. By simulating the interaction process between light and material, the optical properties that the material should exhibit under a specific temperature environment are calculated, including color values, brightness values, and surface texture characteristics, which are data that can be directly used for image editing.

[0151] Step 703: Using the target optical attribute data, edit the optical attribute data of the original image to form an initial edited image.

[0152] In step 703, optical property data editing refers to replacing the visual features of the corresponding areas of the original image with new material properties. The initial edited image refers to the intermediate result image after only basic material replacement, which may have local inconsistencies.

[0153] In this embodiment, the material regions that need to be replaced in the original image are first determined, and then the target optical property data are applied to the image point by point according to the region to replace the original optical properties and generate a preliminary material replacement result image.

[0154] Step 704: Based on the physical constraints of the reflection attribute parameters, perform spatial consistency optimization processing on the initial edited image and output the target edited image.

[0155] In step 704, physical constraints refer to the fundamental physical principles that material property changes must follow, including energy conservation (reflectivity not exceeding 1) and temperature-reflectivity monotonicity (the reflectivity of some materials decreases as temperature increases). These constraints are derived from a database of material physical properties and are embedded into an optimization algorithm in the form of a preset rule set to ensure that the edited material changes do not violate natural laws. Spatial consistency optimization processing refers to naturalizing the material transition areas to ensure that the editing results conform to physical laws.

[0156] In this embodiment, optical discontinuities at the material boundaries in the initial edited image are first detected. Then, based on the physical constraints set by the reflection property parameters, the transition area is gradually adjusted to make the connection between different materials natural and smooth, and finally, the target edited image that conforms to physical reality is output.

[0157] Here is a specific example:

[0158] In a specific embodiment where a ceramic cup is replaced with a metal cup, the extracted ceramic material reflection attribute parameters, including a diffuse reflectance coefficient of 0.8 and a specular intensity of 0.3, are first physically combined with the real-time acquired ambient temperature data of 25 degrees Celsius. The temperature influence coefficient is set to 0.02 per degree Celsius. The optical parameter set of the metal material is calculated using the formula: Metal Reflectance Parameter = Ceramic Reflectance Parameter × (1 + Temperature Influence Coefficient × Temperature Difference), resulting in a metal diffuse reflectance coefficient of 0.88 and a specular intensity of 0.33. These optical parameters are then input into a physical rendering model to convert them into directly applicable target optical attribute data, including the RGB values ​​of the metal surface color (210, 210, 210) and a gloss parameter of 0.85. These attribute data are then used to replace the optical characteristics of each pixel in the cup area of ​​the original image, generating the initial edited image. At this point, a significant material abrupt change was observed at the edge of the cup. Therefore, based on the physical constraints of the reflection property parameters, a gradient processing was performed within a 5-pixel width of the edge to smoothly transition the optical properties of the metal and wood according to the distance ratio, ultimately obtaining a physically realistic target edited image. The light and shadow effects of the metal cup will automatically adjust with changes in ambient temperature. When the temperature rises to 30 degrees Celsius, the diffuse reflection coefficient of the metal surface will be adjusted to 0.92 accordingly, maintaining temperature response characteristics consistent with the real world.

[0159] In the embodiments of this application, the method achieves the preservation of physical properties during the material replacement process through precise modeling of physical laws and multi-stage optimization processing, so that the editing results can present realistic light and shadow effects under different temperature environments, and solves the problem of material distortion in complex environments by traditional methods.

[0160] Figure 2 This application provides a schematic diagram of the structure of an image generation and editing system based on a large model, as shown in the embodiments of this application. Figure 2 As shown, the system includes:

[0161] The acquisition module 21 is used to acquire the original image and dynamic temperature environment information of the target object. The original image includes the thermal radiation data of the material in the infrared band.

[0162] The forming module 22 is used to find the reflection data corresponding to the thermal radiation data from a preset mapping table, and form a thermal radiation texture map based on the reflection data and the dynamic temperature environment information.

[0163] Extraction module 23 is used to analyze the thermal radiation texture map using a material transfer method to identify the material feature regions of the target object and extract reflection attribute parameters from the material feature regions.

[0164] The editing module 24 is used to edit the physical properties of the original image based on the reflection attribute parameters and the dynamic temperature environment information to obtain the target edited image.

[0165] Figure 2 The aforementioned image generation and editing system based on a large model can perform... Figure 1 The implementation principle and technical effects of the large-model-based image generation and editing method described in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit of the large-model-based image generation and editing system in the above embodiments perform operations have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0166] In one possible design, Figure 2 The large-model-based image generation and editing system shown in the embodiment can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0167] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 32.

[0168] The processing component 32 is used to perform the above. Figure 1 The embodiment describes an image generation and editing method based on a large model.

[0169] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0170] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0171] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0172] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0173] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0174] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0175] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The illustrated embodiment presents an image generation and editing method based on a large model.

[0176] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0177] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image generation and editing method based on a large model, characterized in that, include: Acquire raw images and dynamic temperature environment information of the target object, wherein the raw images include thermal radiation data of the material in the infrared band; The reflection data corresponding to the thermal radiation data is found in the preset mapping table, and a thermal radiation texture map is formed based on the reflection data and the dynamic temperature environment information. The thermal radiation texture map is analyzed by the material transfer method to identify the material feature regions of the target object and extract the reflection attribute parameters from the material feature regions. Based on the reflection attribute parameters and the dynamic temperature environment information, the original image is physically edited to obtain the target edited image; The step of physically editing the original image based on the reflection attribute parameters and the dynamic temperature environment information to obtain the target edited image includes: The reflection attribute parameters and the dynamic temperature environment information are combined according to their physical correspondence to generate an optical parameter set; Convert the set of optical parameters into target optical property data; Using the target optical attribute data, the optical attribute data of the original image is edited to form an initial edited image; Based on the physical constraints of the reflection attribute parameters, the initial edited image is subjected to spatial consistency optimization processing to output the target edited image.

2. The method according to claim 1, characterized in that, The step of parsing the thermal radiation texture map using a material transfer method to identify the material feature regions of the target object and extracting reflection attribute parameters from the material feature regions includes: The texture units in the thermal radiation texture map are matched with the mapping table using a material transfer method, and a material feature region is generated based on the matching result. Identify the material boundary from the material feature region; The region data within the material boundary is converted into reflection property parameters.

3. The method according to claim 2, characterized in that, The step of matching texture units in the thermal radiation texture map with the mapping table using a material transfer method, and generating material feature regions based on the matching results, includes: Extract reflection feature values ​​from the texture unit; The material transfer method is used to match the reflection feature value with the reference value range in the mapping table; Based on the matching results, the material characteristics corresponding to each texture unit are determined, and a material characteristic tag is assigned to each texture unit according to the material characteristics. Texture units with the same material property markers are aggregated to form material feature regions.

4. The method according to claim 2, characterized in that, The process of identifying material boundaries from the material feature region includes: Calculate the difference value between adjacent texture units in the material feature region; When the difference value exceeds a preset difference threshold, the boundary position corresponding to the difference value is marked as a boundary point; The boundary points are connected in spatial order to form the material boundary.

5. The method according to claim 1, characterized in that, The step of forming a thermal radiation texture map based on the reflection data and the dynamic temperature environment information includes: The dynamic temperature environment information is converted into temperature distribution data; Based on the reflection data and the temperature distribution data, multiple texture units are generated; The texture units are arranged and combined according to the spatial coordinate order of the original image to form a thermal radiation texture map.

6. The method according to claim 5, characterized in that, The step of generating multiple texture units based on the reflection data and the temperature distribution data includes: Establish a spatial mapping relationship between the reflection data and the temperature distribution data; Based on the spatial location mapping relationship, the values ​​of the reflection data located in the same spatial location are bound to the values ​​of the temperature distribution data to generate multiple texture feature pairs; Based on the coordinate information and data content of each texture feature pair, a corresponding texture unit is constructed.

7. An image generation and editing system based on a large model, characterized in that, include: The acquisition module is used to acquire the original image and dynamic temperature environment information of the target object. The original image includes the thermal radiation data of the material in the infrared band. The forming module is used to look up the reflection data corresponding to the thermal radiation data from a preset mapping table, and form a thermal radiation texture map based on the reflection data and the dynamic temperature environment information; The extraction module is used to analyze the thermal radiation texture map using a material transfer method to identify the material feature regions of the target object and extract reflection attribute parameters from the material feature regions. The editing module is used to edit the physical properties of the original image based on the reflection attribute parameters and the dynamic temperature environment information to obtain the target edited image; The step of physically editing the original image based on the reflection attribute parameters and the dynamic temperature environment information to obtain the target edited image includes: The reflection attribute parameters and the dynamic temperature environment information are combined according to their physical correspondence to generate an optical parameter set; Convert the set of optical parameters into target optical property data; Using the target optical attribute data, the optical attribute data of the original image is edited to form an initial edited image; Based on the physical constraints of the reflection attribute parameters, the initial edited image is subjected to spatial consistency optimization processing to output the target edited image.

8. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the image generation and editing method based on a large model as described in any one of claims 1 to 6.

9. A computer storage medium, characterized in that, The system contains a computer program that, when executed by a computer, implements an image generation and editing method based on a large model as described in any one of claims 1 to 6.

Images