Image processing method, apparatus, device, medium, and product

Abstract

An image processing method, device, equipment, medium and product are provided in the present application, which are applied to the technical field of image processing. The method comprises: receiving at least one first image; displaying a first identification image and a first three-dimensional model; wherein the first identification image and the first three-dimensional model are generated based on the first image, the first identification image comprises a plurality of image sub-regions, and the image sub-regions are associated with at least part of the regions of the first three-dimensional model; obtaining a first identification model, the first identification model comprises a plurality of model sub-regions, the model sub-regions are generated based on the image sub-regions, and are associated with the image sub-regions; displaying a second three-dimensional model, the second three-dimensional model comprises a plurality of first sub-models, and the first sub-models are generated based on the first three-dimensional model and the first identification model. The present application solves the problem of poor visual content quality of the generated three-dimensional model, and can improve the detail richness and accuracy of the generated three-dimensional visual content, and improve the overall content creation efficiency.

Description

Technical Field

[0001] This article relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, medium, and product. Background Technology

[0002] The technology of generating 3D (Three-Dimensional) image content is used in the field of visual content processing technology, but the image quality of complex 3D visual content still needs to be further improved. Summary of the Invention

[0003] This paper provides an image processing method, apparatus, device, medium, and product that can improve the content quality of 3D visual content processing results and enhance the resolution and accuracy of detailed areas of complex content objects.

[0004] Firstly, an image processing method is provided, the method comprising: Receive at least one first image; Display a first identification image and a first three-dimensional model; wherein the first identification image and the first three-dimensional model are generated based on a first image, the first identification image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the region of the first three-dimensional model; A first identifier model is obtained, which includes multiple model sub-regions. The model sub-regions are generated based on and associated with the image sub-regions. The second three-dimensional model is displayed. The second three-dimensional model includes multiple first sub-models, which are generated based on the first three-dimensional model and the first identifier model.

[0005] Secondly, an image processing apparatus is also provided, the apparatus comprising: The first module is used to receive at least one first image; The second module is used to display a first identification image and a first three-dimensional model; wherein the first identification image and the first three-dimensional model are generated based on a first image, the first identification image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the region of the first three-dimensional model; The third module is used to obtain a first identifier model, which includes multiple model sub-regions. The model sub-regions are generated based on and associated with the image sub-regions. The fourth module is used to display the second three-dimensional model, which includes multiple first sub-models. The first sub-models are generated based on the first three-dimensional model and the first identifier model.

[0006] Thirdly, an electronic device is also provided, comprising: One or more processors; Storage device, used to store program instructions When the processor reads and executes program instructions, it implements any of the image processing methods described in this specification.

[0007] Fourthly, a non-transitory computer-readable storage medium is also provided, wherein computer program instructions are stored internally; when the processor executes the computer program instructions, it performs an image processing method as described in any of the contents of this specification.

[0008] Fifthly, a computer program product is also provided, including a computer program that, when loaded and run by the processor of an electronic device, controls the processor to execute an image processing method as described in any of the contents of this specification. Attached Figure Description

[0009] The above and other features, advantages, and aspects herein will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0010] Figure 1 This is an exemplary system architecture that can implement the image processing methods described in this paper; Figure 2 This is a flowchart illustrating one of the image processing methods presented in this paper; Figure 3 This is a schematic diagram of an interactive interface for inputting or generating an original artwork (first image) provided in this article; Figure 4 This is a schematic diagram of the interactive interface for displaying a first identifier image and a first 3D model provided in this article; Figure 5 This is a schematic diagram of the final 3D model display interface provided in this article; Figure 6 This is a flowchart illustrating another image processing method presented in this paper. Figure 7 This is a schematic diagram illustrating the identification model and interactive interface provided in this article; Figure 8 This is a flowchart illustrating another image processing method presented in this paper. Figure 9 This is a schematic diagram of the structure of an image processing device provided in this article; Figure 10 This is a schematic diagram of the structure of an electronic device used to implement the image processing method described in this paper. Detailed Implementation

[0011] The implementation of the methods described herein will now be described in more detail with reference to the accompanying drawings. While some implementations of the methods described herein are shown in the drawings, it should be understood that the methods described herein can be implemented in various forms and should not be construed as limited to those described herein. Rather, these implementations are provided to facilitate a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and implementations are for illustrative purposes only and are not intended to limit the scope of this invention.

[0012] It should be understood that the steps described in the method embodiments herein may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this document is not limited in this respect.

[0013] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one method implementation" means "at least one method implementation"; the term "another method implementation" means "at least one additional method implementation"; the term "some method implementations" means "at least some method implementations". Definitions for other terms will be given in the following description.

[0014] It should be noted that the concepts of "first" and "second" mentioned in this article are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions performed by these devices, modules or units or their interdependencies.

[0015] It should be noted that the terms "one" and "more" used in this document are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0016] The names of messages or information exchanged between multiple devices in the embodiments herein are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0017] It is understandable that before using any of the solutions described in this article, users should be informed of the type, scope of use, and usage scenarios of the user information involved in this article in an appropriate manner in accordance with relevant laws and regulations, and their authorization should be obtained.

[0018] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require access to and use of their user information. This allows the user to autonomously choose whether to provide user information to the software or hardware, such as the electronic device, application, server, or storage medium executing the scheme described herein, based on the prompt message.

[0019] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide user information to the electronic device.

[0020] It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation method described in this article. Other methods that comply with relevant laws and regulations may also be applied to the implementation method described in this article.

[0021] It is understood that the data involved in this article (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0022] It should also be noted that the system mentioned in this article can be... Figure 1 The system shown can implement the image processing methods described in this paper.

[0023] Figure 1 An exemplary system architecture is shown that can be applied to the image processing methods described in this paper.

[0024] The technical solutions described in this paper can be applied to the image processing system architecture shown in Figure 1. In practical applications, the image processing system may include a client 101 and a server 102.

[0025] Client 101 can be a browser, client application (App), Hyper Text Markup Language 5 (HTML5) application, lightweight application (also known as a mini-program, a type of lightweight application), or cloud application. Client 101 can be deployed on an electronic device and relies on the operation of that device or certain applications on the device to implement its functions. The aforementioned electronic device can be, for example, a device with a display screen that supports information browsing, such as a smartphone, tablet, personal computer (PC), extended reality device, or other client terminal. For ease of understanding, Figure 1 The client is primarily represented in the form of a device. Various applications can also be configured on electronic devices, such as multimedia applications, e-commerce platform applications, search applications, instant messaging applications, etc.

[0026] Server 102 can be one or more servers providing various services. That is, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; in addition, it can also be a server of a distributed system, a server integrating blockchain technology, a cloud server, or an intelligent cloud computing server or intelligent cloud host with machine learning technology, etc.

[0027] In the implementation described in this paper, server 102 can interact with client 101 to obtain application function running data of a specified application on the client. This data is used for analysis and mining to guide the improvement of application function performance. Server 102 can also synchronize the function configuration parameters determined based on the data analysis results to client 101 to optimize the running performance of the specified application function on client 101 and optimize business decisions related to the application function.

[0028] Understandable Figure 1 The number of clients and servers shown is for illustrative purposes only. Any number of clients and servers can be configured to meet specific implementation requirements.

[0029] It should be noted that the implementation methods described in this article may involve the use of user data. In practical applications, within the scope permitted by applicable laws and regulations, the personal data of specific users may be used in the scheme described in this article, provided that the relevant laws and regulations of the user's region are met (such as obtaining the user's explicit consent and ensuring that the user receives appropriate notification).

[0030] Figure 2 This paper presents a flowchart illustrating an image processing method applicable to 3D object modeling and generation based on generative machine learning models, particularly in scenarios where intermediate products of the generated 3D target can be obtained. This method can be executed by an image processing device, which can be implemented in software and / or hardware, optionally through an electronic device such as a mobile terminal, PC, or server.

[0031] like Figure 2 As shown, an image processing method in one scenario may specifically include: S210, Receive at least one first image.

[0032] In one scenario, the first image can refer to at least one of the following: a 2D concept art, a 2D concept sketch, or a 2D rendering of the target 3D visual content object. A concept sketch can be a creative concept image from the early design phase, primarily used to determine the overall shape, style, color scheme, and design direction of the content object. Its structural expression is relatively simple, with lower detail completeness, focusing on presenting the design atmosphere and creative solution, and has not yet formed a standardized, finalized structural setting. A concept sketch can be a standardized 2D setting image after the shape has been finalized. The outer contour, main structure, and color ratio of the content object are clearly defined, serving as the benchmark reference material for the creation of 3D assets (3D visual content objects). Concept sketches are mostly single-view images, capable of fully presenting the main form of the object's front, but lacking structural information on occluded areas such as the sides and back. They are standard input material for image-to-3D generation tasks. A rendering is a refined rendered image with complete lighting, material, and perspective representation. It can be generated from a 3D model rendering or hand-drawn rendering, directly reflecting the object's three-dimensional appearance and surface material details. Compared to concept sketches and concept sketches, it contains richer 3D lighting and structural hints. In another scenario, the first image can also be a concept sketch generated based on a 2D concept sketch.

[0033] When multiple first images are received, they can be two-dimensional original drawings showing the same object from different perspectives, such as a front view, a side view, and a top view of a mechanical scorpion. Of course, in some cases, six or more views from different perspectives can also be received.

[0034] In one scenario, receiving at least one first image can be based on a single original artwork. Users can upload a frontal concept design or original artwork of a content object via the import button in the tool interface. The system receives this single image as the first image, stores it in the processing queue, and displays it in the preview area. In another scenario, receiving at least one first image can also be based on batch reception of original artwork from multiple perspectives. Users can drag and drop three or more images onto the tool interface at once, representing the front view, side view, and three-quarter view of the content object, or including images from other perspectives. The system identifies multiple received images belonging to different perspectives of the same object, receives them simultaneously as a group of first images, and labels and groups them according to perspective category for storage.

[0035] For example, the interactive page that receives the first image can be Figure 3 The interactive page shown allows users to import images or generate original artwork based on sketches.

[0036] S220, Display the first identification image and the first three-dimensional model.

[0037] The first identifier image and the first three-dimensional model are generated based on the first image. The first identifier image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the region of the first three-dimensional model.

[0038] In one scenario, the first identifier image can refer to a two-dimensional partitioned identifier map generated based on the first image, i.e., a 2D identifier map. It contains multiple image sub-regions, each of which can be identified by a different color or label. For example, the pixels in the original artwork of the content object (mechanical scorpion) can be divided into "left claw region," "right claw region," "head region," "thorax region," "tail region," etc., with each region being an image sub-region. Each sub-region can correspond to a functional component of the content.

[0039] In one scenario, the first 3D model can refer to a 3D geometric structure model generated based on a first image, such as a rough 3D model of a complete content object generated from a multi-view original artwork of the content object. The association between an image sub-region and at least a portion of the first 3D model can mean that there is a correspondence between the image sub-region and at least a portion of the first 3D model, i.e., a component region in a 2D partition is mapped to a corresponding vertex or facet on the surface of the 3D model.

[0040] In one scenario, the system can input the received original frontal image of a single content object into a first model and a second model. Both the first and second models are machine learning models. The first model performs semantic segmentation on the original image, outputting a first identifier image, on which different regions of the content object, i.e., image sub-regions, are marked with different colors. The second model reconstructs a complete first 3D model from the original image. In another scenario, the system can display these two results side-by-side on an interactive interface. When the user clicks on any sub-region in the first identifier image, the corresponding part on the first 3D model is simultaneously highlighted, visually demonstrating the relationship between the two.

[0041] In one scenario, the system can input multiple original artworks of content objects from different perspectives into the first model to obtain multiple first identifier images corresponding to those perspectives. Simultaneously, the system utilizes multi-view stereo reconstruction technology to input the three original artworks into the second model, generating a first 3D model with a more accurate structure under multi-view constraints. During display, the system presents multiple first identifier images and the 3D model preview window together. When the user switches perspectives to view the original artworks, the 3D model automatically rotates to the corresponding angle, demonstrating the relationships in real time.

[0042] In one scenario, the first model can refer to a trained machine learning model used for semantic segmentation of a 2D image, capable of recognizing different functional regions of an object in the image. The first labeled image refers to the 2D partitioned label map output by the first model, which categorizes pixels in the first image according to different parts; for example, it divides a character's original artwork into "head region," "torso region," and "limbs region," and is a 2D label map with the same resolution as the first image. The second model can refer to a trained machine learning model used to directly generate a complete 3D model from a 2D image. The first 3D model refers to the 3D geometric structure model of the complete object output by the second model, without being segmented, such as a complete character model containing a head, torso, and limbs connected as a single unit.

[0043] In one scenario, the process of obtaining the first identifier image and the first 3D model can involve using a character concept art as the first image and inputting it in parallel into the first and second models. The first model employs a semantic segmentation network to directly predict the first identifier image that distinguishes regions such as the head, torso, and limbs. The second model uses an image-to-3D generative reconstruction network based on a converter model to generate a complete and surface-coherent first 3D model of the character based on the overall concept art.

[0044] In another scenario, the process of obtaining the first identifier image and the first 3D model can also be based on component recognition and voxel reconstruction. For example, a front view of a car is used as the first image input. The first model outputs a two-dimensional first identifier image that distinguishes components such as the car body, tires, and headlights through object detection and instance segmentation. Simultaneously, the second model uses a voxel prediction-based 3D generation network to infer the 3D voxel structure of the entire vehicle from the front view, thereby extracting the complete first 3D model.

[0045] In one scenario, the first identifier image and the first 3D model can be displayed as follows: Figure 4 The interface allows users to select the tool for generating the 3D model and the number of generation attempts in the 3D model display area, and then preview the result. In the first marker image display area, users can configure parameters for the image generation process and modify the displayed marker image to optimize the image partitioning results.

[0046] S230, Obtain the first identification model.

[0047] The first identifier model includes multiple model sub-regions, which are generated based on and associated with image sub-regions.

[0048] The first identification model can refer to a 3D partition identification model, or 3D ID Map, which marks multiple model sub-regions on the surface of the first 3D model, each identified by a different label. Specifically, the partition labels of the first identification model are directly bound to the 3D geometric carrier (triangular mesh) and are 3D geometric attribute data stored on vertices or faces. For example, the surface vertices of the complete content object's 3D model can be labeled as belonging to parts such as "head," "chest," and "left claw."

[0049] A model sub-region can refer to a 3D partition divided within the first identified model. Each partition corresponds to a portion of vertices or faces on the surface of the first 3D model, and different part areas or functional areas on the model are distinguished by different identifier numbers. 3D identifier data can be mapped to colors for visualization rendering. For example, in a mechanical scorpion 3D model, the model sub-region consisting of all vertices belonging to the "tail" corresponds to the first color, and the model sub-region consisting of all vertices belonging to the "head" corresponds to the second color.

[0050] In one scenario, the model sub-region is generated based on the image sub-region, and the association with the image sub-region can mean that each model sub-region corresponds to a certain image sub-region in the first identifier image. That is, the three-dimensional component partition is mapped from the two-dimensional component partition, and the two are semantically consistent.

[0051] For example, the system can input a first identifier image and a first 3D model into a third model. The third model uses the geometric information of the first 3D model as a spatial guiding condition to diffuse the two-dimensional labels of each image sub-region in the first identifier image onto the surface of the 3D model through projection mapping and neighborhood propagation. For example, the label of the "left forefoot" image sub-region in the original image is mapped to the corresponding foot surface vertex on the 3D model, generating a first identifier model containing labels of all components.

[0052] In another scenario, when multiple first-viewpoint labels exist, the third model maps the sub-region labels of each first-viewpoint label image to the corresponding viewpoint surface of the first 3D model, obtaining a preliminary multi-viewpoint labeling result. Then, the third model uses a fusion network to integrate the labeling information from all viewpoints, utilizing the consistency constraints of the 3D surface to eliminate label conflicts between different viewpoints, generating a unified and complete model sub-region partition, forming the first-viewpoint labeling model. Alternatively, the third model can use geometric information such as the surface normals, curvature, and concave feature lines of the first 3D model as auxiliary guiding conditions. When propagating the labels of image sub-regions to the 3D surface, it prioritizes placing the boundaries of model sub-regions in geometrically significant locations, such as the natural concavities at joint connections. The resulting first-viewpoint labeling model will have boundaries between its model sub-regions that naturally conform to the structural transitions of the object.

[0053] In one scenario, the third model can be a pre-trained machine learning model that maps 2D partitioning information onto a 3D model, propagating 2D labels to a 3D surface. In another scenario, the third model can be a generative machine learning network trained with 3D geometry awareness, called a texture generation model. The first identifier image can be used as the main input to the texture generation model, while the first 3D model is used as the geometric guidance input. The texture generation model performs semantic encoding on the first identifier image and geometric feature encoding on the first 3D model. It integrates the partition boundaries and part identifier semantic information carried by the first identifier image with the surface topology and spatial coordinate information of the first 3D model. The model learns to establish the mapping relationship between single-view 2D partition identifiers and the complete surface of the 3D model, inferring the partitioning of the visible area and the occluded area of ​​the first 3D model; predicting the segmentation identifiers corresponding to each triangular facet of the first 3D model; and finally obtaining a 3D spatial partition map attached to the surface of the first 3D model, i.e., the first identifier model.

[0054] S240. Display the second three-dimensional model, which includes multiple first sub-models.

[0055] The first sub-model is generated based on the first 3D model and the first identifier model.

[0056] In one scenario, the second 3D model can refer to a 3D model composed of multiple individual first sub-models. Each first sub-model can refer to a separate component model within the second 3D model, corresponding to a sub-region of a model within the first identifier model. For example, a 3D model of a mechanical scorpion composed of individual component models such as "head," "thorax," "left claw," "right claw," "legs," and "tail," where each component can be selected and edited individually. The second 3D model can be displayed in... Figure 5 The interface shown is used for demonstration purposes.

[0057] In one scenario, 3D model segmentation can be performed directly based on the first representation model. For example, the system inputs the first 3D model and the first identifier model into the fourth model. The fourth model, based on the model sub-region division in the first identifier model, performs cutting operations on the triangular faces of the first 3D model along the boundaries of each sub-region. For example, all faces marked "head" are extracted from the overall model, and the holes generated by the cutting are automatically filled and stitched to generate a closed "head" first sub-model. After all parts are processed in this way, the system assembles the various first sub-models together according to their original spatial positions, displaying them as the second 3D model.

[0058] In one scenario, 3D model segmentation can be achieved based on implicit field separation and reconstruction. For example, the fourth model uses the first 3D model as a geometric reference and the labels of each model sub-region in the first identifier model as guiding conditions to generate an implicit field, such as an occupancy field or an SDF field (Signed Distance Field), for each label. For instance, only positive SDF values ​​are predicted for the spatial region covered by the "tail" label. The corresponding part surfaces are extracted from each implicit field to generate a series of first sub-models, which are then combined and displayed as the second 3D model. Alternatively, the fourth model can transform the first 3D model into a graph structure, using the boundaries of each model sub-region in the first identifier model as cutting line constraints. Optimal cutting is performed along the constraint lines using a graph cut algorithm to separate the individual parts. Local surface smoothing and boundary optimization are performed on each separated part to generate first sub-models with higher geometric quality. Finally, the system presents the second 3D model composed of all the first sub-models in the 3D viewport, and the user can click to select any first sub-model for individual viewing, moving, or exporting.

[0059] The fourth model involved in the above scenarios can be a pre-trained machine learning model that can decompose a single overall geometric model into multiple individual geometric models based on three-dimensional partitioning labels.

[0060] In summary, in one scenario described in this paper, the technical solution can be achieved by receiving at least one first image; displaying a first identifier image and a first 3D model; wherein the first identifier image and the first 3D model are generated based on the first image, the first identifier image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the first 3D model; obtaining a first identifier model, the first identifier model including multiple model sub-regions, the model sub-regions being generated based on the image sub-regions and associated with the image sub-regions; and displaying a second 3D model, the second 3D model including multiple first sub-models, the first sub-models being generated based on the first 3D model and the first identifier model. The solution in this paper solves the problem of insufficient detail and quality degradation caused by resolution limitations when modeling complex objects as a whole in existing technologies. It combines 2D image semantic segmentation with the overall reconstruction of the 3D model, gradually upgrading 2D partitions to 3D and guiding precise model decomposition, providing higher processing resolution opportunities for each component model and texture, automatically deriving the decomposition logic and processing each component separately, significantly reducing the cost of subsequent manual correction and asset reconstruction.

[0061] Based on the above scenarios, this paper also provides a scenario in which the system can receive a first operation, wherein the first operation is used to modify the first identification image; and display a second identification image, wherein the second identification image is obtained based on the modification.

[0062] The first operation can refer to user modifications to the first identifier image, such as using a brush to expand the area of ​​a sub-region, erasing incorrectly defined boundary pixels with an eraser tool, or performing batch corrections based on semantic instructions. The second identifier image is a more accurate two-dimensional partition identifier image obtained after modifications based on the user's first operation. For example, after a user redraws the originally blurry boundary between the "left foreleg" and the "thorax," a more accurate partition identifier image is generated. Specifically, in one scenario, after the system displays the first identifier image of a mechanical scorpion, the user notices that some pixels in the "left claw" sub-region have been incorrectly assigned to the "thorax" sub-region. The user selects the brush tool, upsamples the color of the "left claw" sub-region, and then paints over the incorrectly assigned pixel area. The system recognizes this painting trajectory as the first operation, updates the label value of the painted pixels to the label value of the "left claw," and refreshes the display of the modified second identifier image in real time. In another scenario, the system can overlay the boundary lines of each image sub-region onto the first identifier image. The user noticed that the boundary line between the "tail" and "abdomen" did not follow the object's natural contour. They selected and dragged this boundary line to move it to the correct position. The system recognized this dragging operation as the first operation, reclassified the pixel assignments on both sides based on the new boundary position, and generated a second identifier image with a more fitting boundary. In another scenario, the user could enter the correction command "Merge the right middle foot and right hind foot into one region" in the text box of the modification panel. The system recognized this text input as the first operation, analyzed the semantics of the command, searched for the corresponding image sub-regions for "right middle foot" and "right hind foot" in the identifier image, unified their labels with the new merge label, and refreshed the display to show the merged second identifier image.

[0063] Based on the above scenarios, this paper also provides a scenario where the system can display multiple first identification images, each corresponding to a different projected image of a first 3D model; receive a first operation on the multiple first identification images, wherein the first operation is used to modify the first identification images; and display a second identification image in response to a second operation, wherein the second identification image is obtained based on multiple modifications. The display of the second identification image in response to the second operation may include displaying first information in response to the second operation, the first information being generated based on multiple first operations; and displaying the second identification image in response to a third operation on the first information.

[0064] Here, multiple first identifier images can refer to multiple two-dimensional partition identifier images corresponding to different projection directions of the first 3D model. For example, the system projects and renders the first 3D model from three directions: front, side, and top, and generates a corresponding first identifier image for each projection image. Multiple first identifier images can also be generated based on multiple first images received from different perspectives. The first operation can refer to the modification operation performed by the user on multiple first identifier images with different projection directions or perspectives. The modifications on each image may be consistent, complementary, or conflicting with each other.

[0065] The second operation can refer to the user's action of triggering the generation of the final second identification image, such as clicking the "Synthesize and Modify" button. The first information can refer to the system's summary of modifications or conflict prompts generated after summarizing multiple first operations, i.e., the summary of modifications made from different perspectives. For example, the first information can be presented in the form of a list, with a prompt stating "The front view assigns area A to the left forefoot, and the side view assigns area A to the chest, indicating a conflict." The third operation can refer to the user's confirmation or adjustment of the first information, such as selecting which view's modification to follow on the conflict prompt interface, or manually specifying the final assignment of the conflicting area. Correspondingly, the second identification image is the final, consistent two-dimensional partition identification map generated after integrating modifications from multiple projection directions and resolving conflicts.

[0066] In one scenario, the system can obtain a second identification image based on the summary of modifications made to multiple views and conflict alerts. For example, the system displays a first identification image for the front, side, and top projections of a mechanical scorpion. The user expands the area of ​​the "left claw" in the front view and the side view, but leaves the top view unchanged. The user clicks the "Merge Modifications" button, triggering a second operation. The system summarizes the modifications made to the three images, finds that the modifications in the front and side views target the same area and are in the same direction, and automatically merges them; the unmodified parts in the top view remain unchanged. The system displays the first message "Modifications in the front and side views have been merged, no conflict." After the user confirms and clicks "Apply," a third operation is triggered, and the system generates the merged second identification image.

[0067] In one scenario, the system can ultimately obtain a second labeling image based on conflict detection and user decision-making. For example, a user might assign a boundary region to "chest" on a first labeling image in a frontal view and to "left forefoot" in a side view. After the user triggers a second action, the system detects a label conflict in that region, generates first information, and displays the conflicting region in a list and view highlight format, labeled "Covered as chest in frontal view, labeled as left forefoot in side view." After reviewing this, the user can select "Use side view as reference" through a third action, and the system resolves the conflict accordingly, generating a second labeling image.

[0068] In one scenario, the system can intelligently synthesize a second label image based on voting and automatic fusion. For example, a user modifies the first label image in three projection directions, triggering a second operation. The system maps the modifications in each projection direction back to the surface of the 3D model and counts the label voting results from different views for each facet. Regions with consistent votes are automatically adopted, while regions with divergent votes generate conflict warnings as the first piece of information. The user then uses a third operation to adjudicate a small number of conflicting regions one by one. After confirmation, the system generates a second label image that fuses all modifications and is conflict-free.

[0069] Furthermore, after obtaining the modified second identifier image, a first identifier model can be obtained. Accordingly, the first identifier model includes multiple model sub-regions, which are generated based on and associated with image sub-regions in the second identifier image.

[0070] Figure 6 This document provides a flowchart illustrating another image processing method, which belongs to the same inventive concept as the above scenario. Based on the above scenario, it further describes the processing procedure for modifying the first identifier model. This method can be executed by an image processing device, which can be implemented in software and / or hardware, optionally through an electronic device, such as a mobile terminal, PC, or server.

[0071] like Figure 6 As shown, in one scenario, the image processing method may specifically include the following steps: S310, Receive at least one first image.

[0072] S320, Display the first identification image and the first three-dimensional model.

[0073] The first identifier image and the first three-dimensional model are generated based on the first image. The first identifier image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the region of the first three-dimensional model.

[0074] In one scenario, the process of obtaining a first 3D model based on a first image may include the following steps: First, in response to a first interactive operation, a second model is obtained; the first interactive operation is an operation that interacts with a preset list of models. For example, in Figure 4 The system retrieves a list of preset models using a model selection control and allows users to make a selection.

[0075] The first interactive operation can refer to the user's interaction with the list of preset models displayed on the system interface, such as mouse click, touch screen selection, or voice command.

[0076] The preset model list can refer to a list of multiple 3D generative models that the system has pre-trained and registered, which are available for users to choose from. Each entry in the list represents a second model with different characteristics.

[0077] In one scenario, the system can render a drop-down list control in the model selection area of ​​the user interface, displaying options such as Model 1, Model 2, and Model 3. When the user moves the mouse cursor to the "Model 3" item and clicks, the system recognizes this click event as the first interactive operation, obtains the identifier of the selected model, and loads the corresponding second model accordingly.

[0078] Second, in response to the second interactive operation, the number of model calls is obtained; wherein, the second interactive operation is an interaction with the first control, and the first control is used to input the number of model calls. The first control can be... Figure 4 The control for the number of times a card is drawn is named "number of draws" because it corresponds to the card-drawing phenomenon generated by the model.

[0079] In one scenario, the second interactive operation can refer to the user's interaction with the first control on the system interface used for inputting numerical values, such as typing a number in an input box, dragging a slider, or clicking an increase or decrease button.

[0080] The first control refers to the UI (User Interface) component on the interface that receives the number of times the user inputs the model is called, such as a numeric input box, a slider, or a stepper.

[0081] The number of model calls can refer to the number of times the system will repeatedly call the second model to perform the 3D generation task, as specified by the user.

[0082] Third, based on the number of model calls, the first image is repeatedly input into the second model to obtain multiple 3D model generation results; the number of 3D model generation results is the same as the number of model calls.

[0083] The generated 3D model can refer to a complete 3D model output each time the second model is called. The generated models from different calls may have random differences in geometric details.

[0084] In one scenario, repeatedly inputting the first image into the second model to obtain multiple 3D model generation results can be based on a sequential generation process using loop calls. For example, after the system obtains the model call count "5", it enters a loop structure. Within the loop, the system keeps the weights of the second model and the first image unchanged, but resets the random number seed before each call, inputs the first image into the second model, and obtains one 3D model generation result. After the loop executes 5 times, the system collects 5 different generation results.

[0085] In one scenario, repeatedly inputting the first image into the second model to obtain multiple 3D model generation results can be a parallel generation process based on batch inference. The system constructs an input tensor with a batch size of 3 based on the model call count "3", containing 3 copies of the first image. This batch of data is then fed into the second model deployed on the GPU (Graphics Processing Unit) for parallel inference. The second model utilizes its batch processing capabilities to simultaneously output 3 3D model generation results, which the system then splits and stores separately.

[0086] Fourth, in response to the third interactive operation, a first three-dimensional model is obtained; wherein the first three-dimensional model is one of the multiple three-dimensional model generation results.

[0087] The third interactive operation can refer to the user selecting one of the multiple 3D model generation results displayed by the system as the final output, such as clicking a thumbnail or clicking the confirmation button next to the result to obtain the first 3D model to be displayed.

[0088] S330. Obtain the first identifier model, which includes multiple model sub-regions.

[0089] The model sub-regions are generated based on and associated with the image sub-regions.

[0090] S340, Display the first identification model.

[0091] The first identification model is the model defined in S330, which refers to the three-dimensional partition identification model. Its surface can be marked with different colors to identify each model sub-region. For example, the surface of the three-dimensional model of the content object is marked with blue for "first sub-region", green for "second sub-region", and red for "third sub-region", etc.

[0092] In one scenario, the system can load the first identifier model into the central 3D viewport and display it as a semi-transparent overlay on top of the original first 3D model. The model sub-regions are outlined with prominent boundary lines, and the user can rotate, zoom, and pan the viewpoint to observe the coverage and boundary orientation of each model sub-region on the 3D surface from any angle. In another scenario, the system displays a component list next to the 3D viewport, listing the names of all model sub-regions. When the user clicks on any name in the list, that model sub-region is highlighted in the 3D viewport, while the remaining areas become semi-transparent, allowing the user to check the accuracy of each partition individually.

[0093] In one scenario, the first identifier model can be displayed on the interface together with the first identifier image, or the first identifier model can be displayed on the interface together with a modified second identifier image. This visually demonstrates the correspondence between the image sub-regions in the first or second identifier image and the model sub-regions of the first identifier model.

[0094] In one scenario, the first identifier model can be displayed as follows: Figure 7 The interactive interface shown is displayed in the 3D identification model display area.

[0095] S350, Receive a fourth operation, wherein the fourth operation is used to modify the first identifier model.

[0096] The fourth operation can refer to the user's modification operation on the first identifier model, for example, through... Figure 7 The representation model modification tool shown can be used to modify the model. You can use a brush to paint on the surface of the 3D model to adjust the boundaries of the model sub-regions, or select two adjacent model sub-regions to merge.

[0097] In one scenario, the fourth operation can be a surface smearing correction operation based on a 3D brush. For example, a user notices that the dividing boundary between "left foreleg" and "chest" on the first-labeled model does not follow the joint recess but instead appears as a diagonal line. The user selects the 3D brush tool and smears along the correct joint recess position on the model surface, relabeling some vertices that were incorrectly labeled as "chest" as "left foreleg." The system recognizes this smearing trajectory as the fourth operation, updates the labels of the smeared vertices in real time, and refreshes the partition display on the model surface.

[0098] In one scenario, the fourth operation can be merging and splitting based on region selection. For example, a user finds that the "tail" area is overly segmented into three fragmented model sub-regions and wants to merge them into one. The user holds down the shortcut key and clicks on each of the three sub-regions in sequence to select them, then clicks the "Merge Regions" button. The system recognizes this operation as the fourth operation, unifies the labels of the three sub-regions into a single "tail" label, and completes the merge modification.

[0099] S360, Display the second identifier model, which is obtained based on the modification.

[0100] The second identification model refers to a more accurate 3D partition identification model obtained after modification based on the user's fourth operation. For example, after the user corrects the boundary of "left forefoot" to the joint depression, the generated model has a more accurate 3D identification model for sub-region division.

[0101] In one scenario, the display of the second identifier model can be based on real-time updates of modification operations. For example, during the user's fourth operation, the system continuously captures every brush stroke, region selection, or boundary dragging event, updates the label values ​​of the affected vertices in real time, and instantly refreshes the model surface color in the 3D viewport. Once the user completes all modifications and clicks the "Confirm Modifications" button, the system saves the current partition state and displays it as the second identifier model.

[0102] In one scenario, displaying the second identifier model can also be based on a status retrospective confirmation of the modification history. The system records each of the user's fourth actions as a modification step, displaying a list of modification history in the sidebar of the interface. After completing a series of adjustments, the user can review the effects of each step, and once confirmed, click "Apply All Modifications." The system then merges all modification steps, generating and displaying the final second identifier model.

[0103] Alternatively, in one scenario, after the user completes the modifications, the system can provide a split-screen comparison view. The left screen displays the first identifier model before modification, and the right screen displays the modified effect. The user can simultaneously rotate the two views to compare area by area. After confirming that the modifications are correct, the user clicks "Confirm," and the system officially saves the modified result as the second identifier model and displays it in full screen.

[0104] S370. Display the second three-dimensional model, which includes multiple first sub-models. The first sub-models are generated based on the first three-dimensional model and the second identification model.

[0105] The second 3D model refers to a 3D model composed of multiple first sub-models. For example, a 3D model of a mechanical scorpion is composed of parts such as the "head," "thorax," "left claw," "right claw," "legs," and "tail." Each part can be selected and edited individually. The first sub-model refers to the individual component models in the second 3D model. Each first sub-model corresponds to a model sub-region in the second identifier model. For example, the "left claw" part model with a complete closed surface is separated from the complete mechanical scorpion model.

[0106] In one scenario, the system can input the first 3D model and the second identifier model into a fourth model. The fourth model, based on the user-modified sub-region divisions in the second identifier model, performs a cutting operation on the triangular faces of the first 3D model along the precise boundaries of each sub-region. For example, all faces marked "head" are extracted from the overall model along the modified boundaries, and the resulting holes are automatically filled and stitched to generate a closed "head" first sub-model. After all components are processed in this way, the system assembles the various first sub-models according to their original spatial positions, displaying them as the second 3D model.

[0107] In summary, in one scenario described herein, the technical solution can be achieved by receiving at least one first image; displaying a first identifier image and a first 3D model; wherein the first identifier image and the first 3D model are generated based on the first image, the first identifier image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the first 3D model; obtaining a first identifier model, the first identifier model including multiple model sub-regions, the model sub-regions being generated based on the image sub-regions and associated with the image sub-regions; displaying the first identifier model; receiving a fourth operation, wherein the fourth operation is used to modify the first identifier model; displaying a second identifier model, the second identifier model being obtained based on the modification; and displaying a second 3D model, the second 3D model including multiple first sub-models, the first sub-models being generated based on the first 3D model and the second identifier model. The solution in this paper solves the problem of insufficient detail and quality degradation caused by resolution limitations when modeling complex objects as a whole in the prior art. It adds a display and manual modification step after the 3D identifier model is generated, allowing users to correct the automatically segmented component boundaries based on intuitive 3D geometric judgment, ensuring that the 3D segmentation results conform to the actual structure and manufacturing specifications. This collaborative model of "automatic generation and manual refinement" retains the efficiency of automated processes while giving users the final say on the division of 3D components. This makes the subsequent model disassembly results more accurate, avoids downstream rework caused by segmentation errors, and significantly improves the efficiency and quality of 3D asset production.

[0108] Figure 8 This paper presents a flowchart illustrating another image processing method, which belongs to the same inventive concept as the above scenario. Building upon the above scenario, it further describes the process of obtaining a refined 2D original drawing of a model part during the generation of 3D assets based on a machine learning model. This method can be executed by an image processing device, which can be implemented in software and / or hardware, optionally through an electronic device such as a mobile terminal, PC, or server.

[0109] like Figure 8 As shown, in one scenario, the image processing method may specifically include the following steps: S410, Receive at least one first image.

[0110] S420, Display the first identification image and the first three-dimensional model.

[0111] The first identifier image and the first three-dimensional model are generated based on the first image. The first identifier image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the region of the first three-dimensional model.

[0112] S430, Obtain the first identification model.

[0113] The first identifier model includes multiple model sub-regions, which are generated based on and associated with image sub-regions; S440, Display the second 3D model.

[0114] The second three-dimensional model includes multiple first sub-models, which are generated based on the first three-dimensional model and the first identifier model.

[0115] S450: Display multiple second images, each of which is associated with an image sub-region.

[0116] The second image is generated based on the first image and the first sub-model.

[0117] A second image can refer to a detailed two-dimensional image associated with a specific sub-region of an image, with each second image corresponding to a specific component. For example, a high-resolution two-dimensional original drawing showing only the details of the "left claw" component of a mechanical scorpion could contain rich information such as pincer texture, rivet positions, and metal scratches.

[0118] The first sub-model, as defined in S440, can refer to the individual component models separated from the first 3D model. It can serve as a geometric projection guide for generating the second image, ensuring that the generated 2D texture or image can be precisely aligned with the surface of the model without texture distortion.

[0119] In one scenario, obtaining the second image can be based on UV unwrapping and texture regeneration of the component model. Here, U represents the horizontal axis of the two-dimensional texture space, and V represents the vertical axis. Texture unwrapping can also be called UV display. For example, the system inputs the first sub-model of the "left claw" and the original first image into the fifth model. The fifth model first performs UV unwrapping on the first sub-model to obtain its two-dimensional texture space. Then, combining the color and texture style of the "left claw" image sub-region in the first image, it uses a diffusion model to redraw on the UV unwrapped image, generating a second image without background interference, rich in texture detail, and containing only the left claw component. Other components are processed in the same way, and the system displays thumbnails of all generated second images in the interface according to the component list.

[0120] In one scenario, obtaining the second image can be based on differentiable rendering and detail restoration. For example, the fifth model, guided by the geometric normals and depth information of the first sub-model, generates a rough view map of the part using a differentiable renderer. Then, using an image restoration network and combining the real texture features of the corresponding image sub-regions in the first image, super-resolution and detail enhancement are performed on the rough map to generate a high-fidelity second image. The system repeats this process for each part and displays the results one by one.

[0121] In the above scenario, the fifth model can be a pre-trained machine learning model capable of rendering or generating corresponding 2D texture images based on the 3D model, or a model that aligns and fuses the spatial information of the 3D model with the 2D image information. The second image is the 2D image output by the fifth model that precisely corresponds to each of the first sub-models; it can also be understood as a more refined 2D original artwork corresponding to the substructures in the first 3D model. For example, it could be an unfolded UV map containing only the "head" component, or a high-resolution view of the "head" component rendered from a specific perspective.

[0122] In one scenario, the second image can be displayed in Figure 7 The area shown displays the original artwork images of the parts. The original artwork images are compared with their corresponding labeled models.

[0123] S460: Display the third 3D model, which consists of multiple second sub-models.

[0124] The second sub-model is generated based on the second image, the first sub-model, and the first spatial positional relationship; the first spatial positional relationship is the relative positional relationship between the first sub-model and the second three-dimensional model.

[0125] A third 3D model can refer to a final 3D model that is refined and enhanced by combining multiple second sub-models. For example, a mechanical scorpion model is a 3D visual content object with fine geometric details and assembly of all parts.

[0126] The second sub-model can refer to a more detailed component model obtained by reconstructing the first sub-model. For example, an enhanced component model could be created by adding fine geometric structures such as pincer teeth, hydraulic rod threads, and joint rivets to the original "left claw" first sub-model. The first spatial positional relationship can refer to the relative positional relationship of the first sub-model in the second 3D model. For example, it could be a transformation matrix that records the translation, rotation, and scaling parameters of the "left claw" component relative to the mechanical scorpion's "thorax," or its bounding box information in the global coordinate system.

[0127] In one scenario, the process of generating the second sub-model can be to input the second image, the first sub-model, and the first spatial position relationship into the first machine learning model to obtain the second sub-model; wherein the second image is the main input, and the first sub-model and the first spatial position relationship are the conditional inputs.

[0128] The first machine learning model can refer to a trained machine learning model used for refined reconstruction of the component model. It can generate a detailed component model by comprehensively referencing the refined image of the component, the basic geometric model of the component, and the spatial position of the component in the whole. Here, the second image serves as the main input to provide guidance on texture and surface details, while the first sub-model and the first spatial positional relationship serve as conditional inputs to provide basic geometric constraints and spatial context constraints.

[0129] The first machine learning model can be a pre-trained machine learning model used for fine reconstruction of the 3D model of individual parts. It can comprehensively refer to the part model, part texture and its spatial position in the original whole to generate a part model with enhanced details.

[0130] The second sub-model refers to the individual component 3D model output by the first machine learning model, which has been enhanced and refined in detail. For example, based on the first sub-dimensional model of the "robotic arm", a model with fine geometric details such as rivets at the joints and cable grooves has been added.

[0131] The first sub-model and the model spatial location information serve as guiding conditions for generating the second sub-model. They can provide the basic shape of the components and the spatial context constraints provided by the model spatial location information during the generation of the first machine learning model. Together, they guide the refinement process.

[0132] In one scenario, the process of the first machine learning model generating the second sub-model specifically includes the following steps: First, a first encoder based on a first machine learning model encodes the features of the second image to obtain the first feature.

[0133] Second, the second encoder, based on the first machine learning model, encodes the features of the first sub-model to obtain the second feature.

[0134] Third, the third encoder, based on the first machine learning model, performs feature encoding on the first airdrop location relationship to obtain the third feature.

[0135] Fourth, the conditional flow matching module based on the first machine learning model performs feature fusion on the first feature, the second feature, the third feature and the initial latent feature to obtain the first feature fusion result; wherein, the initial latent feature is obtained based on the second three-dimensional model, or the initial latent feature is obtained based on random Gaussian latent vectors.

[0136] Fifth, the decoder based on the first machine learning model decodes the first feature fusion result to obtain the second sub-model.

[0137] The first, second, and third steps can be performed simultaneously.

[0138] In one scenario, the first encoder can refer to the neural network in the first machine learning model used to extract appearance and texture features from the second image, such as a convolutional neural network or a visual converter model. The first feature refers to the feature vector or feature map output by the first encoder that characterizes information such as color, texture, and material of the second image, for example, features containing information about metal scratches and stains on the surface of a robotic arm.

[0139] The second encoder can refer to the neural network in the first machine learning model used to extract geometric structural features from the second 3D model, such as a graph neural network or a 3D convolutional network. The second feature refers to the feature vector or feature map output by the second encoder that characterizes information such as the basic shape and surface curvature of the component, such as features containing geometric information of the spherical groove at the joint of the robotic arm.

[0140] The third encoder can refer to the network in the first machine learning model used to encode the spatial position information of the model, typically a multilayer perceptron or position encoding layer. The third feature refers to the feature vector output by the third encoder that represents the spatial position and orientation of the component within the whole, such as encoding the pose matrix of a robotic arm into a high-dimensional spatial feature.

[0141] The conditional flow matching module can refer to the core generation module of the first machine learning model. Based on the flow matching generation paradigm, it gradually transforms simple distributions into complex distributions and is guided by various conditional features to achieve high-quality 3D shape generation.

[0142] The initial latent features can refer to the initial three-dimensional latent variables input to the conditional flow matching module, or they can be low-precision geometric features extracted from the second three-dimensional model, or vectors randomly sampled from a Gaussian distribution, serving as the starting point for the generation process.

[0143] The first feature fusion result refers to the fused feature obtained by the conditional flow matching module after iteratively updating the initial latent features under the guidance of multiple conditions. It contains all the geometric and texture information required for the final refined model.

[0144] A decoder can refer to a network in the first machine learning model that is used to transform fused features into a final 3D model, such as outputting a high-resolution SDF field or mesh model.

[0145] In one scenario, the first encoder uses a convolutional network to encode the second image of the robotic arm, obtaining the first texture feature. The second encoder uses a 3D sparse convolutional network to encode the first sub-model, obtaining the second geometric feature. The third encoder uses a positional encoding network to map the first spatial positional relationship into a high-dimensional third feature. Then, the initial latent features obtained based on the voxelization of the first sub-model, along with the first, second, and third features, can be input into a conditional flow matching module. This module, guided by the three conditional features and using an ordinary differential equation solver, progressively transforms the initial latent features into a first feature fusion result. The decoder uses a 3D deconvolutional network to decode this result, generating a high-resolution SDF field from which a refined second sub-model is extracted.

[0146] In another scenario, the first encoder uses a visual transformer to extract first features from the second image. The second encoder uses a point cloud model to extract second geometric features from the point cloud sampled from the first sub-model. The third encoder uses a multilayer perceptron to encode spatial location information to obtain third features. A latent vector is randomly sampled from a Gaussian distribution as an initial latent feature, which, along with the three conditional features, is input into a conditional flow matching module. This module, through a flow network based on a transformer model, transforms the Gaussian noise into a structured first feature fusion result under conditional guidance. The decoder directly predicts the coordinate offset of the dense point cloud, optimizing the coarse point cloud into a second sub-model with rich details.

[0147] By extracting image texture, geometric shape and spatial location features by three separate encoders, and then using a conditional flow matching module for deep fusion, the generation process can simultaneously consider the surface details of the part, its own structure and its spatial relationship with the whole, thus generating a refined part model with high geometric consistency and strong detail expression.

[0148] Each second sub-model corresponds to a second image.

[0149] The third 3D model can be a complete, high-precision final 3D model obtained by combining and assembling all the refined individual component models according to their respective spatial position information, such as a complete mechanical scorpion model in which all components have fine details and fit together perfectly.

[0150] In one scenario, the process of combining each second sub-model to obtain the third 3D model can be a direct assembly process based on a spatial position matrix. For example, the system can read the spatial position information of each second sub-model used during its generation, such as the translation, rotation, and scaling matrices of the robotic arm component relative to the vehicle body. It can then directly apply this transformation matrix to transform the robotic arm model to the correct position in the vehicle body coordinate system, assemble all components one by one, and combine them into the third 3D model.

[0151] In one scenario, the process of combining each second sub-model to obtain the third three-dimensional model can be a combination assembly process based on constraint solving. The system can treat each second sub-model as a rigid body and use the spatial position information between them to define assembly constraints.

[0152] In one scenario, multiple second images and a third 3D model can be displayed on the same interface. Since each second sub-model in the third 3D model is generated based on a second image, multiple second images and the third 3D model can be displayed on the same interface. If the generated result of one of the second sub-models is unsatisfactory, the corresponding second image can be selected to regenerate a new second sub-model.

[0153] In summary, in one scenario described herein, the technical solution can be achieved by receiving at least one first image; displaying a first identifier image and a first three-dimensional model; wherein the first identifier image and the first three-dimensional model are generated based on the first image, the first identifier image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the first three-dimensional model; obtaining a first identifier model, the first identifier model includes multiple model sub-regions, the model sub-regions are generated based on the image sub-regions and associated with the image sub-regions; displaying a second three-dimensional model, the second three-dimensional model includes multiple first sub-models, the first sub-models are generated based on the first three-dimensional model and the first identifier model; displaying multiple second images, each second image is associated with an image sub-region, the second images are generated based on the first image and the first sub-models; and displaying a third three-dimensional model, the third three-dimensional model is composed of multiple second sub-models, the second sub-models are generated based on the second images, the first sub-models, and a first spatial positional relationship; the first spatial positional relationship is the relative positional relationship between the first sub-models and the second three-dimensional model. The proposed solution addresses the issue of insufficient detail and quality degradation caused by resolution limitations in the overall modeling of complex objects in existing technologies. Based on a workflow of 2D segmentation, 3D decomposition, individual generation, spatially-aware refinement, and precise assembly, the solution breaks down complex objects into individual components, generating models and textures for each component at high resolution. This systematically overcomes the resolution bottleneck of overall modeling, resulting in richer details for each component. Furthermore, the final refinement step incorporates spatial positional information to guide the assembly, ensuring natural connection and fit between the enhanced components, preventing misalignment and fragmentation, and ultimately improving the overall quality of generated complex 3D models.

[0154] In an application example of the image processing methods described above, a 3D asset intelligent generation tool is provided. This tool can be a desktop-based 3D asset intelligent generation tool for game asset artists or other users who perform 3D visual content processing. The tool encapsulates the aforementioned technical solutions into a three-step guided workflow, allowing users to complete the entire production process from concept art to the final high-precision 3D model through an interactive interface. The following section uses the generation of a game object asset as an example to illustrate the tool's usage and internal implementation in detail.

[0155] Step 1: Generating the original artwork After launching the tool, it defaults to the first step's introductory page. The interface includes a resource import area, where users can drag and drop or click the "Import Reference Image" button to import a hand-drawn sketch of a game object. For example, the game object could be a mechanical scorpion. The right side of the interface features a prompt input area with text input boxes and preset style labels for users to choose from. A progress bar can also be displayed at the top of the interface; for example, currently highlighted as "Step 1: Concept Art Generation".

[0156] Suppose a user imports a pencil sketch of a game object as a reference image and enters "high-precision game object, steampunk style, metal shell, hydraulic joints, brass and cast iron material, front view, concept design" in the prompt box. The user selects the "hard surface mechanical" tab from the style label bar as a supplementary condition.

[0157] After receiving the reference image and prompt, the system invokes the built-in image-generated model based on the user's selection. The system uses the reference image as initial guidance and the prompt as generation conditions. The image-generated model, while preserving the reference composition, performs detail filling and stylized rendering based on descriptions such as "steampunk," "hydraulic joints," and "brass material." After multiple iterations of noise reduction, the model outputs a high-precision concept art image of the game object. The generated result is displayed in the preview area in the center of the interface. If the user is not satisfied with the initial result, they can click the "Regenerate" button to adjust the prompt and generate again until a satisfactory concept art image is obtained. Clicking "Confirm Concept Art" proceeds to the next step.

[0158] Of course, you can also directly import the original artwork used to generate the 3D visual content, corresponding to at least one first image described in the above situations.

[0159] Step 2: Rough 3D Model and Part Disassembly Upon entering the second step, the layout of the interactive interface changes, allowing the simultaneous display of the first identifier image and the first 3D model. For example, the center of the interface features a preview viewport of the 3D model, supporting rotation, scaling, and panning operations. The left panel is the "2D ID Editing Area," displaying a preview of the first identifier image (2D ID Map) and a list of editable partitions. The right panel is the "Model Generation Settings Area," providing options for model selection and setting the number of generation attempts. A toggle label for the "3D ID Editing Area" is located in the lower left corner.

[0160] First, in the "Preset Model List" drop-down menu on the right panel, the user selects "Hard Surface Mechanical Specific Model" as the second model through the first interactive operation. Then, in the "Number of Generations" input box, the user enters the number "3" through the second interactive operation. Based on the model call count of 3, the tool repeatedly inputs the first image into the selected second model, obtaining three slightly different 3D geometric model generation results. These three results are arranged in thumbnail form in the "Historical Results" area. The user previews and compares them one by one through the third interactive operation, and finally clicks to select the model with the clearest joint structure and the most harmonious overall proportions.

[0161] Simultaneously, the tool inputs the first image into the first model. The first model performs semantic segmentation on the original image and outputs a first identifier image, on which different colors are used to mark image sub-regions such as the head, thorax, abdomen, left claw, right claw, left foreleg, right foreleg, left middle leg, right middle leg, left hind leg, right hind leg, and tail. The system displays the first identifier image in the left panel of the interface and displays the first 3D model selected by the user side by side in the central 3D viewport. When the user clicks on any image sub-region on the left, the corresponding surface area on the first 3D model is simultaneously highlighted, intuitively showing the relationship.

[0162] Subsequently, the user discovered that some pixels at the boundary between the "left middle foot" and the "chest" in the first identification image were incorrectly divided. The user selected the brush tool in the left panel and painted over the first identification image to correct the incorrectly divided pixels, re-marking them as the correct sub-regions. The system received this painting operation as the first operation, updated the boundaries of the sub-regions in the image in real time, and displayed the corrected second identification image.

[0163] Based on the second identifier image, the first identifier model can be obtained and displayed.

[0164] In one scenario, after confirming the two-dimensional partitioning is correct, the user clicks "Generate 3D ID". The tool inputs the second identifier image and the first 3D model into the third model. The third model uses the geometric information of the first 3D model as spatial guidance conditions, and diffuses the labels of each image sub-region in the second identifier image onto the surface of the 3D model through projection mapping and neighborhood propagation to obtain the first identifier model, i.e., the 3D ID Map.

[0165] The system displays the first identified model in the central 3D viewport, with different colors used to mark each sub-region. If the user rotates the viewport and finds that the dividing boundary between the "left forefoot" and the "chest" does not follow the natural concavity at the joint, the user selects the 3D brush tool in the 3D viewport and paints along the joint concavity to correct it. The system receives this painting operation as the fourth operation and updates the labels of the painted vertices in real time. After the user confirms all modifications and clicks "Apply Modifications," the system displays the corrected second identified model.

[0166] The second identifier model can be further used to obtain and display a second three-dimensional model.

[0167] When the user clicks "Disassemble Model," the tool inputs the first 3D model and the second identifier model into the fourth model. Based on the user-modified sub-region divisions in the second identifier model, the fourth model cuts, fills, and stitches the first 3D model along the boundaries of each sub-region, generating multiple first sub-models. These sub-models are then assembled and displayed as the second 3D model according to their original spatial positions. The user can select any first sub-model in the viewport to view it.

[0168] The user selects all parts for which detailed images need to be generated and clicks "Generate Part Images." The tool then imports each first sub-model and its first image into the fifth model. The fifth model generates a unique second image for each part; for example, it generates a high-resolution image showing the details of the jaws, hydraulic rods, and rivets for the "left claw." These second images are displayed in the left panel along with their corresponding part names. The user can right-click on any image thumbnail and select "Export" to save it as an image file for use in other projects.

[0169] The above process integrates automatic segmentation, 2D correction, 3D correction, and model decomposition based on machine learning models into a smooth interactive pipeline. Users can intervene and adjust the partitioning results in both 2D and 3D dimensions to ensure that the boundaries of each component conform to manufacturing specifications. At the same time, component-level refined images can be exported and reused separately, significantly improving pipeline flexibility and asset utilization efficiency.

[0170] Step 3: Refined 3D Model of Parts and Final Assembly In the third step, the tool interface switches to the "Part Assembly Workbench" mode. The central 3D viewport simultaneously displays all the first sub-models, the left side shows the part list, and each part name displays a thumbnail of the second image generated in the second step. The right side is the refinement parameter panel. At the bottom are buttons for two execution modes: "Enhance All" and "Enhance Part by Part".

[0171] Users can select components one by one from the list on the left and set enhancement parameters, or they can directly click the "Enhance All" button to process them in batches with the default settings.

[0172] Taking the "left claw" component as an example, the tool takes the second image of the left claw as the main input, and the first sub-model of the left claw and the first spatial position relationship (recording the pose transformation matrix of its assembly on the left side of the chest) as conditional inputs, and feeds them into the first machine learning model.

[0173] The first machine learning model internally performs a refined generation process, using the second image as the main input and extracting texture and surface detail features as the primary generation guide. It uses the first sub-model as the basic geometric constraint and the first spatial positional relationship as the spatial context constraint, both serving as conditional inputs. Under the guidance of the main input's details and the spatial constraints of the conditional inputs, the conditional flow matching module iteratively fuses and optimizes the initial latent features, ultimately decoding and generating the second sub-model. This new model adds fine geometric details such as the serrated edges of the pincers, the threads of the hydraulic rods, and the rivets at the joints, while naturally matching the interface structure of the chest. All components undergo refined processing step-by-step according to this process.

[0174] Once all components are refined, the user clicks the "Assemble" button. The tool reads the first spatial positional relationship carried by each second sub-model, assembles multiple refined second sub-models into the same coordinate system according to their respective transformation matrices, and automatically fine-tunes the tiny gaps between adjacent components to obtain the final third 3D model—a complete mechanical scorpion model with rich details in all components and precise, well-fitting joints.

[0175] Users can freely rotate the viewport in the final preview to observe the finished product. After confirming that everything is correct, click "Export Assets" to export the 3D model and the texture maps of its various components in a standard format for use in the game engine.

[0176] This tool employs a three-step guided workflow, breaking down the complex model-based 3D generation pipeline into three stages: concept art generation, part decomposition and refinement, and final assembly. Each stage provides intuitive visual feedback and flexible human intervention interfaces, significantly lowering the barrier to entry for artists. The tool allows users to make collaborative decisions at key nodes such as 2D segmentation, 3D segmentation, and model optimization, retaining the automation efficiency of model-based systems while ensuring the final output fully meets professional production standards. Furthermore, the ability to export refined concept art for individual parts allows intermediate assets generated by the tool to be flexibly reused across different pipelines, significantly improving the overall efficiency and utilization rate of 3D asset production. This tool can be extended directly from simple and medium-sized objects to complex models, resulting in accurate and richly detailed 3D models.

[0177] Figure 9 This paper presents a schematic diagram of an image processing device applicable to the generation of 3D content based on machine learning models.

[0178] like Figure 9 As shown, the image processing device includes: a first module 510, a second module 520, a third module 530, and a fourth module 540.

[0179] The first module 510 is used to receive at least one first image; the second module 520 is used to display a first identifier image and a first three-dimensional model; wherein the first identifier image and the first three-dimensional model are generated based on the first image, the first identifier image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the first three-dimensional model; the third module 530 is used to obtain a first identifier model, the first identifier model includes multiple model sub-regions, the model sub-regions are generated based on the image sub-regions and are associated with the image sub-regions; the fourth module 540 is used to display a second three-dimensional model, the second three-dimensional model includes multiple first sub-models, the first sub-models are generated based on the first three-dimensional model and the first identifier model.

[0180] In one scenario described herein, the technical solution can receive at least one first image; display a first identifier image and a first 3D model; wherein the first identifier image and the first 3D model are generated based on the first image, the first identifier image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the first 3D model; obtain a first identifier model, the first identifier model including multiple model sub-regions, the model sub-regions being generated based on the image sub-regions and associated with the image sub-regions; and display a second 3D model, the second 3D model including multiple first sub-models, the first sub-models being generated based on the first 3D model and the first identifier model. The solution in this paper solves the problem of insufficient detail and quality degradation caused by resolution limitations when modeling complex objects as a whole in existing technologies. It combines 2D image semantic segmentation with overall 3D model reconstruction, gradually elevating 2D partitions to 3D and guiding precise model decomposition, providing each component model and texture with exclusive full-resolution processing opportunities, automatically deriving decomposition logic and processing each component individually, significantly reducing the cost of subsequent manual correction and asset reconstruction.

[0181] In one scenario described herein, the second module 520 may optionally also be used for: Receive a first operation, wherein the first operation is used to modify the first identifier image; Display a second identifier image, wherein the second identifier image is obtained based on modifications.

[0182] In one scenario described herein, the second module 520 may optionally also be used for: Display multiple first identification images, each of which corresponds to a different projection image of the first three-dimensional model; Receive a first operation on a plurality of first identification images, wherein the first operation is used to modify the first identification images; In response to the second operation, a second identification image is displayed, wherein the second identification image is obtained based on multiple modifications.

[0183] In one scenario described herein, the second module 520 may optionally also be used for: In response to the second operation, first information is displayed, which is generated based on multiple first operations; In response to a third operation on the first information, a second identifier image is displayed.

[0184] In one scenario described herein, the third module 530 may optionally also be used for: Display the first identification model; Receive a fourth operation, wherein the fourth operation is used to modify the first identifier model; The second identifier model is displayed, which is obtained based on the modification.

[0185] In one scenario described herein, the image processing apparatus may optionally further include a fifth module for: Multiple second images are displayed, each associated with a sub-region of an image. The second images are generated based on the first image and the first sub-model.

[0186] In one scenario described herein, the image processing apparatus may optionally further include a sixth module for: The third 3D model is displayed. The third 3D model is composed of multiple second sub-models. The second sub-models are generated based on the second image, the first sub-model, and the first spatial position relationship. The first spatial position relationship is the relative position relationship between the first sub-model and the second 3D model.

[0187] In one scenario described herein, the image processing apparatus may optionally further include a sixth module, which can also be used for: The second image, the first sub-model, and the first spatial position relationship are input into the first machine learning model to obtain the second sub-model; wherein the second image is the main input, and the first sub-model and the first spatial position relationship are the conditional inputs.

[0188] The image processing apparatus provided in this paper can execute the image processing method implemented in any of the ways described in this paper, and has the corresponding functional modules and beneficial effects for executing the image processing method.

[0189] It is worth noting that the various units and modules included in the above-mentioned device are divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the implementation method in this paper.

[0190] The following is for reference. Figure 10 This document illustrates a schematic diagram of the structure of an electronic device (e.g., a terminal device or server) 600 suitable for implementing the present invention. The terminal device described herein may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Pads), PMPs (Portable Multimedia Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs (Televisions), desktop computers, etc. Figure 10 The electronic device shown is merely an example and should not impose any limitations on the functionality and scope of this article.

[0191] like Figure 10 As shown, electronic device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of electronic device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. An input / output (I / O) interface 606 is also connected to bus 604.

[0192] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 10An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0193] Specifically, according to the implementation described herein, the process referred to in the flowchart above can be implemented as a computer software program. For example, the implementation herein includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an implementation, the computer program can be downloaded and installed from a network via communication device 609, or installed from storage device 608, or installed from ROM 602. When the computer program is executed by processing device 601, it performs the functions defined in the method herein.

[0194] The names of messages or information exchanged between multiple devices in this document are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0195] The electronic device provided herein and the image processing method described above belong to the same inventive concept. Technical details not described in detail herein can be found in the above description, and have the same beneficial effects as the above description.

[0196] This document provides a non-transitory computer-readable storage medium that internally stores computer program instructions; when a processor executes the computer program instructions, it implements the image processing method provided by the above-described implementation.

[0197] It should be noted that the computer-readable medium described above can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, (but not limited to) an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this document, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (Radio Frequency), etc., or any suitable combination thereof.

[0198] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include Local Area Networks (LANs), Wide Area Networks (WANs), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0199] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0200] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: Receive at least one first image; Display a first identification image and a first three-dimensional model; wherein the first identification image and the first three-dimensional model are generated based on a first image, the first identification image includes multiple image sub-regions, and the image sub-regions are associated with at least a portion of the region of the first three-dimensional model; A first identifier model is obtained, which includes multiple model sub-regions. The model sub-regions are generated based on and associated with the image sub-regions. The second three-dimensional model is displayed. The second three-dimensional model includes multiple first sub-models, which are generated based on the first three-dimensional model and the first identifier model.

[0201] Computer program code for performing the operations described herein may be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0202] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various implementations described herein. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the figures. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0203] The modules and units described herein can be implemented in software or hardware. The names of modules and units do not necessarily limit the specific unit; for example, a media data editing module can also be described as "a module for editing first audio data and obtaining first display data of target display text associated with the first audio data."

[0204] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Array (FPGA), Application-Specific Integrated Circuit (ASIC), Application-Specific Standard Product (ASSP), System on Chip (SoC), Complex Programmable Logic Device (CPLD), and so on.

[0205] In the context of this document, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0206] This document also provides a computer program product, including a computer program that, when executed by a processor, implements an image processing method as provided by any of the implementations herein.

[0207] In the implementation of a computer program product, computer program code for performing the operations described herein can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0208] The above description is merely a preferred implementation and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this document is not limited to technical solutions formed by specific combinations of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions in this document.

[0209] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be executed in the specific order shown or in a sequential order. Multitasking and parallel processing may be advantageous in certain contexts. Similarly, while some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this paper. Certain features described in the context of a single implementation may also be implemented in combination within a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0210] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image processing method, comprising: Receive at least one first image; Display a first identification image and a first three-dimensional model; wherein the first identification image and the first three-dimensional model are generated based on the first image, the first identification image includes a plurality of image sub-regions, the image sub-regions being associated with at least a portion of the region of the first three-dimensional model; A first identifier model is obtained, the first identifier model includes multiple model sub-regions, the model sub-regions are generated based on the image sub-regions and are associated with the image sub-regions; The second three-dimensional model is displayed. The second three-dimensional model includes multiple first sub-models, which are generated based on the first three-dimensional model and the first identifier model.

2. The method according to claim 1, further comprising: Receive a first operation, wherein the first operation is used to modify the first identification image; Display a second identifier image, wherein the second identifier image is obtained based on the modification.

3. The method according to claim 1, further comprising: Display multiple first identification images, each of which corresponds to a different projection image of the first 3D model; Receive a first operation on a plurality of the first identification images, wherein the first operation is used to modify the first identification image; In response to the second operation, a second identification image is displayed, wherein the second identification image is obtained based on the plurality of said modifications.

4. The method of claim 3, wherein displaying the second identification image in response to the second operation comprises: In response to the second operation, first information is displayed, the first information being generated based on a plurality of the first operations; In response to a third operation on the first information, a second identification image is displayed.

5. The method according to claim 1, further comprising: Display the first identifier model; A fourth operation is received, wherein the fourth operation is used to modify the first identifier model; Display a second identifier model, which is obtained based on the modification.

6. The method according to any one of claims 1-5, further comprising: Display multiple second images, each of which is associated with a sub-region of the image, and the second images are generated based on the first image and the first sub-model.

7. The method of claim 6, comprising: Display a third 3D model, which is composed of multiple second sub-models, and the second sub-models are generated based on the second image, the first sub-model, and the first spatial positional relationship; The first spatial positional relationship is the relative positional relationship between the first sub-model and the second three-dimensional model.

8. The method according to claim 7, wherein the process of generating the second sub-model comprises: The second image, the first sub-model, and the first spatial position relationship are input into the first machine learning model to obtain the second sub-model; wherein the second image is the main input, and the first sub-model and the first spatial position relationship are the conditional inputs.

9. An image processing apparatus, comprising: The first module is used to receive at least one first image; The second module is used to display a first identification image and a first three-dimensional model; wherein the first identification image and the first three-dimensional model are generated based on the first image, and the first identification image includes multiple image sub-regions, the image sub-regions being associated with at least a portion of the region of the first three-dimensional model; The third module is used to obtain a first identifier model, which includes multiple model sub-regions, the model sub-regions being generated based on the image sub-regions and associated with the image sub-regions; The fourth module is used to display a second three-dimensional model, which includes multiple first sub-models generated based on the first three-dimensional model and the first identifier model.

10. An electronic device, the electronic device comprising: One or more processors; Storage device, used to store program instructions When the processor reads and executes the program instructions, it implements the image processing method as described in any one of claims 1-8.

11. A non-transitory computer-readable storage medium, wherein computer program instructions are stored internally; when a processor executes the computer program instructions, it performs the image processing method as described in any one of claims 1-8.

12. A computer program product comprising a computer program that, when loaded and run by a processor of an electronic device, controls the processor to execute the image processing method as described in any one of claims 1-8.

Images