Object map generation method and apparatus, and device, computer-readable storage medium and computer program product

Abstract

Provided in the present application are an object map generation method and apparatus, and a device, a computer-readable storage medium and a computer program product. The method comprises: acquiring a plurality of initial object maps of a virtual object, and adjustment description text used for describing a preset adjustment effect on the virtual object; controlling a virtual camera to photograph the virtual object from a plurality of different photographing angles, so as to obtain captured images respectively corresponding to the photographing angles; on the basis of the adjustment description text, performing image adjustment on each captured image, so as to obtain a reference image corresponding to each captured image, wherein the reference image meets the preset adjustment effect; selecting from among reference images a target reference image corresponding to each initial object map, wherein the virtual camera can capture the corresponding initial object map at the photographing angle corresponding to the target reference image; and on the basis of the target reference image and the initial object map, generating a target object map which meets the preset adjustment effect.

Description

Method, device, apparatus, computer-readable storage medium, and computer program product for generating object maps

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number 2023112437155 and application date September 22, 2023, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a method, apparatus, device, computer-readable storage medium, and computer program product for generating an object map. Background Art

[0004] In recent years, with the development of computer technology, industries such as gaming, film, and virtual reality have flourished. Virtual scenes are populated with virtual objects, virtual light sources, and virtual cameras. Virtual objects are interactive figures of people and objects within a virtual scene, or movable objects within a virtual scene. Virtual objects consist of an object skeleton and an object texture. By attaching an object texture to the object skeleton, a virtual object is created within the virtual scene.

[0005] In related technologies, to adjust the appearance of a virtual object, it is usually necessary to manually adjust the initial object map of the virtual object to generate an adjusted object map. Due to the large number of object maps of virtual objects, the efficiency of generating the object map is extremely low.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a method, device, electronic device, computer-readable storage medium, and computer program product for generating an object map, which can effectively improve the efficiency of generating an object map.

[0008] The technical solution of the embodiment of the present application is implemented as follows:

[0009] The present invention provides a method for generating an object map, including:

[0010] Acquire multiple initial object maps of a virtual object and adjustment description text for describing a preset adjustment effect on the virtual object, wherein the initial object map is the smallest unit constituting a surface map of the virtual object;

[0011] Controlling a virtual camera to shoot the virtual object from a plurality of different shooting angles to obtain shooting images corresponding to the respective shooting angles;

[0012] Based on the adjustment description text, performing image adjustment on each of the captured images to obtain a reference image corresponding to each of the captured images, wherein the reference image satisfies the preset adjustment effect;

[0013] Selecting target reference images corresponding to the initial object maps from the reference images, wherein the virtual camera can capture the corresponding initial object maps at the shooting angle corresponding to the target reference images;

[0014] A target object map that meets the preset adjustment effect is generated based on the target reference image and the initial object map.

[0015] The present invention provides a device for generating an object map, comprising:

[0016] an acquisition module configured to acquire a plurality of initial object maps of a virtual object and adjustment description text for describing a preset adjustment effect on the virtual object, wherein the initial object map is a minimum unit constituting a surface map of the virtual object;

[0017] a shooting module configured to obtain a plurality of initial object maps of a virtual object and adjustment description text for describing a preset adjustment effect on the virtual object, wherein the initial object map is a minimum unit constituting a surface map of the virtual object;

[0018] an adjustment module configured to perform image adjustment on each of the captured images based on the adjustment description text to obtain a reference image corresponding to each of the captured images, wherein the reference image satisfies the preset adjustment effect;

[0019] a selection module configured to select, from the reference images, target reference images corresponding to the initial object maps, wherein the virtual camera can capture the corresponding initial object maps at the shooting angle corresponding to the target reference images;

[0020] A generating module is configured to generate a target object map that meets the preset adjustment effect based on the target reference image and the initial object map.

[0021] In the above scheme, the above adjustment module is also configured to call the encoding layer, encode the image content of each of the captured images based on the adjustment description text, and obtain the image features corresponding to each of the captured images; call the decoding layer, adjust the image content of each of the captured images based on the image features, and obtain the candidate images corresponding to each of the captured images.

[0022] In the above solution, the generation module is further configured to replace each of the initial object maps on the virtual object with the corresponding target object map to obtain a target virtual object, and the target virtual object satisfies a preset adjustment effect.

[0023] In the above solution, the acquisition module is further configured to generate an adjustment description text for describing a preset adjustment effect on the virtual object in response to an editing operation on the adjustment description text.

[0024] An embodiment of the present application provides an electronic device, including:

[0025] a memory configured to store computer-executable instructions or computer programs;

[0026] The processor is configured to implement the object map generation method provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0027] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions configured to cause a processor to execute the instructions to implement the object map generation method provided in the embodiment of the present application.

[0028] An embodiment of the present application provides a computer program product, comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the object map generation method described in the embodiment of the present application.

[0029] The embodiments of the present application have the following beneficial effects:

[0030] By obtaining multiple initial object maps of the virtual object and adjusting the description text, and controlling the virtual camera to shoot the virtual object from multiple different shooting angles, the captured images corresponding to each shooting angle are obtained. Based on the adjusted description text, the captured images are adjusted to obtain reference images corresponding to each captured image. From the reference images, the target reference images corresponding to each initial object map are selected. Based on the target reference images and the initial object map, a target map that meets the preset adjustment effect is generated. In this way, since the virtual object is photographed and the photographed images corresponding to each photographing angle are obtained, the number of initial object maps of the virtual object collected is at least one, and the photographed images are adjusted respectively. The obtained reference image can realize content adjustment of at least one initial object map corresponding to the photographed image. By using the reference image corresponding to the initial object map, that is, the target map generated based on the target reference image, the adjustment of at least one initial object map can be realized, so there is no need to perform image adjustment on each initial object map separately. It is only necessary to perform image adjustment on the photographed images separately to realize adjustment of all initial object maps, thereby effectively reducing the number of image adjustments and at the same time enabling the generated target object map to meet the preset adjustment effect, thereby effectively improving the generation efficiency of the object map. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1 is a schematic diagram of the architecture of a system for generating an object map according to an embodiment of the present application;

[0032] FIG2 is a schematic structural diagram of an electronic device for generating an object map according to an embodiment of the present application;

[0033] FIG3 is a flow chart of a method for generating an object map according to an embodiment of the present application;

[0034] FIG4 is a second flow chart of a method for generating an object map according to an embodiment of the present application;

[0035] FIG5 is a third flow chart of a method for generating an object map according to an embodiment of the present application;

[0036] FIG6 is a fourth flow chart of a method for generating an object map according to an embodiment of the present application;

[0037] FIG7 is a fifth flow chart of a method for generating an object map according to an embodiment of the present application;

[0038] FIG8 is a first schematic diagram of the principle of a method for generating an object map provided in an embodiment of the present application;

[0039] FIG9 is a second schematic diagram of the principle of the method for generating an object map provided in an embodiment of the present application;

[0040] FIG10 is a schematic diagram of the principle of an undirected graph provided in an embodiment of the present application;

[0041] FIG11 is a schematic diagram of the structure of an image adjustment network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0043] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0044] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0046] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0047] 1) Virtual scene: This is a virtual scene displayed or provided when an application is running on a terminal. The virtual scene can be a simulation of the real world, a semi-simulation and semi-fictitious virtual environment, or a purely fictitious virtual environment. The virtual scene can be any of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The embodiments of the present application do not limit the dimensions of the virtual scene. For example, the virtual scene can include the sky, land, ocean, etc. The land can include environmental elements such as deserts and cities, and users can control virtual objects to move in the virtual scene.

[0048] 2) Artificial Intelligence (AI): This refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics.

[0049] 3) Virtual objects: These refer to objects that are moving or stationary in a virtual scene. Moving virtual objects include animals, vehicles, and people in the virtual scene. Stationary virtual objects include walls, rocks, and the ground in the virtual scene.

[0050] 4) Virtual camera: It is a "camera" set up in computer animation software or virtual engine. The role of virtual camera in expressing viewpoint during animation is equivalent to that of traditional camera. The shooting objects of virtual camera and physical camera are completely different, but the functions are extremely similar. Physical camera shoots real people or actually built scenes, while virtual camera shoots models built in 3D software, which can realize infinite possibilities. Virtual camera is presented in the form of icon in virtual engine, and also has parameters such as lens, focal length, focus, aperture, depth of field, etc. It can realize camera actions such as "push, pull, shake, move, follow, swing, rise, fall, and comprehensive movement", and can achieve shooting effects that are difficult or even impossible to achieve with physical camera, such as: passing through walls, through keyholes, through objects, etc. The parameters that need to be adjusted for physical camera are distributed on the body of physical camera and require manual operation. The camera parameters of the virtual camera are buttons or numerical input bars integrated on the panel. The operator only needs to input the parameters or drag the mouse. Sometimes a few key frames can determine the motion path of the virtual camera. In actual shooting, physical cameras often require a stabilizer or motion control system. Even so, the shaking of the picture still exists.

[0051] 5) Virtual Engine: A virtual engine refers to some already-written editable computer virtual systems or core components of some interactive real-time image applications. These systems provide virtual scene designers with various tools needed to write virtual scenes. The purpose is to enable designers to write programs easily and quickly. Virtual engines include rendering engines (rendering engines include two-dimensional rendering engines and three-dimensional rendering engines), physics engines, collision detection engines, sound engines, script engines, animation engines, artificial intelligence engines, network engines, and scene management engines.

[0052] 6) Computer vision technology (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing so that the computer processing becomes an image that is more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition. It can be understood that the real-time global illumination rendering method in this application is based on computer vision technology to perform image processing on virtual scenes, thereby effectively improving the lighting rendering efficiency and lighting rendering effect in virtual scenes.

[0053] 7) Virtual Objects: Images of various people and objects that can interact in a virtual scene, or movable objects in a virtual scene. These movable objects can be virtual people, virtual animals, cartoon characters, etc., such as people, animals, plants, oil drums, walls, rocks, etc. displayed in a virtual scene. These virtual objects can be virtual avatars that represent users in the virtual scene. A virtual scene can include multiple virtual objects, each with its own shape and volume, occupying a portion of the space within the virtual scene. Optionally, these virtual objects can be user characters controlled through client operations, artificial intelligence (AI) trained to compete in a virtual scene, or non-player characters (NPCs) used in virtual scene interactions. Optionally, these virtual objects can be virtual characters that engage in adversarial interactions in the virtual scene. Optionally, the number of virtual objects participating in the interaction in the virtual scene can be pre-set or dynamically determined based on the number of clients participating in the interaction.

[0054] 8) Convolutional Neural Networks (CNN): These are a type of feed-forward neural network (FNN) with a deep structure that incorporates convolutional computations. They are a representative algorithm in deep learning. CNNs possess representation learning capabilities and can perform shift-invariant classification of input images based on their hierarchical structure.

[0055] 9) Primitives: Also known as rendering primitives, these are the graphical data used to describe various graphic elements in the graphics rendering development interface. These correspond to the visible entities on the drawing interface and include the geometric information required for rendering, such as vertex data, line segments, and polygons. A primitive must contain at least one vertex; a vertex defines a point in a 2D or 3D coordinate system and also defines several attributes that affect how the vertex is rendered on the screen.

[0056] During the implementation of the embodiments of this application, the applicant discovered that the related technology has the following problems:

[0057] In related technologies, to adjust the appearance of a virtual object, it is usually necessary to manually adjust the initial object map of the virtual object to generate an adjusted object map. Due to the large number of object maps of virtual objects, the efficiency of generating the object map is extremely low.

[0058] Embodiments of the present application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating an object map, which can effectively improve the efficiency of generating an object map. The following describes an exemplary application of the object map generation system provided by the embodiments of the present application.

[0059] See Figure 1, which is a schematic diagram of the architecture of the object map generation system 100 provided in an embodiment of the present application. The terminal (terminal 400 is shown as an example) is connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0060] The terminal 400 is configured to allow a user to use a client 410 to display a target object map on a graphical interface 410-1 (graphic interface 410-1 is shown as an example). The terminal 400 and the server 200 are connected to each other via a wired or wireless network.

[0061] In some embodiments, the server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal 400 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart TV, a smart watch, a car terminal, etc., but is not limited to this. The electronic device provided in the embodiment of the present application can be implemented as a terminal or as a server. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiment of the present application.

[0062] In some embodiments, the server 200 obtains multiple initial object maps of the virtual object, and adjusts the description text, and determines the photographed images corresponding to each shooting angle, and sends the photographed images to the terminal 400. The terminal 400 selects the target reference images corresponding to each initial object map from the reference image, and generates a target map that meets the preset adjustment effect based on the target reference image and the initial object map.

[0063] In other embodiments, the server 200 obtains multiple initial object maps of the virtual object and adjustment description text, and determines the shooting images corresponding to each shooting angle, selects the target reference images corresponding to each initial object map from the reference images, generates a target map that meets the preset adjustment effect based on the target reference image and the initial object map, and sends the target map to the terminal 400.

[0064] Referring to Figure 2, Figure 2 is a schematic diagram of the structure of an electronic device 500 for generating object maps provided in an embodiment of the present application, wherein the electronic device 500 shown in Figure 2 can be the server 200 or the terminal 400 in Figure 1, and the electronic device 500 shown in Figure 2 includes: at least one processor 430, a memory 450, and at least one network interface 420. The various components in the electronic device 500 are coupled together through a bus system 440. It can be understood that the bus system 440 is configured to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus system 440 in Figure 2.

[0065] The processor 430 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0066] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 430.

[0067] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0068] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0069] Operating system 451, including system programs configured to handle various basic system services and perform hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., configured to implement various basic services and handle hardware-based tasks;

[0070] The network communication module 452 is configured to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include Bluetooth, Wireless Fidelity (WiFi), and Universal Serial Bus (USB).

[0071] In some embodiments, the object map generation device provided in the embodiments of the present application can be implemented using software. FIG2 shows an object map generation device 455 stored in memory 450. This device 455 can be software in the form of a program or plug-in, and includes the following software modules: an acquisition module 4551, a shooting module 4552, an adjustment module 4553, a selection module 4554, and a generation module 4555. These modules are logical and can be arbitrarily combined or further separated according to the functions implemented. The functions of each module will be described below.

[0072] In other embodiments, the object map generation device provided in the embodiments of the present application can be implemented in hardware. As an example, the object map generation device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the object map generation method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0073] In some embodiments, the terminal or server can implement the object map generation method provided in the embodiments of the present application by running a computer program or computer executable instructions. For example, the computer program can be a native program in the operating system (for example, a dedicated map generation program) or a software module, for example, a map generation module that can be embedded in any program (such as an instant messaging client, a photo album program, an electronic map client, a navigation client); for example, it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run. In short, the above-mentioned computer program can be any form of application, module or plug-in.

[0074] The method for generating an object map provided in the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the server or terminal provided in the embodiment of the present application.

[0075] Refer to Figure 3, which is a flow chart of the object map generation method provided in an embodiment of the present application. It will be explained in conjunction with steps 101 to 105 shown in Figure 3. The object map generation method provided in an embodiment of the present application can be implemented by a server or a terminal alone, or by a server and a terminal in collaboration. The following will be explained using the server alone as an example.

[0076] In step 101 , a plurality of initial object maps of a virtual object are obtained, and an adjustment description text for describing a preset adjustment effect on the virtual object is obtained.

[0077] In some embodiments, the initial object map is the smallest unit that constitutes the surface map of the virtual object.

[0078] In some embodiments, a virtual object refers to the images of various people and objects that can interact in a virtual scene, or an animated object in the virtual scene. The animated object may be a virtual person, virtual animal, or cartoon character, for example, a person, animal, plant, oil drum, wall, stone, etc. displayed in the virtual scene. The virtual object may be a virtual avatar in the virtual scene that represents the user. The virtual scene may include multiple virtual objects, each of which has its own shape and volume and occupies a portion of the space in the virtual scene. Optionally, the virtual object may be a user character controlled by operations on the client, an artificial intelligence (AI) trained to play in the virtual scene, or a non-player character (NPC) set in the virtual scene interaction. Optionally, the virtual object may be a virtual person that engages in adversarial interaction in the virtual scene. Optionally, the number of virtual objects participating in the interaction in the virtual scene may be pre-set or dynamically determined based on the number of clients participating in the interaction.

[0079] In some embodiments, object maps, also known as rendering primitives, are graphical data used to describe various graphical elements in the graphics rendering development interface. These correspond to visible entities on the drawing interface and include the geometric information required for rendering, such as vertex data, line segments, and polygons. A primitive must contain at least one vertex; a vertex defines a point in a 2D or 3D coordinate system and also defines several attributes that affect how the vertex is rendered on the screen.

[0080] In some embodiments, the adjustment description text is used to describe the preset adjustment effect on the virtual object, that is, to describe the preset adjustment effect on the initial object map of the virtual object. The adjustment description text plays an important role in the adjustment process of the virtual object and is used to describe the preset adjustment effect on the virtual object. In the context of the virtual object, mapping refers to the process of mapping a texture or pattern to the surface of an object. The adjustment description text is used to describe the preset adjustment effect on such a map, such as changing the position, scaling, rotating, or replacing the map with a different texture.

[0081] For example, adjustment description text can be used for texture adjustments. The adjustment description text may describe the specific operation performed on the texture, such as "Move the horizontal position of texture A 10 units to the right" or "Increase the brightness of texture B by 20%." Adjustment description text can also be used for texture attribute adjustments. The adjustment description text may also include adjustments to texture attributes, such as "Set the transparency of texture C to 80%" or "Increase the texture detail of texture D by 50%." Adjustment description text can also be used for texture replacement. The adjustment description text can also describe the replacement of an existing texture, such as "Replace texture E with the new texture F."

[0082] In some embodiments, a preset adjustment effect refers to the intended goal or result of adjusting a virtual object (such as a computer graphic, an object in virtual reality, a digital model, etc.). It describes the specific change or improvement that a user or system hopes to achieve through a certain operation or algorithm. For example, in a virtual reality environment, a preset adjustment effect may refer to changing the size, position, color, or texture of a virtual object to make it more suitable for a specific application scenario or user needs.

[0083] In some embodiments, adjustment description text is specific textual information used to describe the preset adjustment effect on the virtual object. It typically includes a detailed description of the preset adjustment effect, including the adjusted parameters, target value, and expected effect. Adjustment description text serves as a communication bridge between humans and computer systems or algorithms, helping the system understand the user's specific needs and perform corresponding operations accordingly. In technical documentation or code, adjustment description text may appear in the form of code comments, configuration files, user input, etc.

[0084] For example, in an image editing software, if a user wishes to move a virtual object (such as a graphic logo) from one location to another and change its size, the preset adjustment effect may be: the user desires the object to move from its current location to a specified new location and to be 50% of its original size. The adjustment description text may be: a descriptive text or code comment in the following form: Preset adjustment effect: Move object A to coordinates (x=100, y=200) and resize it to 50% of its original size.

[0085] In some embodiments, the acquisition of the adjustment description text for describing the preset adjustment effect on the virtual object may be achieved as follows: in response to an editing operation on the adjustment description text, generating the adjustment description text for describing the preset adjustment effect on the virtual object.

[0086] In some embodiments, it is necessary to identify the editing operations performed by the user on the adjustment description text. These operations may include adding new adjustment instructions, modifying existing instructions, deleting instructions, or adjusting the parameters of instructions. Parse the edited adjustment description text to understand the user's intention to adjust the virtual object. This usually requires the use of natural language processing technology, including lexical analysis, syntax analysis, etc., to extract specific adjustment instructions and related parameters. Based on the parsing results, the system generates an intermediate representation, which is usually a data structure such as an object, dictionary or JSON, which accurately describes the adjustment operation expected by the user. Convert the intermediate representation into a new adjustment description text. This process involves converting the data and instructions in the intermediate representation into human-readable text form to facilitate user understanding or further editing when necessary.

[0087] As an example, a user can edit the adjustment description text through a graphical user interface (GUI) or a text editor. These editing operations, such as text insertion, deletion, and replacement, need to be detected. The edited text is analyzed to extract the adjustment instructions and parameters. For example, if the user changes "Increase the width of object A by 10%" to "Increase the width of object A by 20%", it is necessary to identify the specific value of the width adjustment from 10% to 20%. An intermediate representation is created to store the parsed adjustment instructions and parameters. For example, a dictionary structure may contain a key "object_id" corresponding to the value "A", a key "attribute" corresponding to the value "width", and a key "change" corresponding to the value "+20%". The intermediate representation is converted into the adjustment description text. This may involve template filling, string concatenation, or other text processing techniques. For example, the above intermediate representation may be converted into the following adjustment description text: "Adjust the width of object A by 20%".

[0088] In some embodiments, when generating adjustment description text for describing a preset adjustment effect on a virtual object, the following factors may also be considered: the generated adjustment description text needs to conform to specific grammatical rules to ensure that the virtual object system can correctly parse and execute it. The adjustment description text should be consistent with the user's editing intention to avoid ambiguity or misunderstanding. The adjustment description text should be easy for users to read and understand, especially for non-technical users. Ultimately, the generated adjustment description text will become the interface between the user and the virtual object system, guiding the system to adjust the virtual object according to the user's intention.

[0089] In this way, the adjustment description text can clearly convey the intention of adjusting the virtual object, which helps ensure that the adjustment of the virtual object meets design requirements and user expectations. In actual applications, this adjustment description text may exist in the form of programming interface call parameters, configuration files, user input instructions, etc., and is used to guide the rendering and interaction process of the virtual object.

[0090] In step 102 , a virtual camera is controlled to shoot a virtual object from a plurality of different shooting angles to obtain shooting images corresponding to each shooting angle.

[0091] In some embodiments, a virtual camera is a "camera" set up in computer animation software or a virtual engine. The role of a virtual camera in expressing viewpoints during animation is equivalent to that of a traditional camera. The virtual camera and the physical camera shoot completely different objects, but their functions are extremely similar. A physical camera shoots real people or actually built scenes, while a virtual camera shoots models built in three-dimensional software, which can achieve infinite possibilities. The virtual camera is presented in the form of an icon in the virtual engine, and also has parameters such as lens, focal length, focus, aperture, and depth of field. It can achieve camera actions such as "push, pull, shake, move, follow, swing, rise, fall, and comprehensive movement", and can achieve shooting effects that are difficult or even impossible to achieve with a physical camera, such as: passing through walls, through keyholes, through objects, etc. The parameters that need to be adjusted for the physical camera are distributed on the body of the physical camera and require manual operation. The camera parameters of the virtual camera are buttons or numerical input bars integrated on the panel. The operator only needs to input the parameters or drag the mouse. Sometimes a few key frames can determine the motion path of the virtual camera. In actual shooting, physical cameras often require a stabilizer or motion control system. Even so, the shaking of the picture still exists.

[0092] In some embodiments, the virtual object is located at a target position in the virtual scene, and the virtual camera is controlled to shoot the virtual object at the target position from multiple different shooting angles to obtain shooting images corresponding to each shooting angle.

[0093] For example, create a virtual scene containing the virtual object you want to film. Define multiple shooting angles. These angles can be fixed, predefined, or dynamically calculated based on specific needs. For example, you might define the following shooting angles: Head-on view: The camera faces the object directly. Side view: The camera is positioned to the side of the object. Top view: The camera is positioned directly above the object. Oblique view: The camera is positioned at a corner of the object, creating an oblique effect. For each shooting angle, set the position and orientation of the virtual camera. Render the scene using the rendering capabilities of the virtual engine. During rendering, the virtual engine calculates the scene's lighting, shadows, and perspective based on the camera's position and orientation. After rendering, an image representing the specific shooting angle is generated. This image can be 2D or 3D, depending on the capabilities of the virtual engine and the complexity of the scene. Store the images captured from each angle for future use. These images can be used for various purposes, such as virtual reality applications, animation, and game development. This allows you to capture images of the virtual object from multiple angles, each reflecting the perspective of the virtual camera at a specific angle. These images can be used to create dynamic visual effects and enhance the user's sense of immersion in the virtual environment.

[0094] In some embodiments, referring to FIG. 4 , FIG. 4 is a second flow chart of a method for generating an object map provided in an embodiment of the present application. Step 102 shown in FIG. 3 can be implemented through steps 1021 to 1023 shown in FIG. 4 .

[0095] In step 1021 , the center position of the geometric center of the virtual object in the virtual scene is obtained.

[0096] In some embodiments, the central position of the geometric center of the virtual object in the virtual scene is used to indicate the position coordinates of the geometric center of the virtual object in the virtual scene.

[0097] In some embodiments, the center position is used to perform standardization processing on the virtual object to obtain a standardized target virtual object.

[0098] As an example, suppose you have a virtual scene containing one or more virtual objects. You want to find the geometric center of one of these objects and place that center point at the center of the scene. You need to define the virtual object. This is usually done by creating a 3D model consisting of vertices, faces, and other geometric information. Calculate the average position of all the object's vertices. If the object is composed of multiple parts, you may need to calculate the center of each part separately, and then calculate the overall center based on the weights of the parts. You need to determine the center position of the virtual scene, which is usually achieved by calculating the average position of the centers of all objects in the scene.

[0099] In step 1022 , based on the center position, the virtual object is standardized in the virtual scene to obtain a target virtual object.

[0100] In some embodiments, the above-mentioned standardization processing refers to the process of standardizing the virtual object to obtain a standardized target virtual object. Standardization processing is a processing method for processing virtual objects in a virtual scene, which aims to unify and standardize the position data of the object by referring to the center position. Since different object parts of the virtual object may have protrusions, resulting in an uneven surface of the virtual object, the protruding parts easily cause visual occlusion of the concave parts, resulting in the subsequent captured images being unable to accurately capture the concave parts of the virtual object. In order to make the subsequent captured images more complete and comprehensive, by performing standardization on the virtual object, the subsequent captured images can accurately capture the concave parts before the standardization processing, so that the captured images can accurately and comprehensively describe the virtual object from the corresponding shooting angle.

[0101] In some embodiments, the above-mentioned step 1022 can be implemented as follows: obtain the positions of multiple object parts of the virtual object in the virtual scene, determine the distance between each position and the center position, and determine the maximum distance as the reference distance; perform the following processing on each object part of the virtual object to obtain the target virtual object: divide the distance corresponding to the object part by the reference distance to obtain the standard position of the object part in the virtual scene; adjust the position of the object part to the standard position.

[0102] For example, it is necessary to determine the specific locations of different parts of a virtual object in a virtual scene. These parts can be specific points on the object, such as vertices, centroids, or any point with spatial coordinates. For example, if the virtual object is a human model, parts might include the head, chest, arms, and legs. The distance between each object part and the center of the virtual scene is calculated. This is typically accomplished using the Euclidean distance formula, which calculates the straight-line distance between two points in three-dimensional space. The largest distance calculated is selected as the distance. A reference distance serves as a scale, defining the relative sizes of the object parts during the normalization process. The distance between the object part and the center is divided by the reference distance to produce a normalized ratio value between 0 and 1. This ratio value represents the normalized position of the object part relative to the center. Based on the calculated normalized ratio value, the position of the object part is adjusted. This typically involves adding the actual coordinates of the object part to the coordinates of the center and multiplying the result by the normalized ratio value. Through this process, each part of the virtual object is repositioned to a normalized distance relative to the center, creating a consistent, center-referenced coordinate system within the virtual scene. This is crucial for ensuring an even distribution of virtual objects within the scene.

[0103] As an example, the expression for the standard position of the above object part in the virtual scene can be:

[0104] in, Used to indicate the standard position of the object in the virtual scene. The horizontal coordinate used to indicate the standard position in the virtual scene, The vertical coordinate used to indicate the standard position in the virtual scene, The vertical coordinate used to indicate the standard position in the virtual scene, X i The horizontal axis component used to indicate the distance corresponding to the object part, Y i The vertical axis component used to indicate the distance corresponding to the object part, Z i The vertical axis component used to indicate the distance corresponding to the object part, d max Used to indicate reference distance, X mean The horizontal coordinate used to indicate the center position, Y mean The vertical coordinate used to indicate the center position, Z mean The vertical coordinate used to indicate the center position.

[0105] As an example, the expression of the above reference distance can be:

[0106] d max =max{d1,d2,d i …,d N} (2)

[0107] Among them, d max Used to indicate reference distance, d1, d2, di…, d N Used to indicate the distances corresponding to each object part.

[0108] As an example, the distance corresponding to object part i can be expressed as:

[0109] Among them, d i Used to indicate the distance corresponding to object part i, X i The horizontal coordinate used to indicate the position of object part i in the virtual scene, Y i The vertical coordinate used to indicate the position of object part i in the virtual scene, Z i The vertical coordinate used to indicate the position of object part i in the virtual scene, X mean The horizontal coordinate used to indicate the center position, Y mean The vertical coordinate used to indicate the center position, Z mean The vertical coordinate used to indicate the center position.

[0110] In this way, by obtaining the position of each part of the virtual object in the virtual scene, it is possible to accurately place and position the virtual object, ensuring that the spatial relationship of the virtual object in the virtual environment meets expectations. By calculating the distance of each part from the center position and determining the maximum distance as the reference distance, a unified benchmark can be provided for subsequent position adjustments. By dividing the position distance of each part by the reference distance, the original position information can be converted into proportional information, so that the position relationship of each part can remain consistent even in virtual scenes of different sizes. By adjusting the position of the parts to a standard position based on the reference distance, the virtual object can be made more visually balanced and coordinated, improving the overall aesthetics. It can adapt to different virtual environments because it is adjusted based on the distance ratio, so as long as the consistency of the reference distance is guaranteed, the consistency of the object can be maintained in different scenes.

[0111] In step 1023 , the virtual camera is controlled to shoot the target virtual object from a plurality of different shooting angles to obtain shooting images corresponding to the respective shooting angles.

[0112] In some embodiments, the virtual camera can only capture part of the target virtual object within the viewing angle from different shooting angles. By controlling the virtual camera to shoot the target virtual object from multiple different shooting angles, the captured images corresponding to each shooting angle can fully display the virtual object.

[0113] In some embodiments, the above-mentioned step 1023 can be implemented as follows: determine multiple shooting positions in the virtual scene whose distance from the center position is equal to the target distance, and the shooting angles correspond one-to-one to the shooting positions; for each shooting position, control the virtual camera to be at the shooting position and face the target virtual object, shoot the target virtual object, and obtain the shooting image corresponding to the corresponding shooting angle.

[0114] In some embodiments, the numerical value of the target distance can be formulated according to the actual acquisition accuracy requirements. The numerical value of the target distance is positively correlated with the acquisition accuracy. The larger the numerical value of the target distance, the greater the corresponding acquisition accuracy. The smaller the numerical value of the target distance, the smaller the corresponding acquisition accuracy. The larger the target distance, the closer the virtual camera is to the virtual object, and the virtual camera can achieve more fine-grained acquisition of the virtual object.

[0115] In some embodiments, the numerical value of the target distance is negatively correlated with the collection range of the virtual camera for the virtual object. The smaller the numerical value of the target distance, the larger the collection range of the virtual camera for the virtual object. The larger the numerical value of the target distance, the smaller the collection range of the virtual camera for the virtual object. The larger the target distance, the closer the virtual camera is to the virtual object, and the smaller the collection range of the virtual camera for the virtual object.

[0116] For example, consider a virtual museum displaying numerous valuable virtual artworks. To allow visitors to appreciate these artworks from various perspectives, it's necessary to control the virtual camera to capture them from multiple angles to obtain comprehensive images. Control the virtual camera to capture the artwork from the front. This angle can highlight the artwork's frontal features, such as a person's facial expression or the main facade of a building. Adjust the camera angle to capture the artwork from the side. This can reveal the artwork's profile or a person's body posture. Capture the artwork from the back so that the viewer can see the design or structure behind the artwork. Capture images from above or below to show the artwork's visual effects from different perspectives. For example, shooting from above can reveal the overall layout of the artwork, while shooting from below can emphasize details. In addition to static shooting angles, dynamic shooting techniques, such as panning and rotating the camera, can also be used to showcase the dynamic effects of the artwork. By capturing images from these multiple angles, a series of images can be generated. These images not only fully display the virtual object's external features but also provide viewers with an immersive experience within the virtual environment. Applications are not limited to museums; they also have broad applications in product displays, architectural design reviews, virtual tours, and other fields. By controlling the virtual camera to shoot the target virtual object from multiple different shooting angles, the display effect of the virtual object can be greatly improved, bringing users a richer and more realistic virtual experience.

[0117] In this way, since there may be protrusions on different parts of the virtual object, resulting in an uneven surface of the virtual object, the protruding parts can easily cause visual occlusion of the concave parts, resulting in the subsequent captured images being unable to accurately capture the concave parts of the virtual object. In order to make the subsequent captured images more complete and comprehensive, the virtual object is standardized so that the subsequent captured images can accurately capture the concave parts before the standardization processing, so that the captured images can accurately and comprehensively describe the virtual object from the corresponding shooting angle.

[0118] In step 103 , image adjustments are performed on each captured image based on the adjustment description text to obtain a reference image corresponding to each captured image.

[0119] In some embodiments, the reference image satisfies a preset adjustment effect described in the adjustment description text, and image adjustment refers to a process of adjusting the image content of a captured image to satisfy the preset adjustment effect.

[0120] In some embodiments, in virtual reality, game development, or any scenario requiring highly realistic images, captured images are adjusted to ensure they meet specific quality standards or conform to pre-defined visual effects. Image adjustments are performed on each captured image based on the adjustment description text, primarily involving two aspects: image content adjustment and super-resolution processing. Image content adjustment can include color correction: adjusting the image's color balance to better match real-world standards or meet the requirements of a specific artistic style. Contrast enhancement can also be performed: enhancing image contrast can make image details clearer and visually more vivid. Brightness adjustment can also be performed: adjusting the image's brightness to ensure that the image maintains appropriate display quality under varying lighting conditions. Cropping and rotation can also be performed: cropping the image, removing excess content, or rotating the image for a better perspective as needed. Image filtering can also be performed: applying various filtering algorithms to smooth the image or remove noise, which is particularly common in photos taken under low-light conditions. Super-resolution reconstruction can also be performed: inferring high-resolution details from low-resolution images. For captured images, if they are low-resolution, super-resolution processing can be used to improve image clarity and detail. Detail enhancement can also be performed: while maintaining the image content, the algorithm increases the image's details to make the image appear higher definition. Through these image adjustments, the resulting reference image better matches the desired text description and can better serve the final application scenario. For example, in game development, reference images may need to reflect a specific artistic style; in virtual games, reference images may need to have sufficient detail and a clear field of view to provide users with an immersive experience.

[0121] In some embodiments, referring to FIG. 5 , FIG. 5 is a flowchart diagram three of a method for generating an object map provided in an embodiment of the present application. Step 103 shown in FIG. 3 can be implemented through steps 1031 to 1032 shown in FIG. 5 .

[0122] In step 1031, the image adjustment network is called to adjust the image content of each captured image based on the adjustment description text to obtain candidate images corresponding to each captured image.

[0123] In some embodiments, the candidate image satisfies the preset adjustment effect described by the adjustment description text.

[0124] In some embodiments, the image adjustment network may be a convolutional neural network (CNN). CNNs are a type of feedforward neural network (FNN) with a deep structure that incorporates convolutional computations and is a representative algorithm in deep learning. CNNs possess representation learning capabilities and can perform shift-invariant classification on input images based on their hierarchical structure.

[0125] As an example, refer to Figure 11, which is a structural diagram of the image adjustment network provided in an embodiment of the present application. The image adjustment network includes an encoding layer 1 and a decoding layer 2. The above step 1031 can be implemented as follows: calling the encoding layer 1, based on the adjustment description text, encoding the image content of each captured image, and obtaining the image features corresponding to each captured image; calling the decoding layer 2, based on the image features, adjusting the image content of each captured image, and obtaining the candidate images corresponding to each captured image.

[0126] In some embodiments, the above-mentioned image adjustment network can be trained in the following manner: obtain an initial image adjustment network, and obtain an adjustment description text sample, a label image, and a captured image sample, wherein the label image satisfies the preset adjustment effect described by the text sample; call the initial image adjustment network, and adjust the image content of the captured image sample based on the adjustment description text sample to obtain an adjusted image corresponding to the captured image sample; determine the loss value of the initial image adjustment network based on the adjusted image and the label image, and train the initial image adjustment network based on the loss value to obtain the image adjustment network.

[0127] In some embodiments, a predefined neural network is required that can adjust image content based on text descriptions. This network may be a convolutional neural network (CNN) specifically designed for image content adjustment. A set of sample data is required, including text descriptions (adjusted description text samples), corresponding adjusted images (labeled images), and original, unadjusted images (captured image samples). The labeled images should meet the preset adjustment effect described by the text samples. Using an initial image adjustment network, the captured image samples are adjusted based on the adjusted description text samples. This network attempts to adjust the images to better match the preset adjustment effect of the text descriptions. After processing by the network, a set of adjusted images is generated. These images should be closer to the label images and meet the preset adjustment effect of the text descriptions. Based on the adjusted and labeled images, a loss value is calculated for the initial image adjustment network. This loss value is a metric that represents the difference between the adjusted and labeled images. Based on the calculated loss value, the initial image adjustment network is trained. During training, the network learns how to more accurately adjust image content based on text descriptions. This process involves adjusting the network weights to reduce the loss value. Through this process, the initial image adjustment network gradually learns how to adjust image content based on text descriptions, thereby improving its accuracy and effectiveness. This process can be repeated, and after each training, the performance of the network will be improved, which can better meet the user's preset adjustment effects.

[0128] In some embodiments, the loss value of the initial image adjustment network may be the similarity between the adjusted image and the label image.

[0129] In step 1032, super-resolution processing is performed on each candidate image to obtain a reference image corresponding to each candidate image.

[0130] In some embodiments, super-resolution processing, also known as image super-resolution, refers to the process of restoring a high-resolution image from a low-resolution image or image sequence. Super-resolution processing includes super-resolution restoration and super-resolution reconstruction. Super-resolution processing can improve the resolution of the original image through hardware or software methods. The process of obtaining a high-resolution image from a series of low-resolution images is super-resolution reconstruction. The core idea of ​​super-resolution reconstruction is to trade temporal bandwidth (obtaining a multi-frame image sequence of the same scene) for spatial resolution, achieving the conversion of temporal resolution to spatial resolution.

[0131] In some embodiments, the image resolution of the reference image is greater than the image resolution of the corresponding candidate image, and the image content of the reference image is completely identical to the image content of the corresponding candidate image.

[0132] In this way, by super-resolution processing each candidate image, a reference image corresponding to each candidate image is obtained. The image resolution of the candidate image can be significantly improved while keeping the image content of the original candidate image unchanged, thereby effectively improving the image quality of the candidate image, so that the reference image can reflect the virtual object more clearly and accurately.

[0133] In step 104, target reference images corresponding to the initial object maps are selected from the reference images.

[0134] In some embodiments, the virtual camera can capture the corresponding initial object map at a shooting angle corresponding to the target reference image.

[0135] In some embodiments,

[0136] In some embodiments, referring to FIG. 6 , FIG. 6 is a fourth flow chart of a method for generating an object map provided in an embodiment of the present application. Step 104 shown in FIG. 3 can be implemented through steps 1041 to 1043 shown in FIG. 6 .

[0137] In step 1041 , the reference shooting positions corresponding to the reference images are obtained, and the following steps 1042 to 1043 are performed for each initial object map.

[0138] In some embodiments, the reference shooting position is used to indicate the position of the virtual camera in the virtual scene when shooting the image corresponding to the reference image.

[0139] In step 1042, for each reference image, based on the reference shooting position corresponding to the reference image and the initial object map, when it is determined that the virtual camera can capture the initial object map at the shooting angle corresponding to the reference image, the reference image is determined as a candidate image for the initial object map.

[0140] In some embodiments, when it is determined that the virtual camera cannot capture the initial object map at the shooting angle corresponding to the reference image, the reference image is not determined as a candidate image for the initial object map.

[0141] As an example, for reference image A, based on the reference shooting position and initial object map corresponding to reference image A, when it is determined that the virtual camera can capture the initial object map at the shooting angle corresponding to reference image A, reference image A is determined as a candidate image for the initial object map; for reference image B, based on the reference shooting position and initial object map corresponding to reference image B, when it is determined that the virtual camera cannot capture the initial object map at the shooting angle corresponding to reference image B, reference image B is determined as a candidate image for the initial object map.

[0142] In some embodiments, before step 1042, it is also possible to determine whether the virtual camera can capture the initial object map at the shooting angle corresponding to the reference image by: obtaining the map position of the initial object map in the virtual scene, and connecting the map position and the reference shooting position in the virtual scene to obtain a virtual detection line; when the virtual detection line does not pass through other initial object maps in the virtual scene, it is determined that the virtual camera can capture the initial object map at the shooting angle corresponding to the reference image; when the virtual detection line passes through other initial object maps in the virtual scene, it is determined that the virtual camera cannot capture the initial object map at the shooting angle corresponding to the reference image.

[0143] In some embodiments, the virtual detection line is used to detect whether the virtual camera can capture the initial object map at a shooting angle corresponding to the reference image.

[0144] In some embodiments, when the virtual detection line does not pass through other initial object maps in the virtual scene, it indicates that no other initial object maps in the virtual scene block the line of sight of the current initial object map, thereby enabling the virtual camera to capture the initial object map at the shooting angle corresponding to the reference image. When the virtual detection line passes through other initial object maps in the virtual scene, it indicates that other initial object maps in the virtual scene block the line of sight of the current initial object map, thereby preventing the virtual camera from capturing the initial object map at the shooting angle corresponding to the reference image.

[0145] In step 1043, a target reference image corresponding to the initial object map is determined from the candidate images of the initial object map.

[0146] In some embodiments, step 1043 may be implemented as follows: when the number of candidate images for the initial object map is one, the candidate image is determined as the target reference image corresponding to the initial object map; when the number of candidate images for the initial object map is multiple, the degree of association between the initial object map and each candidate image is determined; and the candidate image corresponding to the greatest degree of association is determined as the target reference image corresponding to the initial object map.

[0147] In some embodiments, a larger value of the correlation degree indicates a greater correlation between the initial object map and the candidate image, and a smaller value of the correlation degree indicates a smaller correlation between the initial object map and the candidate image.

[0148] In some embodiments, the above-mentioned determination of the degree of association between the initial object map and each candidate image can be determined in the following manner: among multiple initial object maps of the virtual object, determine the adjacent object map of the initial object map, and perform the following processing on each candidate image: based on the adjacent object map and the candidate image, determine a first degree of association of the candidate image; determine the imaging area of ​​the initial object map in the candidate image, and determine a second degree of association of the candidate image based on the area of ​​the imaging area; sum the first degree of association and the second degree of association to obtain the degree of association between the initial object map and the candidate image.

[0149] In some embodiments, the value of the second degree of association is positively correlated with the area of ​​the imaging region, and the first degree of association is associated with the adjacent object map.

[0150] As an example, the expression of the above correlation degree can be:

[0151] Among them, E(l) is used to indicate the degree of association, Used to indicate the second associated program, Used to indicate the first degree of association.

[0152] In some embodiments, the above-mentioned determination of the first degree of association of the candidate image based on the adjacent object map and the candidate image can be achieved as follows: when the candidate image is a candidate image of the adjacent object map, the first degree of association of the candidate image is determined as a first value, and the first value is a non-zero constant; when the candidate image is not a candidate image of the adjacent object map, the first degree of association of the candidate image is determined as a second value.

[0153] In some embodiments, the first value is a non-zero constant, and the second value is equal to zero.

[0154] In some embodiments, when the candidate image is not a candidate image of the adjacent object map, it means that the adjacent image does not interfere with the current candidate image. At this time, the value of the first degree of association can be determined as zero. When the candidate image is a candidate image of the adjacent object map, it means that the adjacent image interferes with the current candidate image. At this time, the value of the first degree of association can be determined as a non-zero constant.

[0155] In this way, when there are multiple candidate images for the initial object map, the candidate image corresponding to the largest correlation degree is determined as the target reference image corresponding to the initial object map. Since the correlation degree can accurately reflect the correlation relationship between the candidate image and the initial object map, the most correlated candidate image is determined as the target reference image of the initial object map, so that the target object map of the initial object map subsequently determined based on the candidate image can accurately express the preset adjustment effect, and the obtained target object map is more accurate.

[0156] In step 105 , a target object map that meets a preset adjustment effect is generated based on the target reference image and the initial object map.

[0157] In some embodiments, the initial object map corresponds to the target object map one-to-one, and the above step 105 can be implemented as follows: for each initial object map, based on the initial object map and the corresponding target reference image, the initial object map is adjusted to obtain a target object map corresponding to the initial object map that meets a preset adjustment effect.

[0158] In some embodiments, referring to FIG. 7 , FIG. 7 is a flowchart diagram five of a method for generating an object map provided in an embodiment of the present application. Step 105 shown in FIG. 3 can be implemented by executing steps 1051 to 1054 shown in FIG. 7 for each initial object map.

[0159] In step 1051, an imaging area of ​​the initial object map in the target reference image is determined from the target reference image corresponding to the initial object map.

[0160] In some embodiments, the virtual camera corresponding to the target reference image can capture the initial object map of a portion of the virtual object from a corresponding shooting angle, so that the target reference image includes imaging areas corresponding to each captured initial object map, so that the imaging area of ​​the initial object map in the target reference image can be determined from the target reference image corresponding to the initial object map.

[0161] In some embodiments, object texturing is a common technique in computer graphics and virtual reality technology for adding texture and color to three-dimensional models to make them appear more realistic. This process typically involves mapping a two-dimensional image (texture) onto the surface of the three-dimensional model. When determining the imaging area of ​​the initial object texture in the target reference image from the target reference image corresponding to the initial object texture, this actually refers to mapping a specific area on the texture to a corresponding area in the target reference image.

[0162] In step 1052, the image content of the initial object map is adjusted to the image content of the imaging area in the target reference image to obtain a candidate map corresponding to the initial object map.

[0163] In some embodiments, since the target reference image includes imaging areas corresponding to each collected initial object map, and since the target reference image satisfies a preset adjustment effect, each imaging area in the target reference image satisfies the preset adjustment effect. By adjusting the image content of the initial object map to the image content of the imaging area in the target reference image, a candidate map corresponding to the initial object map is obtained, so that the obtained candidate map can meet the preset adjustment effect.

[0164] In some embodiments, the target reference image is analyzed to determine the area that needs to be mapped. This usually involves image analysis techniques such as edge detection and image segmentation to identify the imaging area in the target reference image. There is an initial object map, which is a two-dimensional image that contains the texture information that needs to be mapped to the target reference image. According to the imaging area of ​​the target reference image, the image content of the initial object map is adjusted. This may include adjusting the color, brightness, contrast, saturation, etc. to make the map better adapt to the visual effect of the target reference image. After adjusting the map content, the map needs to be mapped to the imaging area of ​​the target reference image and merged with other areas. This usually involves image blending and edge fusion technology to create a smooth and natural transition effect. After the above steps, a candidate map corresponding to the initial object map is obtained. This map can be used for image synthesis or texture mapping to enhance the realism and details of the target reference image.

[0165] For example, if you want to map a wood texture onto a virtual desktop model, you analyze the desktop image to determine the areas that require the texture. Then, you adjust the color and brightness of the wood texture to match the lighting and color of the desktop. Finally, the texture is mapped onto the desktop and blended with other parts to create a realistic wooden desktop image.

[0166] In step 1053 , an adjacent object map of the initial object map is determined among the multiple initial object maps of the virtual object.

[0167] In some embodiments, the adjacent object texture of the initial object texture refers to an initial object texture that is adjacent to the initial object texture on the virtual object, that is, the adjacent object texture of the initial object texture and the initial object texture are adjacent to each other on the virtual object.

[0168] In some embodiments, the number of neighboring object maps of the initial object map is a positive integer greater than or equal to one.

[0169] In some embodiments, in computer graphics, texture mapping is a process of mapping a texture (usually a two-dimensional image) to the surface of a three-dimensional model to create a more realistic visual effect. When a virtual object is composed of multiple initial object maps, these maps may need to be connected to each other to maintain overall consistency and continuity. Determining the adjacent object maps of the initial object map refers to identifying and determining other initial object maps that are adjacent to a specific initial object map on the surface of the virtual object. For each initial object map, its adjacent area on the surface of the virtual object is identified, and the seams or transition areas between adjacent maps are determined through edge detection, feature matching and other technologies.

[0170] As an example, suppose there is a face model composed of multiple maps, each of which represents a different area of ​​the face, such as the eyes, nose, mouth, etc. When mapping these maps to the face model, it is necessary to ensure that the eye map is correctly adjacent to the nose and mouth maps to maintain the overall coherence of the face.

[0171] In step 1054, the candidate texture corresponding to the initial object texture is smoothed based on the candidate texture corresponding to the adjacent object texture to obtain a target object texture corresponding to the initial object texture.

[0172] In some embodiments, the above-mentioned smoothing processing refers to adjusting the color values ​​of the candidate maps so that the color difference between the candidate maps corresponding to the initial object map and the candidate maps of the adjacent object maps tends to be stable, so as to avoid map distortion due to excessive color difference, thereby effectively improving the color value stability of the target object map.

[0173] In some embodiments, the above step 1054 can be implemented as follows: the following processing is performed on each pixel point in the candidate map corresponding to the initial object map to obtain the target object map: when the minimum distance between the pixel point and the candidate map corresponding to the adjacent object map is less than a distance threshold, the pixel point is determined as the pixel point to be smoothed; from the pixel points of the candidate map corresponding to the adjacent object map, the reference pixel point closest to the pixel point to be smoothed is determined; the color value of the reference pixel point and the color value of the pixel point to be smoothed are averaged to obtain an average color value; and the color value of the pixel point to be smoothed is adjusted to the average color value.

[0174] As an example, consider a 3D model whose surface is composed of multiple initial object maps. These maps need to be smoothly blended together to create a continuous appearance. This process may encounter holes or sharp edges on the model's surface where the distance between adjacent maps can be very small, resulting in noticeable seams in the mapped image. To address this issue, smoothing can be used to gently blend adjacent maps. The 3D model's surface needs to be analyzed to determine which maps are adjacent. For example, the left and right sides of the model may be composed of different maps, yet they are continuous on the model's surface. For each pixel in a map, the minimum distance between it and the corresponding candidate map in the adjacent map is calculated. This distance can be calculated by calculating the Euclidean distance between the two pixels. If the minimum distance between a pixel and the candidate map corresponding to the adjacent object map is less than a set distance threshold, the pixel is identified as a candidate for smoothing. From the pixels in the candidate map corresponding to the adjacent object map, the reference pixel closest to the pixel to be smoothed is found. This reference pixel is used for smoothing. The color values ​​of the reference pixel and the pixel to be smoothed are averaged. This average color value is used as the new color value for the pixel to be smoothed. The color value of the pixel to be smoothed is adjusted to the calculated average color value. This smoothes out pixels that might have previously appeared as seams, and the overall model has a more natural appearance. For example, suppose there is a pixel on one side of the model that is very close to a pixel on an adjacent map, but not completely aligned. Smoothing can bring the color values ​​of these two pixels closer together, eliminating the seam.

[0175] In some embodiments, the above-mentioned determination of the reference pixel point closest to the pixel point to be smoothed from the pixel points of the candidate map corresponding to the adjacent object map can be achieved by obtaining the distance between each pixel point and the pixel point to be smoothed, and determining the pixel point closest to the pixel point to be smoothed from the pixel points of the candidate map corresponding to the adjacent object map as the reference pixel point.

[0176] In some embodiments, assume that there is a three-dimensional model whose surface is composed of multiple initial object maps. When mapping these maps to the model surface, it is noted that in some areas, the pixels between adjacent maps may not be completely aligned, which will result in the appearance of seams or discontinuities. In order to smooth these seams, it is necessary to find the pixel on the adjacent object map that is closest to the pixel to be smoothed and use it as a reference pixel. Suppose there is a pixel on the map on one side of the model that is very close to the pixel on the adjacent map, but not completely aligned. Through calculation, it is found that a pixel on the adjacent map is closest to this pixel. Then, this closest pixel is used as the reference pixel, and its color value is used to smooth the pixel to be smoothed.

[0177] In this way, smoothing processing refers to adjusting the color values ​​of the candidate maps so that the color difference between the candidate map corresponding to the initial object map and the candidate map of the adjacent object map tends to be stable, so as to avoid map distortion due to excessive color difference, thereby effectively improving the color value stability of the target object map.

[0178] In some embodiments, after obtaining the target object map corresponding to the initial object map, the following processing can also be performed: each initial object map on the virtual object is replaced with the corresponding target object map to obtain a target virtual object, and the target virtual object satisfies the preset adjustment effect.

[0179] In this way, by obtaining multiple initial object maps of the virtual object, adjusting the description text, and controlling the virtual camera to shoot the virtual object from multiple different shooting angles, the captured images corresponding to each shooting angle are obtained. Based on the adjusted description text, the captured images are adjusted respectively to obtain reference images corresponding to each captured image. From the reference images, the target reference images corresponding to each initial object map are selected. Based on the target reference images and the initial object map, a target map that meets the preset adjustment effect is generated. In this way, since the virtual object is photographed and the photographed images corresponding to each photographing angle are obtained, the number of initial object maps of the virtual object collected is at least one, and the photographed images are adjusted respectively. The obtained reference image can realize content adjustment of at least one initial object map corresponding to the photographed image. By using the reference image corresponding to the initial object map, that is, the target map generated based on the target reference image, the adjustment of at least one initial object map can be realized, so there is no need to perform image adjustment on each initial object map separately. It is only necessary to perform image adjustment on the photographed images separately to realize adjustment of all initial object maps, thereby effectively reducing the number of image adjustments and at the same time enabling the generated target object map to meet the preset adjustment effect, thereby effectively improving the generation efficiency of the object map.

[0180] Below, an exemplary application of the embodiment of the present application in an actual application scenario of transforming a virtual object will be described.

[0181] Texture generation aims to generate corresponding 3D model texture maps based on text prompts entered by users, and has broad application prospects in games, film and television, virtual reality, 3D printing and other fields.

[0182] In the application scenario of games, the embodiments of the present application can help game producers quickly generate the model texture map assets they want, thereby speeding up the development of games and improving game production efficiency; it can help players generate texture maps of three-dimensional models of interest and import them into the game for use, thereby enhancing user participation.

[0183] In product design and manufacturing, it can help designers quickly generate texture maps corresponding to 3D models, and can use this as a basis to further optimize the generated model textures. These 3D models with texture information can then be used to guide the production process, such as manufacturing physical products through 3D printing.

[0184] In virtual and augmented reality, the embodiments of the present application can quickly generate texture maps for a large number of three-dimensional models, enrich the virtual world scene, and thus create a more realistic virtual reality (VR) and augmented reality (AR) experience.

[0185] In education and training, the embodiments of the present application can generate texture maps of three-dimensional models of objects of interest to learners, thereby obtaining a more three-dimensional and realistic 3D experience, helping learners to better observe and understand specific object concepts and enhance learning fun.

[0186] This embodiment of the application first generates a multi-view album based on a user-provided 3D model and a textual description of the target texture mapping effect. Based on the multi-view album, a consistent model texture map is then generated. This not only enables rapid model texture generation but also ensures sufficient richness of the generated model texture.

[0187] Typically, a complete three-dimensional model consists of two parts: geometry and texture. Geometry is generally represented by a triangular mesh; texture is represented by a two-dimensional image. If a three-dimensional model has geometric information but no texture information, it is called a white model. The embodiment of the present application aims to generate reasonable texture maps for a textureless white model. The core parts are a multi-view book generation method based on Depth2Image and a model texture generation method based on graph optimization. The multi-view book generation method based on Depth2Image is responsible for generating observation images of the three-dimensional model at different angles according to the text prompts given by the user. The observation images obtained at different angles are generally consistent, but there will be differences in local details. In order to reduce the impact of such differences on the generated texture, a model texture generation method based on graph optimization is further proposed to obtain a more visually coordinated model texture.

[0188] In some embodiments, referring to FIG8 , FIG8 is a schematic diagram of the principle of the method for generating an object map provided by an embodiment of the present application. Assuming that the given white model is M raw , then the process of generating the multi-view book is as follows: first, standardize the white film by normalizing the white model coordinates and standardizing the data input. First, calculate the white model M raw The geometric center is the geometric center of the M raw The average coordinate of all vertex coordinates, assuming that the coordinate is V mean =[Xmean ,Y mean ,Z mean Then update M raw Vertex coordinates, assuming M raw There are N vertices, V i =[X i ,Y i ,Z i ] is any vertex of it, then the corresponding vertex coordinates after update are The calculation process is as follows:

[0189] d max =max{d1,d2,…,d N} (2)

[0190] Among them, s = 0.8 is the scaling factor of the normalized coordinates, and the standardized white model is marked as M norm , d i Used to indicate the vertex V i With the mean value coordinate V mean The distance between It is used to indicate the standard position of the object parts described above in the virtual scene. The horizontal coordinate indicating the standard position in the virtual scene described above, The vertical coordinate used to indicate the standard position in the virtual scene described above, The vertical coordinate used to indicate the standard position in the virtual scene described above, X i The horizontal axis component, Y, indicates the distance corresponding to the object part described above. i The vertical axis component, Z, is used to indicate the distance corresponding to the object part described above. i The vertical axis component, d, indicates the distance corresponding to the object part described above. max Used to indicate the reference distance described above, X mean The horizontal coordinate used to indicate the center position described above, Y mean The vertical coordinate used to indicate the center position described above, Z mean The vertical coordinate used to indicate the center position described above.

[0191] In some embodiments, referring to FIG8 , for multi-view depth atlas generation, based on the standardized M norm Calculate the depth maps of multiple perspectives and stitch them into a complete multi-view depth atlas. First, determine the R rendering perspectives and calculate the corresponding camera parameter list {C1, C2, ..., C R}. Then use M normThe geometric center of the sphere is the coordinate origin, and the camera is rotated with a fixed sphere radius to render the 3D model, and R depth maps {D1, D2, ..., D R Then extract the valid areas of the depth maps from all viewing angles (image areas with depth values ​​greater than 0) and stitch them into a multi-view depth atlas A. Depth For the depth map D r , the embodiment of this application uses These five parameters are used to mark its valid area, r represents the index of the corresponding depth map, U r 、V r Indicates that the effective area is D r The starting position in W r 、H r It represents the length and width of the bounding box of the valid area.

[0192] In some embodiments, referring to FIG8 , for multi-view album generation, based on the multi-view depth atlas A in the previous step Depth Calculate the corresponding multi-view book. Assume that the trained Depth2Image model is f D2I , then the generated multi-view album can be expressed as A Image :

[0193] A Image =f D2I (A Depth ,prompt ) (7)

[0194] Here, prompt represents the text prompt input by the user for controlling the generation of the multi-view book.

[0195] In some embodiments, referring to FIG8 , for multi-view image restoration, the multi-view album A generated by segmentation is Image To obtain images of the same resolution at each rendering perspective. Segment the images corresponding to the valid areas of the depth maps at each perspective from the multi-view book and replace the valid areas of the corresponding depth maps to obtain the corresponding multi-view image list {I′1, I′2, …, I′ R}, I r Corresponding depth map D r .

[0196] In some embodiments, referring to FIG8 , for multi-view image upsampling, the resolution of each multi-view image is increased to improve the quality of model texture generation. Assuming that the image super-resolution network is f super , then the multi-view image after resolution improvement is {I1,I2,…,I R}, then:

[0197] I r =f super (I′r ) (8)

[0198] In some embodiments, for the training of the multi-viewbook generation network, the training data is: first, a batch of 3D models with textures are collected; then, the white models are normalized and the camera parameters {C1, C2, ..., C R} Render the model to obtain the depth map and rendering image of the corresponding viewpoint, and obtain the corresponding multi-view depth map and multi-view album based on the rendered depth map and rendering image; then use CLIP technology to extract the text description of the rendering image, and use manual review to ensure the accuracy of the text description. Therefore, for each 3D model M i , the following triple information can be obtained:

[0199] TrainSample i Indicates that from M i The obtained training sample data, represents the multi-view depth atlas, Indicates multiple viewbooks, prompt i Indicates the corresponding text prompt.

[0200] In some embodiments, as for the training strategy, the embodiment of the present application adopts the Depth2Image neural network model based on Diffusion to generate the multi-view album. Therefore, during training, in order to fully utilize the image generation capability of the original network model, the embodiment of the present application adopts a network fine-tuning scheme based on LORA (Low-Rank Adaptation) to quickly train the neural network model f that generates the multi-view album based on the multi-view depth map. D2I .

[0201] In some embodiments, referring to FIG9 , FIG9 is a schematic diagram showing the principle of a method for generating an object map according to an embodiment of the present application, and obtaining a multi-view image set {I1, I2, …, I R} and its corresponding camera parameters {C1, C2, ..., C R After that, an undirected graph model is first constructed based on the Markov random field model to determine the appropriate texture image source for each triangle on the white model and generate an initial texture map. Then, the initial texture map is optimized to improve the overall map quality. The object map generation method provided in the embodiment of the present application can implement graph-optimized model texture generation through steps 201 to 203 shown in FIG9 . Steps 201 to 203 are described below.

[0202] In step 201 , an undirected graph for selecting an initial texture is constructed.

[0203] In some embodiments, see FIG10, which is a schematic diagram of the principle of the undirected graph provided by the embodiment of the present application, for constructing the undirected graph required for the initial texture selection. k , are all regarded as a vertex of the undirected graph, and all vertices are represented as set V1; and F k Each triangle with a common edge forms an edge, and all edges are represented as a set Φ1. For the white mold example in Figure 10, the white mold includes triangles F1, F3, Fk, and F2, which correspond to vertices F1, F3, Fk, and F2 in the undirected graph of the white mold, respectively.

[0204] In step 202, an initial texture selection is performed based on graph optimization.

[0205] In some embodiments, the initial texture selection based on graph optimization is constructed and minimized to obtain the initial texture information. First, a label is assigned to each image in the multi-view image set. Assume that image I i Corresponding label l i First, construct the Markov random field and express its energy as E(l):

[0206] Among them, E data (F k ,l i ) means label l i Assigned to triangle F k The energy consumed, E smooth (F k ,F j ,l k ,l j ) means label l i Assigned to triangle F k , and at the same time label l j Assigned to triangle F j The energy required.

[0207] In some embodiments, E data (F k ,l i ) is defined as follows:

[0208] E data (F k ,l i )=exp(-Area(F k ,I i ,C i )) (11)

[0209] Among them, Area(Fk ,I i ,C i ) represents triangle F k Passing Camera C i In image I i The projected area on the surface of the Earth is exp, which represents the exponential function with the natural constant e as the base.

[0210] In some embodiments, E smooth (F k ,F j ,l k ,l j ) is defined using the Potts model:

[0211] E smooth (F k ,F j ,l k ,l j )=[l k ≠l j ] (12)

[0212] Where [·] is the Iverson Bracket. Then, the multi-label graph cut algorithm is used to minimize the energy E(l). The solution is obtained, which is each triangle F of the white model. k The most reasonable label is selected, that is, the most reasonable texture source, so that the initial texture of the white model can be obtained.

[0213] In step 203, initial texture optimization is performed.

[0214] In some embodiments, the texture quality of the white model is improved. After obtaining the initial texture of the white model, if the textures of adjacent triangles of the model come from images with different perspectives, visual gaps may occur. In this embodiment of the application, such triangles are referred to as edge triangles, and the common sides of the triangles are referred to as edge edges.

[0215] To solve this problem, the embodiment of the present application will optimize the colors of the vertices at both ends of the edge. First, the vertex of each edge is split, and for any vertex v of the edge, i , splitting it into and And denote all the vertices of the edge as set V2, and all the edges as set Φ2. Then construct the following formula to optimize the texture:

[0216] in, Represent the vertices after splitting and The color value of Represent the vertices after splitting and The color change value, Represents the vertices v at both ends of the edge i 、v j The color change value, Δ represents the parameter set composed of the color change value to be optimized. For any edge vertex v i , the color change value calculation rules are as follows:

[0217] in, Represented by vertex v i The color change values ​​of all the vertices split out, Indicates the area of ​​the triangle where the corresponding split vertex is located, N i Indicates v i The maximum value of all split vertex labels. f(Δ) can be converted into a large linear system and solved using the conjugate gradient method. After texture optimization is complete, the color values ​​of the corresponding vertices are updated to the texture image to obtain the final texture map asset corresponding to the white model.

[0218] In this way, the texture of the white model is automatically generated: the user only needs to give a text description of the target model texture to generate the corresponding texture map for the white model. Therefore, the user does not need to pay attention to complex operation processes such as UV unfolding of the white model, and can obtain a large number of optional model texture maps. The model texture is generated quickly: when the embodiment of the present application is applied, the multi-view album generation stage only involves an efficient neural network reasoning process, and the multi-label graph cut algorithm and conjugate gradient method relied on in the texture generation stage are fast and stable. Therefore, the texture generation speed of the model is fast, and the target texture map of the white model can be quickly obtained. A variety of texture maps can be generated: the embodiment of the present application can generate a variety of texture maps based on the text prompts given by the user, which can meet the different needs of different users.

[0219] In this way, by obtaining multiple initial object maps of the virtual object, adjusting the description text, and controlling the virtual camera to shoot the virtual object from multiple different shooting angles, the captured images corresponding to each shooting angle are obtained. Based on the adjusted description text, the captured images are adjusted respectively to obtain reference images corresponding to each captured image. From the reference images, the target reference images corresponding to each initial object map are selected. Based on the target reference images and the initial object map, a target map that meets the preset adjustment effect is generated. In this way, since the virtual object is photographed and the photographed images corresponding to each photographing angle are obtained, the number of initial object maps of the virtual object collected is at least one, and the photographed images are adjusted respectively. The obtained reference image can realize content adjustment of at least one initial object map corresponding to the photographed image. By using the reference image corresponding to the initial object map, that is, the target map generated based on the target reference image, the adjustment of at least one initial object map can be realized, so there is no need to perform image adjustment on each initial object map separately. It is only necessary to perform image adjustment on the photographed images separately to realize adjustment of all initial object maps, thereby effectively reducing the number of image adjustments and at the same time enabling the generated target object map to meet the preset adjustment effect, thereby effectively improving the generation efficiency of the object map.

[0220] It is understandable that in the embodiments of the present application, when the embodiments of the present application are applied to specific products or technologies, data related to the initial object map of the virtual object, etc., need to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0221] The following continues to describe an exemplary structure of the object map generation device 455 provided in an embodiment of the present application implemented as a software module. In some embodiments, as shown in FIG2 , the software modules stored in the object map generation device 455 of the memory 450 may include: an acquisition module 4551, configured to acquire multiple initial object maps of a virtual object, and an adjustment description text for describing a preset adjustment effect on the virtual object, wherein the initial object map is the smallest unit constituting the surface map of the virtual object; a shooting module 4552, configured to acquire multiple initial object maps of a virtual object, and an adjustment description text for describing a preset adjustment effect on the virtual object, wherein the initial object map is the smallest unit constituting the surface map of the virtual object; The map is the smallest unit of the surface map of the virtual object; the adjustment module 4553 is configured to perform image adjustments on each of the captured images based on the adjustment description text to obtain a reference image corresponding to each of the captured images, and the reference image satisfies the preset adjustment effect; the selection module 4554 is used to select the target reference image corresponding to each of the initial object maps from the reference image, and the virtual camera can capture the corresponding initial object map at the shooting angle corresponding to the target reference image; the generation module 4555 is configured to generate a target object map that meets the preset adjustment effect based on the target reference image and the initial object map.

[0222] In some embodiments, the above-mentioned shooting module is further configured to obtain the center position of the geometric center of the virtual object in the virtual scene, and based on the center position, standardize the virtual object in the virtual scene to obtain the target virtual object; control the virtual camera to shoot the target virtual object from multiple different shooting angles to obtain the captured images corresponding to each shooting angle.

[0223] In some embodiments, the above-mentioned shooting module is further configured to determine multiple shooting positions in the virtual scene whose distance from the center position is equal to the target distance, and the shooting angles correspond one-to-one to the shooting positions; for each of the shooting positions, the virtual camera is controlled to be at the shooting position and face the target virtual object to shoot the target virtual object, and obtain a shooting image corresponding to the corresponding shooting angle.

[0224] In some embodiments, the above-mentioned shooting module is further configured to obtain the positions of multiple object parts of the virtual object in the virtual scene, determine the distance between each of the positions and the center position, and determine the maximum distance as the reference distance; perform the following processing on each of the object parts of the virtual object to obtain the target virtual object: divide the distance corresponding to the object part by the reference distance to obtain the standard position of the object part in the virtual scene; adjust the position of the object part to the standard position.

[0225] In some embodiments, the above-mentioned adjustment module is further configured to call an image adjustment network, and based on the adjustment description text, adjust the image content of each of the captured images to obtain candidate images corresponding to each of the captured images, and the candidate images satisfy the preset adjustment effects described in the adjustment description text; and perform super-resolution processing on each of the candidate images to obtain the reference images corresponding to each of the candidate images.

[0226] In some embodiments, the above-mentioned object map generation device further includes: a training module, configured to obtain an initial image adjustment network, and obtain an adjustment description text sample, a label image, and a captured image sample, wherein the label image satisfies the preset adjustment effect described by the text sample; calling the initial image adjustment network, and adjusting the image content of the captured image sample based on the adjustment description text sample to obtain an adjusted image corresponding to the captured image sample; determining the loss value of the initial image adjustment network based on the adjusted image and the label image, and training the initial image adjustment network based on the loss value to obtain the image adjustment network.

[0227] In some embodiments, the selection module is further configured to obtain a reference shooting position corresponding to each of the reference images, the reference shooting position being used to indicate the position of the virtual camera in the virtual scene when shooting the image corresponding to the reference image; and perform the following processing for each of the initial object maps: for each of the reference images, based on the reference shooting position corresponding to the reference image and the initial object map, when it is determined that the virtual camera can capture the initial object map at the shooting angle corresponding to the reference image, determine the reference image as a candidate image of the initial object map; and determine the target reference image corresponding to the initial object map from the candidate images of the initial object map.

[0228] In some embodiments, the selection module is further configured to obtain the mapping position of the initial object map in the virtual scene, and connect the mapping position and the reference shooting position in the virtual scene to obtain a virtual detection line; when the virtual detection line does not pass through other initial object maps in the virtual scene, it is determined that the virtual camera can capture the initial object map at the shooting angle corresponding to the reference image; when the virtual detection line passes through other initial object maps in the virtual scene, it is determined that the virtual camera cannot capture the initial object map at the shooting angle corresponding to the reference image.

[0229] In some embodiments, the selection module is further configured to, when the number of candidate images of the initial object map is one, determine the candidate image as the target reference image corresponding to the initial object map; when the number of candidate images of the initial object map is multiple, determine the degree of association between the initial object map and each of the candidate images; and determine the candidate image corresponding to the largest degree of association as the target reference image corresponding to the initial object map.

[0230] In some embodiments, the selection module is further configured to determine the adjacent object map of the initial object map among the multiple initial object maps of the virtual object, and perform the following processing for each candidate image: determine a first degree of association of the candidate image based on the adjacent object map and the candidate image; determine an imaging area of ​​the initial object map in the candidate image, and determine a second degree of association of the candidate image based on the area of ​​the imaging area, wherein the value of the second degree of association is positively correlated with the area of ​​the imaging area; sum the first degree of association and the second degree of association to obtain the degree of association between the initial object map and the candidate image.

[0231] In some embodiments, the selection module is further configured to determine the first degree of association of the candidate image as a first value, which is a non-zero constant, when the candidate image is a candidate image of the adjacent object map; and to determine the first degree of association of the candidate image as a second value, which is equal to zero, when the candidate image is not a candidate image of the adjacent object map.

[0232] In some embodiments, the above-mentioned generation module is further configured to perform the following processing on each of the initial object maps: determining the imaging area of ​​the initial object map in the target reference image corresponding to the initial object map; adjusting the image content of the initial object map to the image content of the imaging area in the target reference image, and obtaining a candidate map corresponding to the initial object map; determining the adjacent object map of the initial object map among the multiple initial object maps of the virtual object; and smoothing the candidate map corresponding to the initial object map based on the candidate map corresponding to the adjacent object map, and obtaining a target object map corresponding to the initial object map.

[0233] In some embodiments, the generation module is further configured to perform the following processing on each pixel point in the candidate map corresponding to the initial object map to obtain the target object map: when the minimum distance between the pixel point and the candidate map corresponding to the adjacent object map is less than a distance threshold, determine the pixel point as a pixel point to be smoothed; determine the reference pixel point closest to the pixel point to be smoothed from the pixel points of the candidate map corresponding to the adjacent object map; average the color value of the reference pixel point and the color value of the pixel point to be smoothed to obtain an average color value; and adjust the color value of the pixel point to be smoothed to the average color value.

[0234] In some embodiments, the above-mentioned adjustment module is further configured to call the encoding layer, encode the image content of each of the captured images based on the adjustment description text, and obtain the image features corresponding to each of the captured images; and call the decoding layer, adjust the image content of each of the captured images based on the image features, and obtain the candidate images corresponding to each of the captured images.

[0235] In some embodiments, the generation module is further configured to replace each of the initial object maps on the virtual object with the corresponding target object map to obtain a target virtual object, and the target virtual object satisfies a preset adjustment effect.

[0236] In some embodiments, the acquisition module is further configured to generate an adjustment description text for describing a preset adjustment effect on the virtual object in response to an editing operation on the adjustment description text.

[0237] An embodiment of the present application provides a computer program product, comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the object map generation method described in the embodiment of the present application.

[0238] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the processor will execute the object map generation method provided by the embodiment of the present application, for example, the object map generation method shown in Figure 3.

[0239] In some embodiments, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), a flash memory, a magnetic surface memory, an optical disk, or a CD-ROM; or it may be various electronic devices including one or any combination of the above memories.

[0240] In some embodiments, computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0241] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0242] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0243] By way of example, computer-executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed across multiple sites and interconnected by a communication network.

[0244] In summary, the embodiments of the present application have the following beneficial effects:

[0245] (1) By obtaining multiple initial object maps of a virtual object and adjusting description texts, and controlling a virtual camera to shoot the virtual object from multiple different shooting angles, a shooting image corresponding to each shooting angle is obtained. Based on the adjusted description texts, the shooting images are adjusted to obtain reference images corresponding to each shooting image. From the reference images, a target reference image corresponding to each initial object map is selected. Based on the target reference image and the initial object map, a target map that meets the preset adjustment effect is generated. In this way, since the virtual object is photographed and the photographed images corresponding to each photographing angle are obtained, the number of initial object maps of the virtual object collected is at least one, and the photographed images are adjusted respectively. The obtained reference image can realize content adjustment of at least one initial object map corresponding to the photographed image. By using the reference image corresponding to the initial object map, that is, the target map generated based on the target reference image, the adjustment of at least one initial object map can be realized, so there is no need to perform image adjustment on each initial object map separately. It is only necessary to perform image adjustment on the photographed images separately to realize adjustment of all initial object maps, thereby effectively reducing the number of image adjustments and at the same time enabling the generated target object map to meet the preset adjustment effect, thereby effectively improving the generation efficiency of the object map.

[0246] (2) The above-mentioned standardization processing refers to the process of standardizing the virtual object to obtain a standardized target virtual object. Since different object parts of the virtual object may have protrusions, resulting in an uneven surface of the virtual object, the protrusions easily cause visual occlusion of the concave parts, resulting in the subsequent captured images being unable to accurately capture the concave parts of the virtual object. In order to make the subsequent captured images more complete and comprehensive, by performing standardization processing on the virtual object, the subsequent captured images can accurately capture the concave parts before the standardization processing, so that the captured images can accurately and comprehensively describe the virtual object from the corresponding shooting angle.

[0247] (3) When there are multiple candidate images for the initial object map, the candidate image corresponding to the largest correlation degree is determined as the target reference image corresponding to the initial object map. Since the correlation degree can accurately reflect the correlation relationship between the candidate image and the initial object map, the most correlated candidate image is determined as the target reference image of the initial object map, so that the target object map of the initial object map determined subsequently based on the candidate image can accurately express the preset adjustment effect, so that the obtained target object map is more accurate.

[0248] (4) By super-resolution processing each candidate image, a reference image corresponding to each candidate image is obtained. While keeping the image content of the original candidate image unchanged, the image resolution of the candidate image can be significantly improved, thereby effectively improving the image quality of the candidate image, so that the reference image can reflect the virtual object more clearly and accurately.

[0249] (5) Smoothing processing refers to adjusting the color values ​​of the candidate maps so that the color difference between the candidate maps corresponding to the initial object map and the candidate maps of the adjacent object maps tends to be stable, so as to avoid distortion of the map due to excessive color difference, thereby effectively improving the color value stability of the target object map.

[0250] (6) Automatic generation of white model texture: The user only needs to provide a text description of the target model texture to generate the corresponding texture map for the white model. Therefore, the user does not need to pay attention to the complex operation process such as UV unfolding of the white model to obtain a large number of optional model texture maps. Fast model texture generation speed: When the embodiment of the present application is applied, the multi-view album generation stage only involves an efficient neural network reasoning process, and the multi-label graph cut algorithm and conjugate gradient method relied on in the texture generation stage are fast and stable. Therefore, the model texture generation speed is fast, and the target texture map of the white model can be quickly obtained. Ability to generate diverse texture maps: The embodiment of the present application can generate diverse texture maps based on the text prompts given by the user, which can meet the different needs of different users.

[0251] (7) By using adjustment description text, the intention of adjusting the virtual object can be clearly conveyed, which helps ensure that the adjustment of the virtual object can meet the design requirements and user expectations. In practical applications, such adjustment description text may exist in the form of programming interface call parameters, configuration files, user input instructions, etc., and is used to guide the rendering and interaction process of the virtual object.

[0252] (8) Create a virtual scene that includes the virtual object you want to shoot, and determine multiple shooting angles. These angles can be fixed predefined positions or dynamically calculated according to specific needs. For example, you can define the following shooting angles: Frontal perspective: The camera faces the object directly. Side perspective: The camera is located to the side of the object. Top-down perspective: The camera is located directly above the object. Oblique perspective: The camera is located at a corner of the object, creating an oblique effect. For each shooting angle, set the position and direction of the virtual camera. Use the rendering function of the virtual engine to render the scene. During rendering, the virtual engine calculates the lighting, shadows, and perspective of the scene based on the position and direction of the camera. After rendering, an image representing the specific shooting angle will be obtained. This image can be 2D or 3D, depending on the capabilities of the virtual engine and the complexity of the scene. The images obtained from each angle are stored for subsequent use. These images can be used for various purposes, such as virtual reality applications, animation production, game development, etc. In this way, images of the virtual object can be obtained from multiple angles, and each image reflects the perspective of the virtual camera at a specific angle. These images can be used to create dynamic visual effects and enhance the user's immersion in the virtual environment.

[0253] (9) By obtaining the position of each part of the virtual object in the virtual scene, the virtual object can be accurately placed and positioned to ensure that the spatial relationship of the virtual object in the virtual environment meets expectations. By calculating the distance between each part and the center position and determining the maximum distance as the reference distance, a unified benchmark can be provided for subsequent position adjustments. By dividing the position distance of each part by the reference distance, the original position information can be converted into proportional information, so that the position relationship of each part can remain consistent even in virtual scenes of different sizes. By adjusting the position of the part to a standard position based on the reference distance, the virtual object can be made more visually balanced and coordinated, improving the overall aesthetics. It can adapt to different virtual environments because it is adjusted based on the distance ratio. Therefore, as long as the consistency of the reference distance is guaranteed, the consistency of the object can be maintained in different scenes.

[0254] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A method for generating an object map, the method comprising: Acquire multiple initial object maps of a virtual object, and acquire adjustment description text for describing a preset adjustment effect on the virtual object, wherein the initial object map is a minimum unit constituting a surface map of the virtual object; Controlling a virtual camera to shoot the virtual object from a plurality of different shooting angles to obtain shooting images corresponding to the shooting angles; Based on the adjustment description text, perform image adjustment on each of the captured images to obtain a reference image corresponding to each of the captured images, wherein the reference image satisfies the preset adjustment effect; Selecting target reference images corresponding to the initial object maps from the reference images, wherein the virtual camera can capture the corresponding initial object maps at the shooting angle corresponding to the target reference images; A target object map that satisfies the preset adjustment effect is generated based on the target reference image and the initial object map.

2. The method according to claim 1, wherein: The controlling the virtual camera to shoot the virtual object from a plurality of different shooting angles to obtain shooting images corresponding to the shooting angles respectively includes: Acquire the center position of the geometric center of the virtual object in the virtual scene, and based on the center position, perform standardization processing on the virtual object in the virtual scene to obtain a target virtual object; The virtual camera is controlled to shoot the target virtual object from a plurality of different shooting angles to obtain the shooting images corresponding to the respective shooting angles.

3. The method according to claim 1 or 2, wherein: The controlling the virtual camera to shoot the target virtual object from a plurality of different shooting angles to obtain the shooting images corresponding to the respective shooting angles includes: Determine a plurality of shooting positions in the virtual scene whose distances from the center position are equal to the target distance, wherein the shooting angles correspond to the shooting positions one by one; For each of the shooting positions, the virtual camera is controlled to be at the shooting position and face the target virtual object to shoot the target virtual object, thereby obtaining a shooting image corresponding to the corresponding shooting angle.

4. The method according to claim 1 or 2, wherein: The step of performing standardization processing on the virtual object in the virtual scene based on the center position to obtain a target virtual object includes: Acquire positions of multiple object parts of the virtual object in the virtual scene, determine the distances between each of the positions and the center position, and determine the largest distance as a reference distance; The following processing is performed on each object part of the virtual object to obtain the target virtual object: Dividing the distance corresponding to the object part by the reference distance to obtain a standard position of the object part in the virtual scene; The position of the target part is adjusted to the standard position.

5. The method according to claim 1, wherein: The step of performing image adjustment on each of the captured images based on the adjustment description text to obtain a reference image corresponding to each of the captured images includes: Calling an image adjustment network to adjust the image content of each of the captured images based on the adjustment description text to obtain candidate images corresponding to each of the captured images, wherein the candidate images satisfy the preset adjustment effect described in the adjustment description text; Super-resolution processing is performed on each of the candidate images to obtain the reference images corresponding to each of the candidate images.

6. The method according to claim 5, wherein: The method further comprises: Acquire an initial image adjustment network, and acquire an adjustment description text sample, a label image, and a captured image sample, wherein the label image satisfies a preset adjustment effect described by the text sample; Calling the initial image adjustment network, and adjusting the image content of the captured image sample based on the adjustment description text sample to obtain an adjusted image corresponding to the captured image sample; Based on the adjusted image and the label image, a loss value of the initial image adjustment network is determined, and based on the loss value, the initial image adjustment network is trained to obtain the image adjustment network.

7. The method according to claim 5, wherein: The image adjustment network includes a coding layer and a decoding layer. The calling of the image adjustment network adjusts the image content of each of the captured images based on the adjustment description text to obtain candidate images corresponding to each of the captured images, including: Calling the coding layer to perform image content coding on each of the captured images based on the adjustment description text to obtain image features corresponding to each of the captured images; The decoding layer is called to adjust the image content of each of the captured images based on the image features to obtain the candidate images corresponding to each of the captured images.

8. The method according to claim 1, wherein: The step of selecting target reference images corresponding to the initial object maps from the reference images comprises: Acquire a reference shooting position corresponding to each of the reference images, wherein the reference shooting position is used to indicate a position of the virtual camera in the virtual scene when shooting an image corresponding to the reference image; The following processing is performed for each of the initial object maps: For each of the reference images, based on a reference shooting position corresponding to the reference image and the initial object map, when it is determined that the virtual camera can capture the initial object map at the shooting angle corresponding to the reference image, the reference image is determined as a candidate image of the initial object map; A target reference image corresponding to the initial object map is determined from the candidate images of the initial object map.

9. The method according to claim 8, wherein: The method further comprises: Acquire a mapping position of the initial object mapping in the virtual scene, and connect the mapping position and the reference shooting position in the virtual scene to obtain a virtual detection line; When the virtual detection line does not pass through other initial object maps in the virtual scene, determining that the virtual camera can capture the initial object map at the shooting angle corresponding to the reference image; When the virtual detection line passes through other initial object maps in the virtual scene, it is determined that the virtual camera cannot capture the initial object map at the shooting angle corresponding to the reference image.

10. The method according to claim 8, wherein: The step of determining a target reference image corresponding to the initial object map from candidate images of the initial object map comprises: When the number of candidate images of the initial object map is one, determining the candidate image as a target reference image corresponding to the initial object map; When there are multiple candidate images for the initial object map, determining the degree of association between the initial object map and each of the candidate images; The candidate image corresponding to the greatest degree of association is determined as the target reference image corresponding to the initial object map.

11. The method according to any one of claims 8 to 10, wherein: Determining the degree of association between the initial object map and each of the candidate images includes: Among the multiple initial object maps of the virtual object, determine an adjacent object map of the initial object map, and perform the following processing for each candidate image: Determining a first degree of association of the candidate image based on the neighboring object map and the candidate image; Determine an imaging region of the initial object map in the candidate image, and determine a second association degree of the candidate image based on an area of ​​the imaging region, wherein a value of the second association degree is positively correlated with the area of ​​the imaging region; The first correlation degree and the second correlation degree are summed to obtain the correlation degree between the initial object map and the candidate image.

12. The method according to any one of claims 8 to 11, wherein: The determining, based on the adjacent object map and the candidate image, a first degree of association of the candidate image, comprises: When the candidate image is a candidate image of the adjacent object map, determining a first association degree of the candidate image as a first value, wherein the first value is a non-zero constant; When the candidate image is not a candidate image of the neighboring object map, the first association degree of the candidate image is determined as a second value, and the second value is equal to zero.

13. The method according to claim 1, wherein: The step of generating a target object map that satisfies the preset adjustment effect based on the target reference image and the initial object map comprises: The following processing is performed for each of the initial object maps: Determining, from the target reference image corresponding to the initial object map, an imaging area of ​​the initial object map in the target reference image; Adjusting the image content of the initial object map to the image content of the imaging area in the target reference image to obtain a candidate map corresponding to the initial object map; determining, among the plurality of initial object maps of the virtual object, an adjacent object map of the initial object map; Based on the candidate maps corresponding to the adjacent object maps, the candidate maps corresponding to the initial object maps are smoothed to obtain a target object map corresponding to the initial object map.

14. The method according to claim 13, wherein: The step of smoothing the candidate texture corresponding to the initial object texture based on the candidate texture corresponding to the adjacent object texture to obtain the target object texture corresponding to the initial object texture includes: The following processing is performed on each pixel point in the candidate map corresponding to the initial object map to obtain the target object map: When the minimum distance between the pixel point and the candidate map corresponding to the adjacent object map is less than a distance threshold, the pixel point is determined as a pixel point to be smoothed; Determine, from the pixels of the candidate map corresponding to the adjacent object map, a reference pixel point closest to the pixel point to be smoothed; Averaging the color value of the reference pixel and the color value of the pixel to be smoothed to obtain an average color value; The color value of the pixel to be smoothed is adjusted to the average color value.

15. The method according to claim 1, wherein: After generating the target object map satisfying the preset adjustment effect based on the target reference image and the initial object map, the method further includes: Each of the initial object maps on the virtual object is replaced with the corresponding target object map to obtain a target virtual object, and the target virtual object satisfies a preset adjustment effect.

16. The method according to claim 1, wherein: The obtaining of an adjustment description text for describing a preset adjustment effect on the virtual object includes: In response to an editing operation on the adjustment description text, an adjustment description text for describing a preset adjustment effect on the virtual object is generated.

17. A device for generating an object map, the device comprising: An acquisition module configured to acquire a plurality of initial object maps of a virtual object and acquire an adjustment description text for describing a preset adjustment effect on the virtual object, wherein the initial object map is a minimum unit constituting a surface map of the virtual object; A shooting module, configured to control a virtual camera to shoot the virtual object from a plurality of different shooting angles, and obtain shooting images corresponding to the shooting angles; an adjustment module, configured to perform image adjustment on each of the captured images based on the adjustment description text, to obtain a reference image corresponding to each of the captured images, wherein the reference image satisfies the preset adjustment effect; A selection module is configured to select target reference images corresponding to the initial object maps from the reference images, and the virtual camera can capture the corresponding initial object map at the shooting angle corresponding to the target reference image; A generating module is configured to generate a target object map satisfying the preset adjustment effect based on the target reference image and the initial object map.

18. An electronic device, comprising: a memory configured to store computer executable instructions or computer programs; The processor is configured to implement the method for generating an object map according to any one of claims 1 to 16 when executing the computer executable instructions or computer program stored in the memory.

19. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method for generating an object map according to any one of claims 1 to 16.

20. A computer program product, comprising a computer program or a computer executable instruction, wherein when the computer program or the computer executable instruction is executed by a processor, the method for generating an object map according to any one of claims 1 to 16 is implemented.

Images