Three-dimensional model generation method and device

Abstract

The present disclosure provides a three-dimensional model generation method and device, which relates to the field of artificial intelligence technology, specifically computer vision, augmented reality, virtual reality, deep learning and other technical fields, and can be applied to scenes such as artificial intelligence content generation and virtual images. A specific implementation of the method includes: obtaining an image of an animal; selecting a low-rank adaptation LoRA of a large language model corresponding to at least one category to which the animal belongs; adjusting a basic map generation model based on the LoRA corresponding to at least one category to obtain a map generation model; inputting a normal map of the animal's tunica albuginea into the map generation model to obtain a texture map of the animal; and attaching the texture map to the tunica albuginea to generate a three-dimensional model of the animal.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically computer vision, augmented reality, virtual reality, deep learning and other technical fields, and can be applied to scenes such as artificial intelligence content generation and virtual images. Background Art

[0002] Usually, people store pictures of animals on various electronic devices. In the era of VR (Virtual Reality) / AR (Augmented Reality), people are not satisfied with only saving and displaying animals in two-dimensional form, but will explore the possibility of displaying them in three-dimensional form. Summary of the invention

[0003] The embodiments of the present disclosure provide a three-dimensional model generation method, apparatus, device, storage medium, and program product.

[0004] In a first aspect, an embodiment of the present disclosure proposes a three-dimensional model generation method, including: obtaining an image of an animal; selecting a low-rank adapted LoRA of a large language model corresponding to at least one category to which the animal belongs; adjusting a basic texture generation model based on the LoRA corresponding to at least one category to obtain a texture generation model, wherein one category corresponds to one LoRA; inputting a normal map of the animal's albuginea into the texture generation model to obtain a texture map of the animal; and fitting the texture map to the albuginea to generate a three-dimensional model of the animal.

[0005] In the second aspect, an embodiment of the present disclosure proposes a three-dimensional model generation device, including: an acquisition module, configured to acquire an image of an animal; a selection module, configured to select a low-rank adaptation LoRA of a large language model corresponding to at least one category to which the animal belongs; an adjustment module, configured to adjust a basic map generation model based on the LoRA corresponding to at least one category to obtain a map generation model, wherein one category corresponds to one LoRA; a generation module, configured to input a normal map of the animal's white membrane into the map generation model to obtain a texture map of the animal; a fitting module, configured to fit the texture map to the white membrane to generate a three-dimensional model of the animal.

[0006] In a third aspect, an embodiment of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.

[0007] In a fourth aspect, an embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to cause a computer to execute the method described in the first aspect.

[0008] In a fifth aspect, an embodiment of the present disclosure proposes a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0009] The three-dimensional model generation method provided by the embodiment of the present disclosure can generate corresponding texture maps based on the animal images provided by the user on the basis of a given animal white membrane, perform texture coloring, and realize the generation of personalized animal three-dimensional models, which can be applied to scenes such as three-dimensional asset creation or three-dimensional printing figures. Moreover, for each category of animals, only one LoRA needs to be trained, and there is no need to train LoRA for each animal image provided by the user, which reduces the training cost and training time of LoRA. In addition, the LoRA corresponding to each category can be pre-trained. When it is necessary to generate the texture map of the animal, it is only necessary to find the corresponding LoRA. There is no need to train LoRA in real time for each animal image provided by the user, which greatly shortens the time consumed in generating the three-dimensional model of the animal.

[0010] The key or important features of the disclosed embodiments are not intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, purposes and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. Among them:

[0012] Figure 1 is a flow chart of an embodiment of a method for generating a three-dimensional model according to the present disclosure;

[0013] Figure 2 is a flow chart of another embodiment of a three-dimensional model generating method according to the present disclosure;

[0014] Figure 3 is a flow chart of another embodiment of a three-dimensional model generating method according to the present disclosure;

[0015] Figure 4 is a structural schematic diagram of an embodiment of a three-dimensional model generating device according to the present disclosure;

[0016] Figure 5 It is a block diagram of an electronic device used to implement the three-dimensional model generating method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0018] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0019] Figure 1 A process 100 of an embodiment of a three-dimensional model generation method according to the present disclosure is shown. The three-dimensional model generation method comprises the following steps:

[0020] Step 101, obtaining an image of an animal.

[0021] In this embodiment, the user can input an image of an animal, wherein the source of the image of the animal is various, including but not limited to: an image of the animal taken by the user himself, an image of the animal downloaded from the Internet, and the like.

[0022] It should be noted that the animal images provided by the user usually contain only one type of animal. If the image contains multiple types of animals, the image needs to be segmented into multiple images containing only one type of animal, and then the subsequent steps are performed separately.

[0023] Step 102: Select a LoRA corresponding to at least one category to which the animal belongs.

[0024] In this embodiment, the animal in the image input by the user may belong to at least one category. Here, the LoRA (Low-Rank Adaptation of Large Language Models) corresponding to at least one category to which the animal belongs is obtained. Among them, at least one category to which the animal belongs may be determined based on the image analysis of the animal, or may be input by the user. For example, the user may input the category to which the animal belongs while inputting the image of the animal.

[0025] LoRA is a lightweight model fine-tuning training method. It fine-tunes the model based on the original large model to generate specific characters, objects or styles. This method has the characteristics of fast training speed, moderate model size, and low training configuration requirements. It can train the desired style effect with a small number of pictures.

[0026] The sample image set corresponding to each category can be used to train a LoRA. The granularity of the animal category division can be selected according to the needs. For example, when animals are divided into coarse-grained categories, the differences between different categories of animals are large, such as animals can be divided into basic categories such as cats, dogs, pigs, and cows. For another example, when animals are divided into fine-grained categories, the differences between different categories of animals are small, such as pet cats can be divided into detailed categories such as Persian cats, ragdoll cats, Garfield cats, and Chinese rural cats.

[0027] Step 103: Adjust the basic texture generation model based on the LoRA corresponding to at least one category to obtain a texture generation model.

[0028] In this embodiment, based on the LoRA corresponding to at least one category, the basic texture generation model can be adjusted to obtain a texture generation model.

[0029] The basic map generation model can be used to generate texture maps of various categories of animals. It is trained using sample texture maps of various categories of animals. First, initialize the generation model; then, input the normal map of the white membrane of animals corresponding to various categories in various postures into the generation model to obtain the predicted texture maps corresponding to various categories; then, calculate the loss value based on the predicted texture maps and sample texture maps corresponding to various categories; finally, adjust the parameters of the generation model based on the loss value until the model converges to obtain the basic map generation model.

[0030] By using LoRA corresponding to at least one category, the model is fine-tuned on the basis of the basic texture generation model, and the obtained texture generation model can generate texture maps of at least one category of animals in a targeted manner. For example, by using LoRA corresponding to Persian cats to fine-tune the basic texture generation model, the obtained texture generation model can generate texture maps of Persian cats.

[0031] Step 104 , inputting the normal map of the animal's tunica albuginea into a texture generation model to obtain a texture map of the animal.

[0032] In this embodiment, the white membrane of the animal is obtained, the normal map of the animal in various postures is rendered, and the normal map is input into the map generation model to obtain the texture map of the animal.

[0033] Step 105, attaching the texture map to the white film to generate a three-dimensional model of the animal.

[0034] In this embodiment, the texture map of an animal is attached to the albuginea of ​​the animal to generate a three-dimensional model of the animal. The texture map is a two-dimensional image, and the albuginea of ​​the animal is a three-dimensional model. The two-dimensional texture map can be attached back to the three-dimensional albuginea of ​​the animal using the MVS-Texture technology to obtain a three-dimensional model of the animal. MVS-Texturing is an open source code library for three-dimensional reconstruction, mainly used for model texture attachment.

[0035] The three-dimensional model generation method provided by the embodiment of the present disclosure can generate corresponding texture maps based on the animal images provided by the user on the basis of a given animal white membrane, perform texture coloring, and realize the generation of personalized animal three-dimensional models, which can be applied to scenes such as three-dimensional asset creation or three-dimensional printing figures. Moreover, for each category of animals, only one LoRA needs to be trained, and there is no need to train LoRA for each animal image provided by the user, which reduces the training cost and training time of LoRA. In addition, the LoRA corresponding to each category can be pre-trained. When it is necessary to generate the texture map of the animal, it is only necessary to find the corresponding LoRA. There is no need to train LoRA in real time for each animal image provided by the user, which greatly shortens the time consumed in generating the three-dimensional model of the animal.

[0036] Continue to refer Figure 2 , which shows a process 200 of another embodiment of a three-dimensional model generation method according to the present disclosure. The three-dimensional model generation method comprises the following steps:

[0037] Step 201, obtaining an image of an animal.

[0038] In this embodiment, the specific operation of step 201 is already described in Figure 1 In the illustrated embodiment, step 101 is described in detail and will not be repeated here.

[0039] Step 202, based on the image, determine the confidence that the animal belongs to each category.

[0040] In this embodiment, the images of animals are classified using a traditional image classification method or a deep learning-based image classification method to determine the confidence that the animals belong to each category.

[0041] The steps of traditional image classification methods mainly include: feature extraction and classifier training.

[0042] In the feature extraction stage, you can try various common features or self-designed features to extract features from the image. Select the backup features, try the features and weigh the pros and cons. For computers, the better the uniqueness and universality of the features, the greater the amount of calculation, so it is necessary to select the best features while ensuring speed.

[0043] After selecting features that have a moderate amount of computation and can achieve the required accuracy, a classification model is trained using traditional machine learning methods. This classification model can be used for image classification. When selecting a model, you need to try all the alternative models, select the model with the best effect, adjust the parameters, and wait for the model to converge to obtain the classification model.

[0044] Image classification methods based on deep learning mainly use neural networks for image classification.

[0045] The neural network is constructed by many neurons into layers of networks, and the activation layer makes the model have strong nonlinear fitting capabilities. You only need to input the image, and the model will automatically learn feature extraction and result mapping. Therefore, you only need to design the network results so that the better the features automatically extracted by the network, the better the effect will be and the higher the accuracy will be.

[0046] Step 203: Based on the confidence level, select the LoRA corresponding to at least one category to which the animal belongs.

[0047] In this embodiment, based on the confidence, the LoRA corresponding to at least one category to which the animal belongs may be selected. For example, the LoRA corresponding to the category with the highest confidence may be selected. For another example, the LoRA corresponding to the category with a confidence greater than a preset threshold may be selected.

[0048] Step 204: Adjust the basic texture generation model based on the LoRA corresponding to at least one category to obtain a texture generation model.

[0049] Step 205 , inputting the normal map of the animal's tunica albuginea into a texture generation model to obtain a texture map of the animal.

[0050] Step 206, attaching the texture map to the white film to generate a three-dimensional model of the animal.

[0051] In this embodiment, the specific operations of steps 204-206 are already described in Figure 1 In the illustrated embodiment, steps 103 - 105 are described in detail and will not be repeated here.

[0052] from Figure 2 It can be seen that Figure 1 Compared with the corresponding embodiment, the 3D model generation method in this embodiment highlights the LoRA selection step. Therefore, the scheme described in this embodiment can automatically classify animal images and select LoRA based on the classification results, without the need for the user to manually input the animal category, thereby reducing the user's operating costs.

[0053] Further references Figure 3, which shows a process 300 of another embodiment of a three-dimensional model generation method according to the present disclosure. The three-dimensional model generation method comprises the following steps:

[0054] Step 301, obtaining an image of an animal.

[0055] In this embodiment, the specific operation of step 301 is already described in Figure 2 In the illustrated embodiment, step 201 is described in detail and will not be described again here.

[0056] Step 302, input the image into a pre-trained classification model to obtain the confidence that the animal belongs to each category.

[0057] In this embodiment, the image of the animal is input into a pre-trained classification model to obtain the confidence that the animal belongs to each category.

[0058] The classification model can be used to classify animal images. It is trained based on deep learning methods using sample image sets corresponding to various categories. First, initialize the neural network (such as ResNet (Residual Neural Network)); then, input the sample animal images in the sample image sets corresponding to various categories into the neural network to obtain the predicted category of the animal in the input sample animal image; then, calculate the loss value based on the predicted category and the true category of the animal in the input sample animal image; finally, adjust the parameters of the neural network based on the loss value until the model converges to obtain the classification model.

[0059] Step 303: Determine whether there is a confidence level greater than a preset threshold.

[0060] In this embodiment, the confidence that the animal belongs to each category is compared with a preset threshold (e.g., 0.9) to determine whether there is a confidence greater than the preset threshold. If there is a confidence greater than the preset threshold, step 304 is executed; if there is no confidence greater than the preset threshold, step 306 is executed.

[0061] Step 304: Select the LoRA corresponding to the category with a confidence level greater than a preset threshold.

[0062] In this embodiment, if there is a confidence level greater than a preset threshold, the LoRA corresponding to the category with a confidence level greater than the preset threshold is selected.

[0063] For example, among the confidences of animals belonging to various categories, the confidence of Persian cats is greater than 0.9, indicating that the animal is a Persian cat, and the LoRA corresponding to the Persian cat is directly selected.

[0064] It should be noted that among the confidences that an animal belongs to each category, there is usually only one confidence that is greater than a preset threshold.

[0065] Step 305 , updating the weight of the basic map generation model based on the additional weight of LoRA corresponding to the category with a confidence greater than a preset threshold, to obtain a map generation model.

[0066] In this embodiment, the basic map generation model can be adjusted using the LoRA corresponding to the category with a confidence greater than a preset threshold to obtain a map generation model. Specifically, the weight of the basic map generation model is updated using the additional weight of the LoRA to obtain the map generation model.

[0067] If the weight of UNet in the basic map generation model is W, then LoRA will train an additional weight ΔW = AB T Finally, the weight of UNet in the map generation model is updated to W'=W+aΔW. Among them, a is generally 0.8, and A and B are both low-rank matrices. Unet is a variant of FCN (Fully Convolutional Networks), which is an Encoder-Decoder structure and can solve problems in biomedical images.

[0068] Step 306: Select the LoRA corresponding to the category ranked in the top n in terms of confidence.

[0069] In this embodiment, if there is no confidence greater than the preset threshold, the confidences of the animals belonging to each category can be sorted in descending order, and n confidences (such as p1, p2, ..., p3) are selected from the side with the largest value. n ), and obtain the LoRA corresponding to the selected confidence category. Where n is a positive integer less than the total number of animal categories.

[0070] For example, the confidence that an animal belongs to each category is less than 0.9, indicating that the animal may be a hybrid of multiple categories. The LoRA corresponding to the top two categories (such as Persian cats and Chinese rural cats) is selected.

[0071] Step 307: Mix the LoRAs corresponding to the categories ranked in the top n in terms of confidence to generate a mixed LoRA.

[0072] In this embodiment, the LoRAs corresponding to the categories ranked in the top n in confidence are mixed to generate a mixed LoRA. By mixing multiple LoRAs, the diversity of the map generation model can be further expanded. In other words, the map generation model obtained by fine-tuning a single LoRA can generate a texture map of a certain category of animals. The map generation model obtained by fine-tuning the mixed LoRA can generate a texture map of a hybrid breed of animals. The texture of a hybrid breed of animals is usually a mixture of textures of multiple species of animals.

[0073] In some embodiments, an additional weight of the hybrid LoRA may be generated based on the additional weights of the LoRAs corresponding to the top n confidences and the top n confidence categories.

[0074] Taking the mixing of two LoRA as an example, the additional weight of one LoRA is ΔW1=A1B1 T , the confidence of the corresponding category is p1; the additional weight of another LoRA is ΔW2=A2B2 T , the confidence of the corresponding category is p2, then the additional weight of the hybrid LoRA is ΔW 12 =(p1A1+p2A2)(p1B1+p2B2) T For multiple LoRA, the mixing method is similar.

[0075] Step 308: Adjust the basic texture generation model based on the hybrid LoRA to obtain a texture generation model.

[0076] In this embodiment, the basic map generation model is adjusted based on the hybrid LoRA to obtain the map generation model. Specifically, the weight of the basic map generation model is updated based on the additional weight of the hybrid LoRA to obtain the map generation model.

[0077] If the weight of UNet in the basic map generation model is W, the weight of the hybrid LoRA is ΔW 12 , then the weight of UNet in the map generation model is updated to W'=W+aΔW 12 Among them, a is generally taken as 0.8.

[0078] Step 309 , inputting the normal map of the animal's tunica albuginea into the texture generation model to obtain the texture map of the animal.

[0079] Step 310, attaching the texture map to the tunica albuginea to generate a three-dimensional model of the animal.

[0080] In this embodiment, the specific operations of steps 309-310 are already described in Figure 2 In the illustrated embodiment, steps 205-206 are described in detail and will not be repeated here.

[0081] from Figure 3 It can be seen that Figure 2 Compared with the corresponding embodiment, the three-dimensional model generation method in this embodiment highlights the classification step and the LoRA selection step. Therefore, the scheme described in this embodiment uses an image classification method based on deep learning, which only needs to design the network effect, without designing the extraction feature, and can take less time to achieve a better model effect. Selecting LoRA based on confidence can not only generate texture maps of purebred animals, but also generate texture maps of hybrid animals, expanding the diversity of the map generation model.

[0082] Further references Figure 4 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a three-dimensional model generation device. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0083] like Figure 4 As shown, the three-dimensional model generation device 400 of this embodiment may include: an acquisition module 401, a selection module 402, an adjustment module 403, a generation module 404 and a fitting module 405. Among them, the acquisition module 401 is configured to acquire an image of an animal; the selection module 402 is configured to select a low-rank adaptation LoRA of a large language model corresponding to at least one category to which the animal belongs; the adjustment module 403 is configured to adjust the basic map generation model based on the LoRA corresponding to at least one category to obtain a map generation model, wherein one category corresponds to one LoRA; the generation module 404 is configured to input the normal map of the animal's tunica albuginea into the map generation model to obtain the animal's texture map; the fitting module 405 is configured to fit the texture map to the tunica albuginea to generate a three-dimensional model of the animal.

[0084] In this embodiment, the specific processing of the acquisition module 401, the selection module 402, the adjustment module 403, the generation module 404 and the fitting module 405 and the technical effects thereof can be referred to in detail. Figure 3 The relevant descriptions of steps 101-105 in the corresponding embodiment are not repeated here.

[0085] In some optional implementations of this embodiment, the selection module 402 includes: a determination submodule, configured to determine the confidence that the animal belongs to each category based on the image; and a selection submodule, configured to select the LoRA corresponding to at least one category to which the animal belongs based on the confidence.

[0086] In some optional implementations of this embodiment, the determination submodule is further configured to: input the image into a pre-trained classification model to obtain the confidence that the animal belongs to each category.

[0087] In some optional implementations of this embodiment, the selection submodule is further configured to: if there is a confidence level greater than a preset threshold, select the LoRA corresponding to the category with a confidence level greater than the preset threshold; and the adjustment module 403 includes: an update submodule, configured to update the weight of the basic map generation model based on the additional weight of the LoRA corresponding to the category with a confidence level greater than the preset threshold, to obtain a map generation model.

[0088] In some optional implementations of this embodiment, the selection submodule is further configured to: if there is no confidence greater than a preset threshold, select the LoRA corresponding to the categories ranked in the top n in confidence, where n is a positive integer less than the total number of animal categories; and the adjustment module 403 includes: a mixing submodule, configured to mix the LoRA corresponding to the categories ranked in the top n in confidence to generate a mixed LoRA; an adjustment submodule, configured to adjust the basic map generation model based on the mixed LoRA to obtain a map generation model.

[0089] In some optional implementations of this embodiment, the mixing submodule is further configured to: generate additional weights of the mixed LoRA based on the additional weights of the LoRA corresponding to the top n confidences and the top n categories with confidences; and the adjustment submodule is further configured to: update the weights of the basic map generation model based on the additional weights of the mixed LoRA to obtain the map generation model.

[0090] In some optional implementations of this embodiment, LoRA corresponding to various categories are obtained by training using sample image sets corresponding to various categories.

[0091] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0092] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0093] Figure 5A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0094] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0095] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0096] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the three-dimensional model generation method. For example, in some embodiments, the three-dimensional model generation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the three-dimensional model generation method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the three-dimensional model generation method in any other appropriate manner (e.g., by means of firmware).

[0097] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0099] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0100] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0101] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0102] A computer system may include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises through computer programs running on respective computers and having a client-server relationship to each other. The server may be a cloud server, a server in a distributed system, or a server combined with a blockchain.

[0103] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions provided by this disclosure can be achieved, and this document does not limit this.

[0104] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A three-dimensional model generation method, comprising: Obtain images of animals; Based on the image, determining a confidence that the animal belongs to each category; Based on the confidence, selecting a low-rank adaptation LoRA of a large language model corresponding to at least one category to which the animal belongs; Adjusting the basic map generation model based on the LoRA corresponding to the at least one category to obtain a map generation model, wherein one category corresponds to one LoRA; Inputting the normal map of the tunica albuginea of ​​the animal into the texture generation model to obtain a texture map of the animal; attaching the texture map to the albuginea to generate a three-dimensional model of the animal; The step of adjusting the basic map generation model based on the LoRA corresponding to the at least one category to obtain the map generation model includes: If the LoRA corresponding to the category with a confidence level greater than a preset threshold is selected, the weight of the basic map generation model is updated based on the additional weight of the LoRA corresponding to the category with a confidence level greater than the preset threshold to obtain the map generation model.

2. The method according to claim 1, wherein: Determining the confidence that the animal belongs to each category based on the image includes: The image is input into a pre-trained classification model to obtain the confidence that the animal belongs to each category.

3. The method according to claim 1, wherein: The selecting, based on the confidence, a LoRA corresponding to at least one category to which the animal belongs, comprises: If there is no confidence greater than the preset threshold, select the LoRA corresponding to the category with the top n confidence rankings, where n is a positive integer less than the total number of categories of the animal; and The step of adjusting the basic map generation model based on the LoRA corresponding to the at least one category to obtain the map generation model includes: Mix the LoRAs corresponding to the categories ranked in the top n in confidence to generate mixed LoRAs; The basic map generation model is adjusted based on the hybrid LoRA to obtain the map generation model.

4. The method according to claim 3, wherein: The step of mixing the LoRAs corresponding to the categories ranked in the top n in terms of confidence to generate a mixed LoRA includes: Generate an additional weight of the hybrid LoRA based on the additional weights of the LoRAs corresponding to the top n confidences and the top n categories of confidence; and The step of adjusting the basic map generation model based on the hybrid LoRA to obtain the map generation model includes: The weight of the basic map generation model is updated based on the additional weight of the hybrid LoRA to obtain the map generation model.

5. The method according to any one of claims 1 to 4, wherein: The LoRA corresponding to various categories is trained using the sample image sets corresponding to various categories.

6. A three-dimensional model generating device, comprising: an acquisition module configured to acquire an image of an animal; a determination module configured to determine, based on the image, a confidence level that the animal belongs to each category; A selection module is configured to select a low-rank adaptation LoRA of a large language model corresponding to at least one category to which the animal belongs based on the confidence; An adjustment module is configured to adjust the basic map generation model based on the LoRA corresponding to the at least one category to obtain a map generation model, wherein one category corresponds to one LoRA; A generation module, configured to input the normal map of the tunica albuginea of ​​the animal into the texture generation model to obtain a texture map of the animal; A fitting module, configured to fit the texture map onto the tunica albuginea to generate a three-dimensional model of the animal; Wherein, the adjustment module includes: The update submodule is configured to update the weight of the basic map generation model based on the additional weight of the LoRA corresponding to the category with a confidence greater than a preset threshold if the LoRA corresponding to the category with a confidence greater than a preset threshold is selected, so as to obtain the map generation model.

7. The device according to claim 6, wherein: The determination module is further configured to: The image is input into a pre-trained classification model to obtain the confidence that the animal belongs to each category.

8. The device according to claim 6, wherein: The selection module is further configured to: If there is no confidence greater than the preset threshold, select the LoRA corresponding to the category with the top n confidence rankings, where n is a positive integer less than the total number of categories of the animal; and The adjustment module comprises: The mixing submodule is configured to mix the LoRAs corresponding to the categories ranked in the top n in confidence to generate a mixed LoRA; The adjustment submodule is configured to adjust the basic map generation model based on the hybrid LoRA to obtain the map generation model.

9. The device according to claim 8, wherein: The mixing submodule is further configured to: Generate an additional weight of the hybrid LoRA based on the additional weights of the LoRA corresponding to the top n confidences and the top n categories in the confidence ranking; as well as The adjustment submodule is further configured to: The weight of the basic map generation model is updated based on the additional weight of the hybrid LoRA to obtain the map generation model.

10. The device according to any one of claims 6 to 9, wherein: The LoRA corresponding to various categories is trained using the sample image sets corresponding to various categories.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images