Generation method and device of stylized map and related product
The texture map of the initial element texture map of the navigation map is modified through the texture map generation model to generate a stylized map, which solves the problem of single image style of the navigation map and improves the user interaction experience and map personalization.
Patent Information
- Application Number
- CN202411085096.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-12-06
AI Technical Summary
The navigation map has a single image style, poor user interaction experience, and lacks a diverse selection of image styles.
通过获取初始元素纹理图集和用户输入的目标文本,利用纹理图生成模型进行风格修改,生成风格化地图元素和风格化地图,实现导航地图的图像风格多元化。
Improves the interactive experience between users and navigation maps, allows users to customize the map style, and enhances the personalization and aesthetics of navigation maps.
Smart Images

Figure CN119085698A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of map generation technology, and in particular to a method and device for generating a stylized map and related products. Background Art
[0002] Navigation maps are used to support navigation needs of different users. Currently, navigation maps have a relatively simple image style, and the image styles that can be switched are also relatively few, that is, the image styles of navigation maps are mostly the same for all users, resulting in a poor interaction experience between users and navigation maps. Therefore, how to improve the user interaction experience is a key issue that technicians in this field are concerned about. Summary of the invention
[0003] Based on the above problems, the present application provides a method, device and related products for generating a stylized map to improve the user's interactive experience.
[0004] The embodiments of the present application disclose the following technical solutions:
[0005] The first aspect of the present application provides a method for generating a stylized map. The method for generating a stylized map comprises:
[0006] Acquire an initial element texture atlas and a target text input by a user, wherein an initial element texture map in the initial element texture atlas corresponds to an initial map element, and the target text includes text prompting to modify the style of the initial map element;
[0007] Extracting the initial element texture atlas according to the target text to obtain an initial element texture map in the initial element texture atlas corresponding to the initial map element indicated in the target text;
[0008] Using a texture map generation model to perform style modification processing on the initial element texture map corresponding to the initial map element according to the target text and the initial element texture map corresponding to the initial map element, so as to generate an element style texture map, wherein the texture map generation model is used to perform style modification on the initial element texture map;
[0009] A style map element is generated according to the element style texture map, and a stylized map is generated according to the style map element, wherein the style map element is an initial map element after the style is modified.
[0010] In an achievable implementation, the texture map generation model includes a first generation model and a second generation model, and the texture map generation model is used to perform style modification processing on the initial element texture map corresponding to the initial map element according to the target text and the initial element texture map corresponding to the initial map element to generate an element style texture map, including:
[0011] Using the first generation model in the texture map generation model, encoding the target text and the initial element texture map corresponding to the initial map element to obtain a fused feature vector;
[0012] Using the first generation model in the texture map generation model to modify the style of the initial element texture map corresponding to the initial map element according to the fused feature vector, so as to generate an initial style texture map;
[0013] The initial style texture map is style optimized by using a second generation model in the texture map generation model to generate an element style texture map.
[0014] In an achievable implementation, before using the first generation model in the texture map generation model to modify the style of the initial element texture map corresponding to the initial map element according to the fused feature vector to generate the initial style texture map, the method further includes:
[0015] Generate a noise texture map according to the initial element texture map corresponding to the initial map element using the first generation model in the texture map generation model;
[0016] The first generation model includes a texture map generation module, and the first generation model in the texture map generation model is used to modify the style of the initial element texture map corresponding to the initial map element according to the fused feature vector to generate an initial style texture map, including:
[0017] The texture map generation module of the first generation model in the texture map generation model is used to perform image denoising on the noise texture map according to the fused feature vector to generate an initial style texture map.
[0018] In an achievable implementation, the first generation model further includes a text encoding module, an image encoding module and a fusion encoding module, and the first generation model in the texture map generation model is used to encode the target text and the initial element texture map corresponding to the initial map element to obtain a fusion feature vector, including:
[0019] Using the text encoding module of the first generation model in the texture map generation model, the target text is encoded to obtain a text feature vector;
[0020] Using the image encoding module of the first generation model in the texture map generation model, image encoding is performed on the initial element texture map corresponding to the initial map element to obtain an image feature vector;
[0021] The text feature vector and the image feature vector are fused using a fusion coding module of a first generation model in the texture map generation model to obtain a fused feature vector.
[0022] In an achievable implementation, the first generation model is trained and obtained according to the following method:
[0023] Obtaining a first style label texture map, a sample text, and an element sample texture map corresponding to the sample text;
[0024] Inputting the sample text and the element sample texture map into the first generation model to be trained, processing the sample text and the element sample texture map using the first generation model to be trained to obtain a fusion vector, and processing the element sample texture map to obtain a noise texture map corresponding to the element sample texture map;
[0025] Using the first generation model to be trained, according to the fusion vector, to perform denoising prediction on the noise texture map corresponding to the element sample texture map, to generate a first style prediction texture map;
[0026] According to the difference between the first style label texture map and the first style prediction texture map, the parameters of the first generation model to be trained are adjusted until the adjusted model meets the training cutoff condition, and the training is completed to obtain the first generation model.
[0027] In an achievable implementation, the second generation model is trained and obtained according to the following method:
[0028] Acquire a second style label texture map and a style sample texture map, wherein the second style label texture map is a style-optimized texture map, and the style sample texture map is a non-style-optimized texture map;
[0029] Inputting the style sample texture map into the second generation model to be trained, and using the second generation model to be trained to perform style prediction optimization on the style sample texture map to obtain a second style prediction texture map;
[0030] According to the difference between the second style label texture map and the second style prediction texture map, the parameters of the second generation model to be trained are adjusted until the adjusted model meets the training cutoff condition, and the training is completed to obtain the second generation model.
[0031] In a feasible implementation, the first generation model includes a diffusion model, and the second generation model includes a lora model.
[0032] The second aspect of the present application provides a device for generating a stylized map. The device for generating a stylized map includes:
[0033] A texture map text acquisition unit, used to acquire an initial element texture atlas and a target text input by a user, wherein an initial element texture map in the initial element texture atlas corresponds to an initial map element, and the target text includes text prompting to modify the style of the initial map element;
[0034] An element texture map extraction unit is used to extract the initial element texture map set according to the target text to obtain an initial element texture map in the initial element texture map set corresponding to the initial map element prompted in the target text;
[0035] a style texture map generating unit, configured to perform style modification processing on the initial element texture map corresponding to the initial map element according to the target text and the initial element texture map corresponding to the initial map element by using a texture map generating model to generate an element style texture map, wherein the texture map generating model is used to perform style modification on the initial element texture map;
[0036] The stylized map generating unit is used to generate a style map element according to the element style texture map, and to generate a style map according to the style map element, wherein the style map element is an initial map element after style modification.
[0037] In an achievable implementation, the style texture map generating unit includes:
[0038] A fused feature vector obtaining unit, used to encode the target text and the initial element texture map corresponding to the initial map element using the first generation model in the texture map generation model to obtain a fused feature vector;
[0039] A texture map style modification unit, configured to modify the style of an initial element texture map corresponding to the initial map element according to the fused feature vector by using a first generation model in the texture map generation model to generate an initial style texture map;
[0040] The texture map style optimization unit is used to optimize the style of the initial style texture map by using the second generation model in the texture map generation model to generate an element style texture map.
[0041] In one feasible embodiment, the device further includes:
[0042] A noise texture map generating unit, configured to generate a noise texture map according to an initial element texture map corresponding to the initial map element by using a first generating model in the texture map generating model;
[0043] The texture map style modification unit is specifically used for:
[0044] The texture map generation module of the first generation model in the texture map generation model is used to perform image denoising on the noise texture map according to the fused feature vector to generate an initial style texture map.
[0045] In an achievable implementation, the fused feature vector obtaining unit is specifically used for:
[0046] Using the text encoding module of the first generation model in the texture map generation model, the target text is encoded to obtain a text feature vector;
[0047] Using the image encoding module of the first generation model in the texture map generation model, image encoding is performed on the initial element texture map corresponding to the initial map element to obtain an image feature vector;
[0048] The text feature vector and the image feature vector are fused using a fusion coding module of a first generation model in the texture map generation model to obtain a fused feature vector.
[0049] In an achievable implementation, the first generation model is trained and obtained according to the following method:
[0050] Obtaining a first style label texture map, a sample text, and an element sample texture map corresponding to the sample text;
[0051] Inputting the sample text and the element sample texture map into the first generation model to be trained, processing the sample text and the element sample texture map using the first generation model to be trained to obtain a fusion vector, and processing the element sample texture map to obtain a noise texture map corresponding to the element sample texture map;
[0052] Using the first generation model to be trained, according to the fusion vector, to perform denoising prediction on the noise texture map corresponding to the element sample texture map, to generate a first style prediction texture map;
[0053] According to the difference between the first style label texture map and the first style prediction texture map, the parameters of the first generation model to be trained are adjusted until the adjusted model meets the training cutoff condition, and the training is completed to obtain the first generation model.
[0054] In an achievable implementation, the second generation model is trained and obtained according to the following method:
[0055] Acquire a second style label texture map and a style sample texture map, wherein the second style label texture map is a style-optimized texture map, and the style sample texture map is a non-style-optimized texture map;
[0056] Inputting the style sample texture map into the second generation model to be trained, and using the second generation model to be trained to perform style prediction optimization on the style sample texture map to obtain a second style prediction texture map;
[0057] According to the difference between the second style label texture map and the second style prediction texture map, the parameters of the second generation model to be trained are adjusted until the adjusted model meets the training cutoff condition, and the training is completed to obtain the second generation model.
[0058] In a feasible implementation, the first generation model includes a diffusion model, and the second generation model includes a lora model.
[0059] A third aspect of the present application provides a computer device. The identification device comprises:
[0060] a memory having a computer program stored thereon;
[0061] A processor is used to execute the computer program in the memory to implement the steps of the method for generating a stylized map provided in the first aspect.
[0062] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method for generating a stylized map provided in the first aspect are implemented.
[0063] Compared with the prior art, this application has the following beneficial effects:
[0064] In the present application, the initial element texture atlas and the target text input by the user are first obtained, and the initial element texture atlas is extracted and processed according to the target text to obtain the initial element texture map in the initial element texture atlas corresponding to the initial map element prompted in the target text; thereafter, the texture map generation model is used to perform style modification processing on the initial element texture map corresponding to the initial map element according to the target text and the initial element texture map corresponding to the initial map element to generate an element style texture map; finally, a style map element is generated according to the element style texture map, and a stylized map is generated according to the style map element. It should be noted that an initial element texture map in the initial element texture atlas corresponds to an initial map element, and the target text includes text prompting to modify the style of the initial map element; the texture map generation model is used to modify the style of the initial element texture map; and the style map element is the initial map element after the style modification.
[0065] It can be seen that in this application, the texture map generation model is used to modify the style of the initial element texture map corresponding to the initial map element based on the target text and the initial element texture map corresponding to the initial map element to generate a style-modified element style texture map, and a stylized map is generated based on the element style texture map. In this way, users can DIY the map style by themselves (that is, the texture map generation model in this application can generate an element style texture map based on the target text input by the user), realizing the diversification of the image style of the navigation map and improving the interactive experience between the user and the navigation map. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0067] Figure 1 Map images of navigation maps provided for related technologies;
[0068] Figure 2 A flowchart of a method for generating a stylized map provided in an embodiment of the present application;
[0069] Figure 3 A schematic diagram of the structure of a texture map generation model in a method for generating a stylized map provided in an embodiment of the present application;
[0070] Figure 4 A flow chart of generating an element style texture map in a method for generating a stylized map provided in an embodiment of the present application;
[0071] Figure 5 A schematic diagram of the structure of a first generation model in a method for generating a stylized map provided in an embodiment of the present application;
[0072] Figure 6 A schematic diagram of map style modification in a method for generating a stylized map provided in an embodiment of the present application;
[0073] Figure 7 A schematic diagram of a stylized map in a method for generating a stylized map provided in an embodiment of the present application;
[0074] Figure 8 A schematic diagram of the structure of a stylized map generation device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0075] As described above, navigation maps are used to support navigation needs of different users. Currently, the image style of navigation maps is relatively simple. For example, they can only be generated with fixed colors, the color changes of images are rare, and the image styles that can be switched are also limited. For example, there are only image styles for day mode and night mode. Figure 1 As shown, Figure 1 A map image of a navigation map provided for related technology, in Figure 1 The map elements such as roads in the navigation map are shown in the figure. The image style of the displayed image is relatively simple, that is, the image style of the navigation map is mostly the same for all users, which makes the interaction experience between the user and the navigation map poor. Therefore, how to improve the user's interaction experience is a key issue that technicians in this field are concerned about.
[0076] In view of the above problems, a solution is provided in the embodiment of the present application, and a method, device and related products for generating a stylized map are proposed, aiming to improve the user's interactive experience. In the technical solution of the present application, the initial element texture atlas and the target text input by the user are first obtained, and the initial element texture map corresponding to the initial map element is extracted from the initial element texture atlas according to the target text. After that, the initial element texture map corresponding to the initial map element is style-modified based on the target text and the initial element texture map corresponding to the initial map element using the texture map generation model to generate an element style texture map after style modification. Finally, a style map element is generated according to the element style texture map, and a stylized map is generated according to the style map element. In this way, in the present application, the user can DIY the map style by himself (that is, the texture map generation model in the present application can generate an element style texture map after style modification described by the target text according to the target text input by the user), which realizes the diversification of the image style of the navigation map and improves the interactive experience between the user and the navigation map.
[0077] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0078] Next, the technical terms that may appear in this application are explained.
[0079] Texture map: A texture map is an image containing surface detail information, which is used to map to the surface of a 3D model to increase the visual detail and realism of the 3D body.
[0080] Diffusion model: Also known as the stable diffusion model, it aims to generate new images through a latent diffusion process.
[0081] lora model: Low-Rank Adaptation of Large Language Models, used in this application to fine-tune the generation results of the diffusion model.
[0082] See also Figure 2 , which is a flow chart of a method for generating a stylized map provided in an embodiment of the present application. Figure 2 The method for generating the stylized map shown includes:
[0083] S201: Acquire an initial element texture atlas and a target text input by a user.
[0084] In this step, the initial element texture atlas includes different types of texture maps, and one initial element texture map in the initial element texture atlas corresponds to one initial map element, wherein the initial map element can be understood as elements such as buildings, traffic signs and different types of roads in the navigation map. Under normal circumstances, one initial element texture map can correspond to construct a simpler initial map element, such as a building, etc. It should be noted that multiple initial element texture maps can correspond to construct a more complex initial map element, such as windows and doors of buildings, etc., wherein the initial map element is usually constructed using 3D technology, which will not be described in detail here.
[0085] Furthermore, the target text in the present application includes text prompting to modify the style of the initial map elements. It can be understood that the target text is the text input by the user that requires the style of the initial map elements to be modified, such as: changing a red building to a green building, etc.
[0086] S202: extracting the initial element texture atlas according to the target text to obtain an initial element texture map in the initial element texture atlas corresponding to the initial map element indicated in the target text.
[0087] In this step, the initial element texture atlas can be extracted according to the initial map element mentioned in the target text and requiring style modification, so as to obtain the initial element texture map corresponding to the initial map element in the initial element texture atlas. In this way, it is convenient to subsequently modify the style of the initial element texture map, and generate the style-modified initial map element based on the style-modified initial element texture map.
[0088] It should be noted that if there are different map elements of the same type in the navigation map (such as all are buildings, but the sizes of the buildings are different), and the initial map elements mentioned in the target text are not completely specific (such as the target text only suggests that the style of the building needs to be modified, but does not suggest which size of the building needs to be modified), then the style of all buildings in the navigation map can be modified in the subsequent process.
[0089] S203: Using a texture map generation model, based on the target text and the initial element texture map corresponding to the initial map element, perform style modification processing on the initial element texture map corresponding to the initial map element to generate an element style texture map.
[0090] In this step, the texture map generation model is used to modify the style of the initial element texture map. Figure 3 As shown, Figure 3 A structural diagram of a texture map generation model in a method for generating a stylized map provided in an embodiment of the present application, wherein Figure 3 It is shown that the texture map generation model includes a first generation model and a second generation model, wherein the first generation model is used to perform initial style modification on the initial element texture map, and the second generation model is used to perform style optimization modification on the initial element texture map. It should be noted that the first generation model in the present application specifically includes a diffusion model, and the second generation model specifically includes a lora model. In this way, under the action of the first generation model and the second generation model, the present application enables more stylized map elements and their maps to be generated in the subsequent process.
[0091] like Figure 4 As shown, Figure 4 A flow chart of generating an element style texture map in a method for generating a stylized map provided in an embodiment of the present application is provided. Next, in combination with Figure 4 Steps S2031 to S2033 in the present application introduce the process of generating the element style texture map. Steps S2031 to S2033 are as follows:
[0092] S2031: Using the first generation model in the texture map generation model, encoding the target text and the initial element texture map corresponding to the initial map element to obtain a fused feature vector.
[0093] First of all, it should be noted that Figure 5 As shown, Figure 5 A schematic diagram of the structure of a first generation model in a method for generating a stylized map provided in an embodiment of the present application, Figure 5The first generation model in includes a text encoding module, an image encoding module and a fusion encoding module, wherein the text encoding module is used to encode text, the image encoding module is used to encode images, and the fusion encoding module is used to fuse feature vectors.
[0094] Specifically, in the present application, the text encoding module of the first generation model in the texture map generation model can be used to first perform text encoding on the target text to obtain a text feature vector. After that, the image encoding module of the first generation model in the texture map generation model can be used to perform image encoding on the initial element texture map corresponding to the initial map element to obtain an image feature vector. Finally, the fusion encoding module of the first generation model in the texture map generation model can be used to fuse the text feature vector and the image feature vector to obtain a fused feature vector. In this way, in the subsequent process, image denoising can be performed based on the fused feature vector obtained after the fusion of the text feature vector and the image feature vector.
[0095] It should be noted that the text encoding module in the present application adopts a 6-layer transformer model architecture, so that the target text can be encoded into the text latent space to obtain a text feature vector; the image encoding module in the present application adopts a 6-layer transformer model architecture, so that the initial element texture map corresponding to the initial map element can be encoded into the image latent space to obtain the image feature vector; and the fusion encoding module in the present application mainly uses cross-attention (multi-head attention mechanism) to fuse the text feature vector and the image feature vector. It can be understood that since the text feature vector and the image feature vector belong to different latent spaces, it is necessary to use the fusion encoding module to map the feature vectors in two different latent spaces to the same latent space to obtain a fused feature vector.
[0096] S2032: Using the first generation model in the texture map generation model to modify the style of the initial element texture map corresponding to the initial map element according to the fused feature vector, to generate an initial style texture map.
[0097] It should be noted that before executing step S2032, the present application can also use the first generation model in the texture map generation model to generate a noise texture map according to the initial element texture map corresponding to the initial map element, wherein the noise texture map is a texture map of completely random noise, and the spatial dimension of the noise texture map is the same as the spatial dimension of the initial element texture map, so that no additional computing consumption will be generated when performing image denoising operations in subsequent processes.
[0098] It is further necessary to explain that Figure 5The first generation model in the texture map generation model includes a texture map generation module, wherein the texture map generation module is used to generate an initial style texture map according to the fused feature vector. At this time, the texture map generation module of the first generation model in the texture map generation model can be used to perform image denoising on the noise texture map according to the fused feature vector to generate an initial style texture map. In this way, in the present application, under the guidance of the fused feature vector, the noise texture map after image denoising can be used as the initial style texture map (i.e., the initial element texture map corresponding to the initial map element after the style modification), thereby realizing the initial modification of the style of the initial element texture map corresponding to the initial map element.
[0099] It can be understood that since the first generation model in the present application includes a diffusion model, in the present application, noise can be first gradually added to the initial element texture map corresponding to the initial map element until a noise texture map of completely random noise is obtained (that is, the diffusion process of the diffusion model), and the fused feature vector in the present application is used to guide the generation of the image (that is, image denoising), so that the generated image can conform to the modified image style described by the target text.
[0100] After that, under the guidance of the fused feature vector, the noise in the noise texture map can be gradually removed until the initial style texture map is obtained (i.e., the inverse diffusion process of the diffusion model). In this way, under the constraint of the fused feature vector, the present application denoises the noise texture map containing random noise to obtain an initial style texture map that is clear and meets the requirements of the target text, which improves the user experience to a certain extent.
[0101] like Figure 6 As shown, Figure 6 A schematic diagram of map style modification in a method for generating a stylized map provided in an embodiment of the present application, wherein Figure 6 As shown in Figure 6 (a) represents the map image without style modification. Figure 6 (b) shows the map image after style modification. Figure 6 The rectangle A in (a) represents the initial map element without style modification (which can be understood as building A). Figure 6 Rectangle A in (b) represents the initial map element after style modification, and Figure 6 (a) and Figure 6 The rectangles B in (b) represent the initial map elements without any style modification (which can be understood as building B).
[0102] It can be understood that if the target text at this time is a text indicating that the white building A is changed to the gray building A, then Figure 6 The initial element texture map corresponding to the white building A in (a) is modified in style to obtain Figure 6The gray building A in (b) (i.e. the initial map element after style modification). It should be noted that Figure 6 The figures shown are only some examples and are not limiting.
[0103] S2033: Utilize the second generation model in the texture map generation model to perform style optimization on the initial style texture map to generate an element style texture map.
[0104] It should be noted that before executing step 2033, the present application also needs to obtain the map style type, where the map style types include cartoon style and cyberpunk style, etc., and the map style type can be obtained through the target text input by the user, and the target text input by the user can be generated according to the content clicked by the user on the terminal, and the target text input by the user can also be generated according to the content input by the user by voice on the terminal. It can be understood that buttons of various map style types can be displayed in the front-end map style adjustment interface to facilitate user selection by clicking, and different map style types in the present application have corresponding second generation models. For example: if the map style type clicked by the user is a cartoon style, then the second generation model for style optimization at this time is a second generation model of the cartoon style.
[0105] It should also be noted that the first generation model in this application is mainly used to adjust the style of the initial element texture map according to the target text input by the user to generate an initial style texture map that conforms to the target text description (i.e., the initial element texture map after style modification), and the second generation model in this application is mainly used to optimize the style of the initial style texture map according to the map style type provided by the user to generate a more stylized element style texture map, which is convenient for the subsequent generation of style map elements and their stylized maps. In this way, in this application, it is achieved that map elements of various styles can be generated according to the actual needs of the user.
[0106] Correspondingly, the map style type provided by the user at this time can also be obtained through the target text entered by the user. The target text entered by the user can be generated according to the content clicked by the user on the terminal, and the target text entered by the user can also be generated according to the content input by the user through voice on the terminal. The map style types include cartoon style and cyberpunk style, etc., which are not specifically limited here.
[0107] In some examples, if the target text is a text that requires changing a red building to a green building, then the first generation model can be used to implement the operation of changing the element texture map corresponding to the red building to the element texture map corresponding to the green building, so as to obtain the element texture map corresponding to the green building; thereafter, if the map style type provided by the user is a cartoon style, then the second generation model can be used to change the element texture map corresponding to the green building to an element texture map corresponding to the green building in a cartoon style.
[0108] In other examples, if the target text is a text that requires a cyberpunk style and a bear, kitten and / or puppy is pasted on the building, then the first generation model and the second generation model can be used to generate an element texture map that satisfies the cyberpunk style and pastes bears, kittens and / or puppies on the building. In other examples, if the target text is a text that requires a steampunk style and the grass is full of flowers, then the first generation model and the second generation model can be used to generate an element texture map that satisfies the steampunk style and the grass is full of flowers. In other examples, if the target text is a text that requires a cubist style and the lake presents a ripple shape, then the first generation model and the second generation model can be used to generate an element texture map that satisfies the cubist style and the lake presents a ripple shape. It should be noted that the target texts mentioned in this application are only some examples, which are not specifically limited here, and the target texts can also be arbitrarily set in actual applications.
[0109] S204: Generate a style map element according to the element style texture map, and generate a stylized map according to the style map element.
[0110] In this step, the style map element is the initial map element after the style is modified. It can be understood that after the style modification and style optimization of the initial element texture map are completed and the element style texture map is obtained, the user can preview the element style texture map at this time. If the user is satisfied with the generated element style texture map, the style map element corresponding to the element style texture map can be generated, and the generated style map element can be redisplayed in the navigation map through technologies such as Unity, so as to realize the user-defined personalized navigation map DIY design; if the user is not satisfied with the generated element style texture map, the first generation model and the second generation model are continued to be used to modify and optimize the style of the initial element texture map until the user is satisfied with the generated element style texture map.
[0111] like Figure 7 As shown, Figure 7 A schematic diagram of a stylized map in a method for generating a stylized map provided in an embodiment of the present application, Figure 7The stylized map generated in this application based on the target text entered by the user and the map style type clicked is more stylized and more beautiful, which enhances the interactive experience between the user and the navigation map.
[0112] In the above process, the application process of the first generation model and the second generation model are introduced in detail. Next, the training process of the first generation model and the second generation model is introduced.
[0113] In an achievable implementation, the first generation model is trained and obtained according to the following method:
[0114] In the present application, a first style label texture map, sample text, and an element sample texture map corresponding to the sample text are first obtained, wherein the first style label texture map serves as a training label for a first generation model, and the sample text and the element sample texture map corresponding to the sample text serve as training samples for the first generation model, and the training samples in the present application can be multiple different sample texts and their different types of element sample texture maps.
[0115] It should be noted that the first generation model to be trained includes a text encoding module to be trained, an image encoding module to be trained, a fusion encoding module to be trained, and a texture map generation module to be trained. At this time, the sample text and the element sample texture map can be input into the first generation model to be trained, so as to process the sample text and the element sample texture map according to the text encoding module to be trained, the image encoding module to be trained, and the fusion encoding module to be trained in the first generation model to be trained to obtain a fusion vector, and the element sample texture map can be processed according to the first generation model to be trained to obtain a noise texture map corresponding to the element sample texture map.
[0116] Finally, the fusion coding module to be trained in the first generation model to be trained can be used to perform denoising prediction on the noise texture map corresponding to the element sample texture map according to the fusion vector to generate the first style prediction texture map, and the parameters of the first generation model to be trained can be adjusted according to the difference between the first style label texture map and the first style prediction texture map, until the adjusted model meets the training cutoff condition, and the first generation model is obtained after the training. Among them, the parameters in the first generation model to be trained can be adjusted to adjust the parameters of the text coding module to be trained, the image coding module to be trained, the fusion coding module to be trained, and the texture map generation module to be trained in the first generation model to be trained, so as to obtain the text coding module, image coding module, fusion coding module, and texture map generation module that meet the training conditions. In this way, the style modification of the initial element texture map can be well achieved during the application of the first generation model.
[0117] It should also be noted that in the present application, the parameters corresponding to the text encoding module to be trained, the image encoding module to be trained, the fusion encoding module to be trained and the texture map generation module to be trained are not adjusted independently, but the parameters corresponding to the text encoding module to be trained, the image encoding module to be trained, the fusion encoding module to be trained and the texture map generation module to be trained are adjusted end-to-end at the same time.
[0118] In another achievable implementation, the second generation model is obtained by training according to the following method (in this application, multiple different second generation models can be obtained by training for different map style types, and each different second generation model is not described in detail here. In actual application, multiple different second generation models can also be obtained by training according to the following training process):
[0119] In the present application, a second style label texture map and a style sample texture map can be first obtained, wherein the second style label texture map is a style-optimized texture map, the second style label texture map serves as a training label for the second generation model, the style sample texture map is a non-style-optimized texture map, the style sample texture map serves as a training sample for the second generation model, and one training label corresponds to model training for one map style type.
[0120] After that, the style sample texture map can be input into the second generation model to be trained, and the style sample texture map can be optimized by using the second generation model to be trained to perform style prediction to obtain a second style prediction texture map, and finally, according to the difference between the second style label texture map and the second style prediction texture map, the parameters of the second generation model to be trained are adjusted until the adjusted model meets the training cutoff condition, and the second generation model is obtained after the training. In this way, in the present application, by training the second generation model, the consumption of computing resources can be reduced to a certain extent while maintaining the model performance, and the style optimization of the initial style texture map can be well achieved during the application process.
[0121] It should be noted that the second generation model obtained after training in the present application can be expressed by formula (1) to formula (3), and formula (1) to formula (3) are specifically embodied as follows:
[0122] h=Wx+αBAx Formula (1)
[0123] W∈R d*d Formula (2)
[0124] A∈R r*d , B∈R d*r Formula (3)
[0125] Where x represents the input of the second generative model (i.e., the output of the first generative model during application), h represents the output of the second generative model, W represents the weight of the first generative model, and R d*d represents a matrix of shape (d, d), BA represents the weight of the second generation model, A∈R r*d Represents a low-rank matrix A, B∈R of shape (d, r) d*r Represents a low-rank matrix B of shape (d, r), the rank of low-rank matrix A and low-rank matrix B is r, and since the dimension of r is much smaller than d, the overall training parameter amount of the second generative model can be greatly reduced, that is, the parameter amount changes from d*d to 2*d*r, so in this application, the training of the second generative model can be realized by updating low-rank matrix A and low-rank matrix B. And α is a scaling factor, which is used to control the image style output by the second generative model. If the α value is larger, the generated image style is more biased towards the second generative model; if the α value is smaller, the generated image style is more biased towards the first generative model.
[0126] To summarize, in this embodiment, users can DIY their own map style (that is, the texture map generation model in this application can generate an element style texture map with modified style according to the target text described by the target text input by the user), which realizes the diversification of the image style of the navigation map and improves the interactive experience between the user and the navigation map.
[0127] Based on the method for generating a stylized map provided in the aforementioned embodiment, the present application also provides a device for generating a stylized map accordingly. Figure 8 A schematic diagram of the structure of a stylized map generation device provided in an embodiment of the present application. Figure 8 As shown, the generating device of the stylized map includes:
[0128] The texture image text acquisition unit 801 is used to acquire an initial element texture atlas and a target text input by a user, wherein an initial element texture image in the initial element texture atlas corresponds to an initial map element, and the target text includes text prompting to modify the style of the initial map element;
[0129] An element texture map extraction unit 802 is used to extract the initial element texture map set according to the target text to obtain an initial element texture map in the initial element texture map set corresponding to the initial map element prompted in the target text;
[0130] The style texture map generating unit 803 is used to perform style modification processing on the initial element texture map corresponding to the initial map element according to the target text and the initial element texture map corresponding to the initial map element by using a texture map generating model to generate an element style texture map, wherein the texture map generating model is used to perform style modification on the initial element texture map;
[0131] The style map generating unit 804 is used to generate a style map element according to the element style texture map, and to generate a style map according to the style map element, wherein the style map element is an initial map element after style modification.
[0132] In an achievable implementation, the style texture map generating unit 803 includes:
[0133] A fused feature vector obtaining unit, used to encode the target text and the initial element texture map corresponding to the initial map element using the first generation model in the texture map generation model to obtain a fused feature vector;
[0134] A texture map style modification unit, configured to modify the style of an initial element texture map corresponding to the initial map element according to the fused feature vector by using a first generation model in the texture map generation model to generate an initial style texture map;
[0135] The texture map style optimization unit is used to optimize the style of the initial style texture map by using the second generation model in the texture map generation model to generate an element style texture map.
[0136] In one feasible embodiment, the device further includes:
[0137] A noise texture map generating unit, configured to generate a noise texture map according to an initial element texture map corresponding to the initial map element by using a first generating model in the texture map generating model;
[0138] The texture map style modification unit is specifically used for:
[0139] The texture map generation module of the first generation model in the texture map generation model is used to perform image denoising on the noise texture map according to the fused feature vector to generate an initial style texture map.
[0140] In an achievable implementation, the fused feature vector obtaining unit is specifically used for:
[0141] Using the text encoding module of the first generation model in the texture map generation model, the target text is encoded to obtain a text feature vector;
[0142] Using the image encoding module of the first generation model in the texture map generation model, image encoding is performed on the initial element texture map corresponding to the initial map element to obtain an image feature vector;
[0143] The text feature vector and the image feature vector are fused using a fusion coding module of a first generation model in the texture map generation model to obtain a fused feature vector.
[0144] In an achievable implementation, the first generation model is trained and obtained according to the following method:
[0145] Obtaining a first style label texture map, a sample text, and an element sample texture map corresponding to the sample text;
[0146] Inputting the sample text and the element sample texture map into the first generation model to be trained, processing the sample text and the element sample texture map using the first generation model to be trained to obtain a fusion vector, and processing the element sample texture map to obtain a noise texture map corresponding to the element sample texture map;
[0147] Using the first generation model to be trained, according to the fusion vector, to perform denoising prediction on the noise texture map corresponding to the element sample texture map, to generate a first style prediction texture map;
[0148] According to the difference between the first style label texture map and the first style prediction texture map, the parameters of the first generation model to be trained are adjusted until the adjusted model meets the training cutoff condition, and the training is completed to obtain the first generation model.
[0149] In an achievable implementation, the second generation model is trained and obtained according to the following method:
[0150] Acquire a second style label texture map and a style sample texture map, wherein the second style label texture map is a style-optimized texture map, and the style sample texture map is a non-style-optimized texture map;
[0151] Inputting the style sample texture map into the second generation model to be trained, and using the second generation model to be trained to perform style prediction optimization on the style sample texture map to obtain a second style prediction texture map;
[0152] According to the difference between the second style label texture map and the second style prediction texture map, the parameters of the second generation model to be trained are adjusted until the adjusted model meets the training cutoff condition, and the training is completed to obtain the second generation model.
[0153] In a feasible implementation, the first generation model includes a diffusion model, and the second generation model includes a lora model.
[0154] The stylized map generation device provided in the embodiment of the present application has the same beneficial effects as the stylized map generation method provided in the above embodiment, and can improve the user's interactive experience, which will not be described in detail here.
[0155] The present application also provides a computer device. The identification device includes:
[0156] A memory having a computer program stored therein.
[0157] The processor is used to execute the computer program in the memory to implement some or all of the steps in the method for generating a stylized map introduced in the above embodiment.
[0158] The present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements some or all of the steps in the method for generating a stylized map described in the above embodiment.
[0159] It should be noted that the "first" and "second" (if any) in the names such as "first" and "second" mentioned in the embodiments of the present application are only used as name identifiers and do not represent the first or second in order.
[0160] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same and similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and equipment embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0161] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for generating a stylized map, It is characterized in that include: Acquire an initial element texture atlas and a target text input by a user, wherein an initial element texture map in the initial element texture atlas corresponds to an initial map element, and the target text includes text prompting to modify the style of the initial map element; Extracting the initial element texture atlas according to the target text to obtain an initial element texture map in the initial element texture atlas corresponding to the initial map element indicated in the target text; Using a texture map generation model to perform style modification processing on the initial element texture map corresponding to the initial map element according to the target text and the initial element texture map corresponding to the initial map element, so as to generate an element style texture map, wherein the texture map generation model is used to perform style modification on the initial element texture map; A style map element is generated according to the element style texture map, and a stylized map is generated according to the style map element, wherein the style map element is an initial map element after the style is modified.
2. The method according to claim 1, It is characterized in that The texture map generation model includes a first generation model and a second generation model. The texture map generation model is used to perform style modification processing on the initial element texture map corresponding to the initial map element according to the target text and the initial element texture map corresponding to the initial map element to generate an element style texture map, including: Using the first generation model in the texture map generation model, encoding the target text and the initial element texture map corresponding to the initial map element to obtain a fused feature vector; Using the first generation model in the texture map generation model to modify the style of the initial element texture map corresponding to the initial map element according to the fused feature vector, so as to generate an initial style texture map; The initial style texture map is style optimized by using a second generation model in the texture map generation model to generate an element style texture map.
3. The method according to claim 2, It is characterized in that Before the first generation model in the texture map generation model is used to modify the style of the initial element texture map corresponding to the initial map element according to the fused feature vector to generate the initial style texture map, the method further includes: Generate a noise texture map according to the initial element texture map corresponding to the initial map element using the first generation model in the texture map generation model; The first generation model includes a texture map generation module, and the first generation model in the texture map generation model is used to modify the style of the initial element texture map corresponding to the initial map element according to the fused feature vector to generate an initial style texture map, including: The texture map generation module of the first generation model in the texture map generation model is used to perform image denoising on the noise texture map according to the fused feature vector to generate an initial style texture map.
4. The method according to claim 3, It is characterized in that The first generation model further includes a text encoding module, an image encoding module and a fusion encoding module. The first generation model in the texture map generation model is used to encode the target text and the initial element texture map corresponding to the initial map element to obtain a fusion feature vector, including: Using the text encoding module of the first generation model in the texture map generation model, the target text is encoded to obtain a text feature vector; Using the image encoding module of the first generation model in the texture map generation model, image encoding is performed on the initial element texture map corresponding to the initial map element to obtain an image feature vector; The text feature vector and the image feature vector are fused using a fusion coding module of a first generation model in the texture map generation model to obtain a fused feature vector.
5. The method according to claim 3, It is characterized in that The first generation model is trained in the following manner: Obtaining a first style label texture map, a sample text, and an element sample texture map corresponding to the sample text; Inputting the sample text and the element sample texture map into the first generation model to be trained, processing the sample text and the element sample texture map using the first generation model to be trained to obtain a fusion vector, and processing the element sample texture map to obtain a noise texture map corresponding to the element sample texture map; Using the first generation model to be trained, according to the fusion vector, to perform denoising prediction on the noise texture map corresponding to the element sample texture map, to generate a first style prediction texture map; According to the difference between the first style label texture map and the first style prediction texture map, the parameters of the first generation model to be trained are adjusted until the adjusted model meets the training cutoff condition, and the training is completed to obtain the first generation model.
6. The method according to claim 2, It is characterized in that The second generation model is trained according to the following method: Acquire a second style label texture map and a style sample texture map, wherein the second style label texture map is a style-optimized texture map, and the style sample texture map is a non-style-optimized texture map; Inputting the style sample texture map into the second generation model to be trained, and using the second generation model to be trained to perform style prediction optimization on the style sample texture map to obtain a second style prediction texture map; According to the difference between the second style label texture map and the second style prediction texture map, the parameters of the second generation model to be trained are adjusted until the adjusted model meets the training cutoff condition, and the training is completed to obtain the second generation model.
7. The method according to claim 2, It is characterized in that The first generative model includes a diffusion model, and the second generative model includes a lora model.
8. A device for generating a stylized map, It is characterized in that include: A texture map text acquisition unit, used to acquire an initial element texture atlas and a target text input by a user, wherein an initial element texture map in the initial element texture atlas corresponds to an initial map element, and the target text includes text prompting to modify the style of the initial map element; An element texture map extraction unit is used to extract the initial element texture map set according to the target text to obtain an initial element texture map in the initial element texture map set corresponding to the initial map element prompted in the target text; a style texture map generating unit, configured to perform style modification processing on the initial element texture map corresponding to the initial map element according to the target text and the initial element texture map corresponding to the initial map element by using a texture map generating model to generate an element style texture map, wherein the texture map generating model is used to perform style modification on the initial element texture map; The stylized map generating unit is used to generate a style map element according to the element style texture map, and to generate a style map according to the style map element, wherein the style map element is an initial map element after style modification.
9. A computer device, It is characterized in that include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method for generating a stylized map according to claims 1-7.
10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the steps of the method for generating a stylized map described in claims 1 to 7 are implemented.
Citation Information
Cited By
Information processing method and device, readable storage medium and program product
CN121810865A
