Display device and three-dimensional image generation method based on large model

Abstract

The embodiment of the invention provides a display device and a three-dimensional image generation method based on a large model, and the method comprises the steps: obtaining an image description text corresponding to an image generation instruction in response to the image generation instruction input by a user, and inputting the image description text to a text encoder to obtain a feature code set, the image description text is used for describing the shape and the spatial position of the generated image, the feature code set comprises feature codes, and the feature codes comprise sketch codes, topological codes and stretching codes. And the feature code set is input to an image generator to generate a target image, and the image generator is a generative adversarial network model for generating a three-dimensional image according to the feature code set. According to the method, the image description text input by the user can be converted into the feature code set by using the text encoder, and the three-dimensional image is generated according to the feature code set by using the image generator, so that the accuracy of generating the target image is improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a method for generating a three-dimensional stereoscopic image based on a large model. Background Art

[0002] A display device is a terminal device that can display image information. Through the voice recognition technology and image generation technology of the display device, the function of generating a three-dimensional image according to the user's voice command can be realized. For example, the user inputs a voice command to the display device, "Please generate a three-dimensional image of a cube under a cone", and the display device converts the voice command into text description information, and uses the text-based image model to generate a corresponding image according to the text description information. In the generated image, there is a cube and a cone, and the cube is located below the cone.

[0003] However, due to the limitations of the Wensheng graph model in understanding user semantics, the generated 3D images deviate from the user's description. In addition, the Wensheng graph model cannot accurately control the image layout of the 3D image, resulting in inaccurate position, size and shape of the generated image, affecting the user experience. Summary of the invention

[0004] In some embodiments of the present application, a display device and a method for generating a three-dimensional stereoscopic image based on a large model are provided to solve the problem of low accuracy of three-dimensional stereoscopic images generated by a Wensheng graph model.

[0005] In a first aspect, some embodiments of the present application provide a display device, comprising: a display and a controller. The display is configured to display a user interface, and the controller is configured to perform the following steps:

[0006] In response to an image generation instruction input by a user, acquiring an image description text corresponding to the image generation instruction, wherein the image description text is used to describe the shape and spatial position of the generated image;

[0007] Input the image description text into a text encoder to obtain a feature coding set, wherein the text encoder is a deep learning model that converts the image description text into feature coding according to a coding rule, wherein the feature coding set includes feature coding, wherein the feature coding includes sketch coding, topological coding, and stretch coding, wherein the sketch coding is used to characterize the coordinate information of the generated image, the topological coding is used to characterize the curve type of the generated image, and the stretch coding is used to characterize the spatial structure of the generated image, and the sketch coding, the topological coding, and the stretch coding are associated with each other;

[0008] The feature coding set is input into an image generator to obtain a target image, and the display is controlled to display the target image. The image generator is a generative adversarial network model that generates a three-dimensional stereo image based on the feature coding set.

[0009] In a second aspect, some embodiments of the present application further provide a method for generating a three-dimensional stereoscopic image based on a large model, which is applied to the display device described in the first aspect, wherein the display device includes a display and a controller, and the method includes:

[0010] In response to an image generation instruction input by a user, acquiring an image description text corresponding to the image generation instruction, wherein the image description text is used to describe the shape and spatial position of the generated image;

[0011] Input the image description text into a text encoder to obtain a feature coding set, wherein the text encoder is a deep learning model that converts the image description text into feature coding according to a coding rule, wherein the feature coding set includes feature coding, wherein the feature coding includes sketch coding, topological coding, and stretch coding, wherein the sketch coding is used to characterize the coordinate information of the generated image, the topological coding is used to characterize the curve type of the generated image, and the stretch coding is used to characterize the spatial structure of the generated image, and the sketch coding, the topological coding, and the stretch coding are associated with each other;

[0012] The feature coding set is input into an image generator to obtain a target image, and the display is controlled to display the target image. The image generator is a generative adversarial network model that generates a three-dimensional stereo image based on the feature coding set.

[0013] As can be seen from the above technical solutions, some embodiments of the present application provide a display device and a method for generating a three-dimensional stereoscopic image based on a large model. The method can respond to the image generation instruction input by the user, obtain the image description text corresponding to the image generation instruction, and input the image description text into the text encoder to obtain a feature coding set, and input the feature coding set into the image generator to obtain the target image, and then control the display to display the target image. Among them, the image description text is used to describe the shape and spatial position of the generated image; the text encoder is a deep learning model that converts the image description text into feature coding according to the coding rule, the feature coding set includes feature coding, the feature coding includes sketch coding, topology coding and stretch coding, the sketch coding is used to characterize the coordinate information of the generated image, the topology coding is used to characterize the curve type of the generated image, and the stretch coding is used to characterize the spatial structure of the generated image, and the sketch coding, topology coding and stretch coding are associated with each other; the image generator is a generative adversarial network model that generates a three-dimensional stereoscopic image according to the feature coding set. The method can use the text encoder to convert the image description text input by the user into a feature coding set, and use the image generator to generate a three-dimensional stereoscopic image according to the feature coding set, so as to improve the accuracy of generating the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0016] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0017] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0018] Figure 4 A timing diagram for generating a three-dimensional stereoscopic image provided in some embodiments of the present application;

[0019] Figure 5 A schematic diagram of a process for generating a three-dimensional image provided by some embodiments of the present application;

[0020] Figure 6 A schematic diagram of obtaining a geometric structure for a display device provided in some embodiments of the present application;

[0021] Figure 7 A schematic diagram of generating a feature code set for a display device provided in some embodiments of the present application;

[0022] Figure 8 A schematic diagram of a three-dimensional coordinate system constructed by a display device provided in some embodiments of the present application;

[0023] Fig. 9 A schematic diagram of generating characteristic codes for a display device provided in some embodiments of the present application;

[0024] Fig.10 A schematic diagram of a two-dimensional geometric element provided for some embodiments of the present application;

[0025] Fig.11 A schematic diagram of generating sketch code for a display device provided in some embodiments of the present application;

[0026] Fig.12 A schematic diagram of a display device training image generator provided for some embodiments of the present application. DETAILED DESCRIPTION

[0027] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0028] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and common meanings.

[0029] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0030] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0031] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0032] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0033] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0034] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0035] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0036] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0037] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0038] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0039] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0040] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures, and the image collector can also collect the user interface displayed by the display 260, or the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds, and the sound collector can also collect recorded audio.

[0041] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0042] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0043] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0044] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0045] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0046] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0047] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0048] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0049] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0050] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0051] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0052] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0053] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0054] like Figure 3 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0055] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0056] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0057] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0058] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0059] In some embodiments, the display device 200 can interact with the user through text segmentation technology and image generation technology. For example, the user inputs the text information "please generate an image of a cube with a cylinder underneath", and the display device 200 generates and displays a three-dimensional image that matches the user description.

[0060] In some embodiments, the display device 200 uses a text-to-image generation model to convert the text description input by the user into a corresponding three-dimensional image. For example, the display device 200 is installed with an application for early childhood three-dimensional geometry education, and the display device 200 converts abstract three-dimensional geometry concepts (such as spheres, cubes, and cylinders, etc.) into intuitive and vivid three-dimensional images through the text-to-image generation model.

[0061] Among them, the text graph model is a deep learning model that can convert text into images, including a text encoder (Text Encoder), an image generator (Image Generator) and a noise predictor (Noise Predictor). The text encoder is used to convert the text description input by the user into a text vector that can be understood by the display device 200. The text vector contains the semantic information of the text and is the basis for subsequent image generation. The image generator uses a deep learning network structure to capture the semantic information in the text vector and generate an image that matches the text vector. The noise predictor is used to predict the added noise and gradually generate a pixel-level target image through a denoising process. The target image can be iterated and optimized multiple times during the generation process to improve the clarity of the image and the matching degree of the text description. The user can modify the output target image by adjusting the model parameters and input conditions of the text graph model.

[0062] In some embodiments, the display device 200 first extracts key information of the image description text, such as the shape, material, and color of the object, etc. Then, the text graph model is used to generate two-dimensional images under multiple perspectives according to the key information, and the two-dimensional images under multiple perspectives are synthesized into a three-dimensional point cloud or mesh model by using multi-perspective geometry technology and computer vision technology, and a three-dimensional stereoscopic image is generated based on the three-dimensional point cloud or mesh model.

[0063] In some embodiments, the display device 200 needs to collect a high-quality training data set to perform training on the text graph model. The training data set includes two-dimensional images under multiple viewing angles and detailed text descriptions corresponding to the images. By inputting the training data set into the basic model, configuring training parameters such as learning rate, batch size, and training rounds, and monitoring the loss function and the quality of the generated image, the training parameters of the text graph model can be adjusted to improve the matching degree between the generated image and the description text.

[0064] Among them, the learning rate is used to characterize the step size of the network weight adjustment in the iteration process of the Wensheng graph model. If the learning rate is set too high, the Wensheng graph model will not converge during the training process; if the learning rate is set too low, the training process will be slow. Batch is used to characterize the number of samples used to update the weights of the Wensheng graph model in each iteration. During the training process, the training data set is divided into multiple batch samples. The selection of batches will affect the training speed and generalization ability of the model. If the batch value is set too small, the training process will be unstable; if the batch value is set too large, the model will lack generalization ability. Training rounds are used to characterize the number of times the training data set is traversed and trained. If the training rounds are too few, the Wensheng graph model will not be able to fully learn the features in the data; if the training rounds are too many, the Wensheng graph model will overfit, that is, it will perform well on the training data but poorly on the new data. By setting the training parameters reasonably, the training efficiency and generalization ability of the Wensheng graph model can be improved.

[0065] In the process of synthesizing two-dimensional images from multiple perspectives into a three-dimensional model, the Vincent graph model first extracts the key features in the two-dimensional image, uses the posture trajectory algorithm to simulate the viewing direction of the two-dimensional image, and combines the key features and the viewing direction to project the points in the three-dimensional coordinate system to the pixel positions on the image, and uses the feature matching algorithm to establish the correspondence between the image and the three-dimensional space. Then, the three-dimensional information of the stereo image is generated based on the two-dimensional images of multiple views using the geometric and physical principles, and the three-dimensional information is constructed into a three-dimensional model using the three-dimensional reconstruction algorithm.

[0066] However, due to the relatively simple model architecture of the Wensheng graph model, there are limitations in understanding user semantics, which leads to the Wensheng graph model's inadequacy in understanding user semantics, which in turn leads to the generated three-dimensional stereoscopic images being deviated from the text description in terms of spatial relationship, resulting in the position, size and shape of objects in the image not meeting expectations. Moreover, in the process of generating three-dimensional stereoscopic images, the Wensheng graph model needs to use relatively complex algorithms to simulate physical phenomena such as the propagation of light and the shadows of objects, which consumes a lot of computing resources. Therefore, when the computing power of the application terminal is poor, the generated three-dimensional stereoscopic images will have problems such as perspective distortion and shadow errors, which will reduce the user's visual experience.

[0067] In order to improve the accuracy of the display device 200 generating a three-dimensional image based on the text description input by the user, some embodiments of the present application provide a display device 200, which generates a three-dimensional image corresponding to the image description text by executing a three-dimensional image generation method based on a large model. Figure 4As shown, in response to the image generation instruction, the controller 250 obtains the image description text in the image generation instruction, and performs preprocessing on the image description text to generate a feature vocabulary. The controller 250 inputs the preprocessed image description text into the text encoder so that the text encoder generates a sketch code, a topology code, and a stretch code, and establishes an association relationship between the sketch code, the topology code, and the stretch code to generate a feature code set. The display device 200 inputs the feature code set into the image generator to generate a target image, which is a three-dimensional stereo image corresponding to the image description text. The display 200 displays the target image, and rotates the target image according to the rotation instruction sent by the user to display a three-dimensional stereo image from different perspectives.

[0068] Figure 5 A flow chart of a method for generating a three-dimensional image for a display device is shown in FIG. Figure 5 As shown, the display device 200 includes a display 260 and a controller 250. The display 260 is configured to display a user interface. The controller 250 is configured to execute a three-dimensional stereoscopic image generation method based on a large model, including the following contents:

[0069] S100: In response to an image generation instruction input by a user, obtaining an image description text corresponding to the image generation instruction.

[0070] The display device 200 may receive a variety of different control instructions during operation, wherein some control instructions are used to control the display device 200 to generate a three-dimensional image corresponding to the text description using a text-image model, and these control instructions are called image generation instructions.

[0071] In some embodiments, the display device 200 provides a voice service for the user to enhance the convenience and intelligence level of the user's interaction with the display device 200. The display device 200 receives the voice data input by the user, and recognizes the voice data using voice recognition technology and semantic understanding technology, and converts the voice data into voice text, and uses the voice text as the image description text of the image generation instruction. Through the voice service provided by the display device 200, the user does not need to manually operate the remote control or touch screen, but can control the display device 200 through image generation instructions.

[0072] The image generation instruction includes image description text, which is used to describe the shape and spatial position of the image. In some embodiments, the image description text is a description of a composite body composed of multiple three-dimensional geometric bodies. The image description text includes the parameters of each geometric body (such as side length, radius) and the position of each geometric body in three-dimensional space, so that the display device 200 generates a three-dimensional stereoscopic image corresponding to the image description text. For example, the user inputs, through the image generation instruction, "Generate a cube with a side length of 3 cm. Above this cube is a sphere with a radius of 1 cm, and the sphere is at the center position of the top surface of the cube". The display device 200 inputs the image description text into the text encoder to obtain a set of feature encodings. Then, the set of feature encodings is input into the image generator to obtain the corresponding three-dimensional stereoscopic image, and the display 260 is controlled to display the image.

[0073] In some embodiments, in order to improve the accuracy of the text encoder in generating the set of feature encodings, the display device 200 performs preprocessing on the image description text to identify the structure of the composite geometric body. As Figure 6 shown, the display device 200 first performs word segmentation and part-of-speech tagging on the image description text to perform preprocessing on the image description text; then extracts feature phrases from the preprocessed image description text; and queries the geometric body thesaurus based on the geometric body type to obtain the geometric body structure.

[0074] During the process of the display device 200 performing preprocessing on the image description text, first, a preset word segmentation algorithm is used to obtain the vocabulary of the image description text and identify the boundaries of the vocabulary. Then, words that frequently appear in the image description text but have no actual contribution to text encoding, such as "de", "le", and "zai", are deleted to reduce the noise of the text data and improve the processing efficiency of text encoding. And part-of-speech tagging is performed on the segmented vocabulary to identify the feature phrases in the image description text. The feature phrases include the geometric body type and geometric body parameter information, and the geometric body parameter information includes length parameter information and position parameter information.

[0075] Exemplarily, the image description text is "Generate a cube with a side length of 3 cm. Above this cube is a sphere with a radius of 1 cm, and the sphere is at the center position of the top surface of the cube". Then the display device 200 extracts "cube", "side length", "3 cm", "above", "sphere", "radius", "4 cm", "top surface", "center position" as feature phrases.

[0076] In some embodiments, a geometric word library is stored in the display device 200, and the geometric word library is a database that characterizes the mapping relationship between geometric types and geometric structures. The display device 200 queries the structure corresponding to the geometric type according to the geometric type in the feature vocabulary, and the geometric structure includes two-dimensional geometric elements and geometric transformation methods for generating a single geometric body, and the geometric transformation method includes at least one of stretching, moving, and rotating.

[0077] Exemplarily, the geometric body type is a cube, and the geometric body structure is: the cube has 6 identical faces, each of which is a square; the cube has 12 edges, each of which is of equal length; the cube has 8 vertices, each of which is the intersection of three edges, and the three edges come from three different faces; the cube is generated by translating the square on the bottom upward in the direction of height.

[0078] For example, the geometric body type is a sphere, and the geometric body structure is that the sphere has a continuous spherical surface, and the distances between the points on the spherical surface and the center of the sphere are equal, and are all radii. The sphere is generated by rotating a circle along the diameter.

[0079] S200: Inputting the image description text into a text encoder to obtain a feature encoding set.

[0080] In some embodiments, the preprocessed image description text is a feature phrase, which cannot be directly calculated or analyzed by a machine learning model. Therefore, a text encoder is required to convert non-numeric text data into numerical form. The text encoder is a deep learning model that converts the image description text into feature coding according to coding rules. The feature coding set includes feature coding, and the feature coding includes sketch coding, topological coding, and stretch coding. Among them, the sketch coding is used to characterize the coordinate information of the generated image, the topological coding is used to characterize the curve type of the generated image, and the stretch coding is used to characterize the spatial structure of the generated image. There is an association relationship between the sketch coding, the topological coding, and the stretch coding.

[0081] The text encoder converts the image description text into feature codes to achieve text vectorization. In some embodiments, the three-dimensional image corresponding to the image description text is a complex structure composed of multiple single geometric bodies. The display device 200 obtains the structure, quantity and spatial position relationship of the single geometric bodies that constitute the target complex by extracting feature phrases in the image description text. Figure 7As shown, the display device 200 generates a first quantity based on the feature phrase, and the first quantity is the number of single geometric bodies constituting the target image; and uses the geometric body parameter information to mark the coordinate information of the single geometric body to generate a feature code of the single geometric body; generates a feature code set through a text encoder, wherein the feature code set includes the feature codes of the first quantity and a combination type, and the combination type is used to characterize the spatial position relationship between multiple single geometric bodies in the target image, and the combination type includes at least one of merging, crossing and differencing.

[0082] It should be understood that a geometric complex is a complex geometric shape composed of single geometric bodies through a specific combination method, and these combination methods include one or more of merging, crossing and difference. The display device 200 first separates the geometric complex to be generated into a first number of single geometric bodies according to the feature phrase, and then generates a feature code representing the coordinate information of the single geometric body according to the length parameter information and position parameter information of the single geometric body. And a text encoder is used to generate a feature code set based on the feature code and the combination method, and then the image description text is converted into a feature vector that can be understood by the image generator.

[0083] The merging is also called the union operation. In the process of merging multiple single geometric bodies, the display device 200 retains the non-overlapping parts and merges the overlapping parts into a whole, thereby combining them into a new geometric complex. The crossing is also called the intersection operation. In the process of crossing multiple single geometric bodies, the display device 200 retains the space area occupied by them as a new geometric complex. The difference is also called the difference operation. In the process of differentiating multiple single geometric bodies, the display device 200 subtracts one single geometric body from another single geometric body to obtain a new geometric complex.

[0084] In some embodiments, the display device 200 generates a feature code by mapping a single geometric body into a three-dimensional coordinate system. Figure 8 As shown, the display device 200 first constructs a three-dimensional coordinate system with the upper left vertex of the display 260 as the origin, and the three-dimensional coordinate system includes a first axis, a second axis and a third axis. Among them, the first axis is perpendicular to the second axis, the third axis is perpendicular to the sketch plane, and the sketch plane is the plane where the first axis and the second axis are located. The display device 200 maps the two-dimensional geometric element to the sketch plane according to the position parameter information to obtain the two-dimensional coordinates of the two-dimensional geometric element; and performs a simulated geometric transformation on the two-dimensional geometric element based on the geometric transformation method to generate a simulated single geometric body, and obtain the three-dimensional coordinates of the simulated single geometric body.

[0085] The display device 200 uses a three-dimensional coordinate system to represent the positional relationship of geometric elements such as points, lines and surfaces in a three-dimensional space. The three-dimensional coordinate system in the embodiment of the present application can be a Cartesian coordinate system, in which the upper left vertex of the display 260 is selected as the origin in the three-dimensional space, and three mutually perpendicular coordinate axes are selected as the first axis, the second axis and the third axis. Among them, the plane where the first axis and the second axis are located is the sketch plane.

[0086] In some embodiments, Fig. 9 As shown, a single geometric body is generated by geometric transformation of two-dimensional geometric elements, for example, a sphere is generated by circular rotation; a cylinder is generated by circular movement. The display device 200 obtains the two-dimensional geometric elements and geometric transformation methods corresponding to the single geometric body according to the geometric structure of the single geometric body, maps the two-dimensional geometric elements to the sketch plane, and obtains the two-dimensional coordinates of the two-dimensional geometric elements in the sketch plane. The display device 200 also obtains the three-dimensional coordinates of the simulated single geometric body by simulating the geometric transformation of the two-dimensional geometric elements. Among them, the two-dimensional coordinates are used to generate sketch coding and topological coding, and the three-dimensional coordinates are used to generate stretch coding.

[0087] In the process of generating feature encoding of a single geometric body by the display device 200, image segmentation is first performed on the two-dimensional geometric elements to obtain the surface structure. The surface structure includes a ring structure, and the ring structure includes a line structure. The ring structure is a closed edge composed of one or more line structures. The surface structure is a two-dimensional area composed of one or more ring structures.

[0088] It should be understood that the smallest basic element that constitutes a two-dimensional geometric element is a line. One or more lines can be combined into a ring. A ring is a closed sequence of edges. One or more rings can be combined into a surface. A surface can include an outer edge composed of an outer ring and a hole composed of multiple inner rings. One or more surfaces can be combined into a two-dimensional geometric element.

[0089] For example, Fig.10 As shown, the display device 200 performs image segmentation on the two-dimensional geometric elements to obtain a circular surface structure inside the hexagon, the surface structure includes a hexagonal ring structure and a circular ring structure, the hexagonal ring structure includes six line segments, and the circular ring structure includes a closed curve.

[0090] In some embodiments, the display device 200 generates a sketch code based on the two-dimensional coordinates of the two-dimensional geometric elements, the sketch code includes sequentially spliced ​​surface structure codes and a sketch structure end mark, the surface structure code includes sequentially spliced ​​ring structure codes and a surface structure end mark, the ring structure code includes sequentially spliced ​​line structure codes and a ring structure end mark, the line structure code includes sequentially spliced ​​line structure vectors and a line structure end mark, and the line structure vector is used to represent the key point coordinates of the line structure and the length of the line structure.

[0091] like Fig.11 As shown, the display device 200 marks the key points of the line structure based on the curve type of the line structure, and obtains the two-dimensional point coordinates of the key points in the sketch plane; then obtains the defined length of the line structure according to the length parameter information; and sequentially concatenates the two-dimensional point coordinates and the defined length to generate a line structure vector, and then generates a sketch code.

[0092] For example, see again Fig.10 , the display device 200 generates a sketch code according to the two-dimensional coordinates of the two-dimensional geometric elements. Among them, the key points of the line segment AB are point A (2, 1) and point B (3, 1), and the length is 1; the key points of the line segment BC are point B (3, 1) and point C (4, 2), and the length is √2; the key points of the line segment CD are point C (4, 2) and point D (3, 3), and the length is √2; the key points of the line segment DE are point D (3, 3) and point E (2, 3), and the length is 1; the key points of the line segment EF are point E (2, 3) and point F (1, 2), and the length is √2; the key points of the line segment FA are F (1, 2) and point A (2, 1), and the length is √2; the key point of the circle is the center (2.5, 2), and the length is π.

[0093] The display device 200 concatenates the key point coordinates and the length of the line structure to generate a line structure vector of line segment AB: (2,1), (3,1), (1,1). Since the key point coordinates are two-dimensional vectors and the length is a one-dimensional vector, in order to concatenate the two vectors, the display device 200 repeatedly represents the length value to generate a two-dimensional vector and concatenates it after the coordinate vector. The display device 200 adds a line structure end flag after the line structure vector to generate a line structure code. For example, the line end flag is (3,3).

[0094] For example, the line structure of line segment AB is encoded as (2,1), (3,1), (1,1), (3,3); the line structure of line segment BC is encoded as (3,1), (4,2), (√2,√2), (3,3); the line structure of line segment CD is encoded as (4,2), (3,3), (√2,√2), (3,3); the line structure of line segment DE is encoded as (3,3), (2,3), (1,1), (3,3); the line structure of line segment EF is encoded as (2,3), (1,2), (√2,√2), (3,3); the line structure of line segment FA is encoded as (1,2), (2,1), (√2,√2), (3,3); and the line structure of circle O is encoded as (2.5,2), (π,π), (3,3).

[0095] The display device 200 sequentially concatenates the line structure code and the ring structure end mark to generate a ring structure code. The ring structure end mark is used to indicate the end of the ring, and the display device 200 identifies the ring structure according to the ring end mark. For example, the ring structure end mark is (2,2). Exemplarily, Fig.10 The ring structure of the middle circle is encoded as (2.5,2),(π,π),(3,3),(2,2).

[0096] The display device 200 sequentially concatenates the ring structure code and the surface structure end flag to generate the surface structure code. The surface structure end flag is used to indicate the end of the surface, and the display device 200 identifies the surface structure according to the surface end flag. For example, the surface structure end flag is (1,1). Exemplarily, Fig.10 The mid-surface structure code is a hexagonal ring structure code, a circle O ring structure code and a surface structure end mark that are sequentially spliced.

[0097] The display device 200 sequentially splices the face structure code and the sketch structure end mark to generate a sketch code. The sketch structure end mark is used to indicate the end of the sketch, and the display device 200 identifies the sketch according to the sketch structure end mark. For example, the sketch structure end mark is (0,0). Exemplarily, if the two-dimensional geometric element includes face structure A and face structure B, the sketch code is the face structure code A, face structure code B and sketch structure end mark spliced ​​in sequence.

[0098] In some embodiments, each two-dimensional geometric element corresponds to a stretching operation, and the stretching operation is used to characterize the stretching, moving or rotating of the two-dimensional geometric element along the direction of the third axis to generate a simulated single geometric body. The display device 200 then generates a stretching code based on the three-dimensional coordinates of the simulated single geometric body. The stretching code includes the sequentially spliced ​​stretching distance, moving distance, rotation angle, stretching operation type and stretching end mark. For example, the stretching operation type can be fusion, intersection, cutting and independence. The stretching operation type is used to characterize the splicing relationship between geometric bodies in the process of generating a simulated single geometric body by two-dimensional geometric elements.

[0099] In some embodiments, the feature code also includes a topology code, which is used to characterize the curve type in the two-dimensional geometric element. The display device 200 generates a topology code according to the curve type of the line structure, the topology code includes a sequentially spliced ​​surface topology code and a sketch topology end mark, the surface topology code includes a sequentially spliced ​​ring topology code and a surface topology end mark, the ring topology code includes a sequentially spliced ​​line topology code and a ring topology end mark, and the line topology code is used to characterize the curve type of the line structure; the display device 200 establishes an association relationship between the sketch code, the topology code, and the stretch code to generate the feature code.

[0100] Exemplarily, the curve types of the line structure include line segments, circles, and arcs, wherein the line topology code corresponding to the line segment is (4), the line topology code corresponding to the circle is (5), and the line topology code corresponding to the arc is (6). The display device 200 sequentially concatenates the line topology code and the ring topology end flag to generate a ring topology code, and the ring topology end flag may be (2). See again Fig.10 , the hexagonal ring topology is encoded as (4),(4),(4),(4),(4),(4),(2). The circular ring topology is encoded as (5),(2).

[0101] The display device 200 sequentially concatenates the ring topology code and the surface topology end mark to generate the surface topology code. Exemplarily, the surface topology end mark is (1). Fig.10 The surface topology encoding corresponding to the mid-surface structure is (4), (4), (4), (4), (4), (4), (2), (5), (2), (1).

[0102] The display device 200 sequentially splices the surface topology code and the sketch topology end mark to generate a topology code. Exemplarily, the sketch topology end mark is (0). For example, if the two-dimensional geometric element includes a surface structure A and a surface structure B, the topology code is the sequentially spliced ​​surface topology code A, the surface topology code B, and the sketch topology end mark.

[0103] In some embodiments, the display device 200 establishes an association relationship between the sketch code, the topology code, and the stretch code to generate a feature code, and generates a feature code set based on the feature code. Since each two-dimensional geometric element corresponds to a stretching operation, the sketch code and the topology code are associated. Since the sketch code includes the line structure code and the topology code includes the line topology code, the sketch code and the topology code are associated based on the splicing order of the line structure.

[0104] S300: Inputting the feature encoding set into the image generator to obtain a target image.

[0105] In some embodiments, the display device 200 generates a three-dimensional stereoscopic image based on a feature coding set using an image generator. The image generator is a generative adversarial network model that generates a three-dimensional stereoscopic image based on the feature coding set. The image generator also includes a discriminator, and the display device 200 uses the discriminator to perform discrimination on the target image generated by the image generator to train the image generator to generate a three-dimensional stereoscopic image with high accuracy.

[0106] In order to improve the generalization ability of the image generator, such as Fig.12As shown, the display device 200 first obtains a training data set, then inputs the training feature encoding set into the image generator to obtain an output three-dimensional image; and inputs the real three-dimensional image and the output three-dimensional image into the discriminator, and uses the loss function to calculate the generator loss; and then adjusts the model parameters of the image generator according to the generator loss, thereby training the image generator. Among them, the training data set includes the training feature encoding set and the real three-dimensional image; the generator loss is used to characterize the difference between the real three-dimensional image and the output three-dimensional image.

[0107] The discriminator is a binary classifier that distinguishes between real images and target images generated by the generator. By inputting the real image and the target image generated by the image generator into the discriminator, a probability value indicating that the target image is a real image can be obtained. The probability value is used to indicate the accuracy of the image generated by the image generator.

[0108] In some embodiments, the display device 200 applies a back propagation algorithm to calculate the gradient information of the generator loss, and updates the parameters of the image generator layer by layer to minimize the generator loss. Among them, back propagation is an algorithm for training deep neural networks (such as generators, discriminators) and optimizing their parameters. The display device 200 first calculates the partial derivatives of the loss function with respect to the weights and biases in the deep neural network, and then updates these parameters using the gradient descent algorithm, so that the output result of the deep neural network is as close to the true value as possible. In the back propagation algorithm, the input sample is first passed through each layer of the deep neural network through forward propagation to obtain the output result, and the loss function is calculated with the real label. In the back propagation stage, the parameter gradient between the output layer and the hidden layer is calculated according to the loss function, and the parameter gradient of each layer is calculated forward in turn. Using the calculated gradient, the parameters of each layer are updated by the gradient descent algorithm to minimize the loss.

[0109] In some embodiments, the display device 200 performs iterative training on the image generator, continuously updates parameters through back propagation until the model converges or reaches a preset stopping condition, and then uses an independent test data set to evaluate the performance of the model, such as indicators such as the model's accuracy, precision, and recall rate, to complete data processing.

[0110] In some embodiments, the display device 200 further trains the discriminator by updating the model parameters of the discriminator. The display device 200 inputs the target image and the real image into the discriminator to obtain the identification result; then obtains the real identification result, and calculates the identification loss between the real result and the identification result using the loss function; then based on the identification loss, updates the model parameters of the discriminator.

[0111] The display device 200 maintains a balance between the image generator and the discriminator by alternately training the image generator and the discriminator. If the image generator is trained too strongly, the discriminator cannot provide enough gradient information to guide the image generator to improve; if the discriminator is trained too strongly, the image generator cannot learn useful information from the identification results. Therefore, the display device 200 needs to properly adjust the training ratio of the image generator and the discriminator.

[0112] S400: Control the display to display the target image.

[0113] In some embodiments, after the display device 200 acquires the target image generated by the image generation model, it controls the display 260 to display the target image.

[0114] In some embodiments, the display device 200 generates rotation parameters in response to a rotation instruction input by a user, the rotation parameters including a rotation angle, a rotation direction and a rotation axis, and generates a rotation matrix based on the rotation parameters, and multiplies the rotation matrix with the coordinate information of the feature code to obtain a rotation feature code set; the rotation feature code set is then input into an image generator to obtain a rotated target image.

[0115] The display device 200 may receive various control instructions during operation, some of which are used to control the display device 200 to rotate the target image, and these control instructions are called rotation instructions. The display device 200 obtains the rotation angle and rotation direction in the rotation instruction to construct a rotation matrix.

[0116] The rotation matrix is ​​a coefficient matrix constructed based on Euler angles or quaternions, and is used to describe rotations in three-dimensional space. Through the rotation matrix, each point in a three-dimensional image can be rotated in a specified direction and angle.

[0117] Exemplarily, the display device 200 multiplies the coordinate value of each point in the three-dimensional stereoscopic image by the rotation matrix to obtain the rotated coordinates, and generates a rotation feature code set according to the rotated coordinates. The rotation feature code set is used to update the three-dimensional stereoscopic image.

[0118] Based on the above display device 200, some embodiments of the present application further provide a method for generating a three-dimensional image based on a large model, comprising the following steps:

[0119] In response to an image generation instruction input by a user, acquiring an image description text corresponding to the image generation instruction, wherein the image description text is used to describe the shape and spatial position of the generated image;

[0120] Input the image description text into a text encoder to obtain a feature coding set, wherein the text encoder is a deep learning model that converts the image description text into feature coding according to a coding rule, wherein the feature coding set includes feature coding, wherein the feature coding includes sketch coding, topological coding, and stretch coding, wherein the sketch coding is used to characterize the coordinate information of the generated image, the topological coding is used to characterize the curve type of the generated image, and the stretch coding is used to characterize the spatial structure of the generated image, and the sketch coding, the topological coding, and the stretch coding are associated with each other;

[0121] The feature coding set is input into an image generator to obtain a target image, and the display is controlled to display the target image. The image generator is a generative adversarial network model that generates a three-dimensional stereo image based on the feature coding set.

[0122] As can be seen from the above technical solutions, some embodiments of the present application provide a display device and a method for generating a three-dimensional stereoscopic image based on a large model. The method can respond to the image generation instruction input by the user, obtain the image description text corresponding to the image generation instruction, and input the image description text into the text encoder to obtain a feature coding set, and input the feature coding set into the image generator to obtain the target image, and then control the display to display the target image. Among them, the image description text is used to describe the shape and spatial position of the generated image; the text encoder is a deep learning model that converts the image description text into feature coding according to the coding rule, the feature coding set includes feature coding, the feature coding includes sketch coding, topology coding and stretch coding, the sketch coding is used to characterize the coordinate information of the generated image, the topology coding is used to characterize the curve type of the generated image, and the stretch coding is used to characterize the spatial structure of the generated image, and the sketch coding, topology coding and stretch coding are associated with each other; the image generator is a generative adversarial network model that generates a three-dimensional stereoscopic image according to the feature coding set. The method can use the text encoder to convert the image description text input by the user into a feature coding set, and use the image generator to generate a three-dimensional stereoscopic image according to the feature coding set, thereby improving the accuracy of the target image.

[0123] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0124] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0126] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: a display configured to display a user interface; The controller is configured as: In response to an image generation instruction input by a user, acquiring an image description text corresponding to the image generation instruction, wherein the image description text is used to describe the shape and spatial position of the generated image; Input the image description text into a text encoder to obtain a feature coding set, wherein the text encoder is a deep learning model that converts the image description text into feature coding according to a coding rule, wherein the feature coding set includes feature coding, wherein the feature coding includes sketch coding, topological coding, and stretch coding, wherein the sketch coding is used to characterize the coordinate information of the generated image, the topological coding is used to characterize the curve type of the generated image, and the stretch coding is used to characterize the spatial structure of the generated image, and the sketch coding, the topological coding, and the stretch coding are associated with each other; The feature coding set is input into an image generator to obtain a target image, and the display is controlled to display the target image. The image generator is a generative adversarial network model that generates a three-dimensional stereo image based on the feature coding set.

2. The display device according to claim 1, characterized in that After executing the acquisition of the image description text corresponding to the image generation instruction, the controller is further configured to: Performing word segmentation and part-of-speech tagging on the image description text to perform preprocessing on the image description text; Extracting feature phrases from the preprocessed image description text, wherein the feature phrases include geometric body type and geometric body parameter information, and the geometric body parameter information includes length parameter information and position parameter information; A geometric vocabulary is queried based on the geometric type to obtain a geometric structure, wherein the geometric vocabulary is used to characterize a mapping relationship between the geometric type and the geometric structure, wherein the geometric structure includes two-dimensional geometric elements and geometric transformation methods for generating a single geometric body, wherein the geometric transformation methods include at least one of stretching, moving and rotating.

3. The display device according to claim 2, characterized in that The controller executes inputting the image description text into a text encoder to obtain a feature encoding set, and is specifically configured as follows: Generate a first quantity according to the characteristic phrase, the first quantity being the number of single geometric bodies constituting the target image; Marking the coordinate information of the single geometric body by using the geometric body parameter information to generate a feature code of the single geometric body; A feature coding set is generated by the text encoder, and the feature coding set includes a first number of feature codes and combination types, and the combination type is used to characterize the spatial position relationship between multiple single geometric bodies in the target image, and the combination type includes at least one of merging, crossing and differencing.

4. The display device according to claim 3, characterized in that Before the controller uses the geometric body parameter information to mark the coordinate information of the single geometric body to generate the feature code of the single geometric body, the controller is further configured to: Taking the upper left vertex of the display as the origin, constructing a three-dimensional coordinate system, the three-dimensional coordinate system includes a first axis, a second axis and a third axis, the first axis is perpendicular to the second axis, the third axis is perpendicular to the sketch plane, and the sketch plane is the plane where the first axis and the second axis are located; Mapping the two-dimensional geometric element to the sketch plane according to the position parameter information to obtain the two-dimensional coordinates of the two-dimensional geometric element; Based on the geometric transformation method, a simulated geometric transformation is performed on the two-dimensional geometric elements to generate a simulated single geometric body, and the three-dimensional coordinates of the simulated single geometric body are obtained.

5. The display device according to claim 4, characterized in that The controller uses the geometric body parameter information to mark the coordinate information of the single geometric body to generate a feature code of the single geometric body, and is specifically configured as follows: Performing image segmentation on the two-dimensional geometric element to obtain a surface structure, wherein the surface structure includes a ring structure, the ring structure includes a line structure, the ring structure is a closed edge line composed of one or more of the line structures, and the surface structure is a two-dimensional region composed of one or more of the ring structures; Generate a sketch code according to the two-dimensional coordinates of the two-dimensional geometric element, the sketch code includes sequentially spliced ​​surface structure codes and a sketch structure end mark, the surface structure code includes sequentially spliced ​​ring structure codes and a surface structure end mark, the ring structure code includes sequentially spliced ​​line structure codes and a ring structure end mark, the line structure code includes sequentially spliced ​​line structure vectors and a line structure end mark, and the line structure vector is used to characterize the key point coordinates of the line structure and the length of the line structure; A stretch code is generated according to the three-dimensional coordinates of the simulated single geometric body, wherein the stretch code includes a stretch distance, a moving distance, a rotation angle, a stretch operation type and a stretch end mark which are sequentially spliced.

6. The display device according to claim 5, characterized in that After the controller generates the sketch code according to the two-dimensional coordinates of the two-dimensional geometric element, the controller is further configured to: Generate a topology code according to the curve type of the line structure, the topology code includes sequentially spliced ​​surface topology codes and a sketch topology end mark, the surface topology code includes sequentially spliced ​​ring topology codes and a surface topology end mark, the ring topology code includes sequentially spliced ​​line topology codes and a ring topology end mark, and the line topology code is used to characterize the curve type of the line structure; An association relationship is established among the sketch code, the topology code and the stretch code.

7. The display device according to claim 6, characterized in that The controller generates a sketch code according to the two-dimensional coordinates of the two-dimensional geometric element, and is specifically configured as follows: Marking key points of the line structure based on the curve type of the line structure, and obtaining two-dimensional point coordinates of the key points in the sketch plane; Acquire the defined length of the line structure according to the length parameter information; The two-dimensional point coordinates and the defined length are sequentially concatenated to generate the line structure vector.

8. The display device according to claim 1, characterized in that The image generator includes a discriminator, and before the controller executes inputting the feature encoding set into the image generator to obtain a target image, the controller is further configured to: Acquire a training data set, wherein the training data set includes a training feature encoding set and a real three-dimensional image; Inputting the training feature encoding set into the image generator to obtain an output three-dimensional image; Inputting the real 3D image and the output 3D image into the discriminator, and calculating a generator loss using a loss function, wherein the generator loss is used to characterize the difference between the real 3D image and the output 3D image; Model parameters of the image generator are adjusted according to the generator loss.

9. The display device according to claim 1, characterized in that After the controller controls the display to display the target image, the controller is further configured to: generating a rotation angle and a rotation direction in response to a rotation instruction input by a user, wherein the rotation instruction is used to instruct the target image to rotate along a rotation axis; Based on the rotation angle and the rotation direction, generating a rotation matrix, wherein the rotation matrix is ​​used to characterize a linear transformation of the feature code; Multiplying the rotation matrix with the coordinate information of the feature code to obtain a rotation feature code set; The rotation feature encoding set is input into an image generator to obtain a rotated target image.

10. A method for generating a three-dimensional image based on a large model, characterized in that: Applied to the display device according to any one of claims 1 to 9, the method comprising: In response to an image generation instruction input by a user, acquiring an image description text corresponding to the image generation instruction, wherein the image description text is used to describe the shape and spatial position of the generated image; Input the image description text into a text encoder to obtain a feature coding set, wherein the text encoder is a deep learning model that converts the image description text into feature coding according to a coding rule, wherein the feature coding set includes feature coding, wherein the feature coding includes sketch coding, topological coding, and stretch coding, wherein the sketch coding is used to characterize the coordinate information of the generated image, the topological coding is used to characterize the curve type of the generated image, and the stretch coding is used to characterize the spatial structure of the generated image, and the sketch coding, the topological coding, and the stretch coding are associated with each other; The feature coding set is input into an image generator to obtain a target image, and the display is controlled to display the target image. The image generator is a generative adversarial network model that generates a three-dimensional stereo image based on the feature coding set.

Images