3D model manufacturing method and device based on Gaussian point cloud model, and storage medium

Abstract

The embodiment of the invention provides a 3D model making method and device based on a Gaussian point cloud model and a storage medium, and the method comprises the steps: responding to a touch operation of a model making application, displaying a model making interface, and enabling the model making interface to display a virtual image and a selection cursor; in response to a movement selection operation on the selection cursor through the control operation equipment, displaying a target image area in an area selected by the selection cursor in the virtual image through movement, and displaying an editable Gaussian point cloud model in the target image area, the editable Gaussian point cloud model being generated through a preset model generation model; and in response to an editing operation on the Gaussian point cloud model, executing the editing operation on the Gaussian point cloud model to obtain a target point cloud model. By providing a model making application applied to a virtual reality device for a user, the user can perform editing operation in a virtual image by selecting a cursor to generate an editable Gaussian point cloud model, so that immersive 3D creation by using a VR interface is realized, and 3D GS is edited in a flexible and controllable manner to generate a 3D model.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a 3D model production method, device and storage medium based on a Gaussian point cloud model. Background Art

[0002] Currently, the development of virtual reality (VR) and metaverse applications requires a large amount of 3D model resources to create an immersive environment. There are many artificial intelligence (AI) methods to help generate 3D models, but the models generated by these methods are difficult for users to display and edit to complete downstream tasks. 3D Gaussian splatter technology can use differentiable rendering to reconstruct the corresponding 3D model from a set of pictures. It has attracted widespread attention due to its realistic reconstruction effect and suitability for traditional rendering pipelines. However, there is currently no system that provides an interactive system for controlling the generation and editing of 3DGS, nor is there a related method for controlling the generation and editing of 3DGS. Editing 3DGS in a flexible and controllable way to generate 3D models remains a challenge. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The embodiments of the present application provide a 3D model production method, device and storage medium based on a Gaussian point cloud model, which can facilitate users to flexibly control the image generation and editing of 3DGS.

[0005] In a first aspect, an embodiment of the present application provides a 3D model production method based on a Gaussian point cloud model, which is applied to a virtual reality device, wherein the virtual reality device includes a control operation device, and the method includes: In response to a touch operation on the model making application, displaying a model making interface, the model making interface displaying a virtual image and a selection cursor; In response to a moving selection operation of the selection cursor by the control operation device, the selection cursor is moved in the virtual image to select an area to display a target image area, and an editable Gaussian point cloud model is displayed in the target image area, wherein the editable Gaussian point cloud model is generated by a preset model generation model; In response to the editing operation on the Gaussian point cloud model, the editing operation is performed on the Gaussian point cloud model to obtain a target point cloud model.

[0006] According to the 3D model making method based on the Gaussian point cloud model provided in some embodiments of the present application, the displaying of the editable Gaussian point cloud model in the target image area includes: Generate a corresponding image mask according to the target image area; Calling the preset model generation model to generate an editable Gaussian point cloud model according to the image mask and the image in the target image area; The Gaussian point cloud model is highlighted in a first color in the target image area.

[0007] According to the 3D model production method based on the Gaussian point cloud model provided in some embodiments of the present application, the preset model generation model includes restriction parameters, and the restriction parameters are used to optimize the Gaussian point cloud model.

[0008] According to the 3D model making method based on the Gaussian point cloud model provided in some embodiments of the present application, the generating of the editable Gaussian point cloud model according to the image mask and the image in the target image area includes: Acquire a plurality of images of different viewing angles in the target image area; An editable Gaussian point cloud model is generated according to the multiple images of different viewing angles and the image mask.

[0009] According to the 3D model making method based on the Gaussian point cloud model provided by some embodiments of the present application, before the editing operation is performed on the Gaussian point cloud model in response to the editing operation on the Gaussian point cloud model to obtain the target point cloud model, the method further includes: In response to the moving and selecting operation of the selection cursor on the Gaussian point cloud model by the control operation device, the selected portion of the Gaussian point cloud model is highlighted in a second color.

[0010] According to the 3D model making method based on the Gaussian point cloud model provided in some embodiments of the present application, the control operation device includes an erase control key; in response to the editing operation on the Gaussian point cloud model, performing the editing operation on the Gaussian point cloud model includes: In response to a touch operation on the erase control key, the selected erase image area is displayed by moving the selection cursor through the control operation device; In response to the touch operation on the erase key again, the erase image area is erased.

[0011] According to the 3D model making method based on the Gaussian point cloud model provided in some embodiments of the present application, the control operation device includes a palette control key; in response to the editing operation on the Gaussian point cloud model, performing the editing operation on the Gaussian point cloud model includes: In response to a touch operation on the palette control key, displaying a palette interface, wherein a plurality of colors are displayed in the palette interface; In response to a selection operation on a color in the color palette interface, a corresponding color modification operation is performed on the Gaussian point cloud model.

[0012] According to the 3D model making method based on the Gaussian point cloud model provided in some embodiments of the present application, the control operation device includes a cursor zoom control key; the method also includes: In response to a touch operation on the cursor zoom control key, a corresponding zoom operation is performed on the selection cursor.

[0013] In a second aspect, an embodiment of the present application provides an electronic device, including: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the 3D model production method based on the Gaussian point cloud model as described in the first aspect of the embodiment of the present application is implemented.

[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program executable by a processor. When the computer program executable by the processor is executed by the processor, it is used to implement the 3D model production method based on the Gaussian point cloud model as described in the first aspect of the embodiment of the present application.

[0015] In the technical solution provided by the embodiment of the present application, in response to the touch operation of the model making application, the model making interface is displayed, and the model making interface displays a virtual image and a selection cursor; in response to the moving selection operation of the selection cursor by controlling the operation device, the selection cursor is moved in the virtual image to select the area to display the target image area, and an editable Gaussian point cloud model is displayed in the target image area, and the editable Gaussian point cloud model is generated by the preset model generation model; in response to the editing operation of the Gaussian point cloud model, the editing operation is performed on the Gaussian point cloud model to obtain the target point cloud model. By providing the user with a model making application applied to a virtual reality device, the user can edit the editable Gaussian point cloud model generated by the editing operation of the selection cursor in the virtual image, thereby realizing immersive 3D creation using the VR interface, and editing 3DGS in a flexible and controllable manner to generate a 3D model.

[0016] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained through the structures particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0018] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application; Figure 2 It is a schematic diagram of the steps of a 3D model making method based on a Gaussian point cloud model provided in an embodiment of the present application; Figure 3 It is a schematic diagram of the steps of generating a Gaussian point cloud model provided in an embodiment of the present application; Figure 4 It is a framework schematic diagram of a hierarchical management system of a micro-rendering framework provided in an embodiment of the present application; Figure 5 It is a schematic diagram of a process for generating an editable Gaussian point cloud model provided in an embodiment of the present application; Figure 6a It is a schematic diagram of a process of selecting a partial Gaussian point cloud model provided in an embodiment of the present application; Figure 6b It is a schematic diagram of another process of selecting a partial Gaussian point cloud model provided in an embodiment of the present application; Figure 6c It is a schematic diagram of another process of selecting a partial Gaussian point cloud model provided in an embodiment of the present application; Figure 7 is a schematic diagram of a control operation device provided in an embodiment of the present application; Figure 8 is a panel schematic diagram of a palette interface provided in an embodiment of the present application; Fig. 9 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0020] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0022] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0023] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level technology and software-level technology. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction system, mechatronics, etc. Among them, the pre-trained model is also called the large model or basic model. After fine-tuning, it can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0024] Virtual Reality (VR) is a computer technology that can create and experience a virtual world. It uses software-generated images, sounds, and other sensory stimuli to simulate the user's presence in the physical world. VR technology allows users to immerse themselves in and interact with a three-dimensional environment generated by a computer through devices such as head-mounted displays (HMDs), handle controllers, and sensors.

[0025] The Metaverse is a collective virtual shared space that is a fusion of physical reality and digital virtual world. This concept envisions a persistent, shareable 3D virtual environment in which users can engage in social interaction, work, entertainment and other activities. The concept of the Metaverse originated from the 1992 science fiction novel SnowCrash, but with the development of technology, especially the Internet, virtual reality (VR), augmented reality (AR), blockchain and artificial intelligence, the technical foundation for building the Metaverse has gradually matured.

[0026] 3D Gaussian Splatting (3DGS) is a technology used for real-time rendering and 3D modeling. It combines the advantages of traditional point clouds and the mathematical properties of Gaussian distribution, providing a new way to represent and process 3D data. The core idea of ​​this technology is to use Gaussian functions to represent each point in the point cloud data, and to create a continuous and smooth surface model by superimposing these Gaussian distributions. Compared with traditional triangular mesh or voxel representation methods, 3DGS can process large-scale point cloud data sets more efficiently and support real-time interaction. Due to the use of GPU accelerated computing, 3D Gaussian point clouds are more efficient in processing large-scale point cloud data, and through the smooth characteristics of Gaussian distribution, very realistic rendering effects can be generated.

[0027] Currently, the development of VR and metaverse applications requires a large amount of 3D model resources to create an immersive environment. There are many AI methods to help generate 3D models, but the models generated by these methods are difficult for users to display and edit to complete downstream tasks. 3DGS can use differentiable rendering to reconstruct the corresponding 3D model from a set of pictures. It has attracted widespread attention due to its realistic reconstruction effect and suitability for traditional rendering pipelines. However, there is currently no system that provides an interactive system for controlling the generation and editing of 3DGS, nor is there a related method for controlling the generation and editing of 3DGS. Editing 3DGS in a flexible and controllable way to generate 3D models remains a challenge.

[0028] Based on this, the embodiments of the present application provide a 3D model production method, device and storage medium based on a Gaussian point cloud model, which can facilitate users to flexibly control the image generation and editing of 3DGS.

[0029] Reference Figure 1 As shown, Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application. Figure 1 The illustrated implementation environment includes a first user terminal 101 and a server 102. The first user terminal 101 and the server 102 are directly or indirectly connected via wired or wireless communication, wherein both the first user terminal 101 and the server 102 can be nodes in a blockchain, which is not specifically limited in this embodiment.

[0030] The first user terminal 101 may be an independent virtual reality device. Optionally, the first user terminal 101 may be installed with multiple applications, including a model making application, and the user may touch the model making application through the first user terminal 101. The first user terminal 101 may respond to the selection operation of selecting one of the multiple applications by displaying the application interface corresponding to the selected application on the first user terminal 101, and displaying specific information or interactive controls in the application interface. When the model making application is in operation, the model making application may interact with the server 102 through the first user terminal 101 for data streams.

[0031] The server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN networks, and big data and artificial intelligence platforms. The server 102 may store specific information in a business data stream.

[0032] It should be noted that the 3D model making method based on the Gaussian point cloud model in the present application is applied to a virtual reality device, which includes a VR headset and multiple controllers. The VR headset can be provided with a camera device for capturing the user's current environment. When the user uses the VR headset, the camera device can be automatically used to capture images to obtain a virtual image; the controller includes multiple function control keys, each of which can be set to be bound to a corresponding function. When the user touches the function control key on the controller, in response to the touch operation of the function case, the operation function corresponding to the function control key is executed. For example, the function control key can include a delete function control key. When the user touches the delete function control key, the 3D model part selected by the user in the model making application is deleted. No specific limitation is made here.

[0033] In one embodiment, during the operation of the model making application, the user can touch the model making application through the first user terminal 101. The first user terminal 101 responds to the touch operation of the model making application in the model making application, and displays the model making interface in the virtual reality device. The model making interface can display a virtual image and a selection cursor, and the selection cursor is used to select an area. On the one hand, the virtual image can be based on the real world, and the selection cursor is superimposed on the real world to generate a virtual image; on the other hand, the virtual image can simulate the real world and generate a virtual image corresponding to the real world. The user operates in the virtual image by selecting the cursor. Then, the first user terminal 101 can respond to the movement and selection operation of the selection cursor through the control operation device, and move the selection cursor in the virtual image to display the target image area, and display the editable Gaussian point cloud model in the target image area, wherein the editable Gaussian point cloud model is generated by the preset model generation model. Among them, the image generation model generation is obtained by the first user terminal 101 from the server 102.

[0034] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on the attribute information or attribute information set of the target object (such as a user, etc.) and other data related to the characteristics of the target object, the permission or consent of the target object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiment of the present application needs to obtain the attribute information of the target object, the separate permission or separate consent of the target object will be obtained through a pop-up window or by jumping to a confirmation interface. After clearly obtaining the separate permission or separate consent of the target object, the relevant data of the target object necessary for the normal operation of the embodiment of the present application will be obtained.

[0035] Reference Figure 2 As shown, Figure 2 1 is a schematic diagram of the steps of a 3D model making method based on a Gaussian point cloud model provided in an embodiment of the present application. The 3D model making method based on a Gaussian point cloud model can be executed by a server, or by a user terminal, or by a user terminal and a server together. In the embodiment of the present application, the method is described by taking the method being executed by a user terminal and a server together as an example. Figure 2 The 3D model making and processing method based on the Gaussian point cloud model includes but is not limited to steps S210 to S230.

[0036] Step S210: In response to a touch operation on the model making application, a model making interface is displayed, where the model making interface displays a virtual image and a selection cursor.

[0037] In one embodiment, when the user opens the model making application, a selection cursor is displayed, and the selection cursor can be directly moved and touched; the starting interface of the model making application is the main function interface, and touch controls for other functions are generated in the preset position of the main function interface. The user can control the selection cursor to move and select, and perform touch operations on the touch keys to execute the corresponding functions.

[0038] For example, the touch control keys may include but are not limited to: a new control key, an open control key, etc. By touching the new control key, the user can create a new 3D model file through the model making application for making a 3D model; by touching the open control key, the model making application responds to the touch operation of the open control key and displays an open page. The open page displays multiple 3D model files, so that the user can select the 3D model file that needs to be processed and open it for 3D model making. No specific limitation is made here.

[0039] After the virtual image and the selection cursor are displayed, step S220 may be further executed to edit the virtual image.

[0040] Step S220, in response to the movement selection operation of the selection cursor through the control operation device, the selection cursor is moved in the virtual image to select the target image area, and an editable Gaussian point cloud model is displayed in the target image area. The editable Gaussian point cloud model is generated by a preset model generation model.

[0041] In one embodiment of the present application, a selection cursor is used for mobile selection. The selection cursor moves along with a controller held by a user. The user selects an area by continuously touching the selection function control key on the controller on the virtual reality device. The model making application responds to the movement selection operation of the selection cursor through the control operation device, moves the selection cursor in the virtual image to select the target image area, and displays an editable Gaussian point cloud model in the target image area.

[0042] It should be noted that in the embodiment of the present application, the editable Gaussian point cloud model is generated by a preset model generation model, please refer to Figure 3 , is a schematic diagram of the steps of generating a Gaussian point cloud model provided in an embodiment of the present application, such as Figure 3 As shown, in the embodiment of the present application, displaying the editable Gaussian point cloud model in the target image area may include but is not limited to steps S310 to S330: Step S310, generating a corresponding image mask according to the target image area; It should be noted that the target image area in the present application includes multiple viewpoints selected by the user. When generating an image mask, a mask is projected and generated based on the viewpoints given by the user, which corresponds to the target image area for subsequent generation of a Gaussian point cloud model.

[0043] Step S320, calling a preset model generation model, and generating an editable Gaussian point cloud model according to the image mask and the image in the target image area; It should be noted that in the embodiment of the present application, the preset model generation model combines the hierarchical management system of the micro-rendering framework, please refer to Figure 4 , is a schematic diagram of a hierarchical management system of a micro-rendering framework provided in an embodiment of the present application, such as Figure 4 As shown, in the embodiment of the present application, the hierarchical management system of the micro-rendering framework is mainly used to generate and edit 3D models. The following is a description of each part in the figure: GS Object: represents an object in the graphics system, which can be any generated 3D model or geometry.

[0044] Container: A container used to organize and manage multiple GS Objects to form a hierarchical structure.

[0045] Leaf: A leaf node represents the end object in a hierarchical structure, usually the image model part of a 3D model.

[0046] Training Context: Editing context, used to specify which objects are transformable (Trainable) during the editing process.

[0047] Trainable: Transformable objects whose parameters can be transformed during the editing process. Transformations may include but are not limited to: translation, rotation, and scaling.

[0048] Frozen: Frozen objects whose parameters do not change during editing, often used to provide a stable reference or background.

[0049] Differentiable Rendering: Differentiable rendering, which is the core of the framework, allows to optimize the parameters of the 3D model by calculating gradients.

[0050] Rendering: The rendering process converts the 3D model into a 2D image. The hierarchical management system of the micro-rendering framework allows the generated 3D model to be converted into a 2D object for subsequent plane editing.

[0051] Ground Truth: The true value or target image used for comparison and optimization during the editing process.

[0052] Optimize: The optimization process compares the rendered image with the real value and adjusts the parameters of the editable object to minimize the difference between the two.

[0053] like Figure 4 As shown in the figure, a hierarchical structure in which some GS Objects are marked as trainable and others are frozen. During the editing process, only the editable objects will have their parameters adjusted according to the difference between the rendering result and the true value. This structure allows the model to optimize other parts to improve rendering quality or achieve specific visual effects while keeping some parts stable. The hierarchical structure provides valuable contextual information for the image generation model, allowing users to set the editing context for the target object to ensure that the edited object seamlessly blends into its surroundings. Users can set the overall conditions for editing and then edit local sub-objects.

[0054] On this basis, in an embodiment of the present application, the mask and the image in the target image area are input into a preset image model to form an editable Gaussian point cloud model.

[0055] It should be noted that, in the embodiment of the present application, when the editable Gaussian point cloud model is generated, it can also be optimized through multiple different perspective images, and the perspective image can be an image photo of the environment obtained by the user at any angle, which can be obtained by the camera device carried by the VR head display mentioned above, and will not be repeated here; Exemplarily, in one embodiment of the present application, generating an editable Gaussian point cloud model according to an image mask and an image in a target image area may include the following steps: Acquire multiple images of different viewing angles in the target image area; Generate an editable Gaussian point cloud model based on multiple images from different perspectives and image masks.

[0056] In one embodiment of the present application, when it is necessary to generate an editable Gaussian point cloud model, first, multiple different perspective images of the target image area of ​​the user under multiple perspectives are obtained, specifically, they may include a top-down perspective image, a full-view perspective image, and a side-view perspective image. The multiple different perspective images are used as guide images for generating an editable Gaussian point cloud model; then, the editable Gaussian point cloud model is generated based on the multiple different perspective images and image masks.

[0057] Through multiple images from different perspectives, when generating an editable Gaussian point cloud model, the details of each part of the Gaussian point cloud model can be improved to obtain a Gaussian point cloud model with high restoration degree.

[0058] Specifically, refer to Figure 5In a feasible embodiment of the present application, when generating an editable Gaussian point cloud model in step S320, first, the target image area formed by the viewpoint selected by the user is projected to generate a 2D mask; then, the rendered images under the N camera perspectives are input into the preset model generation model together with the preset text prompt; the preset model generation model generates the edited image based on this information. Finally, the edited image color information is projected back to the 3D point cloud to generate an editable Gaussian point cloud model, completing the entire editing process.

[0059] Through step S320, the generated Gaussian point cloud model is combined with a virtual reality device, so that the user can edit at a designated point cloud area location, thereby improving the flexibility and convenience of editing the 3D model.

[0060] It should be noted that, in the embodiment of the present application, in order to better generate a Gaussian point cloud model, the preset model generation model includes restriction parameters, and the restriction parameters are used to optimize the Gaussian point cloud model.

[0061] In one embodiment of the present application, the Gaussian point cloud model is optimized by limiting the parameters, and a limit loss function is introduced to control the related generation and editing. The generated Gaussian point cloud includes a large number of points, which are represented as , forming a rough representation of the desired shape, and introducing a shape control loss function as a regularization term: ; ... (1) Among them, L shape is the defined shape control loss function, p represents the points being optimized and edited, which are split and optimized from the initial points, and is a restriction parameter used to control the degree of shape restriction. Represents the point set being optimized. Through this restriction, the Gaussian point cloud generated by the editing and generation method will tend to be close to the area where the Gaussian point cloud model is located, thereby achieving a controllable editing and generation effect. This controlled Gaussian point cloud generation and editing can be applied to specific editing operation tasks such as hole filling and object generation without affecting other irrelevant areas or generating irrelevant shapes. The shape control loss function can not only optimize the image when generating the Gaussian point cloud model, but also optimize when the user selects the image area for image editing, avoiding the selection of image areas not selected by the user.

[0062] For example, when generating a Gaussian point cloud model, the Gaussian point cloud includes a large number of points, which are represented as , p represents the point being optimized in the generated Gaussian point cloud model, The point set representing the Gaussian point cloud model is optimized through the shape control loss function so that the generated Gaussian point cloud model can be closer to the Gaussian point cloud model required by the user.

[0063] After the editable Gaussian point cloud model is generated in step S320, the user can be informed that the point cloud image is created and can perform editing operations. The notification method is completed through step S330.

[0064] Step S330: highlighting the Gaussian point cloud model in the target image area with a first color.

[0065] In a feasible embodiment of the present application, after the Gaussian point cloud model is generated, it is located in the target image area. At this time, in order to inform the user of the generated Gaussian point cloud model and that it can be edited, the Gaussian point cloud model is highlighted with a first color. The first color can be a single bright color, such as green, etc. The first color can enable the user to clearly understand the location and editable status of the editable Gaussian point cloud model. The specific color selected is not limited here.

[0066] Step S230: In response to the editing operation on the Gaussian point cloud model, the editing operation is performed on the Gaussian point cloud model to obtain a target point cloud model.

[0067] It should be noted that in the embodiment of the present application, when the user needs to edit the Gaussian point cloud model, he needs to first select the location for editing.

[0068] In an embodiment of the present application, before performing step S230, the 3D model making method based on the Gaussian point cloud model may include: In response to a moving selection operation of a selection cursor on the Gaussian point cloud model by controlling the operation device, a selected portion of the Gaussian point cloud model is highlighted in a second color.

[0069] It can be understood that the second color is used to highlight the point cloud selected by the user for editing in the editable Gaussian point cloud model. The Gaussian point cloud model is highlighted with the second color so that the user can directly view the selected area. The second color can be a single bright color, which needs to be different from the first color to distinguish the entire editable Gaussian point cloud model from the selected part of the Gaussian point cloud model. For example, when the entire editable Gaussian point cloud model is displayed in green, the selected part of the Gaussian point cloud model is displayed in red. The second color allows the user to clearly understand the selected part of the Gaussian point cloud model.

[0070] Please refer to Figures 6a to 6c , is a flowchart of partial Gaussian point cloud model selection provided in an embodiment of the present application.

[0071] In a feasible embodiment of the present application, Figure 6a As shown, a dedicated selection control key can be set on the control operation device. When the user touches the selection control key, the selection cursor can be moved while continuing to touch the selection control key, thereby realizing the control operation device's movement and selection operation of the selection cursor on the Gaussian point cloud model. The Gaussian image point cloud that the selection cursor moves over is selected to obtain a selected area. When the user cancels touching the selection control key, all Gaussian image point clouds that have passed over are selected as part of the Gaussian point cloud model.

[0072] In a feasible embodiment of the present application, Figure 6b As shown, a dedicated selection control key can be set on the control operation device. When the user touches the selection control key, the selection cursor can be moved while continuing to touch the selection control key, thereby realizing the control operation device's movement selection operation of the selection cursor on the Gaussian point cloud model. By moving the selection cursor across the path, a closed figure is formed on the Gaussian image point cloud, and the point cloud within the closed figure is selected as a part of the Gaussian point cloud model. When the user cancels touching the selection control key, it is determined that the point cloud within the closed figure is selected as a part of the Gaussian point cloud model.

[0073] In a feasible embodiment of the present application, if Figure 6c As shown, a dedicated selection control key can be set on the control operation device. When the user touches the selection control key, the selection cursor can be moved while continuing to touch the selection control key, thereby realizing the control operation device's movement selection operation of the selection cursor on the Gaussian point cloud model. The selection cursor selects multiple points in the Gaussian point cloud model through the control operation device, and in response to determining the point selection operation, the point cloud in the image formed by the multiple selected points is determined as a partial Gaussian point cloud model.

[0074] By determining a portion of the Gaussian point cloud model, the user can specify the area of ​​the selected Gaussian point cloud model and perform image editing operations in the specified Gaussian point cloud model area.

[0075] Please refer to Figure 7 , is a schematic diagram of a control operation device provided in an embodiment of the present application, such as Figure 7 As shown, in the embodiment of the present application, the control operation device 700 may include but is not limited to: an erase control key 710 , a palette control key 720 , and a cursor zoom control key 730 .

[0076] In an embodiment of the present application, in response to an editing operation on the Gaussian point cloud model, performing the editing operation on the Gaussian point cloud model may include: In response to the touch operation on the erase control key, the selected erase image area is displayed by controlling the operation device to move the selection cursor; In response to the touch operation on the erase key again, the erase image area is erased.

[0077] It should be noted that in a feasible embodiment of the present application, when the user touches the erase control key, the selection cursor can be moved while continuing to touch the erase control key, thereby realizing the control operation device to move the selection cursor to the Gaussian point cloud model. The Gaussian image point cloud that the selection cursor moves over is selected, and when the user cancels touching the erase control key, all Gaussian image point clouds that have passed over are selected as the erased image area.

[0078] In a feasible embodiment of the present application, when the user touches the erase control key, the selection cursor can be moved while continuing to touch the erase control key, thereby realizing the control operation device to move the selection cursor to the Gaussian point cloud model. By moving the selection cursor across the path, a closed figure is formed on the Gaussian image point cloud. When the user cancels the touch of the erase control key, it is determined that the point cloud within the closed figure is selected as the erased image area.

[0079] In a feasible embodiment of the present application, when the user touches the erase control key, the selection cursor can be moved while continuing to touch the erase control key, thereby realizing the control operation device's movement selection operation of the selection cursor on the Gaussian point cloud model. The selection cursor selects multiple points in the Gaussian point cloud model through the control operation device, and in response to determining the point selection operation, the point cloud in the image formed by the multiple selected points is determined as the erased image area.

[0080] In the embodiment of the present application, the erased image area can be generated in a partial Gaussian point cloud model, and when the user completes the selection of the erased image area, the partial Gaussian point cloud model other than the partial Gaussian point cloud model overlapping with the erased image area can be cancelled, so that the user can intuitively understand the erased image area; on the other hand, the erased image area can also be highlighted in the partial Gaussian point cloud model by a third color, so that the user can intuitively understand the erased image area. There is no specific limitation on this.

[0081] In an embodiment of the present application, part of the Gaussian point cloud model can also be directly determined as the erased image area. After the user determines the part of the Gaussian point cloud model, the user can directly touch the erase key. In response to the user's touch operation on the erase key, the part of the Gaussian point cloud model is erased.

[0082] By using the erase control key, the user can perform image deletion operations on the Gaussian point cloud model in the model making application, delete unnecessary parts of the Gaussian point cloud model, and realize flexible Gaussian point cloud model deletion.

[0083] In an embodiment of the present application, in response to an editing operation on the Gaussian point cloud model, performing the editing operation on the Gaussian point cloud model may further include: In response to a touch operation on a palette control key, a palette interface is displayed, wherein a plurality of colors are displayed in the palette interface; In response to a selection operation on a color in the color palette interface, a corresponding color modification operation is performed on the Gaussian point cloud model.

[0084] Please refer to Figure 8 , is a panel diagram of a palette interface provided in an embodiment of the present application, such as Figure 8 As shown, in the embodiment of the present application, the color palette interface displays colors. Figure 8 As shown, the color palette interface can divide the circle into multiple independent sub-areas, each area is set with the same or different colors. In one embodiment of the present application, the multiple colors in the color palette interface can be common colors set by the user, or can be colors set by the system, which is not specifically limited here.

[0085] In addition, in another embodiment of the present application, the palette interface can be a full-color interface with a gradient setting. Figure 8 The color palette interface shown is different. In this embodiment, the color palette interface is in a continuous circular shape, and all colors are set in the color palette interface in a gradient form. The user can freely select the desired color.

[0086] In one embodiment of the present application, after the color palette interface is displayed, the user selects a color in the color palette interface, and the selected portion of the Gaussian point cloud model can be previewed. Specifically, when the user moves the selection cursor to the orange area in the color palette, the selected portion of the Gaussian point cloud model is changed to orange. At this time, the color modification has not been confirmed, but is in a preview display state. At this time, if the user moves the selection cursor and leaves the orange area, the portion of the Gaussian point cloud model will be restored to its original color. It can be understood that when the user moves the selection cursor to other color areas of the color palette interface, the color of the portion of the Gaussian point cloud model will also be previewed accordingly. In response to the color selection operation in the color palette interface, the corresponding color modification operation is performed on the Gaussian point cloud model, that is, the color of the portion of the Gaussian point cloud model is changed to the corresponding color.

[0087] By previewing the color area to which the selection cursor is moved, the user can intuitively view the effect of the Gaussian point cloud model after the color modification, so that the user can select the desired color for modification.

[0088] In an embodiment of the present application, by setting a color palette button, the user can modify the color of the selected part of the Gaussian point cloud model according to needs. By displaying the color palette page, the user can select the required color in the color palette page and modify it, thereby realizing flexible color editing of the Gaussian point cloud model and meeting the user's needs for color modification.

[0089] In an embodiment of the present application, in response to an editing operation on the Gaussian point cloud model, performing the editing operation on the Gaussian point cloud model may further include: In response to a touch operation on the cursor zoom control key, a corresponding zoom operation is performed on the selection cursor.

[0090] It is understandable that when selecting a point cloud through the selection cursor, due to the characteristics of point clouds being small in size and high in number, if the selection cursor is large in size, the accuracy of point cloud selection will be reduced, but a large selection cursor can select a large range of point clouds more quickly, thus improving the efficiency of point cloud selection; on the contrary, when selecting a point cloud, if a small selection cursor is used, a high degree of selection precision will be obtained, but the efficiency of selecting a large range of point clouds will be reduced.

[0091] In the embodiment of the present application, taking into account the needs of the user when selecting a point cloud, a cursor zoom control key is provided in the control operation device used in the embodiment of the present application. Through the cursor zoom control, the user can adjust the volume size of the selection cursor.

[0092] In one embodiment of the present application, the cursor zoom control key is a knob key, and the user controls the size of the selection cursor by rotating the key. When the user needs to control the size of the selection cursor, the size of the selection cursor can be enlarged by rotating it clockwise, and the size of the selection cursor can be reduced by rotating it counterclockwise. When the user performs a touch operation of rotating the cursor zoom control key clockwise, the displayed size of the selection cursor increases in response to the touch operation of rotating the cursor zoom control key clockwise, and the user can make a large-scale point cloud selection by controlling the selection cursor; when the user performs a touch operation of rotating the cursor zoom control key counterclockwise, the displayed size of the selection cursor decreases in response to the touch operation of rotating the cursor zoom control key counterclockwise, and the user can make a more precise point cloud selection by controlling the selection cursor.

[0093] In the embodiment of the present application, by responding to the clockwise rotation of the cursor zoom control key, the volume enlargement or reduction of the selection cursor can be set according to actual conditions. By responding to the counterclockwise rotation of the cursor zoom control key, the volume enlargement or reduction of the selection cursor can also be set according to actual conditions. No specific restrictions are made here.

[0094] In one embodiment of the present application, the cursor zoom control key is a rocker button, and the user controls the size of the selection cursor by pushing the button. When the user needs to control the size of the selection cursor, the user can enlarge the size of the selection cursor by pushing up, and reduce the size of the selection cursor by pushing down; or, the user can enlarge the size of the selection cursor by pushing left, and reduce the size of the selection cursor by pushing right. There is no specific limitation on this.

[0095] In one embodiment of the present application, the cursor zoom control key is a joystick button. When the user performs a touch operation on the cursor zoom control key, the size of the displayed selection cursor increases in response to the touch operation on the cursor zoom control key, and the user can make a large-scale point cloud selection by controlling the selection cursor; when the user performs a touch operation on the cursor zoom control key, the size of the displayed selection cursor decreases in response to the touch operation on the cursor zoom control key, and the user can make a more precise point cloud selection by controlling the selection cursor.

[0096] In one embodiment of the present application, the cursor zoom control key is two independent keys, one of which corresponds to the cursor zoom-in control key, and the other corresponds to the cursor zoom-out control key. When the user performs a touch operation on the cursor zoom-in control key, in response to the touch operation on the cursor zoom-in control key, the displayed selection cursor increases in size, and the user can select a large range of point clouds by controlling the selection cursor; when the user performs a touch operation on the cursor zoom-out control key, in response to the touch operation on the cursor zoom-out control key, the displayed selection cursor decreases in size, and the user can select a more precise point cloud by controlling the selection cursor.

[0097] In the embodiment of the present application, by setting a cursor zoom control, the user can control the size of the selection cursor to achieve a large-scale and efficient point cloud selection using a large-volume selection cursor, or use the selection cursor proposed by the user to achieve high-precision point cloud selection, thereby flexibly editing 3D Gaussian point cloud graphics.

[0098] It should be noted that in the embodiment of the present application, the number of point clouds of the generated Gaussian point cloud model is huge, so in the embodiment of the present application, the points are retrieved through an octree to achieve efficient point cloud retrieval. The octree is a data structure for efficiently managing and retrieving three-dimensional point cloud data. It recursively divides the space into eight subspaces to effectively organize and query large-scale point cloud data.

[0099] In one embodiment of the present application, first, an octree is constructed to store point cloud data. A root node is created to represent the spatial range of the entire point cloud. If the number of points in a node exceeds a preset threshold or reaches a certain depth, it is divided into eight child nodes; then each point is inserted into the corresponding octree node. Once the octree is constructed, it can be used to perform various types of point cloud retrieval operations, such as: Intra-voxel nearest neighbor search: Find all points that lie within the same voxel as a given point.

[0100] K nearest neighbor search: Find the K points closest to a given point.

[0101] Nearest neighbor search within radius: Find all points whose distance to a given point is less than a given radius.

[0102] By constructing an octree retrieval, a fast point cloud selection response can be achieved in the embodiment of the present application to improve the user experience when editing the point cloud and improve the efficiency of 3D model editing.

[0103] Reference Fig. 9 As shown, the embodiment of the present application further discloses an electronic device, the electronic device 1100 includes: at least one processor 1101; At least one memory 1102, used to store at least one program; When at least one program is executed by at least one processor 1101, the above-mentioned 3D model making and processing method based on Gaussian point cloud model is implemented.

[0104] The embodiment of the present application also discloses a computer-readable storage medium, which stores a computer program executable by a processor. When the computer program executable by the processor is executed by the processor, it is used to implement the 3D model production and processing method based on the Gaussian point cloud model as mentioned above.

[0105] An embodiment of the present application also discloses a computer program product, including a computer program or computer instructions, wherein the computer program or computer instructions are stored in a computer-readable storage medium, and a processor of an electronic device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the electronic device executes the 3D model production and processing method based on the Gaussian point cloud model as described above.

[0106] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0107] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0108] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0109] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0112] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0113] The step numbers in the above method embodiment are only provided for the convenience of explanation and description, and no limitation is imposed on the order of the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

Claims

1. A 3D model making method based on a Gaussian point cloud model, applied to a virtual reality device, wherein the virtual reality device includes a control operation device, characterized in that: The method comprises: In response to a touch operation on the model making application, displaying a model making interface, the model making interface displaying a virtual image and a selection cursor; In response to a moving selection operation of the selection cursor by the control operation device, the selection cursor is moved in the virtual image to select an area to display a target image area, and an editable Gaussian point cloud model is displayed in the target image area, wherein the editable Gaussian point cloud model is generated by a preset image; In response to the editing operation on the Gaussian point cloud model, the editing operation is performed on the Gaussian point cloud model to obtain a target point cloud model.

2. The method according to claim 1, characterized in that Displaying an editable Gaussian point cloud model in the target image area includes: Generate a corresponding image mask according to the target image area; Calling the preset model generation model to generate an editable Gaussian point cloud model according to the image mask and the image in the target image area; The Gaussian point cloud model is highlighted in a first color in the target image area.

3. The method according to claim 2, characterized in that The preset model generation model includes restriction parameters, and the restriction parameters are used to optimize the Gaussian point cloud model.

4. The method according to claim 2, characterized in that The step of generating an editable Gaussian point cloud model according to the image mask and the image in the target image area comprises: Acquire a plurality of images of different viewing angles in the target image area; An editable Gaussian point cloud model is generated according to the multiple images of different viewing angles and the image mask.

5. The method according to claim 1, characterized in that: Before the step of performing the editing operation on the Gaussian point cloud model in response to the editing operation on the Gaussian point cloud model to obtain the target point cloud model, the method further comprises: In response to the moving and selecting operation of the selection cursor on the Gaussian point cloud model by the control operation device, the selected portion of the Gaussian point cloud model is highlighted in a second color.

6. The method according to claim 1, characterized in that The control operation device includes an erase control key; and in response to the editing operation on the Gaussian point cloud model, performing the editing operation on the Gaussian point cloud model includes: In response to a touch operation on the erase control key, the selected erase image area is displayed by moving the selection cursor through the control operation device; In response to the touch operation on the erase key again, the erase image area is erased.

7. The method according to claim 1, characterized in that The control operation device includes a palette control key; the editing operation is performed on the Gaussian point cloud image in response to the editing operation on the image, including: In response to a touch operation on the palette control key, displaying a palette interface, wherein a plurality of colors are displayed in the palette interface; In response to a selection operation on a color in the color palette interface, a corresponding color modification operation is performed on the Gaussian point cloud image.

8. The method according to claim 1, characterized in that: The control operation device includes a cursor zoom control key; the method also includes: In response to a touch operation on the cursor zoom control key, a corresponding zoom operation is performed on the selection cursor.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the 3D model making method based on the Gaussian point cloud model as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program executable by a processor is stored therein, and when the computer program executable by the processor is executed by the processor, it is used to implement the 3D model production method based on the Gaussian point cloud model as described in any one of claims 1 to 8.

Images