Three-dimensional model view generation method and device, storage medium and electronic equipment

Abstract

The invention provides a three-dimensional model view generation method and device, a storage medium and electronic equipment, and is applied to the technical field of computer vision. The method comprises the steps of obtaining a target three-dimensional model; adjusting the orientation of the target three-dimensional model by using a pre-trained orientation discrimination model to obtain the target three-dimensional model with the adjusted orientation; based on the target three-dimensional model with the adjusted orientation, multi-view rendering is conducted on the target three-dimensional model according to preset parameters, a view rendering result is obtained, and the view rendering result comprises multiple views of the target three-dimensional model. According to the method, orientation adjustment is carried out on the target three-dimensional model through an automation technology, and preset parameter rendering is carried out based on the orientation adjustment, so that multiple views are quickly and efficiently generated, large-scale three-dimensional model processing is supported, the generation speed is remarkably improved, the rendering time is shortened, and the requirement for digitization of a large number of cultural relics is met.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a three-dimensional model view generation method, device, storage medium and electronic equipment. Background Art

[0002] With the rapid development of 3D digitalization of cultural relics, the trend of using 3D scanning and modeling technology to digitize cultural relics has become increasingly obvious around the world. Technologies such as laser scanning and photogrammetry are widely used in high-precision modeling of cultural relics. Many international museums have established virtual museums that use 3D models to display their collections, provide detailed background information and interactive functions.

[0003] However, existing technologies still face challenges such as slow loading of three-dimensional models, low efficiency in generating multi-angle views, and inadequate response to diverse demands, which limit the further application and development of cultural relics digitization.

[0004] Therefore, how to efficiently generate views of a three-dimensional model has become a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of the above problems, the present invention provides a method, device, storage medium and electronic device for generating a 3D model view, which overcomes the above problems or at least partially solves the above problems. The technical solution is as follows:

[0006] A method for generating a three-dimensional model view, comprising:

[0007] Obtaining a target three-dimensional model;

[0008] Using a pre-trained orientation discrimination model to adjust the orientation of the target three-dimensional model, to obtain the target three-dimensional model with the adjusted orientation;

[0009] Based on the target three-dimensional model with the adjusted orientation, multi-view rendering is performed on the target three-dimensional model according to preset parameters to obtain a view rendering result, wherein the view rendering result includes multiple views of the target three-dimensional model.

[0010] Optionally, the using a pre-trained orientation discrimination model to adjust the orientation of the target three-dimensional model to obtain the target three-dimensional model with the adjusted orientation includes:

[0011] The target three-dimensional model is input into the orientation discrimination model so that the orientation discrimination model extracts the first geometric feature and the first functional semantic feature of the target three-dimensional model, and performs orientation prediction based on the first geometric feature and the first functional semantic feature to obtain the output first predicted rotation parameter, and the target three-dimensional model is rotated using the first predicted rotation parameter to obtain the target three-dimensional model with adjusted orientation.

[0012] Optionally, the training process of the orientation discrimination model includes:

[0013] Obtaining a three-dimensional model training set, wherein the three-dimensional model training set includes a plurality of standardized three-dimensional training models whose orientations have been adjusted;

[0014] Inputting each of the three-dimensional training models in the three-dimensional model training set into the orientation discrimination model in sequence for training, extracting a second geometric feature and a second functional semantic feature of the three-dimensional training model, and then rotating the three-dimensional training model according to a random rotation parameter to extract a third geometric feature and a third functional semantic feature of the rotated three-dimensional training model;

[0015] Using the second geometric feature and the third geometric feature, obtaining a geometric consistency loss;

[0016] Using the second functional semantic feature and the third functional semantic feature, obtaining a functional semantic constraint loss;

[0017] Based on the third geometric feature and the third functional semantic feature, obtain a second predicted rotation parameter, and use the second predicted rotation parameter and the random rotation parameter to obtain an orientation prediction loss;

[0018] Using the geometric consistency loss, the functional semantic constraint loss and the orientation prediction loss, a comprehensive loss is obtained;

[0019] Based on the comprehensive loss, the network parameters of the orientation discrimination model are adjusted, and retraining is performed until a preset training end condition is reached to obtain the trained orientation discrimination model.

[0020] Optionally, the standardization process includes:

[0021] Obtaining a three-dimensional training model;

[0022] The geometric center of the three-dimensional training model is moved to the target origin, and the three-dimensional training model is adjusted to a preset specification size to obtain the standardized three-dimensional training model.

[0023] Optionally, the geometric features include: local differential geometric features, mesoscale structural features and global structural features.

[0024] Optionally, after performing multi-view rendering on the target three-dimensional model based on the target three-dimensional model with the adjusted orientation according to preset rendering parameters to obtain the view rendering result, the method further includes:

[0025] A cover image selection model is used to select a view from the view rendering results as a cover image of the target three-dimensional model.

[0026] Optionally, the preset parameters include: light source parameters, camera parameters and rendering image parameters.

[0027] A three-dimensional model view generation device comprises: a three-dimensional model obtaining unit, a direction adjusting unit and a view rendering result obtaining unit.

[0028] The three-dimensional model obtaining unit is used to obtain a target three-dimensional model;

[0029] The orientation adjustment unit is used to adjust the orientation of the target three-dimensional model using a pre-trained orientation discrimination model to obtain the target three-dimensional model with the adjusted orientation;

[0030] The view rendering result obtaining unit is used to perform multi-view rendering on the target three-dimensional model according to preset parameters based on the target three-dimensional model with the adjusted orientation, so as to obtain a view rendering result, wherein the view rendering result includes multiple views of the target three-dimensional model.

[0031] A computer-readable storage medium stores a program, which implements the three-dimensional model view generation method when executed by a processor.

[0032] An electronic device comprises at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other via the bus; and the processor is used to call program instructions in the memory to execute the three-dimensional model view generation method.

[0033] By means of the above technical scheme, the present invention provides a method, device, storage medium and electronic device for generating a 3D model view, which obtains a target 3D model; uses a pre-trained orientation discrimination model to adjust the orientation of the target 3D model to obtain a target 3D model with an adjusted orientation; based on the target 3D model with an adjusted orientation, performs multi-view rendering of the target 3D model according to preset parameters to obtain a view rendering result, wherein the view rendering result includes multiple views of the target 3D model. The present invention uses automation technology to adjust the orientation of the target 3D model and performs preset parameter rendering based on the orientation, thereby quickly and efficiently generating multiple views, thereby supporting the processing of large-scale 3D models, significantly improving the generation speed and shortening the rendering time, so as to meet the needs of digitizing a large number of cultural relics.

[0034] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0036] Figure 1 A schematic diagram showing a flow chart of an implementation of a method for generating a 3D model view provided in an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of the model structure of the orientation discrimination model provided by an embodiment of the present invention is shown;

[0038] Figure 3 A schematic diagram showing a flow chart of a training process of a direction discrimination model provided by an embodiment of the present invention;

[0039] Figure 4 A logic block diagram of a self-supervised training process provided by an embodiment of the present invention is shown;

[0040] Figure 5 A logic block diagram of a method for generating a three-dimensional model view provided by an embodiment of the present invention is shown;

[0041] Figure 6 A schematic diagram showing the structure of a three-dimensional model view generating device provided by an embodiment of the present invention is shown;

[0042] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0043] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0044] With the continuous development of the three-dimensional digitalization trend of cultural relics and landscapes, the application of three-dimensional models is increasing. However, three-dimensional models are usually larger than static images, which makes their loading and rendering speed significantly slower than images. This problem is particularly prominent when users search or browse three-dimensional models in information flows. In order to allow users to preview three-dimensional models more intuitively, it is usually necessary to generate cover images and multiple angle views based on three-dimensional data, generally including six different angles: up, down, left, right, and front and back. At present, these covers and six views are mostly generated manually, and the operation process is cumbersome. It is usually necessary to open the three-dimensional model, adjust the cultural relics to the corresponding direction, and render and output, and the overall efficiency is low. The number of manual operations for each cultural relic is as many as seven times, and when the background color needs to be changed, a lot of manpower is also required for repeated operations.

[0045] Globally, the trend of digitization of cultural relics is becoming more and more obvious, and the application of 3D model technology in the protection, display and research of cultural relics is becoming more and more extensive. Foreign countries are in the leading position in the field of 3D scanning and modeling technology, using technologies such as laser scanning, structured light scanning and photogrammetry to carry out high-precision digital modeling of cultural relics. For example, the British Museum and the ruins of the ancient city of Pompeii in Italy both use advanced 3D scanning technology to record and study cultural relics and sites in detail. At the same time, many internationally renowned museums, such as the Louvre and the Metropolitan Museum of Art in New York, have created virtual museums to display their collections through 3D models, allowing global audiences to visit them virtually through the Internet. These platforms not only show the appearance of cultural relics, but also provide detailed background information and interactive functions.

[0046] In China, the Palace Museum is at the forefront of cultural relics digitization. Through the "Digital Duobao Pavilion" project, it has used digitalization, cloud computing and artificial intelligence technologies to create high-precision three-dimensional cultural relic models and display hundreds of digital cultural relics. In addition, the construction of the Longmen Grottoes digital twin platform has also provided tourists with a new digital tourism experience. Domestic cultural and museum institutions have also made significant progress in digital collection and online exhibitions.

[0047] Although existing technologies have made remarkable achievements in 3D scanning and modeling, they still face some challenges in the application and display of 3D models. First, the loading and rendering speed of 3D models is slow, which directly affects the user experience. Second, the generation of covers and multi-angle views still relies on manual operations, which is inefficient and costly. In addition, existing technologies are insufficient in flexibility to cope with diverse needs, and are difficult to adjust and process in batches quickly, which to some extent limits the further development and application of cultural relics digitization.

[0048] Based on this, in an embodiment of the present invention, a method for generating a three-dimensional model view is provided. First, a target three-dimensional model is obtained, and its orientation is adjusted using a pre-trained orientation discrimination model, so as to obtain an adjusted target three-dimensional model. Secondly, based on the adjusted target three-dimensional model, multi-view rendering is performed according to preset parameters to generate a rendering result containing multiple views. The present invention uses automation technology to achieve orientation adjustment of the target three-dimensional model and rendering based on preset parameters, and quickly and efficiently generates multiple views. This process supports the processing of large-scale three-dimensional models, significantly improves the view generation speed and shortens the rendering time, and effectively meets the needs of digitizing a large number of cultural relics.

[0049] like Figure 1 As shown, a flow chart of an implementation of a method for generating a 3D model view provided by an embodiment of the present invention is provided, and the method may include:

[0050] S100: Obtain a target three-dimensional model.

[0051] Among them, a three-dimensional model (3D Model) refers to a three-dimensional computer model created for a specific object (such as cultural relics, buildings, natural landscapes, etc.) during the three-dimensional digitization process. The three-dimensional model captures the shape, structure and details of the object through digital technology so that it can be displayed, analyzed and interacted in a computer environment.

[0052] Specifically, in the process of acquiring the target three-dimensional model, the embodiment of the present invention can use a three-dimensional model processing tool (such as Blender) to normalize the model. This process includes moving the geometric center of the model to the origin and standardizing the length, width, and height of the model. It is worth noting that this step is not limited to a specific three-dimensional model editing tool, and users can also choose other open source libraries and tools. Through this standardized processing, consistent and standardized input data can be provided for the subsequent orientation judgment model, thereby improving the accuracy and efficiency of the processing.

[0053] S110, using a pre-trained orientation discrimination model to adjust the orientation of the target three-dimensional model to obtain a target three-dimensional model with an adjusted orientation.

[0054] The orientation discrimination model is a pre-trained machine learning model used to determine the direction or orientation of a 3D model. The orientation discrimination model can identify the correct orientation of the 3D model by analyzing the geometric features of the 3D model, thereby providing guidance for subsequent processing.

[0055] Specifically, the embodiment of the present invention can judge the orientation of the input target 3D model based on the orientation discrimination model, and then rotate or flip the target 3D model accordingly based on the judgment result to achieve the correct orientation. Through orientation adjustment, the user can obtain a target 3D model with a consistent orientation and in line with the usage scenario, thereby improving the effect and accuracy of the 3D model in display, analysis or other applications.

[0056] Furthermore, the embodiments of the present invention can make full use of the geometric structure (mesh) and functional semantic information (including texture information and shape information) of the three-dimensional model through the orientation discrimination model, and combine the two to realize the judgment of the orientation of the three-dimensional model.

[0057] S120 . Based on the target three-dimensional model with the adjusted orientation, perform multi-view rendering on the target three-dimensional model according to preset parameters to obtain a view rendering result, wherein the view rendering result includes multiple views of the target three-dimensional model.

[0058] Preset parameters refer to a set of parameters set in advance when performing 3D rendering, graphics processing or other computer vision-related tasks on 3D models. These preset parameters are used to define the rendering environment and conditions to ensure the consistency and controllability of the results. By providing preset parameters, users can achieve the expected visual effects during the rendering process and ensure the uniformity and repeatability of the rendering results.

[0059] Optionally, the preset parameters may include: light source parameters, camera parameters and rendering image parameters.

[0060] Among them, light source parameters refer to various parameters about light source settings during the 3D rendering process. Light source parameters affect the lighting effect and visual performance of the 3D model in the scene. Light source parameters can include light source type, light source position, light source direction, light source color, light intensity and attenuation rate.

[0061] Camera parameters refer to various parameters set for the virtual camera during the rendering process. Camera parameters determine the viewing angle and visual effect of the 3D scene. Camera parameters can include camera position, camera orientation, field of view (FOV), focal length, and depth of field.

[0062] Rendering picture parameters refer to the settings when generating the final view of the 3D model. Rendering picture parameters directly affect the output quality and format of the view. Rendering picture parameters can include image resolution, image format, background color, size, compression rate, and post-processing effects.

[0063] Among them, multi-view rendering refers to the process of rendering the same 3D model from multiple different angles and positions. The embodiment of the present invention can generate images of the 3D model from multiple perspectives by adjusting the position and orientation of the virtual camera. Different views show different sides of the 3D model, helping users to more comprehensively understand the shape, details and structure of the 3D model.

[0064] The view rendering result is a set of images generated after multi-view rendering. Each view in the view rendering result is the result of rendering the target 3D model from a specific angle and position. These views together show the appearance of the 3D model from different perspectives and can be used for analysis, display or further processing. The view rendering result can include multiple views of the target 3D model, including the front view, back view, left view, right view, top view, bottom view and the view at a 45-degree angle in front.

[0065] Specifically, the embodiment of the present invention can import the target three-dimensional model with adjusted orientation into the specified rendering software, create a corresponding scene in the rendering software, set the light source and camera perspective according to preset parameters, and then start the rendering process, render the configured camera perspectives one by one, save the corresponding image file after completion, and obtain the view rendering result.

[0066] The present invention provides a method for generating a 3D model view, the method comprising: obtaining a target 3D model; adjusting the orientation of the target 3D model using a pre-trained orientation discrimination model to obtain a target 3D model with an adjusted orientation; performing multi-view rendering of the target 3D model according to preset parameters based on the target 3D model with the adjusted orientation to obtain a view rendering result, wherein the view rendering result includes multiple views of the target 3D model. The present invention uses automation technology to adjust the orientation of the target 3D model and performs preset parameter rendering based on the orientation, thereby quickly and efficiently generating multiple views, thereby supporting the processing of large-scale 3D models, significantly improving the generation speed and shortening the rendering time, so as to meet the needs of digitizing a large number of cultural relics.

[0067] Optional, in the above Figure 1 On the basis of one or more corresponding embodiments, in another optional embodiment provided by the embodiment of the present invention, using a pre-trained orientation discrimination model to adjust the orientation of the target three-dimensional model to obtain the target three-dimensional model with the adjusted orientation may specifically include:

[0068] The target three-dimensional model is input into the orientation discrimination model so that the orientation discrimination model extracts the first geometric feature and the first functional semantic feature of the target three-dimensional model, and performs orientation prediction based on the first geometric feature and the first functional semantic feature to obtain the output first predicted rotation parameter, and the target three-dimensional model is rotated using the first predicted rotation parameter to obtain the target three-dimensional model with adjusted orientation.

[0069] Geometric features refer to the attributes that describe the shape, structure, and spatial relationship of a 3D model. Geometric features usually include information such as the boundaries, angles, surface curvature, volume, and area of ​​a 3D model.

[0070] Among them, functional semantic features refer to attributes related to the function, purpose and meaning of the 3D model. Functional semantic features usually involve the role and usage of the object corresponding to the 3D model in a specific context.

[0071] The rotation parameter is a numerical value used to describe the rotation of the 3D model in the 3D space. When adjusting the orientation, the rotation parameter usually indicates information such as the axis and angle of rotation, so as to correctly orient the 3D model in the desired direction.

[0072] The model structure of the orientation discrimination model provided by the embodiment of the present invention can be as follows: Figure 2 As shown in the figure, after the three-dimensional model is input into the orientation discrimination model, it is subjected to feature extraction by the functional semantic feature module and the multi-scale geometric feature module respectively, and then processed by the encoder and decoder to output the geometric features and functional semantic features of the three-dimensional model, and then the orientation is predicted based on the geometric features and functional semantic features.

[0073] Optionally, the geometric features provided by the embodiments of the present invention may include local differential geometric features, mesoscale structural features and global structural features.

[0074] First, the embodiment of the present invention can extract the local differential geometric features of the three-dimensional model. For example, the principal curvature, mean curvature, Gaussian curvature and normal vector of each point on the three-dimensional model, that is:

[0075]

[0076] in, is a point on the three-dimensional model; For point The local differential geometric characteristics of , For point The principal curvature of For point The mean curvature of ; For point The Gaussian curvature of ; For point The normal vector of .

[0077] Then, the local differential geometric features are connected to obtain the mesoscale structural features, namely:

[0078]

[0079] in, An object composed of multiple points on a three-dimensional model; For object The mesoscale structural characteristics of Represents a graph join operation; For object Points within range; Indicates the connection relationship between components and is used to describe objects The connection relationship between each local element is usually reflected through a topological graph and an adjacency matrix.

[0080] Then, the mesoscale structural features are symmetric and pooled to obtain the global structural features, namely:

[0081]

[0082] in, Represents the global structural features of the 3D model; Represents a pooling operation; Represents a symmetric operation.

[0083] Finally, local differential geometric features, mesoscale structural features and global structural features are used as geometric features of the three-dimensional model, and combined with the functional semantic features of the three-dimensional model for orientation prediction.

[0084] Optionally, in the process of extracting the functional semantic features of the three-dimensional model, the embodiment of the present invention may first divide the three-dimensional model into components, and then perform functional classification on each functional component on this basis, that is:

[0085]

[0086] in, Represents a point on the surface of a given 3D model In the case of Probability of belonging to the component class ; About point The function is used to extract the function that reflects this point Characteristic information related to the area and functional components; is the activation function.

[0087] Then, based on each functional component, a functional relationship diagram is constructed, namely:

[0088]

[0089] in, Represents a functional relationship diagram; Represents a set of vertices in a functional relationship graph, where each vertex corresponds to a functional component; Represents the edge set in the functional relationship graph. Edges are used to connect different vertices and reflect the relationship between functional components.

[0090] Among them, the edge weight representing the functional dependency strength in the functional relationship graph is:

[0091]

[0092] in, Represents the weight of the edge connecting vertices in the functional relationship graph, that is, the edge connecting the first Functional components and The weights of the edges between functional components; For the Feature information related to each functional component; For the Characteristic information related to each functional component; Represents the relationship between functional components.

[0093] Finally, the functional semantic features of the 3D model are combined with the geometric features of the 3D model to predict the orientation.

[0094] Optionally, the embodiment of the present invention can combine the geometric features and functional semantic features of the three-dimensional model into a comprehensive model feature, and then perform orientation prediction based on the comprehensive model feature, that is:

[0095]

[0096] in, Represents the comprehensive characteristics of the model; Represent geometric features; Represents functional semantic features.

[0097] The embodiment of the present invention can extract the first geometric feature and the first functional semantic feature of the target three-dimensional model by inputting the target three-dimensional model into the orientation discrimination model, and perform orientation prediction based on these features to obtain the first predicted rotation parameter. The first predicted rotation parameter is used to rotate the target three-dimensional model, which can effectively adjust the orientation of the model so that it is more consistent and accurate in visual display, thereby helping to improve the efficiency of generating views of the three-dimensional model.

[0098] Optional, such as Figure 3 As shown, a flow chart of a training process of a direction discriminant model provided by an embodiment of the present invention is provided. The training process of the direction discriminant model may include:

[0099] S300: Obtain a three-dimensional model training set, wherein the three-dimensional model training set includes a plurality of standardized three-dimensional training models whose orientations have been adjusted.

[0100] Specifically, the embodiment of the present invention can obtain a three-dimensional training model based on a certain number of existing three-dimensional models with adjusted orientations after standardizing these three-dimensional models. Use these three-dimensional training models to train the orientation discrimination model. After the training is completed, the orientation discrimination model can receive any three-dimensional model data and calculate the required rotation angle to adjust the three-dimensional model to the correct orientation. After adjustment, manual orientation confirmation can be performed, and the confirmed sample data can be included in the three-dimensional model training set, thereby gradually enriching and enhancing the training data of the orientation discrimination model. After multiple iterations, when the accuracy of the orientation discrimination model reaches the predetermined standard, the manual confirmation step can be cancelled to achieve a fully automated orientation adjustment process.

[0101] S310. Input each three-dimensional training model in the three-dimensional model training set into the orientation discrimination model for training in turn, extract the second geometric features and the second functional semantic features of the three-dimensional training model, and then rotate the three-dimensional training model according to the random rotation parameters to extract the third geometric features and the third functional semantic features of the rotated three-dimensional training model.

[0102] S320: Obtain geometric consistency loss using the second geometric feature and the third geometric feature.

[0103] The geometric consistency loss is used to evaluate the consistency of the geometric features of the three-dimensional model in different orientations, that is, the difference between the geometric features of the three-dimensional model before and after rotation. In the embodiment of the present invention, the difference between the second geometric feature and the third geometric feature before and after rotation of the three-dimensional model can be compared to calculate the geometric consistency loss. For example, the geometric consistency loss can be the mean square error or cosine similarity between the second geometric feature and the third geometric feature.

[0104] S330. Obtain functional semantic constraint loss using the second functional semantic feature and the third functional semantic feature.

[0105] Among them, the functional semantic constraint loss is used to evaluate the consistency of the functional semantic features of the three-dimensional model under different orientations, that is, whether the function and meaning of the three-dimensional model remain stable under different viewing angles. In an embodiment of the present invention, the functional semantic constraint loss can be calculated by comparing the difference between the second functional semantic feature and the third functional semantic feature before and after rotation. For example, the functional semantic constraint loss can be the cross entropy loss between the second functional semantic feature and the third functional semantic feature.

[0106] S340: Based on the third geometric feature and the third functional semantic feature, obtain a second predicted rotation parameter, and use the second predicted rotation parameter and the random rotation parameter to obtain an orientation prediction loss.

[0107] The orientation prediction loss is used to evaluate the difference between the rotation parameters predicted by the model and the actual random rotation parameters to reflect the accuracy of the orientation discrimination model in determining the orientation of the object. The embodiment of the present invention can calculate the orientation prediction rotation parameter by comparing the difference between the second predicted rotation parameter and the random rotation parameter.

[0108] S350, using geometric consistency loss, functional semantic constraint loss and orientation prediction loss to obtain a comprehensive loss.

[0109] Among them, the comprehensive loss is a total loss value formed by combining the geometric consistency loss, the functional semantic constraint loss and the orientation prediction loss. The present invention can provide a comprehensive optimization goal by comprehensively considering the consistency of geometric and semantic features and the accuracy of orientation prediction, so as to better guide the parameter update of the model during the training process. For example: the embodiment of the present invention can adopt a weighted average method, that is, multiplying each loss by the corresponding weight and adding them together to form a comprehensive loss.

[0110] S360. Based on the comprehensive loss, adjust the network parameters of the orientation discrimination model, retrain, and reach a preset training end condition to obtain a trained orientation discrimination model.

[0111] The embodiment of the present invention can adjust the network parameters of the orientation discrimination model based on the calculated comprehensive loss. Through the back propagation algorithm, the orientation discrimination model will continuously update the parameters according to the feedback information of the loss. The training process will continue until the preset training end condition is met (for example, a certain loss value or number of training rounds is reached), thereby obtaining a trained orientation discrimination model.

[0112] Optionally, the preset training end condition may be that the comprehensive loss reaches within a preset loss threshold or the number of training rounds reaches a preset round number threshold.

[0113] The training process of the orientation discrimination model provided by the embodiment of the present invention is a self-supervised training process. Figure 4The figure shows a logic block diagram of the self-supervised training process provided by an embodiment of the present invention. In the design of the loss function, the orientation prediction loss, geometric consistency loss and functional semantic constraint loss are calculated separately and combined. The final loss function can be in the form of:

[0114]

[0115] in, is comprehensive loss; Towards predicted losses; is the geometric consistency loss; is the functional semantic constraint loss; and It is a hyperparameter and can be set according to actual needs.

[0116] During the self-supervised training process, a random rotation transformation is applied to each 3D model. The rotated model will generate new comprehensive multi-scale geometric features and functional semantic features, and calculate the corresponding loss based on these features. At the same time, a fully connected multilayer perceptron (MLP) module is used to input the rotated model features and output the predicted rotation parameters. .

[0117] The orientation prediction loss is also called the rotation recovery loss. In the self-supervised training process, the rotation recovery loss is designed as a self-supervised loss to make the rotation parameters predicted by the orientation discriminant model It can be restored to a state that matches the original unrotated 3D model. The goals are:

[0118]

[0119] The design of the prediction loss function is:

[0120]

[0121] in, is the norm calculation; is the identity matrix.

[0122] In order to ensure that the features of the 3D model after the predicted rotation correction are consistent with the features of the original model (i.e., the multi-scale geometric features and functional semantic features of the 3D model before and after the rotation correction are calculated respectively), the designed feature consistency loss function is:

[0123]

[0124] in, is the feature consistency loss; It is the feature extraction network; is the rotated three-dimensional model feature; The original model features.

[0125] The embodiment of the present invention can effectively obtain the geometric and semantic information of the three-dimensional model in different orientations by training a standardized and oriented three-dimensional model training set and extracting the second geometric features and the second functional semantic features using the orientation discrimination model. Further, through random rotation processing, the rotated third geometric features and the third functional semantic features are extracted, so as to calculate the geometric consistency loss and the functional semantic constraint loss, which provides a solid foundation for generating efficient three-dimensional model views. In addition, the model parameters are adjusted through comprehensive loss, so that the trained orientation discrimination model has stronger robustness and generalization ability, so that the orientation adjustment can be performed more accurately in practical applications.

[0126] Optional, in the above Figure 3 On the basis of one or more corresponding embodiments, in another optional embodiment provided by the embodiment of the present invention, the standardization process includes:

[0127] Obtain a three-dimensional training model; move the geometric center of the three-dimensional training model to the target origin, and adjust the three-dimensional training model to a preset specification size to obtain a standardized three-dimensional training model.

[0128] The embodiment of the present invention ensures that the position of the model in space is uniform by calculating the geometric center of the three-dimensional model and moving it to the origin of the coordinate system. This symmetry helps the stability of the three-dimensional model in subsequent processing, especially when analyzing different perspectives or postures, and helps to eliminate the impact of position deviation.

[0129] The embodiment of the present invention adjusts the 3D training model to a preset size, so that different models can be compared and analyzed at the same scale. This unified scale standardization makes the subsequent training and reasoning process more efficient and reduces errors caused by different model sizes.

[0130] The embodiments of the present invention ensure that multiple three-dimensional models have a consistent reference frame in the processing flow by standardizing the position and size. The standardized three-dimensional model simplifies the model data input, reduces the complexity of model preprocessing, and makes the training process more efficient. In addition, the unified positioning and size adjustment of the three-dimensional model helps the machine learning algorithm converge faster, enhancing the learning ability and prediction accuracy towards the discriminant model.

[0131] Optional, in the above Figure 1On the basis of one or more corresponding embodiments, in another optional embodiment provided by an embodiment of the present invention, after performing multi-view rendering on the target three-dimensional model according to preset rendering parameters based on the target three-dimensional model with the adjusted orientation and obtaining the view rendering result, the method may further include:

[0132] Use the cover image to select a model and select a view in the view rendering results as the cover image of the target 3D model.

[0133] Among them, the cover image selection model can be a pre-trained visual aesthetic discrimination model, which judges the aesthetic quality of the view by analyzing the visual features of the view.

[0134] Specifically, the embodiment of the present invention can extract the visual features of each view in the view rendering result through the cover image selection model, such as color, composition, texture, contrast, light and shadow, etc. Then, an aesthetic score is performed based on these visual features, and the view with the highest aesthetic score in the view rendering result is selected as the cover image of the target 3D model.

[0135] The training of cover image selection models usually relies on annotated image datasets that contain images rated by humans to identify their aesthetic quality. The cover image selection model uses machine learning or deep learning algorithms (such as support vector machines, decision trees, and deep neural networks) to train the extracted visual features and establish the relationship between the input visual features and the aesthetic score. After training, the cover image selection model is able to evaluate the views of the 3D model and output an aesthetic score to judge the aesthetic quality of the view.

[0136] The embodiment of the present invention significantly reduces the time for manual screening and evaluation by automating the process of selecting a cover image, thereby supporting the processing of large-scale 3D models, reducing the time from view rendering to cover image selection of the 3D model, and improving the ability to quickly display the 3D model. In addition, the embodiment of the present invention can select a cover image with aesthetic value to attract the user's attention to the 3D model, improve the visual appeal of the 3D model when it is displayed, and thus obtain a better display effect.

[0137] In order to facilitate understanding of the overall process of the 3D model view generation method provided by the embodiment of the present invention, Figure 5 To explain: Figure 5The figure shows a logic block diagram of a 3D model view generation method provided by an embodiment of the present invention. The embodiment of the present invention realizes the automatic generation of the cover image and six views of the 3D model by designing an automated process and using an intelligent algorithm, thereby significantly reducing the need for manual operation. The operation process includes: firstly, standardizing the geometric center and size of the loaded 3D model; then, adjusting the orientation using a trained orientation discrimination model; then, applying preset lighting and camera parameters to complete view rendering; finally, using the cover image selection model to determine the final cover image.

[0138] The orientation discrimination model is a deep learning model built based on the frontal discrimination algorithm. In the early stage, it can be trained by manually confirming sample data, and then the three-dimensional training model of the frontal orientation can be fully automatically supplemented to further optimize the performance of the model. Based on this, the embodiment of the present invention can automatically adjust the angle and direction of the model, quickly generate the required view, support large-scale three-dimensional model processing, thereby improving the generation speed, shortening the rendering time, and meeting the needs of digitizing a large number of cultural relics.

[0139] In addition, the embodiment of the present invention supports flexible configuration options to meet diverse product requirements, including different background colors, resolutions, and formats. Users can easily adjust parameters through a friendly system interface and apply them to multiple models in batches to avoid duplication of work. The introduction of automated processes greatly reduces human errors and ensures the standardization of output results. At the same time, the orientation discrimination model provided by the embodiment of the present invention also has good scalability, can adapt to future technological developments and new requirements, and supports continuous optimization and upgrading.

[0140] Although operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or in a sequential order.Multitasking and parallel processing may be advantageous under certain circumstances.

[0141] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0142] Corresponding to the above method embodiment, the embodiment of the present invention also provides a three-dimensional model view generation device, whose structure is as follows: Figure 6 As shown, it may include: a three-dimensional model obtaining unit 10, a direction adjusting unit 20 and a view rendering result obtaining unit 30.

[0143] A three-dimensional model obtaining unit 10, used to obtain a target three-dimensional model;

[0144] The orientation adjustment unit 20 is used to adjust the orientation of the target three-dimensional model by using a pre-trained orientation discrimination model to obtain the target three-dimensional model with the adjusted orientation;

[0145] The view rendering result obtaining unit 30 is used to perform multi-view rendering on the target three-dimensional model according to preset parameters based on the target three-dimensional model with the adjusted orientation, and obtain a view rendering result, wherein the view rendering result includes multiple views of the target three-dimensional model.

[0146] Optionally, the orientation adjustment unit 20 can be specifically used to input the target three-dimensional model into the orientation discrimination model, so that the orientation discrimination model extracts the first geometric feature and the first functional semantic feature of the target three-dimensional model, and performs orientation prediction based on the first geometric feature and the first functional semantic feature, obtains the output first predicted rotation parameter, and uses the first predicted rotation parameter to rotate the target three-dimensional model to obtain the target three-dimensional model with adjusted orientation.

[0147] Optionally, the 3D model view generating device may further include: a model training unit.

[0148] A model training unit is used to obtain a three-dimensional model training set, wherein the three-dimensional model training set includes a plurality of standardized three-dimensional training models whose orientations have been adjusted; each three-dimensional training model in the three-dimensional model training set is sequentially input into an orientation discrimination model for training, a second geometric feature and a second functional semantic feature of the three-dimensional training model are extracted, and then the three-dimensional training model is rotated according to a random rotation parameter, and a third geometric feature and a third functional semantic feature of the rotated three-dimensional training model are extracted; a geometric consistency loss is obtained using the second geometric feature and the third geometric feature; a functional semantic constraint loss is obtained using the second functional semantic feature and the third functional semantic feature; a second predicted rotation parameter is obtained based on the third geometric feature and the third functional semantic feature, and an orientation prediction loss is obtained using the second predicted rotation parameter and the random rotation parameter; a comprehensive loss is obtained using the geometric consistency loss, the functional semantic constraint loss and the orientation prediction loss; based on the comprehensive loss, the network parameters of the orientation discrimination model are adjusted, and training is performed again until a preset training end condition is reached, thereby obtaining a trained orientation discrimination model.

[0149] Optionally, the three-dimensional model view generating device may further include: a model standardization unit.

[0150] The model standardization unit is used to obtain a three-dimensional training model; the geometric center of the three-dimensional training model is moved to the target origin, and the three-dimensional training model is adjusted to a preset specification size to obtain a standardized three-dimensional training model.

[0151] Optionally, the geometric features include: local differential geometric features, mesoscale structural features and global structural features.

[0152] Optionally, the three-dimensional model view generating device may further include: a cover image selection unit.

[0153] The cover image selection unit is used to select a view from the view rendering results as the cover image of the target three-dimensional model by using the cover image selection model.

[0154] Optionally, the preset parameters include: light source parameters, camera parameters and rendering image parameters.

[0155] The present invention provides a 3D model view generation device, which is used to: obtain a target 3D model; adjust the orientation of the target 3D model using a pre-trained orientation discrimination model to obtain a target 3D model with an adjusted orientation; based on the target 3D model with an adjusted orientation, perform multi-view rendering on the target 3D model according to preset parameters to obtain a view rendering result, wherein the view rendering result includes multiple views of the target 3D model. The present invention uses automation technology to adjust the orientation of the target 3D model and perform preset parameter rendering based on the orientation, thereby quickly and efficiently generating multiple views, thereby supporting the processing of large-scale 3D models, significantly improving the generation speed and shortening the rendering time, so as to meet the needs of digitizing a large number of cultural relics.

[0156] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0157] The three-dimensional model view generating device includes a processor and a memory. The three-dimensional model obtaining unit 10, the orientation adjusting unit 20 and the view rendering result obtaining unit 30 are all stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.

[0158] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to adjust the orientation of the target 3D model through automation technology and render based on preset parameters, so as to quickly and efficiently generate multiple views, thereby supporting the processing of large-scale 3D models, significantly improving the generation speed and shortening the rendering time to meet the needs of digitizing a large number of cultural relics.

[0159] An embodiment of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the three-dimensional model view generating method is implemented.

[0160] An embodiment of the present invention provides a processor, which is used to run a program, wherein the three-dimensional model view generation method is executed when the program is run.

[0161] like Figure 7 As shown, an embodiment of the present invention provides an electronic device 1000, which includes at least one processor 1001, at least one memory 1002 connected to the processor 1001, and a bus 1003; wherein the processor 1001 and the memory 1002 communicate with each other through the bus 1003; the processor 1001 is used to call the program instructions in the memory 1002 to execute the above-mentioned three-dimensional model view generation method. The electronic device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0162] The present invention also provides a computer program product, which, when executed on an electronic device, is suitable for executing a program that initializes the steps of the three-dimensional model view generation method.

[0163] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatuses, electronic devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processor of the computer or other programmable device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0164] In a typical configuration, an electronic device includes one or more processors (CPU), a memory, and a bus. The electronic device may also include an input / output interface, a network interface, and the like.

[0165] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.

[0166] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0167] In the description of the present invention, it should be understood that the terms "up", "down", "front", "back", "left" and "right" etc. indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description. They do not indicate or imply that the positions or elements referred to must have specific directions, be constructed and operate in specific directions. Therefore, they should not be understood as limitations of the present invention.

[0168] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0169] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A method for generating a three-dimensional model view, characterized in that: include: Obtaining a target three-dimensional model; Using a pre-trained orientation discrimination model to adjust the orientation of the target three-dimensional model, to obtain the target three-dimensional model with the adjusted orientation; Based on the target three-dimensional model with the adjusted orientation, multi-view rendering is performed on the target three-dimensional model according to preset parameters to obtain a view rendering result, wherein the view rendering result includes multiple views of the target three-dimensional model.

2. The method according to claim 1, characterized in that: The method of adjusting the orientation of the target three-dimensional model by using a pre-trained orientation discrimination model to obtain the target three-dimensional model with the adjusted orientation includes: The target three-dimensional model is input into the orientation discrimination model so that the orientation discrimination model extracts the first geometric feature and the first functional semantic feature of the target three-dimensional model, and performs orientation prediction based on the first geometric feature and the first functional semantic feature to obtain the output first predicted rotation parameter, and the target three-dimensional model is rotated using the first predicted rotation parameter to obtain the target three-dimensional model with adjusted orientation.

3. The method according to claim 1, characterized in that The training process of the orientation discrimination model includes: Obtaining a three-dimensional model training set, wherein the three-dimensional model training set includes a plurality of standardized three-dimensional training models whose orientations have been adjusted; Inputting each of the three-dimensional training models in the three-dimensional model training set into the orientation discrimination model in sequence for training, extracting a second geometric feature and a second functional semantic feature of the three-dimensional training model, and then rotating the three-dimensional training model according to a random rotation parameter to extract a third geometric feature and a third functional semantic feature of the rotated three-dimensional training model; Using the second geometric feature and the third geometric feature, obtaining a geometric consistency loss; Using the second functional semantic feature and the third functional semantic feature, obtaining a functional semantic constraint loss; Based on the third geometric feature and the third functional semantic feature, obtain a second predicted rotation parameter, and use the second predicted rotation parameter and the random rotation parameter to obtain an orientation prediction loss; Using the geometric consistency loss, the functional semantic constraint loss and the orientation prediction loss, a comprehensive loss is obtained; Based on the comprehensive loss, the network parameters of the orientation discrimination model are adjusted, and retraining is performed until a preset training end condition is reached to obtain the trained orientation discrimination model.

4. The method according to claim 3, characterized in that The standardized process includes: Obtaining a three-dimensional training model; The geometric center of the three-dimensional training model is moved to the target origin, and the three-dimensional training model is adjusted to a preset specification size to obtain the standardized three-dimensional training model.

5. The method according to any one of claims 2 to 4, characterized in that The geometric features include: local differential geometric features, mesoscale structural features and global structural features.

6. The method according to claim 1, characterized in that After performing multi-view rendering on the target three-dimensional model based on the adjusted orientation according to preset rendering parameters to obtain a view rendering result, the method further includes: A cover image selection model is used to select a view from the view rendering results as a cover image of the target three-dimensional model.

7. The method according to claim 1, characterized in that The preset parameters include: light source parameters, camera parameters and rendering image parameters.

8. A 3D model view generation device, characterized in that: include: 3D model acquisition unit, orientation adjustment unit and view rendering result acquisition unit, The three-dimensional model obtaining unit is used to obtain a target three-dimensional model; The orientation adjustment unit is used to adjust the orientation of the target three-dimensional model using a pre-trained orientation discrimination model to obtain the target three-dimensional model with the adjusted orientation; The view rendering result obtaining unit is used to perform multi-view rendering on the target three-dimensional model according to preset parameters based on the target three-dimensional model with the adjusted orientation, so as to obtain a view rendering result, wherein the view rendering result includes multiple views of the target three-dimensional model.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the three-dimensional model view generating method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The electronic device includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the three-dimensional model view generation method as described in any one of claims 1 to 7.

Images