Virtual scene rendering method and device, equipment and storage medium

Abstract

The invention discloses a virtual scene rendering method and device and a storage medium, is used for improving the shielding elimination efficiency, and is applied to the field of artificial intelligence. According to the method, after a preset image obtained by shooting a target model in a target scene at a target pose by a virtual camera is obtained, a target area of the target model in the target scene is shielded, and under the condition that the preset image is changed due to shielding of the target area, vertexes included in the target area are determined as preset vertexes; no shielding exists between the preset fixed point and the virtual camera, determining the primitive of the preset vertex as a preset primitive based on the distance between the preset vertex and the virtual camera, and rendering the preset vertex based on the preset primitive to obtain a target image. Whether the vertex in the target area of the target model is shielded or not is judged by shielding the target area of the target model in the target scene, so that shielding elimination is realized, manual intervention is not needed, and the implementation efficiency of shielding elimination is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the computer technology field, and particularly relates to a virtual scene rendering method and device, equipment and a storage medium. BACKGROUND

[0002] With the continuous development of computer technology and the continuous promotion of digitization, the application scenarios of 3D rendering technology are also increasing. For example, when 3D rendering is applied to a game scenario, in order to make the scene more realistic, the modeling process is constantly expanded to depict details, so that the required calculation amount of rendering increases exponentially, and thus the time required for model rendering is also greatly increased.

[0003] In order to reduce the required calculation amount of rendering, an occlusion culling method is used to remove objects that are not visible in the final rendering image. Occlusion culling mainly relies on manual marking in the application of graphics rendering, which leads to the problem of low efficiency of occlusion culling. SUMMARY

[0004] Embodiments of the present application provide a virtual scene rendering method, device and storage medium, which are used to improve the efficiency of occlusion culling.

[0005] Therefore, the first aspect of the present application provides a virtual scene rendering method, comprising:

[0006] obtaining a preset image, the preset image being an image obtained by a virtual camera at a target pose shooting a target model, the target model being contained in a target scene;

[0007] occluding a target region of the target model in the target scene, and determining a vertex contained in the target region as a preset vertex in a case where occluding the target region causes the preset image to change, the preset vertex having no occlusion between the preset vertex and the virtual camera;

[0008] determining a primitive of the preset vertex as a preset primitive based on a distance between the preset vertex and the virtual camera;

[0009] rendering the preset vertex based on the preset primitive to obtain a target image.

[0010] The second aspect of the present application provides a virtual scene rendering device, comprising:

[0011] an obtaining unit, configured to obtain a preset image, the preset image being an image obtained by a virtual camera at a target pose shooting a target model, the target model being contained in a target scene;

[0012] a processing unit, configured to occlude a target region of the target model in the target scene, and determine a vertex contained in the target region as a preset vertex in a case where occluding the target region causes the preset image to change, the preset vertex having no occlusion between the preset vertex and the virtual camera;

[0013] The processing unit is further configured to determine the primitive of the preset vertex as the preset primitive based on a distance between the preset vertex and the virtual camera.

[0014] The rendering unit is configured to render the preset vertex based on the preset primitive to obtain a target image.

[0015] In a possible implementation manner of the second aspect, the processing unit is specifically configured to:

[0016] perform feature extraction on the preset image to obtain a preset feature vector;

[0017] perform occlusion on a target region in the preset feature vector based on the occlusion mask to obtain a target feature vector;

[0018] perform normalization processing on the target feature vector to determine a preset vertex contained in the target region.

[0019] In a possible implementation manner of the second aspect, the processing unit is further configured to perform spatial feature enhancement on the preset feature vector to obtain a reference feature vector;

[0020] The processing unit is specifically configured to perform occlusion on a target region in the reference feature vector based on the occlusion mask to obtain a target feature vector.

[0021] In a possible implementation manner of the second aspect, the processing unit is specifically configured to determine the primitive of the preset vertex as the preset primitive when a distance between the preset vertex and the virtual camera is greater than or equal to a first distance and less than or equal to a second distance.

[0022] In a possible implementation manner of the second aspect, the rendering unit is specifically configured to:

[0023] perform spatial conversion on the preset vertex to obtain a preset coordinate, the preset coordinate being a coordinate point of the preset vertex mapped on a screen;

[0024] generate a pixel set of the target image based on the preset coordinate, a pixel in the pixel set of the target image corresponding to at least one reference primitive, the at least one reference primitive including the preset primitive;

[0025] select, from the at least one reference primitive corresponding to the target pixel, a primitive with a smallest pixel depth as a target primitive, the target pixel being a pixel in the pixel set of the target image;

[0026] render the target pixel using the target primitive to obtain the target image.

[0027] In a possible implementation manner of the second aspect, the obtaining unit is further configured to obtain a target screen range, the target screen range being a projection range of the virtual camera on the screen based on a target view volume;

[0028] rendering unit, in particular for:

[0029] selecting a coordinate in the target screen range from the preset coordinates to obtain a target coordinate, the target coordinate corresponding to a target vertex;

[0030] generating a pixel set of a target image based on the target coordinate, a pixel in the pixel set of the target image corresponding to at least one reference primitive, the at least one reference primitive including a primitive of the target vertex.

[0031] In a possible implementation of the second aspect, the rendering unit, in particular for:

[0032] determining a target level based on a distance between the target primitive and the virtual camera, the target level being used to describe a degree of detail of the target primitive;

[0033] rendering a target pixel using the target primitive of the target level to obtain the target image.

[0034] In a possible implementation of the second aspect, the processing unit is further configured to:

[0035] pruning a target number of vertices from the preset vertices to obtain updated preset vertices;

[0036] rendering the updated preset vertices based on the updated preset vertices and primitives of the updated preset vertices to obtain a reference image;

[0037] analyzing a visual error between the target image and the reference image;

[0038] when the visual error is less than a preset value, rendering the target image using the updated preset vertices.

[0039] In a possible implementation of the second aspect, the processing unit is further configured to:

[0040] obtaining a blending coefficient;

[0041] performing smoothing processing on the target primitive of the target level in the target image based on the blending coefficient to obtain a blended target primitive set;

[0042] the rendering unit is specifically configured to render the target pixel based on the blended target primitive set to obtain the target image.

[0043] In a possible implementation of the second aspect, the processing unit is further configured to:

[0044] pruning vertices other than the preset vertices in the target scene to obtain updated target scene;

[0045] rendering the target image based on the updated target scene.

[0046] In a possible implementation of the second aspect, the rendering unit is specifically configured to sequentially perform model transformation, view transformation, projection transformation, viewport transformation and perspective division on the preset vertex, project the preset vertex onto a screen, and obtain a preset coordinate.

[0047] The third aspect of the present application provides a computer device, comprising a memory, a processor and a bus system;

[0048] The memory is configured to store a program.

[0049] The processor is configured to execute the program in the memory, and the processor is configured to execute the method of each aspect according to the instructions in the program code.

[0050] The bus system is configured to connect the memory and the processor to enable the memory and the processor to communicate.

[0051] The fourth aspect of the present application provides a computer readable storage medium, and the computer readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method of each aspect.

[0052] The fifth aspect of the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to execute the method provided in each aspect.

[0053] As can be seen from the above technical solutions, after obtaining the preset image of the target model in the target scene captured by the virtual camera at the target pose, the target region of the target model in the target scene is occluded. In the case that the target region is occluded to cause the preset image to change, the preset vertex contained in the target region is determined, the preset vertex is not occluded between the preset vertex and the virtual camera, the primitive of the preset vertex is determined as the preset primitive based on the distance between the preset vertex and the virtual camera, and the preset vertex is rendered based on the preset primitive to obtain the target image. By occluding the target region of the target model in the target scene to determine whether the vertex in the target region of the target model is occluded, occlusion culling is realized, without manual intervention, and the implementation efficiency of occlusion culling is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 An architectural schematic diagram of an application scenario of the rendering method of the virtual scene provided by the embodiments of the present application;

[0055] Figure 2 A flowchart of the rendering method of the virtual scene provided by the embodiments of the present application;

[0056] Figure 3 A schematic diagram of preset image acquisition provided for an embodiment of the present application;

[0057] Figure 4 Another flowchart of a rendering method of a virtual scene provided for an embodiment of the present application;

[0058] Figure 5 Another flowchart of a rendering method of a virtual scene provided for an embodiment of the present application;

[0059] Figure 6 A schematic diagram of a rendering step of a virtual scene provided for an embodiment of the present application;

[0060] Figure 7 A schematic diagram of a rendering device of a virtual scene provided for an embodiment of the present application;

[0061] Figure 8 Another schematic diagram of a rendering device of a virtual scene provided for an embodiment of the present application;

[0062] Figure 9 Another schematic diagram of a rendering device of a virtual scene provided for an embodiment of the present application. DETAILED DESCRIPTION

[0063] The embodiments of the present application provide a rendering method, device and equipment of a virtual scene and a storage medium, which are used for improving the efficiency of occlusion culling.

[0064] The terms "first", "second", "third", "fourth" and the like in the description and claims of the present application and above-mentioned drawings (if any) are used to distinguish similar objects, not necessarily describing a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in other than the order illustrated or described herein. In addition, the terms "include" and "correspond to" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a list of steps or units is not necessarily limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to such processes, methods, products or devices.

[0065] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0066] With the continuous development of computer technology and the continuous advancement of digitization, the application scenarios of 3D rendering technology are also increasing. For example, when 3D rendering is applied to a game scenario, in order to make the scene more realistic, the modeling process is constantly expanding the depiction of details, which exponentially increases the amount of calculations required for rendering, and thus the time required for model rendering is also greatly increased.

[0067] Based on the staggered arrangement of multiple objects in a scene, the objects in the scene are photographed from different angles. The obtained images also show different objects. When the objects in the scene are photographed from view angle A, the visible objects are object A, object B and object C. However, when the objects in the scene are photographed from view angle B, the visible objects become object C and object D. In view angle B, object D blocks objects A and B. As an example, in this case, in the image obtained by photographing the objects in the scene from view angle B, there is occlusion between object D and objects A and B. In the image obtained by rendering the objects in the scene from view angle B, objects A and B are removed by using occlusion removal, thereby reducing the amount of calculations required for rendering.

[0068] However, in the common occlusion removal method, the occlusion removal of the image is mainly realized by the manual marking method. How to eliminate the problem of low efficiency of occlusion removal caused by the manual marking method.

[0069] The present application proposes that after a preset image obtained by a virtual camera at a target pose photographing a target model in a target scene is obtained, an occlusion is performed on a target region of the target model in the target scene. In the case where the occlusion on the target region causes the preset image to change, a preset vertex included in the target region is determined, the preset vertex has no occlusion with the virtual camera, a primitive of the preset vertex is determined as a preset primitive based on a distance between the preset vertex and the virtual camera, and the preset vertex is rendered based on the preset primitive to obtain a target image. By performing the occlusion on the target region of the target model in the target scene to determine whether the vertices in the target region of the target model are occluded, the occlusion removal is realized without human intervention, and the implementation efficiency of the occlusion removal is effectively improved.

[0070] The training method of the image segmentation model or the image segmentation method of each optional embodiment of the present application is implemented based on artificial intelligence technology. Artificial intelligence (AI) is a theory, method, technology and application system for using a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0071] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, automatic driving, intelligent transportation, etc.

[0072] Computer vision (CV) is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further to do image processing, so that the computer processing becomes more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and tries to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional (3D) object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, intelligent transportation, etc. It also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0073] The present application also relates to cloud technology. Among them, cloud technology refers to a hosting technology that unifies system resources such as hardware, software and network in a wide area network or local area network to realize data calculation, storage, processing and sharing.

[0074] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology and other technologies based on cloud computing business model application, which can form a resource pool, and be used on demand, flexibly and conveniently. Cloud computing technology will become an important support. The background service of the technical network system needs a large amount of computing and storage resources, such as video websites, picture websites and more portals. With the high development and application of Internet behavior, every item in the future may have its own identification mark and needs to be transmitted to the background system for logical processing. Different levels of data will be processed separately, and various industry data will need strong system support, which can only be realized through cloud computing. The cloud technology involved in the present application mainly refers to the image recognition between terminal devices or servers through "cloud".

[0075] For the convenience of understanding, part of the nouns in the present application are explained as follows.

[0076] The multi-level detail (LOD) determines the resource allocation of object rendering according to the position and importance of the nodes of the object model in the real environment, reduces the number of surfaces and the degree of detail of unimportant objects, and thus obtains efficient rendering operation.

[0077] ResNet network: its network structure is usually a large-scale convolution, followed by a pooling layer; then a plurality of sub-modules (DenseBlock and TransitinLayer) are connected in succession; and finally a pooling and full connection are connected. Taking ResNet101 as an example, the network structure is described as follows: the number of layers of ResNet101 is 3+4+23+3=33 building blocks, each block is 3 layers, so there are 33x3=99 layers, plus the first layer of convolution conv1 and the last full connection layer (for classification), a total of 99+1+1=101 layers.

[0078] The rendering method and device of a virtual scene and the storage medium provided by the embodiments of the present application can realize occlusion culling without manual marking, so as to improve the efficiency of occlusion culling. The following describes an exemplary application of the electronic device provided by the embodiments of the present application. The electronic device provided by the embodiments of the present application can be implemented as various types of user terminals, or as a server.

[0079] The electronic device runs the rendering method of a virtual scene provided by the embodiments of the present application, and can realize occlusion culling in the image rendering process without manual marking, thereby improving the efficiency of occlusion culling and the efficiency of rendering.

[0080] Referring to Figure 1 , Figure 1An optional architecture diagram in one application scenario of the rendering method of the virtual scene provided in the embodiment of the present application is provided, in order to implement the rendering method of the virtual scene, the terminal device 100 is connected with the server 300 through the network 200, the server 300 is connected with the database 400, and the network 200 can be a wide area network or a local area network, or a combination of the two. The client for implementing the rendering method of the virtual scene is deployed on the terminal device 100, wherein the client can run on the terminal device 100 in the form of a browser, or run on the terminal device 100 in the form of an independent application (APP), and the like. The specific form of the client is not limited herein. The server 300 involved in the present application can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal device 100 can be a smart phone, a tablet computer, a notebook computer, a palm computer, a personal computer, a smart television, a smart watch, a vehicle-mounted device, a wearable device, and the like, but is not limited thereto. The terminal device 100 and the server 300 can be directly or indirectly connected through the network 200 through wired or wireless communication, which is not limited herein. The number of the server 300 and the terminal device 100 is also not limited. The scheme provided in the present application can be independently completed by the terminal device 100, or independently completed by the server 300, or cooperatively completed by the terminal device 100 and the server 300, which is not limited herein. In brief, the database 400 can be regarded as an electronic file cabinet, i.e., a place for storing electronic files, and a user can add, query, update, delete, and the like, the data in the files. The so-called "database" is a collection of data stored together in a certain way, shared by multiple users, with as little redundancy as possible, and independent of application programs. The database management system (DBMS) is a computer software system designed to manage databases, which generally has basic functions such as storage, interception, security assurance, backup, and the like.A database management system can be classified according to the database model it supports, such as relational, Extensible Markup Language (XML); or according to the computer type it supports, such as server cluster, mobile phone; or according to the query language it supports, such as Structured Query Language (SQL), XQuery; or according to the performance impulse focus, such as maximum size, highest running speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, such as supporting multiple query languages at the same time. In this application, the database 400 can be used to store the target scene composed of the target pose of the virtual camera and the target model, and of course, the storage location of the target scene composed of the target pose of the virtual camera and the target model is not limited to the database, for example, it can also be stored in the terminal device 100, the distributed file system of the server 300, or the blockchain.

[0081] In some embodiments, the server 300 and the terminal device 100 can both perform the rendering method of the virtual scene provided in the embodiments of the present application.

[0082] In this embodiment, the specific process can be as follows: the terminal device 100 obtains the target pose of the virtual camera and the target scene containing the target model; then the terminal device 100 can store the target pose of the virtual camera and the target scene containing the target model in the database 400 or in the storage of the terminal device 100; the server 300 obtains the target pose of the virtual camera and the target scene containing the target model from the database 400 or the terminal device 100; then the server 300 obtains a preset image according to the target pose of the virtual camera and the target scene containing the target model, occludes the target region of the target model in the target scene, and determines a vertex contained in the target region as a preset vertex in the case that the occlusion of the target region causes the preset image to change, and the preset vertex is not occluded between the virtual camera, and determines a primitive of the preset vertex as a preset primitive based on the distance between the preset vertex and the virtual camera. Finally, the server 300 can return the preset primitive and the preset vertex to the terminal device 100, so that the terminal device 100 can use the preset primitive to render the preset vertex to obtain a target image.

[0083] In another embodiment, the terminal device 100 independently performs the rendering method of the virtual scene provided in the embodiments of the present application. In this embodiment, the specific process can be as follows: the terminal device 100 acquires a preset image, the preset image being an image obtained by a virtual camera shooting a target model at a target pose, the target model being included in a target scene; the terminal device 100 occludes a target region of the target model in the target scene, and in a case where the occlusion of the target region causes the preset image to change, determines a vertex included in the target region as a preset vertex, the preset vertex being free of occlusion with the virtual camera; the terminal device 100 determines a primitive of the preset vertex as a preset primitive based on a distance between the preset vertex and the virtual camera; and the terminal device 100 renders the target image based on the preset primitive.

[0084] It can be understood that in the specific embodiments of the present application, related data such as input data and output data are involved. When the above embodiments of the present application are applied to specific products or technologies, the permission or consent of the user needs to be obtained, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of the country and region.

[0085] In combination with the above introduction, the rendering method of the virtual scene in the present application will be introduced below. Please refer to Figure 2 One embodiment of the rendering method of the virtual scene in the embodiments of the present application includes the following steps.

[0086] S110, acquiring a preset image, the preset image being an image obtained by a virtual camera shooting a target model at a target pose;

[0087] The target model is included in a target scene, and the target model is a three-dimensional object.

[0088] Specifically, when rendering the target scene, the pose of the virtual camera when shooting and the target model included in the target scene need to be determined first. The acquisition of the preset image is to shoot the target scene under the condition that the virtual camera is at the target pose and the angle of the viewing frustum of the virtual camera is assumed to be 180°, and the obtained image is not limited here.

[0089] For example, please refer to Figure 3 , Figure 3 a schematic diagram of the acquisition of the preset image provided in the embodiments of the present application.

[0090] As Figure 3 can be seen, the target scene includes a target model A, a target model B, a target model C and a target model D. The virtual camera can shoot the target model A, the target model B and the target model C under the condition that the virtual camera is at the target pose, and there is an occlusion relationship between the target model A and the target model B.

[0091] Further, the target scene further includes light sources, lighting modes, textures in the target scene, and many other conditions, which are not limited here.

[0092] It can be understood that the description of the setting of the angle between the virtual camera's view volume when collecting the preset image is only an example, and in actual application, the angle should be an angle of any degree between 0° and 180°, and should be set in combination with the specific application scene, which is not limited here.

[0093] S120, occluding the target region of the target model in the target scene, and in the case that the occlusion of the target region causes the preset image to change, determining the vertex contained in the target region as the preset vertex;

[0094] The preset vertex is free of occlusion from the virtual camera.

[0095] Specifically, taking the target scene shown in the foregoing Figure 3 The target region of the target model in the target scene is occluded, and after the occlusion, the image collected by the virtual camera is compared, and in the case that the occlusion of the target region causes the image collected by the virtual camera to be different from the preset image, the vertex contained in the target region is determined as the preset vertex, and the preset vertex is free of occlusion from the virtual camera, that is, the preset vertex is an unoccluded vertex.

[0096] Exemplarily, the occlusion of the target region of the target model in the target scene and the determination of the preset vertex can be realized based on artificial intelligence technology.

[0097] First, feature extraction is performed on the preset image to obtain a preset feature vector.

[0098] Exemplarily, a ResNet block with a split-attention block can be used for feature extraction of the preset image, that is, a ResNeSt block is used for feature extraction of the preset image.

[0099] Suppose the preset image is X, and the preset feature vector SA(X) obtained by the ResNeSt block performing feature extraction on the preset image can be represented as:

[0100]

[0101] Where S is the number of splits, r s is the attention mechanism weight corresponding to the split, and F s is a feature extraction function.

[0102] Then, the target region in the preset feature vector is occluded based on a feature mask to obtain a target feature vector.

[0103] Exemplarily, a shielding attention mechanism is introduced herein to shield the target region of the target model in the target scene, which is based on a shielding mask M to shield the target region of the target model in the target scene, and obtain a target feature vector O(F, M), wherein F is used to represent a preset feature vector.

[0104] Therefore, shielding the target region in the preset feature vector based on the feature mask to obtain the target feature vector can be described using the following formula:

[0105]

[0106] wherein, is an element-wise multiplication, a is the weight of the shielding attention mechanism, and sigma is used to adjust the influence of the shielding region.

[0107] Finally, the target feature vector is normalized to determine the preset vertex contained in the target region.

[0108] Exemplarily, in order to perform binary classification processing on the shielding result, a normalization function is introduced herein to analyze whether the shielding of the target region has an impact on the preset image.

[0109] Still taking the foregoing Figure 3 For example, since the target model A shields part of the region of the target model B, shielding the shielded region (target region) of the target model B will not cause the preset image to change. Therefore, when it is determined that the shielded region of the target model B is the target region, the target region is shielded, and on this basis, the vertices in the target region are all shielded vertices.

[0110] Similarly, it can be known that shielding any region of model A will cause the preset image to change, so when it is determined that the shielded region of the target model A is the target region, the target region is not shielded, and on this basis, the vertices in the target region are preset vertices, and there is no shielding between the preset vertices and the virtual camera.

[0111] wherein the normalization function can be based on the target feature vector and the distance between the vertex and the near clipping plane to determine whether the target region is shielded, which can be specifically referred to as the following formula:

[0112] f = softmax(O(F, M) / d near );

[0113] wherein d near is the distance between the vertex and the near clipping plane, and the distance between the vertex and the near clipping plane is a preset distance threshold.

[0114] It can be understood that the description of the distance between the vertex introduced in the normalization function and the near clipping plane is only an example, and in actual application, any preset distance value can be used, which is not limited herein.

[0115] The vertex with no occlusion between the virtual camera is selected as the preset vertex by using the normalization function.

[0116] It can be understood that the description of the occlusion of the target region of the target model in the target scene to determine the preset vertex with no occlusion between the virtual camera is only an example, and in actual application, it can be set in combination with specific use scenarios, which is not limited herein.

[0117] In the embodiment of the application, after feature extraction is performed on the preset image to obtain a preset feature vector, the target region in the preset feature vector is occluded based on the occlusion mask to obtain a target feature vector, and then the target feature vector is normalized to determine whether the vertex contained in the target region is a preset point. The way of using the model to determine whether the occlusion of the target region has an impact on the preset image to determine whether the vertex in the target region is a preset vertex improves the efficiency of the scheme implementation and the accuracy of the occlusion determination.

[0118] Further, the occlusion of the target region of the target model in the target scene and the determination of the preset vertex can be based on the feature mask to occlude the target region in the preset feature vector to obtain a target feature vector, and before the spatial feature of the preset feature vector is enhanced to improve the expression of the feature vector in the spatial feature and improve the analysis accuracy of whether the target region is occluded.

[0119] Specifically, before the target region in the preset feature vector is occluded based on the feature mask to obtain a target feature vector, the spatial feature of the preset feature vector is enhanced to obtain a reference feature vector.

[0120] Illustratively, when the spatial feature of the preset feature vector is enhanced, the input preset feature vector can be spatially enhanced by using a graph convolution network to obtain a reference feature vector.

[0121] The calculation of the graph convolution network can refer to the following definition:

[0122] H i+1 =σ(D -0.5 AD -0.5 H (l) W (l) );

[0123] wherein H (l) is the feature representation of the lth layer, A is the adjacency matrix, D is the degree matrix, and W (l)wherein, w is the weight of the masking attention mechanism, and σ is a nonlinear activation function.

[0124] Correspondingly, based on the feature mask, the target region in the reference feature vector is occluded to obtain the target feature vector.

[0125] Therefore, when introducing the occlusion attention mechanism to realize the occlusion of the target region of the target model in the target scene, the occlusion attention mechanism occludes the target region of the target model in the target scene based on the occlusion mask M to obtain the target feature vector O(F', M), wherein F' is used to represent the reference feature vector.

[0126] Therefore, based on the feature mask, the target region in the reference feature vector is occluded to obtain the target feature vector, which can be described by the following formula:

[0127]

[0128] wherein, is an element-wise multiplication, a is the weight of the occlusion attention mechanism, and σ is used to adjust the influence of the occluded region.

[0129] Correspondingly, when using the normalization function to determine whether the target region is occluded, the formula can be updated as:

[0130] f=softmax(O(F',M) / d near );

[0131] wherein, d near is the distance between the vertex and the near clipping plane, and the distance between the vertex and the near clipping plane is a preset distance threshold.

[0132] It can be understood that the description of the distance between the vertex and the near clipping plane introduced in the normalization function is only an example, and in actual application, it can be any preset distance value, which is not limited here.

[0133] In the embodiments of the present application, the graph convolution network is introduced to enhance the dependency relationship between the vertices in the preset image, so as to more fully learn the relationship between the vertices in the preset image and improve the accuracy of subsequent analysis.

[0134] S130, based on the distance between the preset vertex and the virtual camera, determine that the primitive of the preset vertex is the preset primitive;

[0135] For example, the primitive of the preset vertex can be determined as the preset primitive based on the distance between the preset vertex and the virtual camera. The preset vertex can correspond to primitives of different model detail levels. The primitive of the preset vertex can be determined as the preset primitive based on the distance between the preset vertex and the virtual camera, without limitation.

[0136] Specifically, the primitives of the preset vertex can be divided into three different accuracy primitives with the first distance and the second distance as threshold ranges. Please refer to the following formula:

[0137]

[0138] wherein d is the distance between the preset vertex and the virtual camera, d near is the first distance, and d far is the second distance.

[0139] As can be seen from the above formula, when the fine degree of the preset primitive is determined based on the relationship between the distance between the preset vertex and the virtual camera and the first distance and the second distance, the fine degree of the preset primitive can be divided into high, mid and low.

[0140] For example, the first distance can be the distance corresponding to the near clipping plane of the virtual camera, and the second distance can be the distance corresponding to the far clipping plane of the virtual camera, without limitation.

[0141] When the first distance is the distance corresponding to the near clipping plane of the virtual camera, and the second distance is the distance corresponding to the far clipping plane of the virtual camera, generally, when the distance between the preset vertex and the virtual camera is greater than the distance corresponding to the far clipping plane of the virtual camera, according to the conventional visual habit, the distant object should be relatively blurred, and therefore, the preset vertex can be rendered more roughly. When the distance between the preset vertex and the virtual camera is less than the distance corresponding to the near clipping plane of the virtual camera, the preset vertex can be rendered finely according to the requirement, or the preset vertex can be directly removed, without limitation. When the distance between the preset vertex and the virtual camera is greater than the first distance and less than the second distance, the fine degree of the primitive of the preset vertex needs to be considered, and the preset vertex can be reasonably rendered.

[0142] Embodiments of the present application propose that when the distance between the preset vertex and the virtual camera is greater than or equal to the first distance and less than or equal to the second distance, the primitive of the preset vertex is determined as the preset primitive. The preset vertex is rendered for occlusion removal only when the distance between the preset vertex and the virtual camera is within the preset interval, thereby avoiding resource waste caused by rendering the preset vertex for occlusion removal when the distance between the preset vertex and the virtual camera is out of the range.

[0143] S140, obtaining a target image based on rendering a preset vertex according to a preset primitive.

[0144] In the rendering line tube process, the process of obtaining a target image based on rendering a preset vertex according to a preset primitive is also the rendering of converting a three-dimensional image into a two-dimensional image. A series of spatial transformations need to be performed on the preset vertex to obtain the upper coordinates of the preset vertex mapped on the screen.

[0145] Specifically, the preset vertex is sequentially subjected to model transformation, view transformation, projection transformation, viewport transformation and perspective division, so as to project the preset vertex onto the screen and obtain the preset coordinates.

[0146] For example, in the case of V model , the preset vertex is converted from the representation in the model space to the representation in the world space based on the model transformation, specifically as follows:

[0147] V world = M model · V model ;

[0148] wherein M model is a model transformation matrix, and V world is the representation of the preset vertex in the world space.

[0149] Then, the preset vertex is converted from the representation in the world space to the representation in the view space based on the view transformation, specifically as follows:

[0150] V view = M view · V world ;

[0151] wherein M view is a view transformation matrix, and V view is the representation of the preset vertex in the view space.

[0152] Then, the preset vertex is projected from the view space to the representation in the two-dimensional space based on the projection transformation, specifically as follows:

[0153] V proj = M proj · V view ;

[0154] wherein M proj is a projection transformation matrix, and V proj is the representation of the preset vertex in the two-dimensional space.

[0155] Then, in order to integrate these geometric transformations, the preset vertex can be converted from the two-dimensional space to the representation in the screen space based on the viewport transformation, specifically as follows:

[0156] Vscreen = M viewport · V proj ;

[0157] wherein M viewport is a viewport transformation matrix, V screen is a representation of the preset vertex in a screen space.

[0158] Meanwhile, perspective projection can also be performed based on the orthogonal projection to ensure that the projection result can preserve the feature of near large and far small, and the specific implementation is as follows:

[0159] P screen = V proj · w;

[0160] wherein P screen is a screen space coordinate (preset coordinate) of the preset vertex after perspective projection, V proj · w is a w component in the homogeneous coordinate.

[0161] In the embodiments of the present application, the preset vertex is projected onto the screen through the model transformation, the view transformation, the projection transformation, the viewport transformation and the perspective division, and the preset coordinate is obtained, so as to realize the projection of the vertex coordinate onto the screen. The above provides a good technical basis for the implementation of the scheme.

[0162] On this basis, a pixel set of a target image is generated based on the preset coordinate, and the pixels in the pixel set of the target image correspond to at least one reference primitive, and the at least one reference primitive includes the preset primitive.

[0163] That is, the preset coordinate is scan-converted to generate a pixel set of a target image, the pixels in the pixel set of the target image are pixels that need to be rendered based on occlusion culling, and there is a one-to-one correspondence between the pixels in the pixel set of the target image and the pixel set on the screen, so as to accurately fill the pixels corresponding to each primitive. The specific operation can be as follows:

[0164] S(P) = {(x, y) | (x, y) ∈ P screen};

[0165] wherein S(P) is a pixel set on the screen, and P screen is a coordinate of the preset vertex in the screen space.

[0166] In order to avoid that the occlusion culling is not completed sufficiently in the foregoing step S120, it is also necessary to remove the preset vertex occluded by other vertices in the process of rasterization.

[0167] That is, the primitive with the smallest pixel depth is selected as the target primitive from the at least one reference primitive corresponding to the target pixel, and the target pixel is the pixel in the pixel set of the target image.

[0168] The specific performance is as follows:

[0169]

[0170] Wherein, O(P) indicates whether the preset vertex P is occluded, Z buffer (x,y) is the depth value of the target pixel in the depth buffer corresponding to the preset vertex, Z current (x,y) is the depth value of the target pixel corresponding to the current preset vertex.

[0171] That is, from the plurality of preset vertices, the preset vertex corresponding to the primitive with the smallest depth value is selected as the target primitive, and the target pixel is rendered using the target primitive, and each pixel in the target image pixel set is sequentially taken as the target pixel, so that the rendering of the target image is completed, and the target image is obtained.

[0172] It can be understood that the description of the image rendering based on occlusion culling here is only an example, and in actual application, it should be set in combination with the specific application scenario, which is not limited here.

[0173] In the embodiment of the application, the space conversion based on the preset coordinates is implemented, and the mapping between the preset coordinates and the pixels in the pixel set of the target image is implemented, so that the occlusion culling is further performed from the pixel granularity, and the accuracy of the occlusion culling is effectively guaranteed.

[0174] Further, when the target image is obtained based on the preset primitive rendering the preset vertex, based on the foregoing, the target screen range can be further set based on the demand to limit the angle of the viewing frustum of the virtual camera.

[0175] For example, before generating the pixel set of the target image based on the preset coordinates, the target screen range can also be obtained, the target screen range is the projection range of the virtual camera on the screen based on the target viewing frustum, and the viewing frustum angle range of the target viewing frustum is greater than 0° and less than 180°, which is not limited here.

[0176] In this case, the coordinates in the preset coordinates within the target screen range are selected to obtain the target coordinates, and the target coordinates correspond to the target vertex, and the specific operation is as follows:

[0177]

[0178] Wherein, C(V) is used to indicate whether the preset coordinate is included in the target screen range (whether the preset vertex is included in the target viewing frustum), V is the preset coordinate, and when C(V) is 1, the preset coordinate is included in the target screen range; V frustrum is the target screen range (target viewing frustum).

[0179] And generate a pixel set of a target image based on the target coordinates, the pixel set of the target image corresponding to at least one reference primitive, the at least one reference primitive including a primitive of the target vertex.

[0180] It can be understood that the operation of screening the target vertex from the preset vertex is only an example, and in actual application, whether the preset vertex is marked as the target vertex can also be determined by judging whether the preset vertex is in the view pyramid, which is not limited herein.

[0181] In the embodiment of the application, the preset vertex is further screened by the view pyramid of the virtual camera, and the vertex not in the field of view is eliminated, thereby further reducing the number of vertices to be rendered and improving the rendering efficiency.

[0182] Further, on the basis of the foregoing scheme, the target pixel is rendered using the target primitive to obtain the target image, which can further subdivide the rendering level and render the target pixel using the target primitive of the target level to obtain the target image.

[0183] For example, the target level is determined based on the distance between the target primitive and the virtual camera, the target level is used to describe the degree of detail of the target primitive, and the target pixel is rendered using the target primitive of the target level, and the target level is as follows:

[0184]

[0185] Wherein, Lod n is the rendering level, d n is the distance threshold.

[0186] In the embodiment of the application, in the rendering process, the rendering of the target pixel of different accuracy based on the distance between the target primitive and the virtual camera can still be realized based on the multi-layer model, which improves the flexibility of the scheme and provides a strong guarantee for the rendering effect of the scheme.

[0187] Further, in the case of rendering the target pixel based on the subdivided rendering level, in order to avoid visual discontinuity caused by the use of different rendering levels by adjacent pixel points, a detail blending method can also be used to realize smooth transition between different rendering levels.

[0188] For example, a blending coefficient is obtained, wherein the blending coefficient can be pre-set or temporarily set for the target object, which is not limited herein.

[0189] The target primitive of the target level in the target image is smoothed based on the blending coefficient to obtain a blended target primitive set. The specific operation is as follows:

[0190]

[0191] wherein V blended is the mixed target primitive set, and a is a mixing coefficient, is the target primitive of the target level.

[0192] The target image is obtained by rendering a target pixel based on the mixed target primitive set.

[0193] In the embodiments of the present application, by smoothing the target primitive of the target level in the target image, the visual mutation caused by the use of different rendering levels by adjacent pixel points is avoided, and the rendering effect of the target image is further improved.

[0194] In the embodiments of the present application, after obtaining the preset image obtained by the target model in the target pose of the virtual camera shooting the target scene, the target region of the target model in the preset image is occluded. In the case that the target region is occluded, the preset vertex contained in the target region is determined, the preset vertex and the virtual camera are not occluded, the primitive of the preset vertex is determined as the preset primitive based on the distance between the preset vertex and the virtual camera, and the preset vertex is rendered based on the preset primitive to obtain the target image. By occluding the target region of the target model in the preset image to determine whether the vertex in the target region of the target model is occluded, occlusion culling is realized without manual intervention, and the implementation efficiency of occlusion culling is effectively improved.

[0195] Based on the scheme shown in the foregoing Figure 2 , the scheme provided by the present application can also be applied to the scene of rendering the image shot by the virtual camera in the target pose multiple times. In this scene, in order to reduce the rendering efficiency of the scene that needs to be rendered multiple times, the rendering method of the virtual scene provided by the present application can also perform the scheme shown in Figure 4 .

[0196] S210, removing a target number of vertices from the preset vertices to obtain updated preset vertices;

[0197] wherein the target number is less than or equal to a preset number, and the preset number is a threshold number of vertex simplification.

[0198] Specifically, a target number of vertices can be randomly removed from the preset vertices to obtain updated preset vertices, and the specific operation is as follows:

[0199] V simplified = simplify(V original , T);

[0200] wherein V simplified is the updated preset vertex, V original is the preset vertex, and T is the preset number.

[0201] S220, render the updated preset vertex based on the updated preset vertex and the primitive of the updated preset vertex, to obtain a reference image;

[0202] Specifically, the operation of obtaining the reference image based on the updated preset vertex and the primitive of the updated preset vertex is similar to the operation of determining the preset primitive based on the preset vertex and rendering the preset vertex based on the preset primitive to obtain the target image in the steps S130 and S140, and details are not repeated here.

[0203] S230, analyze the visual error between the target image and the reference image;

[0204] After the rendering of the target image and the reference image is completed, the visual error between the target image and the reference image is evaluated. The specific comparison mode is as follows:

[0205]

[0206] Wherein, E(V original ,V simplified ) is the visual error between the target image and the reference image.

[0207] S240, when the visual error is less than a preset value, rendering the target image using the updated preset vertex.

[0208] In the embodiments of the present application, a target number of vertices are removed from the preset vertex to obtain an updated preset vertex, and the updated preset vertex is rendered based on the updated preset vertex and the primitive corresponding to the updated preset vertex to obtain a reference image. When the visual error between the reference image and the target image is within an acceptable range, the updated preset vertex is used for rendering related to the target image in subsequent rendering of the target image, effectively reducing the number of vertices required for rendering and reducing the resources required for rendering the target image.

[0209] Based on the scheme shown in the foregoing Figure 2 , the scheme provided by the present application can also be applied to the scene of rendering the image captured at the target pose of the virtual camera multiple times. In this scene, in order to reduce the rendering efficiency of the scene that needs to be rendered multiple times, the rendering method of the virtual scene provided by the present application can also perform the scheme shown in Figure 5 .

[0210] S310, remove the vertices other than the preset vertex in the target scene to obtain an updated target scene;

[0211] Specifically, first determine the vertices Y included in the target scene, and then determine whether the vertices Y are included in the preset vertex V model .

[0212] If the vertex Y is included in the preset vertex V model , the related operation is not needed to be performed.

[0213] If the vertex Y is not included in the preset vertex V model , the vertex Y is removed from the target scene.

[0214] After all the vertices in the target scene are traversed, the updated target scene is obtained, and the specific judgment basis is as follows.

[0215]

[0216] Wherein, C visible (Y) is used to describe whether the vertex Y is included in the preset vertex.

[0217] It can be understood that the description of the removal standard of the vertices in the target scene except the preset vertex to obtain the updated target scene is only an example, and in actual application, whether the vertex in the target scene is in the target view volume of the virtual camera, or whether the vertex in the target scene appears in the target image can also be judged, which is not limited here.

[0218] S320, rendering the target image based on the updated target scene.

[0219] Specifically, based on the vertices included in the updated target scene, and the primitives corresponding to the vertices, the target image is rendered.

[0220] Wherein, the specific way of rendering is similar to the rendering way of step S140 in the foregoing Figure 2 , which is not limited here.

[0221] In the embodiment of the application, all the vertices except the preset vertex are removed from the target scene to obtain the updated target scene. Since the vertices invisible in the target image are removed from the target scene, the updated target scene is obtained. Therefore, when the target image is rendered based on the updated target scene, the invisible vertices do not need to be considered, which effectively reduces the number of vertices required for rendering and reduces the resources required for rendering the target image.

[0222] In order to facilitate understanding, the role of different steps in the scheme provided by the application and the specific operation in implementation are provided below for further description in combination with Figure 6 .

[0223] The scheme provided by the application includes three key steps:

[0224] Step one, removing the vertices outside the field of view of the virtual camera to obtain the preset vertex;

[0225] Step two, further occlusion culling based on the preset vertex combined rasterization simulation is performed to determine the corresponding primitive of the target pixel point;

[0226] Step three, multi-layer detail simulation is performed to achieve flexible rendering of the target pixel point.

[0227] On this basis, data space structures such as quadtree, octree or thirty-two tree can also be used to organize the target scene to further accelerate the rendering speed of the rendering method of the virtual scene provided in the application.

[0228] For example, a data structure (thirty-two tree) is constructed based on the vertices Y included in the target scene, and the construction result is as follows:

[0229]

[0230] Wherein, O octree (Y) represents the position of the vertex Y in the spatial data structure, Y is the position of the vertex, V noden is the spatial position of the thirty-two tree node.

[0231] The embodiments of the application can also improve the rendering speed by introducing a data space structure, which is not limited here.

[0232] The rendering device of the virtual scene in the application will be described in detail below. Please refer to Figure 7 , Figure 7 Fig. 1 is a schematic diagram of an embodiment of the rendering device of the virtual scene in the application. The rendering device 10 of the virtual scene includes:

[0233] The acquisition unit 110 is configured to acquire a preset image, the preset image being an image obtained by a virtual camera at a target pose shooting a target model, the target model being included in a target scene;

[0234] The processing unit 120 is configured to perform occlusion on a target region of the target model in the target scene. In the case that the occlusion on the target region causes the preset image to change, the vertex included in the target region is determined as a preset vertex, and the preset vertex is not occluded by the virtual camera;

[0235] The processing unit 120 is further configured to determine the primitive of the preset vertex as a preset primitive based on the distance between the preset vertex and the virtual camera.

[0236] The rendering unit 130 is configured to render the target image based on the preset primitive of the preset vertex.

[0237] In the embodiment of the present application, after obtaining the preset image obtained by the target model in the target scene photographed by the virtual camera at the target pose, the target region of the target model in the preset image is occluded. In the case that the occlusion of the target region causes the preset image to change, the preset vertex included in the target region is determined, and there is no occlusion between the preset vertex and the virtual camera. Based on the distance between the preset vertex and the virtual camera, the primitive of the preset vertex is determined as the preset primitive, and the preset vertex is rendered based on the preset primitive to obtain the target image. By occluding the target region of the target model in the preset image to determine whether the vertices in the target region of the target model are occluded, occlusion culling is realized without manual intervention, and the implementation efficiency of occlusion culling is effectively improved.

[0238] Optionally, the processing unit 120 is specifically configured to:

[0239] extracting features of the preset image to obtain a preset feature vector;

[0240] occluding the target region in the preset feature vector based on the occlusion mask to obtain a target feature vector;

[0241] normalizing the target feature vector to determine that the vertex included in the target region is a preset vertex.

[0242] In the embodiment of the present application, after extracting features of the preset image to obtain a preset feature vector, the target region in the preset feature vector is occluded based on the occlusion mask to obtain a target feature vector, and the target feature vector is normalized to determine whether the vertex included in the target region is a preset vertex. The way of using the model to determine whether the occlusion of the target region affects the preset image to determine whether the vertex in the target region is a preset vertex improves the efficiency of the scheme implementation and the accuracy of the occlusion judgment.

[0243] Optionally, the processing unit 120 is further configured to enhance the spatial features of the preset feature vector to obtain a reference feature vector;

[0244] The processing unit 120 is specifically configured to occlude the target region in the reference feature vector based on the occlusion mask to obtain a target feature vector.

[0245] In the embodiment of the present application, the dependency relationship between the vertices in the preset image is enhanced by introducing a graph convolution network, so as to more fully learn the relationship between the vertices in the preset image and improve the accuracy of subsequent analysis.

[0246] Optionally, the processing unit 120 is specifically configured to determine that the primitive of the preset vertex is the preset primitive when the distance between the preset vertex and the virtual camera is greater than or equal to a first distance and less than or equal to a second distance.

[0247] The embodiment of the present application proposes that when the distance between the preset vertex and the virtual camera is greater than or equal to the first distance and less than or equal to the second distance, the preset vertex is determined as the preset primitive, and the rendering related to occlusion culling is performed on the preset vertex only when the distance between the preset vertex and the virtual camera is within the preset interval, thereby avoiding the resource waste caused by the rendering related to occlusion culling on the preset vertex when the distance between the preset vertex and the virtual camera is out of the range.

[0248] Optionally, the rendering unit 130 is specifically configured to:

[0249] perform spatial conversion on the preset vertex to obtain a preset coordinate, the preset coordinate being a coordinate point of the preset vertex mapped on the screen;

[0250] generate a pixel set of a target image based on the preset coordinate, a pixel in the pixel set of the target image corresponding to at least one reference primitive, the at least one reference primitive including the preset primitive;

[0251] select a primitive with the smallest pixel depth from the at least one reference primitive corresponding to the target pixel as a target primitive, the target pixel being a pixel in the pixel set of the target image;

[0252] render the target pixel using the target primitive to obtain the target image.

[0253] In the embodiment of the present application, the spatial conversion based on the preset coordinate realizes the mapping between the preset coordinate and the pixel in the pixel set of the target image, and further performs occlusion culling from the pixel granularity, thereby effectively guaranteeing the accuracy of occlusion culling.

[0254] Optionally, the acquisition unit 110 is further configured to acquire a target screen range, the target screen range being a projection range of the virtual camera on the screen based on a target view volume;

[0255] The rendering unit 130 is specifically configured to:

[0256] select a coordinate within the target screen range from the preset coordinate to obtain a target coordinate, the target coordinate corresponding to a target vertex;

[0257] generate a pixel set of a target image based on the target coordinate, a pixel in the pixel set of the target image corresponding to at least one reference primitive, the at least one reference primitive including a primitive of the target vertex.

[0258] In the embodiment of the present application, the preset vertex is further screened by the view volume of the virtual camera, and the vertex not within the field of view is eliminated, thereby further reducing the number of vertices that need to be rendered and improving the rendering efficiency.

[0259] Optionally, the rendering unit 130 is specifically configured to:

[0260] determine a target level based on the distance between the target primitive and the virtual camera, the target level being used to describe a level of detail of the target primitive;

[0261] render the target pixels using the target primitive of the target level to obtain a target image.

[0262] In the embodiments of the present application, in the rendering process, the distance between the target primitive and the virtual camera can still be used to render target pixels of different accuracy based on the multi-layer model, which improves the flexibility of the scheme and provides a strong guarantee for the rendering effect of the scheme.

[0263] Optionally, the processing unit 120 is further configured to:

[0264] remove a target number of vertices from the preset vertices to obtain updated preset vertices;

[0265] render the updated preset vertices based on the updated preset vertices and the primitives corresponding to the updated preset vertices to obtain a reference image;

[0266] analyze a visual error between the target image and the reference image;

[0267] when the visual error is less than a preset value, render the target image using the updated preset vertices.

[0268] In the embodiments of the present application, by removing a target number of vertices from the preset vertices to obtain updated preset vertices, and rendering the updated preset vertices based on the updated preset vertices and the primitives corresponding to the updated preset vertices to obtain a reference image, when the visual error between the reference image and the target image is within an acceptable range, the updated preset vertices are used for rendering the target image in subsequent rendering of the target image, which effectively reduces the number of vertices required for rendering and reduces the resources required for rendering the target image.

[0269] Optionally, the processing unit 120 is further configured to:

[0270] obtain a blending coefficient;

[0271] smooth the target primitives of the target level in the target image based on the blending coefficient to obtain a blended target primitive set;

[0272] The rendering unit 130 is specifically configured to render the target pixels based on the blended target primitive set to obtain a target image.

[0273] In the embodiments of the present application, by smoothing the target primitives of the target level in the target image, visual discontinuity caused by the use of different rendering levels by adjacent pixel points is avoided, and the rendering effect of the target image is further improved.

[0274] Optionally, the processing unit 120 is further configured to:

[0275] remove all vertices except the preset vertex in the target scene to obtain an updated target scene;

[0276] render the target image based on the updated target scene.

[0277] In the embodiments of the present application, all vertices except the preset vertex are removed from the target scene to obtain an updated target scene. Since all the invisible vertices in the target image are removed from the target scene, the updated target scene is obtained. Therefore, when the target image is rendered based on the updated target scene, the invisible vertices do not need to be considered, the number of vertices required for rendering is effectively reduced, and the resources required for rendering the target image are reduced.

[0278] Optionally, the rendering unit 130 is specifically configured to sequentially perform model transformation, view transformation, projection transformation, viewport transformation and perspective division on the preset vertex, project the preset vertex onto the screen to obtain the preset coordinate.

[0279] In the embodiments of the present application, the preset vertex is projected onto the screen by model transformation, view transformation, projection transformation, viewport transformation and perspective division to obtain the preset coordinate, so as to project the vertex coordinate onto the screen. This provides a good technical basis for the implementation of the scheme.

[0280] The virtual scene rendering device provided in the present application can be used for a server. Please refer to Figure 8 , Figure 8 is a server structure schematic diagram provided by the embodiments of the present application. The server 300 can have great differences due to different configurations or performances, and can include one or more central processing units (CPU) 322 (for example, one or more processors) and a memory 332, one or more storage media 330 (for example, one or more mass storage devices) storing application programs 342 or data 344. Among them, the memory 332 and the storage medium 330 can be temporary storage or persistent storage. The programs stored in the storage medium 330 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the server. Further, the central processing unit 322 can be configured to communicate with the storage medium 330 and execute a series of instruction operations in the storage medium 330 on the server 300.

[0281] Server 300 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0282] The steps performed by the server in the above embodiments can be based on this Figure 8 The server structure shown.

[0283] The virtual scene rendering device provided in this application can be used on terminal devices. Please refer to [link / reference]. Figure 9 For ease of explanation, only the parts relevant to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. In the embodiments of this application, a smartphone is used as an example for illustration:

[0284] Figure 9 This is a block diagram illustrating a portion of the structure of a smartphone related to the terminal device provided in the embodiments of this application. (Reference) Figure 9 The smartphone includes components such as a radio frequency (RF) circuit 410, a memory 420, an input unit 430, a display unit 440, a sensor 450, an audio circuit 460, a wireless fidelity (WiFi) module 470, a processor 480, and a power supply 490. Those skilled in the art will understand that... Figure 9 The smartphone structure shown does not constitute a limitation on smartphones and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0285] The following is combined Figure 9 A detailed introduction to the various components of a smartphone:

[0286] The RF circuit 410 can be used for receiving and sending signals in the process of information or communication, in particular, receiving the downlink information from the base station and processing by the processor 480; in addition, sending the uplink data to the base station. Generally, the RF circuit 410 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 410 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to global system for mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0287] The memory 420 can be used to store software programs and modules, and the processor 480 executes various function applications and data processing of the smart phone by running the software programs and modules stored in the memory 420. The memory 420 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the smart phone (such as audio data, a phone book, etc.), etc. In addition, the memory 420 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid state memory device.

[0288] The input unit 430 can be used to receive input digital or character information, and to generate key signal inputs for user settings and function controls of the smart phone. Specifically, the input unit 430 can include a touch panel 431 and other input devices 432. The touch panel 431, also called a touch screen, can collect touch operations (such as operations of a user using a finger, a stylus, or any suitable object or accessory on or near the touch panel 431) of the user on or near it, and drive the corresponding connection device according to the pre-set program. Optionally, the touch panel 431 can include two parts, a touch detection device and a touch controller. The touch detection device detects the touch position of the user and detects the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch coordinates, and sends it to the processor 480, and can receive the commands from the processor 480 and execute them. In addition, the touch panel 431 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 431, the input unit 430 can also include other input devices 432. Specifically, the other input devices 432 can include one or more of a physical keyboard, function keys (such as volume control buttons, on / off buttons, etc.), trackballs, mice, joysticks, etc.

[0289] The display unit 440 can be used to display information input by the user or information provided to the user, as well as various menus of the smart phone. The display unit 440 can include a display panel 441, which can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 431 can cover the display panel 441, and when the touch panel 431 detects a touch operation on or near it, it transmits to the processor 480 to determine the type of touch event, and then the processor 480 provides corresponding visual output on the display panel 441 according to the type of touch event. Although in the Figure 9 In some embodiments, the touch panel 431 and the display panel 441 can be integrated to realize the input and output functions of the smart phone.

[0290] The smart phone can also include at least one sensor 450, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor can include an ambient light sensor and a proximity sensor, where the ambient light sensor can adjust the brightness of the display panel 441 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 441 and / or the backlight when the smart phone is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and when at rest, can detect the magnitude and direction of gravity, and can be used for applications of identifying the smart phone posture (such as switching between landscape and portrait, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), and the like. As for other sensors that can also be configured on the smart phone, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, they will not be described here.

[0291] The audio circuit 460, the speaker 461, and the microphone 462 can provide an audio interface between the user and the smart phone. The audio circuit 460 can convert the received audio data into an electrical signal, transmit the electrical signal to the speaker 461, and convert the electrical signal into a sound signal output by the speaker 461; on the other hand, the microphone 462 converts the collected sound signal into an electrical signal, which is received by the audio circuit 460 and converted into audio data, and then output to the processor 480 for processing, and then transmitted to another smart phone through the RF circuit 410, or output to the memory 420 for further processing.

[0292] WiFi belongs to a short-range wireless transmission technology, and the smart phone can help the user to send and receive emails, browse web pages, and access streaming media through the WiFi module 470, which provides the user with wireless broadband Internet access. Although Figure 9 The WiFi module 470 is shown, but it can be understood that it does not belong to the essential structure of the smart phone, and can be omitted as needed without changing the essence of the application.

[0293] The processor 480 is the control center of the smart phone, which connects all parts of the smart phone through various interfaces and lines, executes various functions of the smart phone and processes data by running or executing software programs and / or modules stored in the memory 420 and calling data stored in the memory 420, thereby monitoring the smart phone as a whole. Optionally, the processor 480 can include one or more processing units; optionally, the processor 480 can integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, and application program, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 480.

[0294] The smart phone further includes a power supply 490 (such as a battery) for supplying power to each component. Optionally, the power supply can be logically connected to the processor 480 through a power management system, so that the power management system can realize functions such as management of charging, discharging, and power consumption management.

[0295] Although not shown, the smart phone can further include a camera, a Bluetooth module, and the like, which will not be described herein.

[0296] The steps performed by the terminal device in the above embodiments can be based on the terminal device structure shown in the above embodiments. Figure 9 The terminal device structure shown in the above embodiments.

[0297] The computer readable storage medium in the embodiments of the present application stores a computer program, and when the computer program is run on a computer, the computer program causes the computer to execute the method described in the above embodiments.

[0298] The computer program product in the embodiments of the present application includes a program, and when the program is run on a computer, the program causes the computer to execute the method described in the above embodiments.

[0299] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the above method embodiments, which will not be described herein.

[0300] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0301] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to realize the purpose of the embodiment scheme.

[0302] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0303] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods according to the various embodiments of the present application. The aforementioned storage medium includes: various types of U disks, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks, and various other media that can store program codes.

[0304] The above-described embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the technical solutions; even though the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still make modifications to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features; and these modifications or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for rendering a virtual scene, characterized in that, include: Acquire a preset image, wherein the preset image is an image of the target model captured by a virtual camera in the target pose, and the target model is contained within the target scene; The target region of the target model in the target scene is occluded. When the occlusion of the target region causes the preset image to change, the vertices contained in the target region are determined to be preset vertices, and there is no occlusion between the preset vertices and the virtual camera. Based on the distance between the preset vertex and the virtual camera, the primitive of the preset vertex is determined as a preset primitive; The target image is obtained by rendering the preset vertices based on the preset primitives.

2. The method according to claim 1, characterized in that, The step of occluding the target region of the target model in the target scene, and determining the vertices contained in the target region as preset vertices when the occlusion of the target region causes a change in the preset image, includes: Feature extraction is performed on the preset image to obtain a preset feature vector; The target region in the preset feature vector is occluded based on the occlusion mask to obtain the target feature vector; The target feature vector is normalized to determine the vertices contained in the target region as the preset vertices.

3. The method according to claim 2, characterized in that, Before obtaining the target feature vector by occluding the target region in the preset feature vector based on the occlusion mask, the method further includes: Spatial feature enhancement is performed on the preset feature vector to obtain a reference feature vector; The step of occluding the target region in the preset feature vector based on an occlusion mask to obtain the target feature vector includes: The target region in the reference feature vector is occluded based on the occlusion mask to obtain the target feature vector.

4. The method according to any one of claims 1 to 3, characterized in that, The step of determining the preset primitive based on the distance between the preset vertex and the virtual camera includes: When the distance between the preset vertex and the virtual camera is greater than or equal to a first distance and less than or equal to a second distance, the primitive of the preset vertex is determined to be a preset primitive.

5. The method according to any one of claims 1 to 3, characterized in that, The step of rendering the preset vertices based on the preset primitives to obtain the target image includes: The preset vertex is spatially transformed to obtain preset coordinates, which are the coordinate points of the preset vertex mapped on the screen; The pixel set of the target image is generated based on the preset coordinates, and the pixels in the pixel set of the target image correspond to at least one reference primitive, the at least one reference primitive including the preset primitive; The pixel with the smallest pixel depth is selected from the at least one reference pixel corresponding to the target pixel as the target pixel, and the target pixel is a pixel in the pixel set of the target image; The target pixel is rendered using the target primitive to obtain the target image.

6. The method according to claim 5, characterized in that, The method further includes: Obtain the target screen range, which is the projection range of the virtual camera on the screen based on the target view frustum; The pixel set for generating the target image based on the preset coordinates includes: Select coordinates within the target screen range from the preset coordinates to obtain target coordinates, where the target coordinates correspond to target vertices; A pixel set of the target image is generated based on the target coordinates, and the pixels in the pixel set of the target image correspond to the at least one reference primitive, the at least one reference primitive including the primitive of the target vertex.

7. The method according to claim 5 or 6, characterized in that, The step of rendering the target pixel using the target primitive to obtain the target image includes: Based on the distance between the target primitive and the virtual camera, a target level is determined, which describes the level of detail of the target primitive. The target pixel is rendered using the target primitive at the target level to obtain the target image.

8. The method according to claim 1, characterized in that, The method further includes: Remove the target number of vertices from the preset vertices to obtain the updated preset vertices; Render the updated preset vertices based on the updated preset vertices and the primitives of the updated preset vertices to obtain a reference image; Analyze the visual error between the target image and the reference image; When the visual error is less than a preset value, the updated preset vertex is used to render the target image.

9. The method according to claim 7, characterized in that, The method further includes: Obtain the mixing coefficient; Based on the mixing coefficient, the target primitives at the target level in the target image are smoothed to obtain a mixed set of target primitives; The step of rendering the target pixel using the target primitive at the target level to obtain the target image includes: The target pixels are rendered based on the blended set of target primitives to obtain the target image.

10. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Remove all vertices in the target scene except the preset vertices to obtain an updated target scene; The target image is rendered based on the updated target scene.

11. The method according to claim 5, characterized in that, The step of performing spatial transformation on the preset vertex to obtain preset coordinates includes: The preset vertex is sequentially transformed by model transformation, view transformation, projection transformation, viewport transformation and perspective division, and then projected onto the screen to obtain the preset coordinates.

12. A rendering device for a virtual scene, characterized in that, include: An acquisition unit is used to acquire a preset image, wherein the preset image is an image of a target model captured by a virtual camera at a target pose, and the target model is contained within a target scene; The processing unit is used to occlude the target region of the target model in the target scene. When occluding the target region causes the preset image to change, the unit determines the vertices contained in the target region as preset vertices, and there is no occlusion between the preset vertices and the virtual camera. The processing unit is further configured to determine the primitive of the preset vertex as a preset primitive based on the distance between the preset vertex and the virtual camera; A rendering unit is used to render the preset vertices based on the preset primitives to obtain a target image.

13. A computer device, characterized in that, include: Memory, processor, and bus system; The memory is used to store programs; The processor is configured to execute a program in the memory, and the processor is configured to execute the method of any one of claims 1 to 11 according to instructions in the program code; The bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

14. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method as claimed in any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor using the method as described in any one of claims 1 to 11.

Images