A model modeling and rendering processing method based on AI technology and edge computing

Abstract

The present invention discloses a model modeling and rendering processing method based on AI technology and edge computing, which belongs to the technical field of 3D modeling. The method comprises: an edge end receives image data that needs to be modeled and rendered and transmits it to a cloud end; the cloud end performs modeling and processing on the image data through AI technology to generate a 3D scene model; the cloud end divides the 3D scene model into multiple small blocks, which are stored in the cloud end or the edge end, and the small blocks are processed and distributed to each edge end through a distributed computing framework; the edge end is selected to call a rendering engine to render the small blocks to obtain a rendered image; according to the image data and the rendered image that need to be modeled and rendered, the rendering method of the rendering engine is optimized, and then a new rendered image is obtained again; the new rendered image is transmitted to a central end, and the central end merges and iterates the processing results in sequence to form a final 3D scene model, and the final 3D scene model is displayed through a rendering processing engine, etc.

Description

Technical Field

[0001] The present invention relates to the field of 3D modeling technology, and in particular to a model modeling and rendering processing method based on AI technology and edge computing. Background Art

[0002] With the continuous advancement of computer software and hardware technology, 3D modeling technology has gradually been innovated and broken through. Modern 3D modeling technology can already achieve the replication and production of images that are close to reality, allowing designers to more easily achieve 3D modeling and rendering of complex objects, solving the problem of manual mapping being too difficult. Modern 3D modeling technology has also constructed mathematical models with very high observability, using more abstract mathematical methods to model the shape and movement of objects, enriching the application scenarios of 3D modeling technology.

[0003] In the prior art, the Chinese invention patent with application number 202210181223.7 discloses a distributed storage-based real-scene 3D model layered rendering optimization method, system, and storage medium, which realizes the fast rendering process of the real-scene 3D model by dividing the real-scene 3D model into several layers and rendering and loading them separately. The size of the layers divided by the real-scene 3D model is set according to the equipment requirements, which speeds up the rendering process and improves the rendering efficiency; it effectively prevents the 3D model with large amounts of data from being stuck during the rendering process. However, it does not solve how to make the final rendered model more accurate in the process of preventing stuck.

[0004] In view of this, a model modeling and rendering processing method based on AI technology and edge computing is needed. Summary of the invention

[0005] An embodiment of the present invention provides a model modeling and rendering processing method based on AI technology and edge computing, so as to at least solve the technical problem in the related art on how to make the rendering model more accurate.

[0006] According to one aspect of an embodiment of the present invention, a model modeling and rendering processing method based on AI technology and edge computing is provided, including:

[0007] The edge receives and stores the image data that needs to be modeled and rendered, and then transmits the image data to the cloud;

[0008] The cloud uses AI technology to model the image data and generate a corresponding 3D scene model;

[0009] The cloud divides the 3D scene model into multiple small blocks, which are stored in the cloud or edge. The small blocks are processed and distributed to each edge through a distributed computing framework.

[0010] Select the edge end to call the rendering engine to render the small block to obtain a rendered image;

[0011] According to the image data and rendering images for modeling rendering as needed, the rendering method of the rendering engine is optimized, and then a new rendering image is obtained again;

[0012] The new rendered image is transmitted to the central end, and the central end merges and iterates the processing results in sequence until the final 3D scene model is formed. The final 3D scene model is displayed through the rendering processing engine.

[0013] Optionally, it also includes: before the edge end stores the image data, converting, classifying, identifying duplicates, and removing abnormal data.

[0014] Optionally, the image data is modeled by AI technology to generate a corresponding 3D scene model, including:

[0015] Obtain multiple 3D scene models and establish a 3D scene model database;

[0016] Based on the 3D scanning data, the image data that needs to be modeled and rendered is morphologically classified through an intelligent classification algorithm;

[0017] AI automatically identifies various 3D models in image data that need to be modeled and rendered based on morphological classification;

[0018] The designed three-dimensional data is automatically processed and merged to generate the corresponding 3D scene model.

[0019] Optionally, the cloud divides the 3D scene model into multiple small blocks according to the type of model library components, including:

[0020] Step S31, first identify and classify the 3D scene types of the 3D scene model;

[0021] Step S32, selecting one or more scene subjects for each of the divided 3D scene types, with each type being used as a primary image;

[0022] Step S33: using Tile technology and combining the types of model library components to segment the 3D scene model except the scene into small three-dimensional images as secondary images;

[0023] Step S34, classifying the secondary images into levels from large to small, wherein the smaller the image, the larger the level;

[0024] Step S35: Calculate the name of the small three-dimensional image and the corresponding storage path according to the longitude and latitude coordinates of the 3D scene model and the location point, and implement the 3D tiles data fast query and acquisition service.

[0025] Optionally, types of 3D scenes include: earth scene, administrative block map scene, city scene, campus scene, topology scene, and vector map scene.

[0026] Optionally, the scene body includes: a map, a city, a park, a topology or a vector.

[0027] Optionally, the model library components include: sky, main objects, buildings, fixed ground and atmosphere.

[0028] According to another aspect of an embodiment of the present invention, there is also provided a model modeling and rendering processing device based on AI technology and edge computing, including: a cloud server, an edge server and a central server, the cloud server is respectively connected to a plurality of edge servers and the central server;

[0029] The edge server is used to receive and store image data that needs to be modeled and rendered, and then transmit the image data to the cloud server;

[0030] The cloud server models the image data using AI technology to generate a corresponding 3D scene model; the cloud divides the 3D scene model into multiple small blocks, which are stored in the cloud server or edge server, and processed and distributed to each edge server through a distributed computing framework; the edge server is selected to call a rendering engine to render the small blocks to obtain a rendered image;

[0031] The edge server optimizes the rendering method of the rendering engine according to the image data and rendering images for modeling and rendering as needed, and then obtains a new rendering image; the new rendering image is transmitted to the central server, and the central server merges and iterates the processing results in sequence until the final 3D scene model is formed. The final 3D scene model is displayed through the rendering processing engine and can also be stored in the cloud server for other servers or terminals to call.

[0032] According to another aspect of an embodiment of the present invention, a model modeling and rendering processing system based on AI technology and edge computing is also provided. The model modeling and rendering processing system based on AI technology and edge computing includes: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned model modeling and rendering processing methods based on AI technology and edge computing.

[0033] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned model modeling and rendering processing methods based on AI technology and edge computing.

[0034] Compared with the existing technology, the present invention has the following beneficial effects:

[0035] 1. Traditional 3D modeling requires manual design and is time-consuming and labor-intensive. However, the present invention uses AI technology to automatically generate 3D models, effectively reducing labor costs and time costs, and greatly improving the accuracy of the generated model.

[0036] 2. Faster model rendering: By decentralizing the processing flow and utilizing greater computing power and processing power, a series of efficiency improvements in the processing flow are achieved, which speeds up the model rendering process.

[0037] 3. Higher quality 3D models: The present invention adopts local merging and iterative operations to make the generated 3D models more accurate and realistic, and can meet the needs of more fields, such as games, animation, industrial manufacturing, etc.

[0038] 4. Lower processing costs: By processing data at the edge, problems such as data transmission and network latency are reduced, processing efficiency is improved, and costs are greatly reduced.

[0039] 5. More efficient modeling: Traditional 3D modeling requires manual design and is time-consuming and labor-intensive. The present invention uses AI technology to automatically generate 3D models, effectively reducing labor and time costs and greatly improving the accuracy of the generated model.

[0040] 6. The process of the present invention realizes distributed processing and distributed rendering of data, improves processing efficiency and reduces costs when processing small blocks using edge computing, while also ensuring processing accuracy and improving the processing speed and quality of the entire process. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 is a flowchart of a model modeling and rendering processing method based on AI technology and edge computing according to an embodiment of the present invention;

[0043] Figure 2 It is a schematic diagram of a model modeling and rendering processing device based on AI technology and edge computing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0045] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0046] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. Example

[0047] According to an embodiment of the present invention, an embodiment of a model modeling and rendering processing method based on AI technology and edge computing is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] like Figure 1 is a flowchart of a model modeling and rendering processing method based on AI technology and edge computing according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0049] Step S1: The edge receives image data that needs to be modeled and rendered, stores it, and then transmits the image data to the cloud;

[0050] Step S2: The cloud uses AI technology to model the image data and generate a corresponding 3D scene model;

[0051] Step S3: The cloud divides the 3D scene model into multiple small blocks, which are stored in the cloud or edge terminals, and are processed and distributed to each edge terminal through a distributed computing framework;

[0052] Step S4: Select an edge end to call a rendering engine to render the small block to obtain a rendered image;

[0053] Step S5, optimizing the rendering method of the rendering engine according to the image data and the rendered image graph for modeling rendering as needed, and then obtaining a new rendered image graph;

[0054] Step S6: The new rendered image is transmitted to the central end, and the central end merges and iterates the processing results in sequence until a final 3D scene model is formed. The final 3D scene model is displayed through a rendering processing engine, etc.

[0055] As an optional embodiment, in step S1, the edge end converts, classifies, identifies duplicates, and removes abnormal data on the image data before storing the image data.

[0056] Among them, repeated identification is used to select repeated image data and select the clearest image data for modeling processing.

[0057] That is, any edge end can be used for image data collection, conversion, classification, anomaly identification, storage, generation of 3D scene models, and calling of rendering engines for modeling and rendering.

[0058] As an optional embodiment, in step S2, modeling the image data using AI technology to generate a corresponding 3D scene model includes:

[0059] Step S21, obtaining multiple 3D scene models and establishing a 3D scene model database;

[0060] Step S22: morphologically classifying the image data to be modeled and rendered using an intelligent classification algorithm based on the three-dimensional scanning data;

[0061] Step S23, according to the morphological classification AI automatically identifies various types of three-dimensional models in the image data that need to be modeled and rendered for the adaptation design;

[0062] Step S24: automatically process and merge the designed three-dimensional data to generate a corresponding 3D scene model.

[0063] As an optional embodiment, in step S3, the cloud divides the 3D scene model into multiple small blocks according to the type of model library components, including:

[0064] Step S31, first identify and classify the 3D scene types of the 3D scene model, wherein the types of 3D scenes include: earth scene, administrative block map scene, city scene, park scene, topology scene, and vector map scene;

[0065] Step S32, selecting one or more scene subjects for each of the divided 3D scene types, that is, selecting a scene subject that is a combination of one or more of a map, a city, a park, a topology or a vector, and each type is used as a primary image;

[0066] Step S33, using Tile technology, combined with the types of model library components, the 3D scene model other than the scene is divided into small three-dimensional images as secondary images; wherein the model library components include: sky, main features, buildings, fixed ground and atmosphere. The main features can be administrative district landmarks, key features, etc. Fixed ground includes: ground, roads, vegetation, boundaries, pipelines, fences. The atmosphere is adjusted according to the required brightness.

[0067] Step S34, classifying the secondary images into levels from large to small, wherein the smaller the image, the larger the level;

[0068] Step S35: Calculate the name of the small three-dimensional image and the corresponding storage path according to the longitude and latitude coordinates of the 3D scene model and the location point, and implement the 3D tiles data fast query and acquisition service.

[0069] As an optional embodiment, in step S3, processing and distributing the small blocks to each edge terminal through the distributed computing framework includes:

[0070] Step S36, forming hierarchical directory files from the small blocks of three-dimensional images obtained by segmentation in step S35 and storing them in batches into a MongoDB database, which is stored via a cloud server;

[0071] Step S37: construct an index database for accessing or calling data according to the level type obtained in step S34.

[0072] As an optional embodiment, each small block in step S32 is composed of one or several matrix graphic blocks.

[0073] As an optional embodiment, in step S4, an edge end is selected to call a rendering engine to render a small block. According to the load of the hardware resources connected to the edge end, the hardware resource with the smallest load is selected for rendering.

[0074] As an optional embodiment, in step S4, rendering includes the following steps:

[0075] Step S41, obtaining small three-dimensional images, and selecting any small three-dimensional image to obtain rendering feature information from the small three-dimensional image of this type;

[0076] Step S42, constructing a rendering neural network model, inputting rendering feature information into the rendering neural network model, and the rendering neural network model outputs a rendering image;

[0077] Step S43: Repeat steps S41-S42 in order from small to large according to the level of the small three-dimensional image until all small three-dimensional images are rendered. By rendering in small to large levels, it is convenient to combine models after rendering output.

[0078] As an optional embodiment, in step S41, a multi-scale kernel prediction convolutional neural network model is used to reduce noise on a small block of three-dimensional image.

[0079] As an optional embodiment, step S5 specifically includes: obtaining corresponding losses by comparing the image data to be modeled and rendered with the rendered image, and optimizing the neural network model in the rendering engine according to the losses. Example

[0080] According to another aspect of an embodiment of the present invention, a model modeling and rendering processing device based on AI technology and edge computing is also provided. The model modeling and rendering processing device applies the above-mentioned model modeling and rendering processing method based on AI technology and edge computing, such as Figure 2 As shown, the device includes: a cloud server, an edge server and a central server, and the cloud server is connected to multiple edge servers and the central server respectively;

[0081] The edge server is used to receive and store image data that needs to be modeled and rendered, and then transmit the image data to the cloud server;

[0082] The cloud server models the image data using AI technology to generate a corresponding 3D scene model; the cloud divides the 3D scene model into multiple small blocks, which are stored in the cloud server or edge server, and processed and distributed to each edge server through a distributed computing framework; the edge server is selected to call a rendering engine to render the small blocks to obtain a rendered image;

[0083] The edge server optimizes the rendering method of the rendering engine according to the image data and rendering images for modeling and rendering as needed, and then obtains a new rendering image; the new rendering image is transmitted to the central server, and the central server merges and iterates the processing results in sequence until the final 3D scene model is formed. The final 3D scene model is displayed through the rendering processing engine and can also be stored in the cloud server for other servers or terminals to call.

[0084] The present invention is not limited to the above specific implementation methods. The above are only preferred implementation cases of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention. Example

[0085] According to another aspect of an embodiment of the present invention, a model modeling and rendering processing system based on AI technology and edge computing is also provided. The model modeling and rendering processing system based on AI technology and edge computing includes: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned model modeling and rendering processing methods based on AI technology and edge computing. Example

[0086] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned model modeling and rendering processing methods based on AI technology and edge computing.

[0087] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group, and the computer-readable storage medium includes a stored program.

[0088] Optionally, when the program is running, the device where the computer-readable storage medium is located is controlled to perform the following functions: the edge end receives image data that needs to be modeled and rendered, stores it, and then transmits the image data to the cloud; the cloud end models and processes the image data through AI technology to generate a corresponding 3D scene model; the cloud end divides the 3D scene model into multiple small blocks, which are stored in the cloud or the edge end, and the small blocks are processed and distributed to each edge end through a distributed computing framework; the edge end is selected to call the rendering engine to render the small blocks to obtain a rendered image; according to the image data and the rendered image that need to be modeled and rendered, the rendering method of the rendering engine is optimized, and then a new rendered image is obtained again; the new rendered image is transmitted to the central end, and the central end merges and iterates the processing results in sequence until the final 3D scene model is formed, and the final 3D scene model is displayed through the rendering processing engine.

[0089] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0090] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of units or modules can be electrical or other forms.

[0092] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0093] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0094] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nlyMemory), random access memory (RAM, RandomAccessMemory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0095] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A model modeling and rendering processing method based on AI technology and edge computing, characterized in that: include: The edge receives and stores the image data that needs to be modeled and rendered, and then transmits the image data to the cloud; Before the edge stores the image data, it converts, classifies, identifies duplicates, and removes abnormal data. Among them, duplicate identification is used to select duplicate image data and select the clearest image data for modeling processing. That is, any edge is used to collect, convert, classify, identify abnormalities, store, generate 3D scene models, and call the rendering engine for image data for modeling and rendering. The cloud uses AI technology to model the image data and generate a corresponding 3D scene model; including: Obtain multiple 3D scene models and establish a 3D scene model database; Based on the 3D scanning data, the image data that needs to be modeled and rendered is morphologically classified through an intelligent classification algorithm; AI automatically identifies various 3D models in image data that need to be modeled and rendered based on morphological classification; Automatically process and merge the designed three-dimensional data to generate the corresponding 3D scene model; The cloud divides the 3D scene model into multiple small blocks, which are stored in the cloud or edge, and processed and distributed to each edge through a distributed computing framework. The cloud divides the 3D scene model into multiple small blocks, including: Step S31, first identify and classify the 3D scene types of the 3D scene model; Step S32, selecting one or more scene subjects for each of the divided 3D scene types, with each type being used as a primary image; Step S33: using Tile technology and combining the types of model library components to segment the 3D scene model except the scene into small three-dimensional images as secondary images; Step S34, classifying the secondary images into levels from large to small, wherein the smaller the image, the larger the level; Step S35, according to the 3D scene model and the longitude and latitude coordinates of the location point, calculate the name of the small 3D image and the corresponding storage path, so as to realize the fast query and acquisition service of 3D tiles data; Select the edge end to call the rendering engine to render the small block to obtain a rendered image. The rendering includes the following steps: Step S41, obtaining a small three-dimensional image, and selecting any small three-dimensional image to obtain rendering feature information from the small three-dimensional image; using a multi-scale kernel prediction convolutional neural network model to reduce noise on the small three-dimensional image; Step S42, constructing a rendering neural network model, inputting rendering feature information into the rendering neural network model, and the rendering neural network model outputs a rendering image; Step S43, repeating steps S41-S42 in order from small to large according to the levels of the small three-dimensional images, until all small three-dimensional images are rendered; According to the image data and the rendered image graph required for modeling and rendering, the rendering method of the rendering engine is optimized, and then a new rendered image graph is obtained again; including obtaining corresponding losses by comparing the image data required for modeling and rendering with the rendered image graph, and optimizing the neural network model in the rendering engine according to the losses; The new rendered image is transmitted to the central end, and the central end merges and iterates the processing results in sequence until the final 3D scene model is formed. The final 3D scene model is displayed through the rendering processing engine.

2. The model modeling and rendering processing method based on AI technology and edge computing according to claim 1 is characterized in that: The types of 3D scenes include: earth scene, administrative block map scene, city scene, campus scene, topology scene, and vector map scene.

3. The model modeling and rendering processing method based on AI technology and edge computing according to claim 1 is characterized in that: The scene subjects include: maps, cities, parks, topologies or vectors.

4. The model modeling and rendering processing method based on AI technology and edge computing according to claim 1 is characterized in that: The model library components include: sky, main objects, buildings, fixed ground and atmosphere.

5. A model modeling and rendering processing device based on AI technology and edge computing, characterized in that: The model modeling and rendering processing method based on AI technology and edge computing described in any one of claims 1 to 4 is applied, comprising: a cloud server, an edge server and a central server, the cloud server is respectively connected to a plurality of edge servers and a central server; The edge server is used to receive and store image data that needs to be modeled and rendered, and then transmit the image data to the cloud server; The cloud server models the image data using AI technology to generate a corresponding 3D scene model; the cloud divides the 3D scene model into multiple small blocks, which are stored in the cloud server or edge server, and processed and distributed to each edge server through a distributed computing framework; the edge server is selected to call a rendering engine to render the small blocks to obtain a rendered image; The edge server optimizes the rendering method of the rendering engine according to the image data and rendering images for modeling and rendering as needed, and then obtains a new rendering image; the new rendering image is transmitted to the central server, and the central server merges and iterates the processing results in sequence until the final 3D scene model is formed. The final 3D scene model is displayed by the rendering processing engine and stored in the cloud server for call.

6. A model building and rendering processing system based on AI technology and edge computing, characterized in that: The model modeling and rendering processing system based on AI technology and edge computing includes: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by one or more processors, the one or more processors implement the model modeling and rendering processing method based on AI technology and edge computing as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the model modeling and rendering processing method based on AI technology and edge computing as described in any one of claims 1 to 4.

Images