A rapid modeling method and system based on three-dimensional dot matrix rendering

Abstract

The present invention discloses a rapid modeling method and system based on three-dimensional dot matrix rendering, which constructs a three-dimensional dot matrix space model and scans the target object in all directions; then extracts the first frame picture of the scanned video as a reference picture, takes the center line pixel as the reference line, and renders the center line to the center line of the dot matrix space; and decomposes the scanned video frame by frame, and scales the proportions of all frames to be completely consistent with the reference picture; finally, extracts the pixel dot matrix with a width of one pixel to the right of the center line position of the previous frame in the adjacent next frame, and renders and fills it to the right side of the dot matrix center line, and in this way, the dot matrix space is filled 360 degrees counterclockwise based on the center line of the reference photo; and the dot matrix structure data is translated and converted to generate a glTF file; thereby maximally ensuring the proportion and accuracy of the model and the object, and greatly improving the visual effect and modeling efficiency of the three-dimensional model.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional modeling, and in particular to a rapid modeling method and system based on three-dimensional dot matrix rendering. Background Art

[0002] With the gradual development and popularization of technologies such as VR, AR, and 5G, the upgrade of visual dimensions has become a general trend, and more and more people are exposed to 3D. Application software, especially teaching system software, is rapidly developing in the direction of three-dimensionalization, simulation, and gamification, while the basic modeling used to support three-dimensional simulation still mainly relies on manual modeling and drawing, which is inefficient. At the same time, most of the high-end modeling system tool software and methods are occupied by foreign manufacturers (such as 3dmax, maya, etc.). In the current complex international situation, how to make up for the shortcomings in the quality and efficiency of three-dimensional modeling has become the key to the development of three-dimensional applications in my country.

[0003] The traditional 3D modeling process is mainly divided into four steps: 1: Use modeling tools to build models (well-known ones include 3DMAX, ArcGIS, Maya, AutoCAD, etc.). Usually, they will provide some basic geometric elements, such as cubes, spheres, etc., and then use a series of geometric operations (translation, rotation, stretching, etc.) to build complex geometric scenes. 2: Effect editing, add appropriate material textures to the model in turn, perform UV development, assist with various types of maps such as color and normal, create lighting effects, etc. 3: Visual display of the model and formal release. 4: When applied to a specific scene, the model is processed by reducing the surface, developing UV, baking, format conversion, etc. according to the operating environment to reduce the size of the model without destroying the effect itself, and can be displayed quickly.

[0004] However, the traditional 3D modeling process is essentially a manual assembly and painting process based on the modeling tool environment, which inevitably leads to a series of problems. First of all, the accuracy is low. There is a slight gap between the model constructed by the traditional modeling method and the actual object (such as size, curvature, proportion, etc.). In some high-precision process production and training fields, this difference is fatal, causing the 3D simulation system to lose its original value and significance. Secondly, the efficiency is low and the cost is high. Due to the heavy reliance on the work experience and personal ability of the modeling personnel, it operates in a labor-intensive modeling production mode; it is time-consuming and labor-intensive, and it is impossible to complete large-scale modeling work in batches within an effective cycle. Although some solutions for rapid 3D modeling based on 2D plane pictures have appeared on the market in recent years, the overall effect is rough, lacks precision, and cannot fully automatically complete the entire modeling process, and still requires human intervention. There is no actual change and impact on the quality and efficiency of 3D modeling. Summary of the invention

[0005] The purpose of the present invention is to overcome the above technical deficiencies and propose a rapid modeling method and system based on three-dimensional dot matrix rendering to solve the problem that existing modeling relies heavily on manual operation, resulting in low efficiency and accuracy.

[0006] In order to achieve the above technical objectives, the technical solution of the present invention provides a rapid modeling method based on three-dimensional dot matrix rendering in a first aspect, which comprises the following steps:

[0007] Construct a three-dimensional dot matrix space model and perform 360-degree clockwise scanning of the target object;

[0008] Extract the first frame of the target object scanning video as the reference image, filter and remove the background color, take the center line pixels as the reference line, and render the center line to the center line of the dot matrix space;

[0009] Decompose the scanned video into frames and scale all frames to be exactly the same as the reference image.

[0010] Extract the pixel dot matrix with a width of one pixel to the right of the center line of the previous frame in the next adjacent frame, and render and fill it to the right of the center line of the dot matrix. Continue in this way, and fill the dot matrix space 360 ​​degrees counterclockwise based on the center line of the reference photo;

[0011] Translate the dot matrix structure data to generate a glTF file.

[0012] A second aspect of the present invention provides a rapid modeling system based on three-dimensional dot matrix rendering, which includes the following functional modules:

[0013] The modeling and framing module is used to construct a three-dimensional dot matrix space model and perform 360-degree clockwise scanning shooting and framing of the target object;

[0014] The centerline rendering module is used to extract the first frame of the target object scanning video as the reference image, filter and remove the background color, take the centerline pixels as the reference line, and render the centerline to the center line of the dot matrix space.

[0015] The scaling module is used to decompose the scanned video into frames and scale all frames to be exactly the same as the reference image.

[0016] The rendering and filling module is used to extract the pixel dot matrix with a width of one pixel to the right of the center line position of the previous frame in the adjacent next frame image, render and fill it to the right of the center line of the dot matrix, and in this way, the dot matrix space is filled 360 degrees counterclockwise based on the center line of the reference photo;

[0017] The data conversion module is used to translate the dot matrix structure data and generate a glTF file.

[0018] The third aspect of the present invention provides a server, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned rapid modeling method based on three-dimensional dot matrix rendering when executing the computer program.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned rapid modeling method based on three-dimensional dot matrix rendering are implemented.

[0020] Compared with the prior art, the present invention utilizes computer image processing algorithms to automatically complete the process of converting physical objects into three-dimensional models. It takes a different approach to automatically complete model construction in computer memory and directly perform pixel-level replication and transfer of physical objects. Most of the work in traditional modeling is integrated into model construction and completed at the same time, which can maximize the proportion and accuracy of the model and the physical object. At the same time, the original colors are restored to the model, so that the colors of each surface of the model are close to the real object, which greatly improves the visual effect of the three-dimensional model and does not require human participation, greatly improving the modeling efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of a rapid modeling method based on three-dimensional dot matrix rendering according to the present invention;

[0022] Figure 2 It is a flowchart of the steps of a rapid modeling method based on three-dimensional dot matrix rendering according to the present invention;

[0023] Figure 3 It is a module block diagram of a rapid modeling system based on three-dimensional dot matrix rendering described in the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0025] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a rapid modeling method based on three-dimensional dot matrix rendering, which includes the following steps:

[0026] S1. Construct a three-dimensional dot matrix space model and perform 360-degree clockwise scanning shooting of the target object.

[0027] Specifically, first, a three-dimensional dot lattice space model is constructed according to the size of the actual object. The model and the actual object are constructed in a 1:1 size. Preferably, the length, width and height of the three-dimensional dot lattice space model should slightly exceed the maximum length, width and height of the actual object, otherwise the length, width and height ratio of the dot lattice space should be kept consistent with the actual object.

[0028] When constructing a model with a smooth surface, the use of a commonly used pixel cube structure will cause the constructed surface to have a large area of ​​granularity, which will make the model construction of a complex structure appear rough. Therefore, in the three-dimensional lattice space model described in the embodiment of the present invention, the three-dimensional lattice space model is divided at the pixel level, and every four pixels form a triangular pyramid space structure composed of three isosceles right triangles as the minimum element. The triangular pyramid space structure is obtained by performing two diagonal bevels on the cube pixel structure. A cube pixel structure can be decomposed into four triangular pyramid space structures of the same size. By further decomposing a cube pixel space into four space structures, the basic surface processing of the model surface can be made more delicate and accurate.

[0029] Then, the target object is scanned 360 degrees clockwise to take a picture. If the target object is a small object, place the object in the center of the system's framing screen, and select the target object's outline through the contour recognition algorithm. If the target object is a large object such as a ship or a building, you can use a drone to carry a framing device for auxiliary scanning, set the drone's starting position and the circular flight radius, and ensure that the target object is within the circumference. When framing, a circular scan with an equidistant radius and the center of the object is taken to obtain a full-angle equidistant video of the target object.

[0030] S2. Extract the first frame of the target object scanning video as a reference image, filter and remove the background color, take the center line pixels as the reference line, and render the center line to the center line of the dot matrix space.

[0031] That is, the reference image of the 3D model and the starting line (also the ending line) of the rendered pixels are established.

[0032] S3, decomposing the scanned video into frames, and scaling all frames to be completely consistent with the reference image.

[0033] Specifically, the scanned video is decomposed into frames. According to the standard bit rate of 24 frames per second, the number of frame images of the video shot of a small object is about 2000-3000, and the number of frame images of the video shot of a large object is about 8000-15000.

[0034] At least three landmark points capable of constructing a plane space are taken from the reference image, the Y-axis coordinate of at least one of the three points is different from the Y-axis coordinates of the other two points, and the difference between the X-coordinates of the three points does not exceed 100 pixels.

[0035] The algorithm is called to calculate the pixel change characteristic values ​​within the range of the three marker points, that is, to determine the characteristic values ​​in a certain area of ​​the frame image, and then the same area needs to be matched in the adjacent frame images, that is, all pixel values ​​of the adjacent frame images are indexed and compared, and the areas of the adjacent frame images with the same pixel change characteristic values ​​are retrieved; the relative ratio of the area between the two frames is calculated, and the adjacent frame images are scaled accordingly according to the ratio, that is, the entire adjacent frame image is scaled to the size of the reference image; the frame image that has completed the scaling is used to scale the next adjacent frame image, and so on until all the frame images are processed, so that the ratio of all the frame images is scaled to be completely consistent with the reference image, completing the consistent processing of the ratio (camera distance) of all frame images.

[0036] S4. Extract a pixel dot matrix with a width of one pixel to the right of the center line of the previous frame in the next adjacent frame image, render and fill it to the right of the center line of the dot matrix, and continue in this way to fill the dot matrix space 360 ​​degrees counterclockwise based on the center line of the reference photo.

[0037] Take the first frame as the reference image, where the line has been rendered to the center line of the three-dimensional dot matrix; from the next frame image adjacent to the reference image, take the vertical column of pixels corresponding to the right side of the pixel line of the previous frame image, and fill it to the right side of the dot matrix center line. In this way, the dot matrix space is filled 360 degrees counterclockwise based on the center line. After processing all the frame images in sequence, the pixel line taken out of the last frame image is rendered just to the right side of the original pixel line in the dot matrix space, forming a closed loop.

[0038] In the process of rendering and filling the dot matrix, different filling strategies (processing curved surfaces or corners) are selected according to the difference in pixel color depth values. If the color difference is less than the standard value, the complete cube space range composed of four minimum triangular pyramid units is filled; if the color difference is greater than the standard value, a single cone space range is filled on the side close to the adjacent pixel.

[0039] S5. Translate and convert the dot matrix structure data to generate a glTF file.

[0040] The glTF data structure can ensure maximum compatibility and scalability. It is described in json format and can also be compiled into binary content: bglTF. glTF can include scenes, cameras, animations, etc. It can also include meshes, materials, textures, and even rendering techniques, shaders, and shader programs. At the same time, due to the characteristics of the json format, it supports reserved general and vendor-specific extensions. At present, almost all 3D modeling frameworks and tools are compatible with this format.

[0041] The rapid modeling method based on three-dimensional dot rendering described in the present invention uses a computer image processing algorithm to automatically complete the process of converting and producing a physical object into a three-dimensional model. It takes a different approach to automatically complete model construction in a computer memory and directly perform pixel-level replication and transfer of the physical object. Most of the work in traditional modeling is integrated into the model construction and completed at the same time, which can ensure the proportion and accuracy of the model and the physical object to the greatest extent. At the same time, the original color is restored to the model, so that the color of each surface of the model is close to the real object, which greatly improves the visual effect of the three-dimensional model and does not require human intervention, thereby greatly improving the modeling efficiency.

[0042] like Figure 3 As shown, the embodiment of the present invention also discloses a rapid modeling system based on three-dimensional dot matrix rendering, which includes the following functional modules:

[0043] The modeling and framing module is used to construct a three-dimensional dot matrix space model and perform 360-degree clockwise scanning shooting and framing of the target object;

[0044] The centerline rendering module is used to extract the first frame of the target object scanning video as the reference image, filter and remove the background color, take the centerline pixels as the reference line, and render the centerline to the center line of the dot matrix space.

[0045] The scaling module is used to decompose the scanned video into frames and scale all frames to be exactly the same as the reference image.

[0046] The rendering and filling module is used to extract the pixels of the adjacent next frame image corresponding to the previous frame baseline and render and fill them to the right side of the center line of the dot matrix. In this way, one pixel is used as the vertical column width and the center line is used as the reference to fill the dot matrix space 360 ​​degrees counterclockwise;

[0047] The data conversion module is used to translate the dot matrix structure data and generate a glTF file.

[0048] The execution method of the rapid modeling system based on three-dimensional dot matrix rendering in this embodiment is basically the same as the rapid modeling method based on three-dimensional dot matrix rendering mentioned above, so it will not be described in detail.

[0049] The server of this embodiment is a device that provides computing services, generally referring to a computer with high computing power that is provided to multiple consumers through a network. The server of this embodiment includes: a memory, a processor, and a system bus, wherein the memory includes executable programs stored thereon. It can be understood by those skilled in the art that the terminal device structure of this embodiment does not constitute a limitation on the terminal device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0050] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing of the terminal by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, a numerical calculation function, etc.), etc.; the data storage area can store data created according to the use of the terminal (such as image data, text data, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0051] The memory includes an executable program of a rapid modeling method based on three-dimensional dot matrix rendering, the executable program can be divided into one or more modules / units, the one or more modules / units are stored in the memory and executed by the processor to complete the information acquisition and implementation process, the one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the server. For example, the computer program can be divided into a modeling framing module 10, a centerline rendering module 20, a scaling module 30, a rendering and filling module 40, and a data conversion module 50.

[0052] The processor is the control center of the server, which uses various interfaces and lines to connect various parts of the entire terminal device. It executes various functions of the terminal and processes data by running or executing software programs and / or modules stored in the memory, and calling data stored in the memory, thereby monitoring the terminal as a whole. Optionally, the processor may include one or more processing units; preferably, the processor may integrate an application processor and a graphics processor GPU, wherein the application processor mainly processes operating systems, applications, etc., and the graphics processor GPU is mainly used for parallel computing acceleration of deep learning models. It is understandable that the above-mentioned graphics processor GPU may not be integrated into the processor.

[0053] The system bus is used to connect the various functional components inside the computer. It can transmit data information, address information, and control information. Its types can be, for example, PCI bus, ISA bus, VESA bus, etc. The processor's instructions are transmitted to the memory through the bus, and the memory feeds back data to the processor. The system bus is responsible for the data and instruction exchange between the processor and the memory. Of course, the system bus can also be connected to other devices, such as network interfaces, display devices, etc.

[0054] The server should at least include a CPU, a chipset, a memory, a disk system, etc., and other components will not be described in detail here.

[0055] In an embodiment of the present invention, the executable program executed by the processor included in the terminal is specifically: a rapid modeling method based on three-dimensional dot matrix rendering, which includes the following steps:

[0056] Construct a three-dimensional dot matrix space model and perform 360-degree clockwise scanning of the target object;

[0057] Extract the first frame of the target object scanning video as the reference image, filter and remove the background color, take the center line pixels as the reference line, and render the center line to the center line of the dot matrix space;

[0058] Decompose the scanned video into frames and scale all frames to be exactly the same as the reference image.

[0059] Extract the pixels of the next frame image corresponding to the previous frame baseline and render them to fill them to the right of the center line of the dot matrix. Continue in this way, with one pixel as the vertical column width and the center line as the reference to fill the dot matrix space 360 ​​degrees counterclockwise;

[0060] Translate the dot matrix structure data to generate a glTF file.

[0061] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0062] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0063] Those of ordinary skill in the art will appreciate that the modules, units and / or method steps of the various embodiments described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0064] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A rapid modeling method based on three-dimensional dot matrix rendering, characterized in that: The steps include: Construct a three-dimensional dot matrix space model and perform 360-degree clockwise scanning of the target object; Extract the first frame of the target object scanning video as the reference image, filter and remove the background color, take the center line pixels as the reference line, and render the center line to the center line of the dot matrix space; Decompose the scanned video into frames and scale all frames to be exactly the same as the reference image. Extract the pixel dot matrix with a width of one pixel to the right of the center line of the previous frame in the next adjacent frame, and render and fill it to the right of the center line of the dot matrix. Continue in this way, and fill the dot matrix space 360 ​​degrees counterclockwise based on the center line of the reference photo; Translate the lattice structure data to generate a glTF file; The scaling of all frames to be completely consistent with the reference image includes the following: At least three marker points capable of constructing a plane space are taken from the reference image, and the pixel change characteristic values ​​within the region of the three marker points are calculated; All pixel values ​​of adjacent frames are indexed and compared, and regions of adjacent frames with the same pixel change characteristic values ​​are retrieved; Calculate the relative ratio of the area between two frames, and scale the adjacent frames accordingly; The scaled frame is used to scale the next adjacent frame, and so on until all frames are processed.

2. A rapid modeling method based on three-dimensional dot matrix rendering according to claim 1, characterized in that: The three-dimensional dot matrix space model is divided at the pixel level, and every four pixels form a triangular pyramid space structure composed of three isosceles right triangles as the minimum element.

3. The rapid modeling method based on three-dimensional dot matrix rendering according to claim 2, characterized in that: The triangular pyramid space structure is obtained by performing two diagonal bevel cuts on a cubic pixel structure. One cubic pixel structure can be decomposed into four triangular pyramid space structures of the same size.

4. The rapid modeling method based on three-dimensional dot matrix rendering according to claim 1, characterized in that: The Y-axis coordinate of at least one of the three points is different from the Y-axis coordinates of the other two points, and the differences between the X-coordinates of the three points do not exceed 100 pixels.

5. The rapid modeling method based on three-dimensional dot matrix rendering according to claim 1, characterized in that: The method of filling the dot matrix space 360 ​​degrees counterclockwise based on the midline also includes the following contents: After all frames are processed in sequence, the pixel line taken out of the last frame is rendered just to the right of the original pixel line in the dot matrix space, forming a closed loop.

6. The rapid modeling method based on three-dimensional dot matrix rendering according to claim 1, characterized in that: In the process of rendering and filling dot matrix, different filling strategies are selected according to the difference in pixel color depth value. If the color difference is less than the standard value, the complete cube space range composed of four minimum triangular pyramid units is filled; if the color difference is greater than the standard value, a single cone space range is filled on the side close to the adjacent pixel.

7. A rapid modeling system based on three-dimensional dot matrix rendering, characterized in that: Includes the following functional modules: The modeling and framing module is used to construct a three-dimensional dot matrix space model and perform 360-degree clockwise scanning shooting and framing of the target object; The centerline rendering module is used to extract the first frame of the target object scanning video as the reference picture, filter and remove the background color, take the centerline pixels as the reference line, and render the centerline to the center line of the dot matrix space; The scaling module is used to decompose the scanned video into frames and scale all frames to be exactly the same as the reference image. The rendering and filling module is used to extract the pixel dot matrix with a width of one pixel to the right of the center line position of the previous frame in the adjacent next frame image, render and fill it to the right of the center line of the dot matrix, and in this way, the dot matrix space is filled 360 degrees counterclockwise based on the center line of the reference photo; The data conversion module is used to translate the dot matrix structure data and generate a glTF file; The scaling of all frames to be completely consistent with the reference image includes the following: At least three marker points capable of constructing a plane space are taken from the reference image, and the pixel change characteristic values ​​within the region of the three marker points are calculated; All pixel values ​​of adjacent frames are indexed and compared, and regions of adjacent frames with the same pixel change characteristic values ​​are retrieved; Calculate the relative ratio of the area between two frames, and scale the adjacent frames accordingly; The scaled frame is used to scale the next adjacent frame, and so on until all frames are processed.

8. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the rapid modeling method based on three-dimensional dot matrix rendering as claimed in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the rapid modeling method based on three-dimensional dot matrix rendering as claimed in any one of claims 1 to 6 are implemented.

Images