Coding scheme of three-dimensional model data

Abstract

The invention discloses a coding scheme of three-dimensional model data, which is characterized in that the three-dimensional model data and instruction information are coded into a two-dimensional image, and the three-dimensional model data are expressed in a two-dimensional matrix mode of the image. The method is characterized in that a coded two-dimensional image has a complete data format and comprises an identification field, a header file, a data segment, an end character and the like, the image has lossless source data, and original three-dimensional model data can be restored.

Description

Technical Field

[0001] The present application relates to encoding of three-dimensional model data, and in particular to an encoding scheme for three-dimensional model data. Background Art

[0002] Three-dimensional model data refers to data that contains complete geometric information in a real three-dimensional scene. Compared with the two-dimensional data we are familiar with, 3D data has depth information, so it has become the key to achieve high-precision three-dimensional reconstruction and light field display. The expression of three-dimensional model data can be roughly divided into two forms: explicit expression and implicit expression. Among them, explicit expression uses gridded data to express 3D scene information by dividing the space into three dimensions and using volume grids for display. The gridded data is generally in the form of point clouds, patches, and voxels, and the data types are .3ds, .ply, .obj, .stp, etc. Implicit expression mostly uses deep learning networks to store three-dimensional scene information through fully connected network layers or Gaussian functions, spherical harmonic functions, etc., and then only needs to input the position and perspective to obtain the rendering at a specific position and perspective through the network. The data types are mostly network structures such as .npz and .zip. However, no matter what form of expression of three-dimensional model data, it cannot be transmitted using high-speed streaming data due to the limitation of data types.

[0003] The traditional data transmission method for 3D model data greatly limits the data transmission rate and transmission path. The traditional transmission method, that is, transferring 3D model data to the memory card of the light field display through USB, network, etc., has a slow rate and is limited by the format type of the 3D model data. It is impossible to use high-speed streaming data transmission, that is, streaming transmission. Streaming transmission is currently only applicable to the transmission of specific streaming media format data, such as .JPEG, .BMP, .PNG, .H26x, .MPEG, etc., and cannot transmit 3D model data types.

[0004] In view of the transmission problem of 3D model data and the demand for user-defined data, it is urgent to design a new 3D data encoding scheme for streaming data transmission to solve the problem that 3D model data cannot be transmitted due to format type restrictions during transmission. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a coding scheme for 3D model data, which encodes 3D model data, instruction information, format identifier, etc. into a 2D coded image, and expresses the 3D model data in the form of a 2D matrix of the image. The 2D coded image has a complete data format, including an identification field, a header file, a data segment, and a terminator, and the image has lossless source data, and can restore the original data (such as 3D model data, instruction information, data format, etc.).

[0006] The three-dimensional model data supports any format type, including common three-dimensional data types such as .3ds, .ply, .obj, .stp, neural network, etc., and also includes user-defined instruction information, such as zoom in and out instructions, data modification and other operation instructions, execution instructions, etc.; the encoded image output is a standard image format, which is a lossless data format type, including .png and .bmp, etc.; the encoded image includes fields or identifiers for data identification, such as identification fields, header files, data segments, and terminators. The identification field is a field that distinguishes general two-dimensional images from three-dimensional model data. By identifying the identification field, the effective application of the three-dimensional model data carrier is ensured.

[0007] The data identification fields or identifiers include identification fields, header files, data segments, and terminators; they may also include some of the fields or identifiers therein, such as identification fields, data segments, etc.; they may also include but are not limited to identification fields, header files, data segments, and terminators, such as adding new data identification fields or identifiers.

[0008] The identification field is a field that distinguishes the coded image of the general two-dimensional image from the three-dimensional model data. By identifying the identification field, the effective application of the coded image is ensured. The header file corresponds to the header file of the coded three-dimensional model data format. For user-defined data, the header file gives the corresponding field definition.

[0009] The data segment maps the three-dimensional model data and instruction information into one-dimensional storage data, and then maps the one-dimensional storage data into two-dimensional image data, and reserves the identification field and header file for storing the three-dimensional model data in the first row of pixel array; the terminator identifies the starting range of the data segment of the three-dimensional model data to ensure the integrity of the three-dimensional model data.

[0010] The encoding scheme of 3D model data can be applied to the transmission of 3D data and user-defined data. The transmission system includes a host computer encoding module, a data transmission module, and a slave computer decoding module.

[0011] The host computer encoding module is connected to the data transmission module through a transmission interface, calls the software-side encoding software, encodes the three-dimensional model data and instruction information into a coded image through the encoding principle, and forwards the coded image to the transmission interface.

[0012] The data transmission module is connected to the host computer encoding module through a transmission interface, reads the encoded image from the transmission interface, and forwards the data to the lower computer decoding module. The data transmission module receives the encoded image and forwards the encoded image losslessly to the decoding module. The data forwarding form can be a traditional data transmission method such as network and USB, or it can be transmitted by calling a streaming data transmission interface such as HDMI and DP.

[0013] The lower computer decoding module is connected to the data transmission module and is used to decode and restore the original three-dimensional model data and instruction information to complete the data decoding work. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A general schematic diagram of a coding scheme for three-dimensional model data for this application;

[0015] Figure 2 This is a general flow chart of an implementation example of a coding scheme for three-dimensional model data of this application;

[0016] Figure 3 A coding and decoding principle diagram of a coding scheme for three-dimensional model data of this application;

[0017] Figure 4 A coding and decoding flow chart of a coding scheme for three-dimensional model data of this application;

[0018] Figure 5 A schematic diagram of the working of an encoding module of an encoding scheme for three-dimensional model data of the present application;

[0019] Figure 6 A schematic diagram of the decoding module of a three-dimensional model data encoding scheme of the present application; DETAILED DESCRIPTION

[0020] The present application is further described in detail below through specific implementation methods in conjunction with the accompanying drawings.

[0021] refer to Figure 2 As shown, one embodiment of the present invention is a coding scheme for three-dimensional model data, characterized in that:

[0022] The host computer 22 calls the software encoding module 23 to encode the 3D model data 21 into an ordered image sequence 25 through the encoding principle. The ordered image sequence is the sorted encoded image. Then, the data transmission module 24 is called to transmit the encoded image to the data transmission module at high speed and losslessly. The data transmission module specifically includes data lossless reception and data lossless forwarding.

[0023] The data transmission module 24 reads the coded image from the data transmission interface, stores the data losslessly in the high-speed graphics cache, and then organizes the data to prevent data pollution; then calls the data interface through the FrameBuffer frame buffer technology to send the coded image losslessly to the lower computer decoding module. The data transmission module is connected to the lower computer decoding module and is responsible for losslessly transmitting the coded image to the decoding unit to complete the lossless transmission of the coded image.

[0024] The lower computer decoding module 27 is connected to the data transmission module 24. The decoding module reads the coded image forwarded losslessly by the data transmission module, and then stores the data in the high-speed file cache 28, performs data matching and data decoding through the coding principle, restores the original 3D model data 29, and completes the data decoding work.

[0025] refer to Figure 3 As shown in the figure, this coding principle provides a solution of "losslessly encoding 3D model data into a 2D image sequence for data transmission" and is successfully implemented. Under the premise of ensuring confidentiality and data integrity, the coding module can generate a coded image through this principle and send the coded image to the decoding module through the data transmission module. The data transmission module can support high-speed and lossless data transmission, and the decoding module can read the coded image through this principle and decode and restore the original 3D model data.

[0026] Furthermore, the mapping principle is that the three-dimensional model data 301 is fixed as a one-dimensional data format 302 in the computer storage chip, and its storage type is char, and the image pixel value type is uchar. The binary description of char and uchar based on hardware storage is consistent, and the number of bits of both is the same. Therefore, through binary data conversion, the three-dimensional model data can be losslessly compressed into one-dimensional storage data, and then the one-dimensional data is mapped 303 to the two-dimensional pixel matrix of the 4K image, and then the 4K image 304 is filled in the "from left to right, from top to bottom" mapping method. Since the image includes three channels of RGB, the data needs to be read and written with a step size of 3 bytes when mapping the data, where each step size corresponds to the pixel data 305 of the three channels of R, G, and B.

[0027] Furthermore, the encoding module needs to add fields or identifiers including data identification in the encoded image to ensure data uniqueness and security. The header includes an identification field, header file, data segment and terminator, which is conducive to the subsequent reception and decoding of a specific header.

[0028] refer to Figure 3 As shown, the encoding principle includes the following features: a header 1306 storing the file type of the 3D model data, a header 3207 storing the encryption sequence, and N data slices storing the original 3D model data.

[0029] The flag header 1 reserves 300 bytes, including 100 corresponding RGB values. The first 30 bytes are used to identify the field, which is used to distinguish between a general two-dimensional image and a coded image, and can determine whether the image is a coded image; the following 270 bits are used to record the header file of the three-dimensional model data, which corresponds to the header file of the coded three-dimensional model data format. For user-defined instruction information, the header file gives the corresponding field definition.

[0030] The 100-byte length reserved in the marker 2 is used to record the terminator, which identifies the starting range of the data segment of the 3D model data, ensures the integrity of the 3D model data, and is used for the orderly arrangement of the encoded data and data recovery. When checking the image data marker, the terminator of the marker 2 is checked first. Only when the marker 2 matches correctly, the key information of the marker 1 is matched, and the image data is binary demapped 308 to restore the original 3D model data 309, otherwise the image data is cleared.

[0031] refer to Figure 4 As shown, the above coding principle is implemented through a specific coding and decoding process, and the coding process 401 includes the following steps: first, read the three-dimensional model data 402, and then read the storage data 403, the file type 404 and the source data 405 according to the model data. Next, based on the storage model, the data is divided into N pieces 406; based on the file type, the configuration information of the logo header 1 is constructed with reference to the above coding principle 407; based on the established configuration information, the configuration information of the logo header 2 is constructed with reference to the above coding principle 408; finally, the divided data pieces are sorted 409 and filled after the logo headers 1 and 2, and a sequence of N coded images 410 is constructed. Conversely, the decoding process 411 includes the following steps: first, read the N encoded image sequences 412 in order, and match the image's header 2 configuration information 413; after the match is successful, open up storage space 415, and according to the read header 1 configuration information 314, zero-copy the data slices in order to the storage array 416; the N encoded images are processed in turn, and the N data slices are read into the opened storage space, completing the data combination 417, and restoring the three-dimensional model data 418.

[0032] refer to Figure 5 As shown, the encoding module code runs on the host computer to complete the above data encoding and data transmission work. When the host computer runs the encoding module code and selects the 3D model data encoding transmission mode, the host computer has the function of high-speed encoding of the 3D model data and transmits the encoded image to the data transmission module.

[0033] Furthermore, the process is as follows: the first step S501 is to select the 3D model data encoding transmission mode, open the 3D model data, and send the 3D model data storage file path to the encoding software; the second step S502 is to build a large-capacity array binary storage 3D model data; the third step S503 is to build a thread pool and create multiple threads according to the above encoding principle, and re-arrange the encoding of the 3D model data at the same time; the fourth step S504 is to send the encoded image to the data transmission module through the data transmission interface in sequence according to the encoded image number, and the data transmission module completes the transmission work.

[0034] refer to Figure 6 As shown, the decoding module code runs on the lower computer to complete the above data reception and data decoding work.

[0035] Furthermore, the process is as follows: in the first step S601, the lower computer starts the decoding software automatically when it is turned on, and captures the data transmission module interface data to the image frame cache in real time through the VideoCapture framework of the opencv library; in the second step S602, according to the above-mentioned encoding principle, quickly determine whether the image frame cache data is a coded image: if it is a coded image, enter the process S604; otherwise, clear the cache S603 and return to the process S601; in the third step S604, open another asynchronous thread to prevent the decoding time from being too long and affecting the effective data capture. The asynchronous thread opens a large-capacity array to store the image frame cache effective data; in the fourth step S605, the software detects the image frame cache data type. If it jumps from a coded image to ordinary image data, the large-capacity array data is written into the file to restore the original three-dimensional model data and complete the decoding work.

Claims

1. A coding scheme for 3D model data, which encodes 3D model data, instruction information, etc. into a 2D coded image, and expresses the 3D model data in a 2D matrix of an image. It is characterized in that: The two-dimensional coded image has a complete data format, including identification fields, header files, data segments, and terminators, and the image has lossless source data and can restore the original data (such as three-dimensional model data, instruction information).

2. The coding scheme for three-dimensional model data according to claim 1, characterized in that: The three-dimensional model data supports any format type, including common three-dimensional data types such as .3ds, .ply, .obj, .stp, neural networks, etc., and also includes user-defined instruction information, such as zoom in and out instructions, data modification and other operation instructions, execution instructions, etc.; the encoded image output is a standard image format, which is a lossless data format type, including .png and .bmp, etc.; the encoded image includes fields or identifiers for data identification, such as identification fields, header files, data segments, and terminators.

3. The coding scheme for three-dimensional model data according to claim 2, characterized in that: The data identification fields or identifiers include identification fields, header files, data segments, and terminators; they may also include some of the fields or identifiers therein, such as identification fields, data segments, etc.; they may also include but are not limited to identification fields, header files, data segments, and terminators, such as adding new data identification fields or identifiers.

4. The coding scheme for three-dimensional model data according to claim 2, characterized in that: The identification field is a field that distinguishes the coded image of the general two-dimensional image from the three-dimensional model data. By identifying the identification field, the effective application of the coded image is ensured. The header file corresponds to the header file of the coded three-dimensional model data format. For user-defined data, the header file gives the corresponding field definition.

5. The coding scheme for three-dimensional model data according to claim 2, characterized in that: The data segment maps the three-dimensional model data and instruction information into one-dimensional storage data, and then maps the one-dimensional storage data into two-dimensional image data, and reserves the identification field and header file for storing the three-dimensional model data in the first row of pixel array; the terminator identifies the starting range of the data segment of the three-dimensional model data to ensure the integrity of the three-dimensional model data.

6. The coding scheme for three-dimensional model data according to claim 2, characterized in that: The encoding scheme of 3D model data can be applied to the transmission of 3D data and user-defined data. The transmission system includes a host computer encoding module, a data transmission module, and a slave computer decoding module.

7. The coding scheme for three-dimensional model data according to claim 6, characterized in that: The host computer encoding module is connected to the data transmission module through a transmission interface, calls the software-side encoding software, encodes the three-dimensional model data and instruction information into a coded image through the encoding principle, and forwards the coded image to the transmission interface.

8. The coding scheme for three-dimensional model data according to claim 6, characterized in that: The data transmission module is connected to the host computer encoding module through a transmission interface, reads the encoded image from the transmission interface, and forwards the data to the lower computer decoding module. The data transmission module receives the encoded image and forwards the encoded image losslessly to the decoding module. The data forwarding form can be a traditional data transmission method such as network and USB, or it can be transmitted by calling a streaming data transmission interface such as HDMI and DP.

9. The coding scheme for three-dimensional model data according to claim 6, characterized in that: The lower computer decoding module is connected to the data transmission module and is used to decode and restore the original three-dimensional model data and instruction information to complete the data decoding work.

Images