A method, device, system and computer storage medium for creating virtual objects

Abstract

The present invention discloses a method for creating a virtual object, comprising: receiving three-dimensional point cloud data and color image data obtained by three-dimensionally photographing a target object; converting the three-dimensional point cloud data into a virtual object model and converting the color image data into a virtual object texture, and then splicing the two to generate a three-dimensional image of the target object; performing initial attribute configuration on the three-dimensional image of the target object; rendering the three-dimensional image of the target object after the initial attribute configuration to obtain the image of the virtual object to be created. The present invention also discloses a device for creating a virtual object, a computer system for creating a virtual object, and a computer-readable storage medium for creating a virtual object.

Description

Technical Field

[0001] The present invention relates to the field of game technology, and in particular to a game model generation method, device, system and computer storage medium. Background Art

[0002] The game described in the present invention refers to various game programs that can be used by a single machine or multiple machines connected to a network using intelligent terminals such as computers, game consoles, smart phones, tablet computers, AR / VR devices, etc., and a single user or multiple users can use the game program to fight, compete, guess, increase skill levels, etc. The game model described in the present invention is a virtual character, virtual pet, and virtual prop or item with a certain definition in the game, which is used to realize a certain definition of the virtual world role-playing in the game.

[0003] However, in the existing technology, game characters are pre-defined by game developers and stored in the game program. Game developers and their designers need a lot of preparation work before designing, such as creativity and plot, and then prototype design. Concept designers give a flat character concept design that can show the character's body shape, characteristics, and general personality. These concept drawings are further refined and used to provide references for 3D designers using similar three-view drawings (front, side, back). The 3D designer then uses the computer to perform steps such as medium-mode production, high-mode production, and low-mode production to build a three-dimensional model of the game character. Medium-mode production refers to the use of conventional 3D software to generate a model with a medium number of faces; high-mode production refers to the use of specific 3D software to carve the key details of the model to make it more realistic and conform to the normal structural form of the organism; low-mode production requires reducing the number of faces on the basis of the medium model to avoid jamming and downtime caused by the functional limitations of the game device during operation. After the model is made, it is necessary to distribute the UV map reasonably. This step is to bake the normal, including the need to split the UV, adjust the matching degree of the high and low models, and then enter the baking map. Map drawing is a key step in the generation of game characters. It is necessary to select appropriate map materials according to the game style. This requires designers with artistic skills to create, so as to provide the performance and texture of the game. After the model and map are prepared, for the inactive ones, they can basically be used by the game program at this time; but for the active ones, it is necessary to enter the bone binding again, so that the characters in the game are wired according to the muscle direction and joint position in accordance with the organism, which is natural and comfortable, and there will be no anti-space anti-matter structure in the game. The last step is animation adjustment, which requires not only knowledge of human anatomy, bone science, and biology, but also ensures the clarity and rationality of the parent-child relationship of the bones, especially the repeated inspection of the bones involved in the skin; usually one bone is often used on multiple objects, so the binding production is strictly carried out according to the animation process. After completing this step, the entire game character (game character or game pet, etc.) can be imported into the engine for adjustment to see the final effect.

[0004] In the existing technology, professional designers need to perform 3D modeling on computers, which not only places high demands on the functions of the game equipment, but also the image and style of the game characters obtained by pre-3D modeling are difficult to meet the requirements of game players. In addition, we can see that in the existing technology, game developers and designers are developing games. Whether it is the generation of game characters, it is not a simple matter and requires huge resources. This makes the development cost of game models high and the development cycle long.

[0005] In the virtual world, users cannot feel the space, and all objects are homogeneous. For example, any face and image in the game are virtual objects, which have nothing to do with our real world. Such people's sense of identity is often vague. Therefore, by creating sci-fi tasks and environments, and through various parts transformation and adjustment, users and players can feel more immersed. However, no matter how powerful this virtual object is, it still lacks some identity recognition, and the connection with the user himself is not real enough. Summary of the invention

[0006] In response to the above defects or improvement needs of the prior art, the present invention provides a method for creating a virtual object, including: receiving three-dimensional point cloud data and color image data obtained by three-dimensionally photographing a target object; converting the three-dimensional point cloud data into a virtual object model and converting the color image data into a virtual object texture, and then splicing the two to generate a three-dimensional image of the target object; performing initial attribute configuration on the three-dimensional image of the target object; and rendering the three-dimensional image of the target object after the initial attribute configuration to obtain an image of the virtual object to be created.

[0007] Preferably, the method for creating a virtual object further comprises: adjusting the image of the virtual object to be created to obtain a final image of the virtual object.

[0008] Preferably, the method for creating a virtual object further comprises: determining the initial attributes of the virtual object to be created based on existing vacant virtual objects, and entering the virtual object creation screen.

[0009] Preferably, the target object is photographed in three dimensions to obtain three-dimensional point cloud data and color image data of the target object.

[0010] Preferably, the target object is photographed in three dimensions to obtain depth data and image data of the target object, and the three-dimensional point cloud data is restored based on calibrated intrinsic parameters and the color image data is restored based on corrected intrinsic parameters.

[0011] Preferably, the corresponding compensation data is stored in the three-dimensional shooting device in advance, and then the image data collected in real time is corrected and compensated, so as to restore the color image data.

[0012] Preferably, the three-dimensional point cloud data is restored by storing corresponding error compensation in the three-dimensional shooting device in advance and performing coordinate transformation on the depth map after corresponding correction and compensation.

[0013] Preferably, the calibrated intrinsic parameters are written in advance at designated locations in the 3D shooting device, and then the data collected in real time is compensated by calling the intrinsic parameters of the 3D shooting device, thereby restoring the 3D point cloud data with lower geometric errors.

[0014] Preferably, lens uniformity correction parameters are pre-written at a designated position in the three-dimensional shooting device to eliminate the influence of the lens system on brightness uniformity and color uniformity in the image data on image quality.

[0015] Preferably, the fixed noise of the 3D shooting device is corrected and compensated in advance before the temperature drift correction and compensation is performed to eliminate the reduction in ranging accuracy of the 3D shooting device due to self-heating and external temperature changes.

[0016] Preferably, a one-to-one mapping relationship is established between the texture of the virtual object obtained through the color image data and the three-dimensional model, and based on the mapping relationship, the color image data and the three-dimensional model are surface fitted to obtain the three-dimensional image of the target object.

[0017] Preferably, a three-dimensional rectangular coordinate system is established with a certain characteristic point of the point cloud body of the three-dimensional point cloud data as the coordinate origin, other characteristic values ​​of the point cloud body are corresponded to the three-dimensional rectangular coordinate system, and the point cloud body is divided into a number of surface patches, and the three-dimensional model with geometric characteristic curves is obtained after curve fitting.

[0018] Preferably, the initial attribute is one or more of pre-designed virtual object type parameters, virtual object function parameters and virtual object logic parameters.

[0019] On the other hand, the present invention provides a device for creating a virtual object, including: a model generation module, which is used to convert three-dimensional point cloud data obtained by three-dimensionally photographing the target object into a virtual object model and convert color image data obtained by three-dimensionally photographing the target object into a virtual object texture, and then splice the two to generate a three-dimensional image of the target object; an application module, which is used to perform initial attribute configuration on the three-dimensional image of the target object; and a graphics engine module, which is used to render the three-dimensional image of the target object after the initial attribute configuration to obtain the virtual object image to be created.

[0020] Preferably, the application module is further used to adjust the image of the virtual object to be created to obtain a final virtual object image.

[0021] Preferably, the device for creating a virtual object further comprises: a 3D imaging device for photographing the target object in three dimensions to obtain three-dimensional point cloud data and color image data of the target object.

[0022] Preferably, the 3D imaging device performs three-dimensional photography of the target object to obtain depth data and image data of the target object, and restores the three-dimensional point cloud data based on calibrated intrinsic parameters and restores the color image data based on corrected intrinsic parameters.

[0023] Preferably, the 3D imaging device uses pre-stored corresponding compensation data to correct and compensate the image data collected in real time, thereby restoring the color image data.

[0024] Preferably, the 3D imaging device restores the three-dimensional point cloud data by pre-stored corresponding error compensation and coordinate transformation of the depth map after corresponding correction and compensation.

[0025] Preferably, the 3D imaging device writes the calibrated intrinsic parameters in a designated position in advance, and then compensates the data collected in real time by calling the intrinsic parameters, thereby restoring the three-dimensional point cloud data with lower geometric errors.

[0026] Preferably, the 3D imaging device writes lens uniformity correction parameters in advance at a designated position to eliminate the influence of the lens system on brightness uniformity and color uniformity on image quality in the image data.

[0027] Preferably, the 3D imaging device pre-corrects and compensates for fixed noise and then performs temperature drift correction and compensation to eliminate the reduction in ranging accuracy due to self-heating and external temperature changes.

[0028] Preferably, the model generation module obtains the virtual object texture through the color image data and establishes a one-to-one mapping relationship with the three-dimensional model, and based on the mapping relationship, performs surface fitting on the color image data and the three-dimensional model to obtain the three-dimensional image of the target object.

[0029] Preferably, the model generation module uses a characteristic point of the point cloud body of the three-dimensional point cloud data as the coordinate origin, establishes a three-dimensional rectangular coordinate system, corresponds other characteristic values ​​of the point cloud body to the three-dimensional rectangular coordinate system, and divides the point cloud body into a number of surface patches, and obtains the three-dimensional model with geometric characteristic curves after B-spline curve fitting.

[0030] Preferably, the initial attribute is one or more of pre-designed virtual object type parameters, virtual object function parameters and virtual object logic parameters.

[0031] The present invention further provides a computer system for creating virtual objects, the system comprising: a processor; a computer-readable medium; and computer program instructions stored on the computer-readable medium for execution by at least one of the processors, the computer program instructions comprising computer program instructions for executing each step of any of the above methods.

[0032] The present invention further provides a computer-readable storage medium for creating virtual objects, wherein the computer-readable storage medium stores at least one executable computer program instruction, and the computer program instruction includes computer program instructions for executing each step of any of the above methods.

[0033] According to the game model generation method of the present invention, the game model can be quickly generated according to the parameters predefined by the game developer and designer, which greatly facilitates the designers, player users, etc., and also enables the designers, player users, etc. to quickly generate the game model according to their actual needs. In addition, according to the game model generation method of the present invention, game developers and designers do not need to invest huge resources in the process of developing games, reduce the high development cost of the game model, shorten the long development cycle, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the structure of a virtual object creation system according to an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of the structure of a user terminal device according to an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of the structure of a control device of a user terminal device according to an embodiment of the present invention;

[0037] Figure 4 A flowchart of creating a virtual object according to an embodiment of the present invention;

[0038] Figure 5 A flowchart of creating a virtual object image in step S6 of the present invention;

[0039] Figure 6 A schematic diagram of an embodiment of a virtual object creation device of the present invention;

[0040] Figure 7 FIG. 4 is a schematic diagram of another embodiment of the virtual object creation device of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0042] The application mentioned in the present invention may be a game application, a virtual community application, etc., that is, an application for achieving the purpose of the present invention, and the embodiment of the present invention does not limit this. The virtual object mentioned in the present invention may be any three-dimensional target object collected and generated in real time by a 3D imaging device, such as a person, a head, a face, a hand, a cat, a dog, a water cup, a car, etc., and the embodiment of the present invention does not limit this. The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0043] Figure 1 FIG. 1 is a schematic diagram of the structure of a virtual object creation system according to an embodiment of the present invention. Figure 1 As shown, the virtual object creation system includes a user-end device 1 and a background server 2, wherein the user-end device 1 is used for the user to create a virtual object image from a real target object according to his or her own expectations, and the background server 2 is used to perform data communication based on the virtual object image created by the user-end device 1 and provide corresponding services.

[0044] Figure 2 FIG. 1 is a schematic diagram of the structure of a user terminal device according to an embodiment of the present invention. Figure 2 As shown, the user terminal device 1 includes a target object creation device 11, a 3D imaging device 12, a storage device 13 and a display device 14. The target object creation device 11, the 3D imaging device 12, the storage device 13 and the display device 14 are connected via an electronic communication line (whether wired or wireless).

[0045] The 3D imaging device 12 is used to photograph the target object in three dimensions, obtain depth data and image data of the target object, restore three-dimensional point cloud data based on calibrated intrinsic parameters and restore color image data based on corrected intrinsic parameters, and output the three-dimensional point cloud data and color image data.

[0046] In this embodiment, preferably, the 3D imaging device 12 actively transmits a light signal to the target, and can obtain the depth data of the target by using one of a time-of-flight method, a speckle structured light method, a coded structured light method, etc. More preferably, the 3D imaging device 12 actively transmits a light signal to the target and receives a light signal reflected by the target, and obtains the depth data of the target by calculating the flight time of the light signal.

[0047] In this embodiment, preferably, the optical signal emitted by the 3D imaging device 12 to the target is an infrared low-frequency optical signal or a radar high-frequency optical signal.

[0048] In this embodiment, preferably, the above-mentioned 3D imaging device 12 is used to shoot the target object in real time, obtain the depth data and image data of the above-mentioned target object, restore the three-dimensional point cloud data based on the calibrated internal parameters and restore the color image data based on the corrected internal parameters, and output the above-mentioned three-dimensional point cloud data and the above-mentioned color image data.

[0049] In this embodiment, preferably, the above-mentioned 3D imaging device can be externally or internally mounted in the user terminal device 1, including but not limited to data communication via USB protocol, LVDS protocol, MIPI protocol, etc.

[0050] The target object creation device 11 is used to create a virtual object image based on the three-dimensional point cloud data and color image data of the target object. The target object creation device 11 includes a model generation module 111, an application module 112, and a graphics engine module 113, and the model generation module 111, the application module 112 and the graphics engine module 113 are connected to each other in communication.

[0051] The model generation module 111 is used to convert the three-dimensional point cloud data into a virtual object model and the color image data into a virtual object texture, and then splice the two to generate a three-dimensional image of the target object, and the application module 112 is used to perform initial attribute configuration on the three-dimensional image of the target object. The application module 112 imports the three-dimensional image of the target object after the initial attribute configuration into the graphic engine module 113 to obtain the image of the virtual object to be created. The virtual object texture includes information such as the outline, color, and material of the virtual object. The application module 112 decides whether to adjust the image of the virtual object to be created according to actual needs, and outputs a virtual object image that meets the user's aesthetics. The graphic engine module 113 is used to render the three-dimensional image of the target object after the initial attribute configuration imported by the application module 112, so as to obtain the image of the virtual object to be created.

[0052] The storage device 13 is used to store data related to the virtual object image to be created and / or the final virtual object image, as well as to store part or all of the instructions and related parameters during the application of the model generation module 111, the application module 112 and / or the graphic engine module 113.

[0053] The display device 14 is used to display the human-computer interaction interface of the virtual object image to be created and / or the final virtual object image and its corresponding application.

[0054] Figure 3 FIG. 1 is a schematic diagram of the structure of a control device of a user terminal device according to an embodiment of the present invention. Figure 3As shown, the control device 10 is mainly composed of a data processing controller 101, a display 102 and a keyboard 103, which include a CPU, a ROM and a RAM. The data processing controller 101 is mainly composed of a CPU 101a, a ROM 101b, a RAM 101c, a hard disk 101d, a reading device 101e, an input / output interface 101f, a communication interface 101g and a data output interface 101h. The CPU 101a, the ROM 101b, the RAM 101c, the hard disk 101d, the reading device 101e, the input / output interface 101f, the communication interface 101g and the data output interface 101h are connected to each other through a bus 101i, and can send and receive control signals and calculation data to each other.

[0055] The CPU 101 a can execute computer programs stored in the ROM 101 b and computer programs read into the RAM 101 c .

[0056] ROM101b is composed of a read-only memory, PROM, EPROM, EEPROM, etc., and stores computer programs executed by CPU101a and data used therefor. RAM101c is composed of SRAM or DRAM, etc., and is used to read computer programs stored in ROM101b and hard disk 101d. RAM101c can also be used as a workspace when CPU101a executes these computer programs.

[0057] The hard disk 101d stores various computer programs such as an operating system and application programs for the CPU 101a to execute, and data used to execute the computer programs. The application program 10a for creating game virtual objects in this embodiment is also stored in the hard disk 101d.

[0058] The reading device 101e is composed of a floppy drive, a CD-ROM drive, or a DVD-ROM drive, and can read computer programs or data stored in a portable storage medium 6. The portable storage medium 6 stores an application 10a for generating a game model, and the control device 10 can read the application 10a from the portable storage medium 6 and install it into the hard disk 101d.

[0059] The application 10a can be provided not only by the portable storage medium 6, but also downloaded through an electronic communication line from an external machine connected to the electronic communication line (whether wired or wireless) and capable of communicating with the control device 10. For example, the application 10a is stored in the hard disk of a network server, and the control center 10 can access the server, download the application 10a, and install it into the hard disk 101d.

[0060] The hard disk 101d may be installed with an operating system that provides a graphical user interface, such as Windows (registered trademark) produced by Microsoft Corporation of the United States. In the following description, the application program 10a of this embodiment is executed on the above operating system.

[0061] The input / output interface 101f is composed of serial interfaces such as USB, IEEE1394, RS-232C, parallel interfaces such as SCSI, IDE, IEEE1284, and analog signal interfaces composed of D / A converters and A / D converters, etc. The input / output interface 101f is connected to a keyboard 103, and a user can use the keyboard 103 to directly input data such as relevant parameters or execution instructions to the control device 10.

[0062] The communication interface 101g can be used to exchange information and data with the background server 2 and the 3D imaging device 12 .

[0063] The data output interface 101 h is connected to a display 102 composed of an LCD or a CRT, etc., and outputs the relevant data and / or information received from the CPU 101 a to the display 102 .

[0064] Figure 4 FIG. 1 is a flowchart of creating a virtual object according to an embodiment of the present invention. Figure 4 As shown, the user starts the virtual object creation system, and the virtual object creation system determines whether a virtual object creation request is received (step S1). If no virtual object creation request is received (step S1: No), the system continues to wait. If a virtual object creation request is received (step S1: Yes), the user information is sent to the background server 2 to determine whether the virtual object creation request meets the conditions for granting virtual object creation authority (step S2).

[0065] The above-mentioned virtual object creation authority can be decided by the application developer or operator according to their own situation whether to open the interface authority for accessing data for a fee. If it is free, the user can directly initiate the creation request of the virtual object; if it is paid, the user can initiate the creation request of the virtual object after completing the corresponding payment. The above-mentioned user information may include the user's ID information or account information. For example, the above-mentioned backend server 2 can determine whether the user has paid the relevant fees or other restrictive conditions based on the above-mentioned user information, thereby determining whether the user meets the conditions for granting the creation authority of the virtual object.

[0066] If the virtual object creation request does not meet the conditions for granting the virtual object creation authority (step S2: No), the virtual object creation is terminated directly. If the virtual object creation request meets the conditions for granting the virtual object creation authority (step S2: Yes), the user is granted the authority to create the virtual object (step S3), and the process proceeds to step S4.

[0067] In step S4, the initial attributes of the virtual object to be created are determined based on the existing vacant virtual objects, and the virtual object creation screen is entered. The above-mentioned existing vacant virtual objects can be virtual objects to be created that are pre-designed by application developers according to the needs of application content. The three-dimensional model of the virtual object to be created can be close to the three-dimensional model of the initial virtual object, and the virtual object to be created will call the data parameters of the initial virtual object, thereby giving the virtual object to be created the necessary torque parameters, motion parameters, static parameters, etc., which will be determined as the initial attributes of the virtual object to be created. For example, in a game application, the target object desired by the user is himself, that is, he wants to create a virtual object with himself as the target object, then in the game application, he should choose to create a virtual object as a vacant role of human beings. This vacant role will have pre-designed data parameters, such as human body model parameters, torque parameters, motion parameters, static parameters and other necessary data parameters. However, the data parameters of human appearance characteristics, such as appearance parameters, body shape parameters, image parameters, etc. are vacant, and these vacant data parameters are the data that the present invention needs to collect and generate in real time.

[0068] The above initial attributes are pre-designed basic data, including parameters such as virtual object type, virtual object function and virtual object logic, which can give the virtual object the most initial default state, so that the virtual object is the type of creation expected by the user and has certain functions in the application to perform certain tasks. Since the virtual object image is a form of appearance generated by the user based on the target object in reality, its characteristics do not completely satisfy the user, so the user can make secondary adjustments to the generated virtual object image according to his or her own expectations, and change the virtual object image generated by the user through the ready-made components provided in the application, so that the virtual object has a visual effect that combines virtuality and reality, and meets the user's personalized expectations.

[0069] In this embodiment, preferably, the above-mentioned initial attributes can be pre-designed virtual object type parameters. Virtual object types include but are not limited to: characters, animals, objects, or parts of something, etc.; virtual object types can also be single, can be numerous, and can vary according to different applications. In different applications, the application developer or designer will construct virtual object type parameters according to the needs of the application itself, and use them to shape the virtual object type to meet the most basic physical requirements of the application itself. For example, in an application such as a virtual community, there will be many types of virtual objects, which can be a whole character, a part of a character, various animals, various objects, etc. Users can choose the virtual object type according to their preferences, thereby creating the virtual object image to be created that the user expects.

[0070] In this embodiment, preferably, the above-mentioned initial attributes can also be pre-designed virtual object function parameters. Virtual object function parameters include but are not limited to: behavior parameters, expression parameters, tracking parameters, release skill parameters, etc. For example, in an online game, each original character will contain behavior parameters, tracking parameters, etc., and the character model will be configured with multiple fusion deformations, wherein each fusion deformation may correspond to a polygonal mesh or point cloud or any other representation of a geometric three-dimensional surface suitable for body movements or / and facial expressions, so that the original character has the ability to move; at the same time, the tracking parameters are configured, that is, the weights of the fusion deformations are included, and are weightedly combined to generate body movements or / and facial expressions, so that the original character can better realize certain functions in advance according to the requirements of the game plot setting.

[0071] In this embodiment, preferably, the above-mentioned initial attributes can also be pre-designed virtual object logic parameters. Virtual object logic parameters include but are not limited to: sequencers, selectors, circulators and randomizers, etc., which combine virtual object function parameters and conditional parameters according to a certain logical process, thereby giving virtual objects soulful or spiritual functions and making them vivid. For example, in online games, the characters in the game can move, have facial expressions and movements, and even release skills, instead of just simple appearance effects. Game characters can be similar to real people, or even have cooler effects, all of which are inseparable from the support of logical parameters, making game characters more attractive and fascinating, so user players are more interested in playing this game application.

[0072] The target object is photographed to obtain three-dimensional point cloud data and color image data of the target object (step S5). In step S5, the target object is placed within the shooting field of view, and the 3D imaging device 12 can use a three-dimensional scanning device (such as a depth camera) to perform three-dimensional photography of the target object from multiple angles to obtain depth data and image data of the target object, and restore the three-dimensional point cloud data based on the calibrated intrinsic parameters and restore the color image data based on the corrected intrinsic parameters, and output the three-dimensional point cloud data and the color image data.

[0073] During the real-time scanning of the above-mentioned targets, the above-mentioned targets may be fixed or movable, and the shooting location may be anywhere, including but not limited to the office of the game developer, the home of the game designer, the home of the game user, or the game operation location.

[0074] In this embodiment, preferably, the three-dimensional scanning device of the 3D imaging apparatus 12 emits a light signal to a target and receives a light signal reflected by the target, and obtains depth data of the target by calculating the flight time of the light signal.

[0075] In this embodiment, the depth data of the above-mentioned target object is obtained by calculating the flight time of the light signal, and the flight time can be divided into direct and indirect types. The direct flight time is based on the pulse light signal of the light being locked by the timer from the time it is emitted to the time it is reflected back and received, thereby calculating the distance information of the target; the indirect flight time is based on the sinusoidal frequency modulated light signal of the light being emitted to the time it is reflected back and received to detect the phase difference of the waveform, thereby calculating the distance information of the target. When the light is received, due to the non-uniformity of the dark current and the non-uniformity of the photon response, the ranging error of each pixel point is different. Therefore, by shooting each pixel point at different distances, the current value is calculated and compared with the actual value, and then the most direct method is to compensate the corresponding pixels according to the error value in the matrix. The disadvantage of this method is that the amount of calculation is large. Therefore, in this embodiment, preferably, the error value in the flight time of the light signal is fitted and calculated by the quadratic surface equation (g(x,y)=ax2+bxy+cy2+dx+ey+f). This way, the amount of calculation will be less, and only the six unknown parameters (a, b, c, d, e, f) need to be found, and then stored in advance and called during real-time acquisition to obtain the depth data of the above-mentioned target object.

[0076] Furthermore, for indirect time of flight, since it uses a continuous wave method, it is difficult to obtain an ideal sinusoidal harmonic in actual use. Due to hardware limitations, such as circuit delay, gain nonlinearity, etc., the transmitted signal is not a perfect sinusoidal signal. According to the theoretically calculated distance, there will be a swing error. This swing error will appear periodically in the entire measurement range. In this embodiment, before obtaining the depth data of the above-mentioned target, it also includes pre-calculating the measurement value of the position by shooting at different distances, and then comparing this measurement value with the actual distance value to obtain a swing error, and constructing an actual swing error distribution curve, and then fitting it with the theoretical swing error distribution curve through the least squares method, so as to accurately compensate for the swing error of the periodic frequency.

[0077] In this embodiment, preferably, the above-mentioned three-dimensional point cloud data is restored according to the internal parameters calibrated by the three-dimensional shooting device of the above-mentioned 3D imaging device. Due to the imaging reasons of the three-dimensional shooting device (such as a depth camera) of the above-mentioned 3D imaging device itself and the interference of the external environment, such as optical distortion, temperature shift, pixel fixed noise, etc., the measurement accuracy of the distance will be deteriorated, so it is necessary to eliminate the error introduced by the three-dimensional shooting device in advance. In the present invention, preferably, by pre-storing the corresponding error compensation in the three-dimensional shooting device, and performing coordinate conversion on the depth map after the above correction and compensation, including establishing a transformation relationship between the three-dimensional shooting device's focal length f, center coordinates (u0, v0), image plane length and width (L, W), rotation angle (R), radial distortion coefficient (k1, k2, k3, k4, k5) and other three-dimensional shooting device internal coordinates and world coordinates, the above-mentioned three-dimensional point cloud data is obtained.

[0078] In this embodiment, preferably, the color image data is restored based on the corrected internal parameters of the above-mentioned three-dimensional shooting device, and the target object is tracked to calculate the color characteristics and contour feature information of the target object. Due to the imaging reasons of the color three-dimensional shooting device (such as a color depth camera) of the above-mentioned 3D imaging device itself and external environmental interference, such as lens uniformity, color uniformity, etc., image information distortion will be caused, so the distortion caused by the three-dimensional shooting device needs to be eliminated in advance. The present invention obtains color image data by pre-storing the corresponding compensation data in the three-dimensional shooting device, and then correcting and compensating the real-time collected image data.

[0079] In this embodiment, preferably, the above-mentioned color image data can be acquired by real-time acquisition of image data by a color 3D shooting device, and the color image data can be restored according to the corrected internal parameters of the color 3D shooting device, and the target object can be tracked in real time to calculate the color characteristics and contour feature information of the target object.

[0080] In this embodiment, preferably, calibration internal parameters are written in advance at a designated position in the color 3D shooting device, and then the depth image data collected in real time is compensated by calling the internal parameters of the color 3D shooting device, so as to obtain the above-mentioned 3D point cloud data with lower geometric error. The internal parameters of the color 3D shooting device include the focal length f of the depth camera, the center coordinates (u0, v0), the length and width of the image plane (L, W), the rotation angle (R), the radial distortion coefficient (k1, k2, k3, k4, k5), etc. On the basis of the traditional Zhang Zhengyou calibration method, by shooting a calibration image of a chessboard size at different distances and angles, replacing the squares with dots, and performing histogram equalization enhancement on the grayscale image to improve the image contrast, and then performing dot center detection and parameter solution; finally, a transformation relationship is established between the camera internal parameters and the world coordinates.

[0081] In this embodiment, preferably, the parameters in the color 3D shooting device are compensated in advance and then fixed noise correction compensation is performed to eliminate the influence of the ranging error of each pixel point caused by the non-uniform response of each pixel point in the depth camera to the light signal. The error is calculated between the current matrix value collected at different distances for each pixel point and the actual value, and then the error is fitted by the quadratic surface equation to obtain the depth data of the above-mentioned target object.

[0082] In this embodiment, preferably, the fixed noise of the color 3D shooting device is corrected and compensated in advance before the temperature drift correction compensation is performed to eliminate the reduction of the ranging accuracy of the color 3D shooting device due to its own heating and external temperature changes. By establishing a one-to-one corresponding relationship between the working temperature of the depth camera in different temperature environments and the actual ranging, the compensation values ​​of different working temperatures are pre-written into the specified location, and the working temperature is detected online in real time in subsequent use, and then the corresponding compensation value is matched to obtain the depth data.

[0083] In this embodiment, preferably, the lens uniformity correction parameters are pre-written in the designated position in the color 3D shooting device to eliminate the influence of the lens system on the brightness uniformity and color uniformity in the above image data. Image data under at least one color temperature illumination is shot, and the brightness values ​​of the four corner areas are compared with the brightness values ​​of the middle area, and the brightness of the entire picture from the corner to the center is adjusted to be the same, and the brightness quality is more uniform; at the same time, the picture is divided into several areas, and then the values ​​of the R channel, B channel and G channel in each area are calculated, and the ratio of the R / G / B three-color channels in each area is made to converge to the ratio of the middle area, and the color quality is more uniform, thereby improving the image of the color camera to be more realistic.

[0084] The target object creation device 11 is used to create a virtual object image based on the three-dimensional point cloud data and color image data of the target object (step S6).

[0085] Determine whether there is a new virtual object image to be created (step S7). If there is a new virtual object image to be created (step S7: yes), return to step S4 and perform a new round of virtual object image creation. If there is no new virtual object image to be created (step S7: no), then end the virtual object creation procedure.

[0086] Figure 5 FIG. 4 is a flow chart of creating a virtual object image in step S6 of the present invention. Figure 5As shown, the model generation module 111 receives the three-dimensional point cloud data and the color image data output by the 3D imaging device 12 (step S61), converts the three-dimensional point cloud data into a three-dimensional model of a virtual object and converts the color image data into a texture of a virtual object, and then performs splicing processing to generate a three-dimensional image of the target object (step S62).

[0087] In this embodiment, preferably, the random noise removal and abnormal point elimination of the three-dimensional point cloud data are realized by using a bilateral filtering algorithm, and then the damage in the point cloud body is repaired by using the least square method, and the point cloud body is constructed by using surface fitting (such as B-spline surface fitting). The spatial geometric modeling of the target object is constructed by the surface modeling method, and a certain feature point of the point cloud body is used as the coordinate origin, thereby establishing a three-dimensional rectangular coordinate system, and other feature values ​​of the point cloud body are corresponding to this three-dimensional rectangular coordinate system. The point cloud body is divided into a number of surface pieces, and a three-dimensional model with a geometric feature curve is obtained after curve fitting (such as B-spline surface fitting). In addition, a one-to-one mapping relationship is established between the texture (such as contour, color, material, etc.) of the virtual object obtained by the above-mentioned color image data of the target object and the three-dimensional model, and based on the established mapping relationship, the color image data of the target object is surface fitted with the three-dimensional model to obtain the three-dimensional image of the above-mentioned target object that is closest to the target object in reality.

[0088] The application module 112 performs initial attribute configuration on the three-dimensional image of the target object (step S63), and then the graphic engine module 113 renders the three-dimensional image of the target object after the initial configuration to obtain the virtual object image to be created (step S64).

[0089] In this embodiment, the initial attribute configuration of the three-dimensional image of the target object can be performed in the application according to the actual needs of the application. For example, in an online game, if the actual need for the game character is a human face, after the user selects the type of human face, until the three-dimensional image of the human face is generated, the generated three-dimensional image of the face is imported into the graphics engine module 113 for rendering to create a virtual object image based on the human face (i.e., the virtual object image to be created), and the visual effect of this virtual object image is displayed through the above-mentioned display device 14, and the user visually judges whether the current virtual object image meets his expectations.

[0090] Determine whether the user needs to adjust the image of the virtual object to be created (step S65). If the user believes that the current image of the virtual object to be created meets his or her expectations, the image of the virtual object to be created is not adjusted (step S65: No), and the virtual object image is directly output (step S67). If the user believes that the current image of the virtual object to be created does not meet his or her expectations and needs to adjust the image of the virtual object to be created (step S65: Yes), the system adjusts the image of the virtual object to be created to obtain the final virtual object image (step S66), and then outputs the virtual object image (step S67). The application module 112 adjusts the image of the virtual object to be created according to actual needs, so that the final virtual object image obtained meets the user's aesthetic standards. The final virtual object image can be saved in the storage device 13.

[0091] In this embodiment, the image of the virtual object to be created can be adjusted according to the actual needs of the user. For example, in an online game, the image of the virtual object to be created is imported into the graphics engine module 113 for rendering and display. If the user feels that the current image of the virtual object to be created does not match or does not like it, the user can change the hairstyle, hair color, face shape, skin color, and even all the details and outlines of fatness, chest circumference, etc. to shape it according to the user's own aesthetics, so as to obtain an image effect similar to the user's expectations, such as handsome, suave, or graceful, solemn and elegant, etc., so as to obtain the above-mentioned final virtual object image.

[0092] Figure 6 FIG. 1 is a schematic diagram of an embodiment of a virtual object creation device of the present invention. Figure 6As shown, the user terminal device 1 is connected to the 3D imaging device 12 via a USB data cable, and the interactive interface of the user terminal device 1 at least includes a virtual object type area W, a vacant virtual object area V, and an effect display area U. In the application, there is at least one virtual object type, and the virtual object type has at least one vacant virtual object image; in different applications, the virtual object type and the vacant virtual object image can be determined according to the actual situation. For example, in the virtual object type area W, there are multiple types of virtual objects, including but not limited to characters, heads, faces, etc., and the user player can choose the type he wants, so as to create the virtual object he wants. When the user selects a hand model icon from the virtual object type area W, the user terminal device 1 transmits a camera start-up instruction to the 3D imaging device 12 via a USB data cable, and simultaneously starts the depth camera and the color camera for shooting, and places the hand model A within the shooting range. The 3D imaging device 12 completes the data acquisition of the hand model A and outputs the three-dimensional point cloud data restored based on the calibrated intrinsic parameters and the color image data restored based on the corrected intrinsic parameters; after receiving the three-dimensional point cloud data and the color image data, the target object creation device 11 in the user terminal device 1 converts the three-dimensional point cloud data information into the three-dimensional model A' of the hand touch and converts the color image data information into the virtual object texture, and then performs splicing processing to generate a three-dimensional image A" of the target object. The user quickly switches the screen through the effect display area U of the user terminal device 1: from nothing to something, a point cloud body appears, then to the three-dimensional model A' of the virtual object, until the three-dimensional image A" of the target object (i.e., the virtual object image to be created). The user intuitively sees the three-dimensional image A'' of the target object. If the user feels that the three-dimensional image A'' of the target object meets the user's desired aesthetic standards, the user can directly determine the three-dimensional image A'' of the target object and use it as a virtual object image, and save the data of the virtual object image to the above-mentioned storage device 13. If the user feels that the three-dimensional image A'' of the target object does not meet the user's desired aesthetic standards, the user can make the next adjustment, and provide pre-designed virtual object detailed parameters through the above-mentioned application module 112, including but not limited to the shape, skin color, material, etc. of the virtual object, until the user changes the image of the virtual object to meet the user's desired aesthetic standards, the user can determine that the adjusted virtual object image is the final virtual object image, and save the data of the final virtual object image to the above-mentioned storage device 13.

[0093] Figure 7 FIG. 2 is a schematic diagram of another embodiment of the virtual object creation device of the present invention. Figure 7As shown, the user-end device 1 has the 3D imaging device 12 built in the middle portion 1a. When user B stands in front of the user-end device 1 and user B selects the vacant virtual object V as a human body in the virtual object type area W in the interactive interface 1b of the user-end device 1, once the whole body of user B enters the shooting range FOV of the 3D imaging device 12, the 3D imaging device 12 completes data collection of the user B and outputs three-dimensional point cloud data restored based on the calibrated intrinsic parameters and color image data restored based on the corrected intrinsic parameters. After receiving the three-dimensional point cloud data and the color image data, the target object creation device 11 in the user-end device 1 converts the three-dimensional point cloud data information into a three-dimensional model B' of a virtual object and converts the color image data information into a virtual object texture, and then performs splicing processing to generate a three-dimensional image B" of the target object. The user can quickly switch the screen through the effect display area U in the interactive interface 1b of the user-end device 1: from nothing to something, a point cloud body appears, then to the three-dimensional model B' of the virtual object, until the three-dimensional image B" of the target object. The user intuitively sees the three-dimensional image B" of the target object. If the user feels that the three-dimensional image B" of the target object meets the user's desired aesthetic standards, the user can directly determine that the three-dimensional image B" of the target object is the final virtual object image, and save the data of the final virtual object image to the above-mentioned storage device 13. If the user feels that the three-dimensional image B" of the target object does not meet the user's desired aesthetic standards, the user can make the next adjustment, and the above-mentioned application module 112 of the above-mentioned user-end device 1 provides pre-designed virtual object detailed parameters, including but not limited to the shape, skin color, material, etc. of the virtual object, until the user changes the three-dimensional image of the target object to meet the user's desired aesthetic standards, the user can determine that the adjusted virtual object image is the final virtual object image, and save the data of the final virtual object image to the above-mentioned storage device 13.

[0094] Those skilled in the art will appreciate that the various operations, methods, steps, measures, and schemes discussed in the present invention may be alternated, modified, combined, or deleted. Further, other steps, measures, and schemes in the various operations, methods, and schemes discussed in the present invention may also be alternated, modified, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and schemes in the prior art that are similar to those disclosed in the present invention may also be alternated, modified, rearranged, decomposed, combined, or deleted.

[0095] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for creating a virtual object, comprising: Taking three-dimensional photos of the target object to obtain three-dimensional point cloud data and color image data of the target object; receiving the three-dimensional point cloud data and the color image data obtained by three-dimensionally photographing the target object; Converting the three-dimensional point cloud data into a three-dimensional model of a virtual object and converting the color image data into a texture of a virtual object, and then splicing the two to generate a three-dimensional image of the target object; Performing initial attribute configuration on the three-dimensional image of the target object; Rendering the three-dimensional image of the target object after the initial attribute configuration to obtain the image of the virtual object to be created; The target object is photographed in three dimensions to obtain depth data and image data of the target object, and the three-dimensional point cloud data is restored based on the calibrated intrinsic parameters and the color image data is restored based on the corrected intrinsic parameters.

2. The method for creating a virtual object according to claim 1, further comprising: The image of the virtual object to be created is adjusted to obtain a final image of the virtual object.

3. The method for creating a virtual object according to claim 1, further comprising: The initial attributes of the virtual object to be created are determined based on the existing vacant virtual objects, and the virtual object creation screen is entered.

4. The method for creating a virtual object according to claim 1, characterized in that: The corresponding compensation data is stored in the three-dimensional shooting device in advance, and then the image data collected in real time is corrected and compensated, so as to restore the color image data.

5. The method for creating a virtual object according to claim 1, characterized in that: The three-dimensional point cloud data is restored by storing the corresponding error compensation in the three-dimensional shooting device in advance and performing coordinate transformation on the depth map after the corresponding correction and compensation.

6. The method for creating a virtual object according to claim 1, characterized in that: The calibrated intrinsic parameters are written in advance at designated locations in the 3D shooting device, and then the data collected in real time is compensated by calling the intrinsic parameters of the 3D shooting device, thereby restoring the 3D point cloud data with lower geometric errors.

7. The method for creating a virtual object according to claim 1, characterized in that: The lens uniformity correction parameters are pre-written at a designated position in the three-dimensional shooting device to eliminate the influence of the lens system on brightness uniformity and color uniformity in the image data on the image quality.

8. The method for creating a virtual object according to claim 1, characterized in that: The fixed noise of the 3D shooting device is corrected and compensated in advance, and then the temperature drift correction and compensation is performed to eliminate the reduction in the ranging accuracy of the 3D shooting device due to its own heating and external temperature changes.

9. The method for creating a virtual object according to claim 1, characterized in that: A one-to-one mapping relationship is established between the virtual object texture obtained through the color image data and the three-dimensional model, and based on the mapping relationship, the color image data and the three-dimensional model are surface fitted to obtain the three-dimensional image of the target object.

10. The method for creating a virtual object according to claim 9, characterized in that: A three-dimensional rectangular coordinate system is established with a certain feature point of the point cloud body of the three-dimensional point cloud data as the coordinate origin, other feature values ​​of the point cloud body are corresponded to the three-dimensional rectangular coordinate system, and the point cloud body is divided into a number of surface patches. After curve fitting, the three-dimensional model with geometric feature curves is obtained.

11. The method for creating a virtual object according to claim 1, characterized in that: The initial attributes are one or more of pre-designed virtual object type parameters, virtual object function parameters and virtual object logic parameters.

12. A device for creating a virtual object, comprising: A 3D imaging device, used to perform three-dimensional photography of a target object to obtain three-dimensional point cloud data and color image data of the target object; A model generation module, used to convert the three-dimensional point cloud data obtained by three-dimensionally photographing the target object into a three-dimensional model of a virtual object and convert the color image data obtained by three-dimensionally photographing the target object into a virtual object texture, and then perform splicing processing on the two to generate a three-dimensional image of the target object; an application module, used to perform initial attribute configuration on the three-dimensional image of the target object; A graphics engine module, used for rendering the three-dimensional image of the target object after the initial attribute configuration, to obtain the image of the virtual object to be created; The 3D imaging device performs three-dimensional photography on the target object to obtain depth data and image data of the target object, and restores the three-dimensional point cloud data based on calibrated intrinsic parameters and restores the color image data based on corrected intrinsic parameters.

13. The virtual object creation device according to claim 12, characterized in that: The application module is also used to adjust the image of the virtual object to be created to obtain a final virtual object image.

14. The device for creating a virtual object according to claim 12, characterized in that: The 3D imaging device uses the pre-stored corresponding compensation data to correct and compensate the image data collected in real time, thereby restoring the color image data.

15. The device for creating a virtual object according to claim 12, characterized in that: The 3D imaging device restores the three-dimensional point cloud data by pre-storing corresponding error compensation and performing coordinate transformation on the depth map after corresponding correction and compensation.

16. The device for creating a virtual object according to claim 12, characterized in that: The 3D imaging device writes the calibrated intrinsic parameters in a designated position in advance, and then compensates the data collected in real time by calling the intrinsic parameters, thereby restoring the three-dimensional point cloud data with lower geometric errors.

17. The virtual object creation device according to claim 12, characterized in that: The 3D imaging device writes lens uniformity correction parameters in advance at a designated position to eliminate the influence of the lens system on brightness uniformity and color uniformity in the image data on image quality.

18. The virtual object creation device according to claim 12, characterized in that: The 3D imaging device performs correction compensation for fixed noise in advance and then performs temperature drift correction compensation, thereby eliminating the reduction in ranging accuracy due to self-heating and external temperature changes.

19. The device for creating a virtual object according to claim 12, characterized in that: The model generation module obtains the virtual object texture through the color image data and establishes a one-to-one mapping relationship with the three-dimensional model, and based on the mapping relationship, performs surface fitting between the color image data and the three-dimensional model to obtain the three-dimensional image of the target object.

20. The virtual object creation device according to claim 19, characterized in that: The model generation module uses a certain feature point of the point cloud body of the three-dimensional point cloud data as the coordinate origin, establishes a three-dimensional rectangular coordinate system, corresponds other feature values ​​of the point cloud body to the three-dimensional rectangular coordinate system, and divides the point cloud body into a number of surface patches. After B-spline curve fitting, the three-dimensional model with geometric feature curves is obtained.

21. The virtual object creation device according to claim 12, characterized in that: The initial attributes are one or more of pre-designed virtual object type parameters, virtual object function parameters and virtual object logic parameters.

22. A computer system for creating a virtual object, the system comprising: processor; computer readable medium; Computer program instructions stored on the computer readable medium for execution by at least one of the processors, the computer program instructions comprising computer program instructions for performing the steps of the method of any one of claims 1 to 11.

23. A computer-readable storage medium for creating a virtual object, wherein the computer-readable storage medium stores at least one executable computer program instruction, wherein the computer program instruction comprises: Computer program instructions for performing the steps of the method of any one of claims 1 to 11.

Images