Trajectory rendering method, device, electronic device and storage medium

Abstract

The embodiments of the present application disclose a trajectory rendering method, device, electronic device and storage medium. The method includes: obtaining a trajectory to be rendered, the trajectory to be rendered is a sliding trajectory of a touch object in a three-dimensional space; generating three-dimensional model data based on the trajectory to be rendered; and generating a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and a material capture map. In a three-dimensional space, an image is drawn through a material capture map, so that the drawn image can simulate a real physical lighting effect, thereby being more realistic.

Description

Technical Field

[0001] The present application belongs to the field of virtual reality technology, and specifically relates to a trajectory rendering method, device, electronic device and storage medium. Background Art

[0002] During the operation of virtual reality / augmented reality (VR / AR) devices, objects visible to users are rendered. However, the images rendered by the relevant rendering methods are all pure colors, so that the rendered images do not have feedback from ambient light, and thus lack a sense of reality. Summary of the invention

[0003] In view of the above problems, the present application proposes a trajectory rendering method, device, electronic device and storage medium to improve the above problems.

[0004] In a first aspect, an embodiment of the present application provides a trajectory rendering method, the method comprising: obtaining a trajectory to be rendered, the trajectory to be rendered being a sliding trajectory of a touch object in a three-dimensional space; generating three-dimensional model data based on the trajectory to be rendered; and generating a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and a material capture map.

[0005] In the second aspect, an embodiment of the present application provides a trajectory rendering device, which includes: a trajectory acquisition unit, used to acquire the trajectory to be rendered, and the trajectory to be rendered is the sliding trajectory of the touch object in the three-dimensional space; a data generation unit, used to generate three-dimensional model data based on the trajectory to be rendered; a rendering unit, used to generate a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and a material capture map.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, comprising one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, wherein the above method is executed when the program code is run.

[0008] The embodiments of the present application provide a trajectory rendering method, device, electronic device and storage medium. First, the trajectory to be rendered is obtained, and the trajectory to be rendered is the sliding trajectory of the touch object in the three-dimensional space. Then, based on the trajectory to be rendered, three-dimensional model data is generated, and then based on the three-dimensional model data and the material capture map, a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered is generated. Through the above method, in the three-dimensional space, the image is drawn through the material capture map, so that the drawn image can simulate the real physical lighting effect, so that it can be more realistic. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A schematic diagram of an application scenario of a trajectory rendering method proposed in an embodiment of the present application is shown;

[0011] Figure 2 A schematic diagram of a trajectory to be rendered proposed in an embodiment of the present application is shown;

[0012] Figure 3 A schematic diagram of an application scenario of a trajectory rendering method proposed in an embodiment of the present application is shown;

[0013] Figure 4 A schematic diagram of an application scenario of a trajectory rendering method proposed in an embodiment of the present application is shown;

[0014] Figure 5 A flow chart of a trajectory rendering method proposed in an embodiment of the present application is shown;

[0015] Figure 6 A schematic diagram of a trajectory to be rendered in an embodiment of the present application is shown;

[0016] Figure 7 A schematic diagram showing three-dimensional model data generated in an embodiment of the present application is shown;

[0017] Figure 8 A flow chart of a trajectory rendering method proposed in another embodiment of the present application is shown;

[0018] Fig. 9 A schematic diagram of a Bezier curve in another embodiment of the present application is shown;

[0019] Fig.10 A schematic diagram of a trajectory to be rendered in another embodiment of the present application is shown;

[0020] Fig.11 A schematic diagram of obtaining a circle corresponding to a trajectory point in another embodiment of the present application is shown;

[0021] Fig.12 A schematic diagram showing another embodiment of the present application of obtaining a reference point from a circle corresponding to a trajectory point;

[0022] Fig.13 A schematic diagram showing three-dimensional model data generated in another embodiment of the present application is shown;

[0023] Fig.14 A flow chart of a trajectory rendering method proposed in yet another embodiment of the present application is shown;

[0024] Fig.15 A schematic diagram showing preset three-dimensional model data in yet another embodiment of the present application;

[0025] Fig.16 A schematic diagram showing division of preset three-dimensional model data in yet another embodiment of the present application is shown;

[0026] Fig.17 A schematic diagram showing three-dimensional model data generated in yet another embodiment of the present application is shown;

[0027] Fig.18 A flow chart of a trajectory rendering method proposed in another embodiment of the present application is shown;

[0028] Fig.19 A schematic diagram of a MatCap map generated in another embodiment of the present application is shown;

[0029] Fig. 20 A schematic diagram showing a three-dimensional graffiti rendering result generated in another embodiment of the present application;

[0030] Fig.21 A structural block diagram of a trajectory rendering device proposed in an embodiment of the present application is shown;

[0031] Fig. 22 A structural block diagram of a trajectory rendering device proposed in an embodiment of the present application is shown;

[0032] Fig.23 A structural block diagram of an electronic device or a server for executing a trajectory rendering method according to an embodiment of the present application is shown;

[0033] Fig.24 A storage unit in the present application for storing or carrying program codes for implementing the trajectory rendering method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] Virtual Reality / Augmented Reality (VR / AR) is a virtual simulation system that can be built, experienced and interacted with, integrating computer graphics, real-time image processing, precision sensors and human-computer interfaces. The realistic scenes and models give people an immersive experience. Currently, it is mostly used in games, film and television, animation, military and other fields.

[0036] The inventors discovered in their research on related trajectory rendering methods that the related trajectory rendering methods rely heavily on handheld sensor hardware to collect trajectory points drawn by users, and the drawn graphics cannot be well integrated into the VR / AR scene settings, or the collected virtual or real lighting scenes. Therefore, the images drawn by the related rendering methods are mostly pure colors, without feedback from ambient light, and thus lack a sense of reality.

[0037] Therefore, the inventors proposed the trajectory rendering method, device, electronic device and storage medium in the present application. First, the trajectory to be rendered is obtained, and the trajectory to be rendered is the sliding trajectory of the touch object in the three-dimensional space. Then, based on the trajectory to be rendered, three-dimensional model data is generated, and then based on the three-dimensional model data and the material capture map, a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered is generated. Through the above method, in the three-dimensional space, the image is drawn through the material capture map, so that the drawn image can simulate the real physical lighting effect, so that it can be more realistic.

[0038] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of the trajectory rendering method provided in an embodiment of the present application.

[0039] like Figure 1 As shown, the trajectory rendering method provided in the embodiment of the present application can be applied to Figure 1In the application scenario shown, in this application scenario, the user can wear a VR / AR device. While the user is wearing the VR / AR device, the user can draw a track point or image in the three-dimensional space by using a finger or a touch object according to the content displayed in the VR / AR device. For example, when the user is wearing the VR / AR device and playing a game, if the VR / AR device shows that the user needs to control the character in the game to move downward, the user can draw a downward track in the three-dimensional space by using a finger or a touch object, such as Figure 2 As shown, Figure 2 The trajectory in is the downward trajectory drawn by the user in the three-dimensional space using a finger or touch object.

[0040] Optionally, in an embodiment of the present application, the provided trajectory rendering method may be executed by an electronic device. In this manner, all steps in the trajectory rendering method provided in the embodiment of the present application may be executed by the electronic device. Figure 3 As shown, the trajectory acquisition device of the electronic device 100 can acquire the trajectory to be rendered, and then transmit the acquired trajectory to be rendered to the processor, so that the processor can generate three-dimensional model data based on the trajectory to be rendered in real time. Furthermore, the processor can also generate a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map.

[0041] Furthermore, the trajectory rendering method provided in the embodiment of the present application can also be executed by a server (cloud). Correspondingly, in this method executed by the server, the electronic device can obtain the user's finger-drawn image, and synchronously send the user's finger-drawn image to the server, and then the server can recognize the user's finger-drawn image in real time to obtain the trajectory to be rendered, so that the server can generate three-dimensional model data based on the trajectory to be rendered; based on the three-dimensional model data and the material capture map, generate a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered.

[0042] In addition, the electronic device and the server may also perform the process in collaboration. In this manner, some steps of the trajectory rendering method provided by the embodiment of the present application are performed by the electronic device, while other steps are performed by the server.

[0043] For example, Figure 4 As shown, the electronic device 100 can execute the trajectory rendering method including: obtaining the trajectory to be rendered, the trajectory to be rendered is the sliding trajectory of the touch object in the three-dimensional space, and then the server 200 executes to generate three-dimensional model data based on the trajectory to be rendered; based on the three-dimensional model data and the material capture map, generate a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered.

[0044] It should be noted that in this method of collaborative execution by the electronic device and the server, the steps respectively executed by the electronic device and the server are not limited to the methods introduced in the above examples. In actual applications, the steps respectively executed by the electronic device and the server can be dynamically adjusted according to actual conditions.

[0045] It should be noted that the electronic device 100 can be Figure 1 and Figure 2 In addition to the smartphone shown in the figure, it can also be a car device, a wearable device, a tablet computer, a laptop computer, a smart speaker, a VR / AR device, etc. The server 200 can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers.

[0046] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0047] See also Figure 5 , a trajectory rendering method provided in an embodiment of the present application is applied to Figure 3 or Figure 4 The electronic device or server shown, the method comprising:

[0048] Step S110: obtaining a trajectory to be rendered, where the trajectory to be rendered is a sliding trajectory of the touch object in the three-dimensional space.

[0049] In the embodiment of the present application, the trajectory to be rendered is a sliding trajectory that needs to be drawn in three-dimensional graphics. The touch object can be an object that the user can use to draw the sliding trajectory in a certain application scenario. For example, in a VR game scenario, the touch object can be a controller held by the user or a finger of the user, which is not specifically limited here.

[0050] As a method, the trajectory to be rendered can be a sliding trajectory pre-stored in a cloud server or a preset storage area, and the trajectory to be rendered can also be a sliding trajectory obtained in real time. As one of the methods, when the trajectory to be rendered is a sliding trajectory pre-stored in a cloud server or a preset storage area, the sliding trajectories corresponding to different scenes can be pre-acquired, and then the sliding trajectories corresponding to different scenes can be stored in the cloud server or the preset storage area. Among them, when storing the sliding trajectories corresponding to different application scenarios, different identifiers can be set for different application scenarios, and different numbers can be set for the sliding trajectories corresponding to different application scenarios, and the corresponding relationship between the application scenario and the sliding trajectory can be established. Optionally, if there are multiple sliding trajectories corresponding to a certain application scenario, when setting numbers for the multiple sliding trajectories, the multiple sliding trajectories can be numbered according to the preset numbering order. Among them, the preset numbering order can be the order from small to large, the order from large to small, or the order of the time when the sliding trajectory is obtained, according to the order from small to large, etc., which is not specifically limited here. For example, if there are three sliding tracks corresponding to a VR game scene, among which sliding track 1 is obtained on December 1, sliding track 2 is obtained on December 5, and sliding track 3 is obtained on December 2, then if the three sliding tracks are numbered in ascending order according to the chronological order of the acquisition time of the sliding tracks, then sliding track 1 can be numbered 001, sliding track 3 can be numbered 002, and sliding track 2 can be numbered 003.

[0051] After storing the sliding trajectories corresponding to different application scenarios in the above manner, when it is necessary to obtain the trajectory to be rendered, the current application scenario can be identified first, and then when the current application scenario is identified, the sliding trajectory corresponding to the application scenario can be obtained from the cloud server or the preset storage area, and the obtained sliding trajectory can be used as the trajectory to be rendered that needs to be obtained. Optionally, when obtaining the sliding trajectory corresponding to the application scenario, if there are multiple sliding trajectories corresponding to the application scenario, all of the corresponding multiple sliding trajectories can be used as the trajectory to be rendered that needs to be obtained; or, any one of the corresponding multiple sliding trajectories can be used as the trajectory to be rendered that needs to be obtained; or, the sliding trajectory with the smallest or largest number among the corresponding multiple sliding trajectories can be used as the trajectory to be rendered that needs to be obtained, and no specific limitation is made here.

[0052] Furthermore, each application scenario can be divided into multiple sub-application scenarios, and a corresponding relationship between each sub-application scenario and a sliding trajectory can be established, and each sub-application scenario can correspond to a sliding trajectory. Among them, when each application scenario is divided into multiple sub-application scenarios, it can be divided according to different application stages. For example, in a VR game application scenario, the VR game application scenario can be divided into two sub-application scenarios according to whether the game character needs to be operated, one sub-application scenario for which the game character needs to be operated, and one sub-application scenario for which the game character does not need to be operated. Among them, the sliding trajectory corresponding to the sub-application scenario for which the game character needs to be operated is the sliding trajectory for controlling the rotation of the game character; the sliding trajectory corresponding to the sub-application scenario for which the game character does not need to be operated is the sliding trajectory for clicking the equipment.

[0053] After each application scene is divided into a plurality of sub-application scenes, when it is necessary to obtain a trajectory to be rendered, the corresponding sub-application scene can be determined after the application scene is found, and then the corresponding sliding trajectory can be found as the trajectory to be rendered that needs to be obtained.

[0054] As another method, when the trajectory to be rendered is a sliding trajectory obtained in real time, when the user wears the VR / AR device, the user can draw the sliding trajectory in real time in the three-dimensional space through the touch object. Of course, the user can also draw the sliding trajectory according to the game scene running in the VR / AR device.

[0055] Step S120: Generate three-dimensional model data based on the trajectory to be rendered.

[0056] In the embodiment of the present application, a preset method can be used to generate three-dimensional model data having the same shape as the trajectory to be rendered based on the shape of the trajectory to be rendered and the coordinates of each trajectory point in the trajectory to be rendered. Figure 6 As shown, the generated 3D model data is as follows Figure 7 shown.

[0057] Step S130: generating a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map.

[0058] In the embodiment of the present application, the material capture map is also called a MatCap (Material Capture) map. The MatCap map is a pre-set material used to add certain display effects to the model. It is a pre-generated map that stores information such as lighting and reflection. It uses the normal direction for sampling during runtime. The three-dimensional graffiti rendering result is a three-dimensional model data with MatCap material effects.

[0059] As a method, the three-dimensional model data may be rendered according to the MatCap map to generate three-dimensional model data having a MatCap material effect.

[0060] In different application scenarios, the generated material capture map can be different, and then by rendering the 3D model data through different material capture maps, 3D model data with different MatCap material effects can be obtained. For example, when a VR game is running, when it is necessary to display the preset 3D model data in the VR game scene, the preset 3D model data can be rendered according to the target MatCap map to display the preset 3D model data with MatCap material effects in the game scene.

[0061] In an embodiment of the present application, when generating three-dimensional model data with MatCap material effects based on the three-dimensional model data and the material capture map, the three-dimensional model data with MatCap material effects may be generated through a MatCap Shader method.

[0062] Among them, MatCap Shader is a next-generation rendering solution that can replace or even surpass PBR (Physicallly-Based Rendering) at some levels. The basic idea of ​​MatCap Shader is to use the map of a specific material ball as the view-space environment map of the current material to achieve the display of reflective material objects with uniform surface shading. Considering that the projection range of all normals of the object is x(-1,1), y(-1,1), forming a circle, the area in the MatCap map that stores lighting information is a circle. Shader based on the MatCap idea does not need to provide any lighting, but only needs to provide one or more suitable MatCap maps as a "guide" for the lighting results.

[0063] The present application provides a trajectory rendering method, which first obtains the trajectory to be rendered, which is the sliding trajectory of the touch object in the three-dimensional space, then generates three-dimensional model data based on the trajectory to be rendered, and then generates a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map. Through the above method, in the three-dimensional space, the image is drawn through the material capture map, so that the drawn image can simulate the real physical lighting effect, so that it can be more realistic.

[0064] See also Figure 8 , a trajectory rendering method provided in an embodiment of the present application is applied to Figure 3 or Figure 4 The electronic device or server shown, the method comprising:

[0065] Step S210: capturing a touch object drawing image in a three-dimensional space through an image capturing device.

[0066] In the embodiment of the present application, the image acquisition device may be an electronic device with an image acquisition function, such as a mobile phone, a camera, an image sensor, etc. Optionally, the touch object is a user's finger, the image drawn by the touch object is an image drawn by the user's finger, and the image drawn by the user's finger acquired by the image acquisition device may include the trajectory to be rendered, the user's finger, and the background image.

[0067] As a way to reduce hardware costs, a finger and an image acquisition device can be used as input devices. In response to an image acquisition instruction, the image acquisition device starts to acquire an image drawn by the user's finger. The image acquisition instruction can be an instruction sent by an external device that establishes a communication connection with the image acquisition device, or it can be an instruction triggered when a certain application in the image acquisition device is detected to start running, which is not specifically limited here.

[0068] When collecting images drawn by user's fingers, the image acquisition device may collect multiple frames of images drawn by user's fingers continuously, or collect images drawn by user's fingers at preset time intervals. As one of the ways, when the image acquisition device collects multiple frames of images drawn by user's fingers continuously, the images drawn by user's fingers change faster in the time sequence; as another way, when the image acquisition device collects images drawn by user's fingers at preset time intervals, the images drawn by user's fingers change slower in the time sequence. That is, the way to collect images drawn by user's fingers can be determined according to the speed of the change of images drawn by user's fingers in the time sequence. The speed of the change of images drawn by user's fingers can be determined by determining whether the similarity between the collected images drawn by user's fingers of adjacent frames is less than the preset similarity, and whether the time interval of obtaining the images drawn by user's fingers of adjacent frames whose similarity is less than the preset similarity is less than the preset time interval. When the similarity between images drawn by user fingers in adjacent frames is less than a preset similarity, and the time interval for obtaining images drawn by user fingers in adjacent frames is less than a preset time interval, it is determined that the images drawn by user fingers change faster; when the similarity between images drawn by user fingers in adjacent frames is greater than or equal to the preset similarity, and the time interval for obtaining images drawn by user fingers in adjacent frames is greater than or equal to the preset time interval, it is determined that the images drawn by user fingers change slower. The preset similarity is a preset maximum similarity for determining that images drawn by user fingers change faster, and the preset time interval is a preset maximum time interval for determining that images drawn by user fingers change faster.

[0069] For example, the preset similarity can be set to 60%, and the preset time interval can be set to 5s. In the above case, if the similarity between the adjacent frames of user finger-drawn images acquired by the image acquisition device is 55%, and the acquisition time interval between the adjacent frames of user finger-drawn images is 4.5s, it can be determined that the user finger-drawn images change quickly, so when subsequently acquiring the user finger-drawn images, the image acquisition device can continuously acquire multiple frames of user finger-drawn images.

[0070] Optionally, when collecting images drawn by the user's fingers, the image acquisition device may use only one of the methods to obtain the images drawn by the user's fingers, or may use both methods to obtain the images drawn by the user's fingers. For example, in a VR game application scenario, if the images drawn by the user's fingers change slowly in the early stages of the game, and change quickly in the middle and late stages of the game, then when collecting images drawn by the user's fingers, the image acquisition device may first collect images drawn by the user's fingers at preset time intervals, and when reaching the middle and late stages of the game, the image acquisition device may continuously collect multiple frames of images drawn by the user's fingers.

[0071] Step S220: identifying a plurality of track points in the image drawn by the touch object.

[0072] In an embodiment of the present application, the image drawn by the user's finger can be identified through finger recognition and tracking technology, and the position of the user's finger in the three-dimensional space can be identified, that is, the coordinates of the trajectory points of the finger sliding in the image drawn by the current user's finger are identified, and a set of trajectory points is output.

[0073] Step S230: performing smoothing processing on the plurality of trajectory points to obtain trajectories to be rendered corresponding to the plurality of trajectory points.

[0074] In the embodiment of the present application, the trajectory points included in the image drawn by the user's finger can be smoothed by a Bezier curve to obtain a trajectory to be rendered. Fig. 9 As shown in the figure, Bezier curve is a basic tool for computer graphics and one of the most commonly used basic lines for circular modeling. Bezier curve creates and edits graphics by controlling four points on the curve (starting point P0, ending point P3, and two separated middle points P1 and P2).

[0075] Step S240: establishing a trajectory point combination based on every two adjacent trajectory points in the trajectory to be rendered, so as to obtain a plurality of trajectory point combinations.

[0076] In the embodiment of the present application, the trajectory points in the trajectory to be rendered are divided into trajectory point combinations, and all the trajectory points included in the trajectory to be rendered are divided in a manner that every two adjacent trajectory points are divided into one trajectory point combination, thereby obtaining a plurality of trajectory point combinations. Fig.10 As shown, the trajectory to be rendered includes trajectory point a, trajectory point b, trajectory point c, trajectory point d, trajectory point e, trajectory point f, and trajectory point g. When dividing the trajectory point combinations, trajectory point a and trajectory point b are divided into one trajectory point combination, trajectory point b and trajectory point c are divided into one trajectory point combination, trajectory point c and trajectory point d are divided into one trajectory point combination, trajectory point d and trajectory point e are divided into one trajectory point combination, trajectory point e and trajectory point f are divided into one trajectory point combination, and trajectory point f and trajectory point g are divided into one trajectory point combination, so as to obtain 6 trajectory point combinations.

[0077] Step S250: respectively obtaining the circumferences corresponding to the two trajectory points in each trajectory point combination, wherein the center corresponding to the circumference is each trajectory point.

[0078] In the embodiment of the present application, each trajectory point corresponds to a circle, the line connecting the point on the circle and the trajectory point is perpendicular to the vector formed by two adjacent trajectory points, and the line connecting the point on the circle and the trajectory point is equal to the preset radius, such as Fig.11 shown.

[0079] Step S260: With a preset smoothness corresponding to each trajectory point combination, a preset number of reference points are acquired on the circumference corresponding to the trajectory point combination to obtain reference points corresponding to each trajectory point combination.

[0080] In the embodiment of the present application, the preset smoothness is used to characterize the probability that the figure composed of the lines of the acquired reference points is like a circle. The larger the preset smoothness, the more the figure composed of the acquired reference points is like a circle, and the smaller the preset smoothness, the less the figure composed of the acquired reference points is like a circle. The preset smoothness is a preset smoothness used to obtain a preset number of reference points on the circumference corresponding to each trajectory point.

[0081] As a method, a preset number of reference points are taken on the circumference corresponding to each trajectory point combination according to a preset smoothness and at the same interval. Wherein, when a preset number of reference points are taken on the circumference corresponding to each trajectory point combination, reference points are taken at the same position of two corresponding circles. For example, Fig.12 As shown, Fig.12 In the figure, reference point a1 and reference point b1 are at the same position on the corresponding circle, and reference point an and reference point bn are at the same position on the corresponding circle.

[0082] The same interval can be understood as determining the number of reference points to be obtained according to the preset smoothness, and then dividing the circle corresponding to the trajectory point into a corresponding number of arcs, taking a reference point at each division position to obtain a corresponding number of reference points.

[0083] Step S270: generating three-dimensional model data corresponding to the trajectory to be rendered based on the reference point corresponding to each trajectory point combination and the trajectory points included in each trajectory point combination.

[0084] In the embodiment of the present application, the reference points at the same position on the circumference corresponding to each trajectory point combination are connected, and each trajectory point is connected with the reference point on the circumference corresponding to each trajectory point to obtain the three-dimensional model data corresponding to the trajectory to be rendered. Fig.13 As shown, the reference points taken on the circumference of trajectory point a include a1, ..., an, and the reference points taken on the circumference corresponding to trajectory point b include b1, ..., bn. When connecting, a1 and b1 are reference points at the same position, so a1 and b1 are connected; an and bn are reference points at the same position, so an and bn are connected; trajectory point a is connected to a1, ..., an respectively, and trajectory point b is connected to b1, ..., bn respectively.

[0085] Step S280: generating a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map.

[0086] The present application provides a trajectory rendering method, firstly, an image acquisition device is used to acquire a touch object drawing image in a three-dimensional space, a plurality of trajectory points in the touch object drawing image are identified, and a plurality of trajectory points are smoothed to obtain a plurality of trajectory points corresponding to the trajectory to be rendered, then a trajectory point combination is established based on each two adjacent trajectory points in the trajectory to be rendered to obtain a plurality of trajectory point combinations, and the circumferences corresponding to the two trajectory points in each trajectory point combination are respectively obtained, and then a preset number of reference points are taken on the circumference corresponding to the trajectory point combination with a preset smoothness corresponding to each trajectory point combination to obtain the reference points corresponding to each trajectory point combination, and finally, based on the reference points corresponding to each trajectory point combination and the trajectory points included in each trajectory point combination, a three-dimensional model data corresponding to the trajectory to be rendered is generated, and then a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered is generated based on the three-dimensional model data and the material capture map. Through the above method, by taking a preset number of reference points on the circumference corresponding to the trajectory point combination with a preset smoothness, the obtained three-dimensional model data can be closer to the three-dimensional model data of the shape of the trajectory to be rendered, so that the obtained three-dimensional graffiti rendering result corresponding to the trajectory to be rendered is more realistic.

[0087] See also Fig.14, a trajectory rendering method provided in an embodiment of the present application is applied to Figure 3 or Figure 4 The electronic device or server shown, the method comprising:

[0088] Step S310: capturing a touch object drawing image in a three-dimensional space through an image capturing device.

[0089] Step S320: identifying a plurality of track points in the image drawn by the touch object.

[0090] Step S330: performing smoothing processing on the plurality of trajectory points to obtain trajectories to be rendered corresponding to the plurality of trajectory points.

[0091] Step S340: Obtaining preset three-dimensional model data.

[0092] In the embodiment of the present application, the preset 3D model data is a preset cylindrical model with a fixed size and shape. The preset 3D model data can be pre-stored in a cloud server or a preset storage area. When the preset 3D model data is needed, the preset 3D model data can be obtained from the cloud server or the preset storage area. For example, Fig.15 As shown, Fig.15 A preset three-dimensional model data is shown. Optionally, the preset three-dimensional model data may also be a sphere model or a cube model with a fixed size and shape.

[0093] Step S350: Obtain the sum of the distances between every two adjacent trajectory points in the trajectory to be rendered.

[0094] In the embodiment of the present application, the distance between every two adjacent trajectory points is the straight-line distance between the two trajectory points. As a method, the straight-line distance between every two adjacent trajectory points in the trajectory to be rendered is first calculated, and then all the calculated straight-line distances are added together to obtain the sum of the distances.

[0095] Step S360: scaling the length of the preset three-dimensional model data to be equal to the sum of the distances, to obtain scaled preset three-dimensional model data.

[0096] In an embodiment of the present application, if the length of the preset three-dimensional model data is greater than the sum of the calculated distances, the length of the preset three-dimensional model data is compressed to be the same as the sum of the distances; if the length of the preset three-dimensional model data is less than the sum of the calculated distances, the length of the preset three-dimensional model data is enlarged, and the enlargement value is the same as the sum of the distances.

[0097] In the embodiment of the present application, if the preset three-dimensional model data is as follows Fig.15If the preset three-dimensional model data is a cylindrical model as shown, then the length of the preset three-dimensional model data can be the height of the cylindrical model; if the preset three-dimensional model data is a spherical model, then the length of the preset three-dimensional model data can be the diameter of the spherical model; if the preset three-dimensional model data is a cube model, then the length of the preset three-dimensional model data can be the side length of the cube model.

[0098] Step S370: Based on the ratio of the distance between every two adjacent trajectory points to the sum of the distances, the scaled preset three-dimensional model data is vertically divided to obtain reference three-dimensional model data corresponding to every two adjacent trajectory points.

[0099] In the embodiment of the present application, the ratio of the straight-line distance to the sum of the distances between each two adjacent trajectory points is calculated, and the scaled preset three-dimensional model is vertically divided according to the corresponding ratio to obtain the reference three-dimensional model data corresponding to each two adjacent trajectory points. The vertical division refers to dividing the scaled preset three-dimensional model data according to the y-axis direction of the established coordinate system. For example, Fig.16 As shown, the y-axis direction is the direction perpendicular to the ground, and the trajectory to be rendered is abc. According to the straight-line distance L1 between trajectory point a and trajectory point b, and the straight-line distance L2 between trajectory point b and trajectory point c, the sum of the distances L3=L1+L2 is calculated, and then the ratio 1 of L1 to L3 and the ratio 2 of L1 to L3 are calculated. In the y-axis direction, the scaled preset 3D model data is divided into the first reference model data (ab area) and the second reference 3D model data (bc area) according to the ratio 1 and the ratio 2, and the reference 3D model data corresponding to trajectory point a and trajectory point b, and the reference 3D model data corresponding to trajectory point b and trajectory point c are obtained, and then the vertex coordinates of each reference 3D model data can be known. Among them, the vertex of each reference 3D model data is a point on the circumference of the cross section of each reference 3D model data, and the cross section is obtained after longitudinal division according to the ratio.

[0100] Step S380: Based on the coordinates of every two adjacent trajectory points and the slope of the vector formed by every two adjacent trajectory points, the vertices of the reference three-dimensional model data corresponding to every two adjacent trajectory points are calculated to obtain the three-dimensional model data.

[0101] In the embodiment of the present application, a corresponding vector is obtained according to the coordinates of every two adjacent trajectory points, and then a corresponding slope can be calculated based on the calculated vector, so that the coordinates of the vertices of the corresponding reference 3D model data can be calculated according to the coordinates of every two adjacent trajectory points and the corresponding slope. If the calculated coordinates are different from the preset coordinates, the preset coordinates are updated based on the calculated coordinates to obtain the reference 3D model data after the updated coordinates, and then each 3D model data after the updated coordinates is spliced ​​to obtain the 3D model data corresponding to the trajectory to be rendered. Fig.16 After updating the vertex coordinates of the reference 3D model data shown in Fig.17 The three-dimensional model data shown.

[0102] Step S390: generating a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map.

[0103] The present application provides a trajectory rendering method, which first acquires a touch object drawing image in a three-dimensional space through an image acquisition device, identifies multiple trajectory points in the touch object drawing image, and smoothes the multiple trajectory points to obtain a trajectory to be rendered corresponding to the multiple trajectory points, then acquires preset three-dimensional model data, acquires the sum of the distances between every two adjacent trajectory points in the trajectory to be rendered, scales the height of the preset three-dimensional model data to be the same as the sum of the distances, and obtains the scaled preset three-dimensional model data, and then divides the scaled preset three-dimensional model data vertically based on the ratio of the distance between every two adjacent trajectory points to the sum of the distances, and obtains reference three-dimensional model data corresponding to every two adjacent trajectory points, and then calculates the vertices of the reference three-dimensional model data corresponding to every two adjacent trajectory points based on the coordinates of every two adjacent trajectory points and the slope of the vector composed of every two adjacent trajectory points to obtain the three-dimensional model data, and finally generates a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and a material capture map. Through the above method, by dividing the preset three-dimensional model data into multiple reference three-dimensional model data, the vertex coordinates of the reference model data can be updated according to the coordinates of every two trajectory points and the slope of the vector composed of the two trajectory points, so as to obtain three-dimensional model data that is closer to the shape of the trajectory to be rendered, and then the three-dimensional model data is drawn through the material capture map, so that the drawn image can simulate the real physical lighting effect.

[0104] See also Fig.18 , a trajectory rendering method provided in an embodiment of the present application is applied to Figure 3 or Figure 4 The electronic device or server shown, the method comprising:

[0105] Step S410: obtaining a trajectory to be rendered, where the trajectory to be rendered is a sliding trajectory of a touch object in a three-dimensional space.

[0106] Step S420: Generate three-dimensional model data based on the trajectory to be rendered.

[0107] Step S430: Obtain a preset brush material, preset lighting information, and a preset sphere model, and generate a material capture map.

[0108] In an embodiment of the present application, a preset brush material can be generated based on a preset material map and a preset color, wherein the preset material map can be obtained in advance from a material map library, and the preset color can be any preset color. The preset lighting information can be set in advance in a virtual space, or can also be obtained by an AR device, wherein the preset lighting information includes light source coordinates and lighting intensity. The preset sphere model is a commonly used material ball.

[0109] According to the preset brush material, preset lighting information and preset sphere model, a MatCap map with specific material and lighting information effects can be generated. The generated MatCap map can be used as follows Fig.19 shown.

[0110] Step S440: generating a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map.

[0111] In the embodiment of the present application, the generated three-dimensional graffiti rendering result can be as follows: Fig. 20 As shown, through Fig. 20 It can be seen that the 3D graffiti rendering result can well reflect the physical lighting effect.

[0112] Step S450: changing the style of the three-dimensional graffiti rendering result.

[0113] In the embodiment of the present application, for many graffiti beginners, they are always unable to draw the lines and curves they like, and the line color selection is often not satisfactory. At this time, the graffiti auxiliary function can be used to intelligently smooth and straighten the trajectory results input by the user, and even when graffitiing some numbers or fonts, they can be automatically replaced with stylized fonts and numbers through real-time communication technology (RTC technology). After graffiti, the style can be transferred according to the graffiti environment to make the picture more artistic. Optionally, the terminal can also obtain a preset style locally, and then the style of the three-dimensional graffiti rendering result can be replaced with the preset style.

[0114] Step S460: sharing and displaying the three-dimensional graffiti rendering result through the network.

[0115] In an embodiment of the present application, in a multi-person interactive scenario in the virtual world, the three-dimensional graffiti rendering result can be shared with other people through the network, and the process of generating the three-dimensional graffiti rendering result can be reproduced synchronously in front of other people, which can serve as a new way of socializing in the 3D virtual world, similar to leaving a message on a bulletin board. Compared with ordinary old-fashioned font display, graffiti is more personalized and stylized.

[0116] The present application provides a trajectory rendering method, which first obtains the trajectory to be rendered, generates three-dimensional model data based on the trajectory to be rendered, then obtains the preset brush material, preset lighting information and preset sphere model, generates a material capture map, and then generates a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map, and then the style of the three-dimensional graffiti rendering result can be changed through real-time communication technology, and the three-dimensional graffiti rendering result can be shared. Through the above method, the image is drawn through the material capture map, so that the drawn image can simulate the real physical lighting effect.

[0117] See also Fig.21 , an embodiment of the present application provides a trajectory rendering device 500, the device 500 comprising:

[0118] The trajectory acquisition unit 510 is used to acquire a trajectory to be rendered, where the trajectory to be rendered is a sliding trajectory of the touch object in the three-dimensional space.

[0119] As a method, the trajectory acquisition unit 510 is specifically used to capture the touch object drawing image in the three-dimensional space through an image acquisition device; identify multiple trajectory points in the touch object drawing image; and smooth the multiple trajectory points to obtain the trajectories to be rendered corresponding to the multiple trajectory points.

[0120] The data generating unit 520 is configured to generate three-dimensional model data based on the trajectory to be rendered.

[0121] As a method, the data generation unit 520 is specifically used to establish a trajectory point combination based on every two adjacent trajectory points in the trajectory to be rendered to obtain multiple trajectory point combinations; respectively obtain the circles corresponding to the two trajectory points in each trajectory point combination, wherein the center corresponding to the circle is each trajectory point; with a preset smoothness corresponding to each trajectory point combination, obtain a preset number of reference points on the circle corresponding to the trajectory point combination to obtain the reference points corresponding to each trajectory point combination; based on the reference points corresponding to each trajectory point combination and the trajectory points included in each trajectory point combination, generate three-dimensional model data corresponding to the trajectory to be rendered.

[0122] As another method, the data generation unit 520 is specifically used to obtain preset three-dimensional model data; obtain the sum of the distances between every two adjacent trajectory points in the trajectory to be rendered; scale the length of the preset three-dimensional model data to be the same as the sum of the distances to obtain the scaled preset three-dimensional model data; based on the ratio of the distance between every two adjacent trajectory points to the sum of the distances, vertically divide the scaled preset three-dimensional model data to obtain reference three-dimensional model data corresponding to every two adjacent trajectory points; calculate the vertices of the reference three-dimensional model data corresponding to every two adjacent trajectory points based on the coordinates of every two adjacent trajectory points and the slope of the vector composed of every two adjacent trajectory points to obtain the three-dimensional model data.

[0123] The rendering unit 530 is used to generate a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map.

[0124] See also Fig. 22 , the device 500 further includes:

[0125] The texture generation unit 540 is used to obtain a preset brush material, preset lighting information and a preset sphere model to generate a material capture texture.

[0126] The processing unit 550 is used to change the style of the three-dimensional graffiti rendering result through real-time communication technology.

[0127] Optionally, the processing unit 550 is specifically configured to share and display the three-dimensional graffiti rendering result through a network.

[0128] It should be noted that the device embodiment in the present application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.

[0129] The following will be combined Fig.23 An electronic device or server provided in the present application is described.

[0130] See also Fig.23 Based on the above trajectory rendering method and device, the embodiment of the present application also provides another electronic device or server 800 that can execute the above trajectory rendering method. The electronic device or server 800 includes one or more (only one is shown in the figure) processors 802, a memory 804 and a network module 806 that are coupled to each other. Among them, the memory 804 stores a program that can execute the content of the above embodiment, and the processor 802 can execute the program stored in the memory 804.

[0131] Among them, the processor 802 may include one or more processing cores. The processor 802 uses various interfaces and lines to connect various parts within the entire server 800, and executes various functions and processes data of the server 800 by running or executing instructions, programs, code sets or instruction sets stored in the memory 804, and calling data stored in the memory 804. Optionally, the processor 802 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 802 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 802, but may be implemented separately through a communication chip.

[0132] The memory 804 may include a random access memory (RAM) or a read-only memory (ROM). The memory 804 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 804 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device or server 800 during use.

[0133] The network module 806 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, such as communicating with an audio playback device. The network module 806 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM) cards, memories, etc. The network module 806 may communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices via wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. For example, the network module 806 may exchange information with a base station.

[0134] Please refer to Fig.24 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 900 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0135] The computer-readable storage medium 900 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium 900 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 900 has storage space for program code 910 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 910 can be compressed, for example, in an appropriate form.

[0136] The present application provides a trajectory rendering method, device, electronic device and storage medium, which first obtains the trajectory to be rendered, which is the sliding trajectory of the touch object in the three-dimensional space, then generates three-dimensional model data based on the trajectory to be rendered, and then generates a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map. Through the above method, in the three-dimensional space, the image is drawn through the material capture map, so that the drawn image can simulate the real physical lighting effect, so that it can be more realistic.

[0137] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A trajectory rendering method, characterized in that: The method comprises: Acquire a trajectory to be rendered, where the trajectory to be rendered is a sliding trajectory of the touch object in the three-dimensional space; Based on the trajectory to be rendered, generate three-dimensional model data; The generating three-dimensional model data based on the trajectory to be rendered includes: Establishing a trajectory point combination based on every two adjacent trajectory points in the trajectory to be rendered to obtain a plurality of trajectory point combinations; Respectively obtain the circumferences corresponding to the two trajectory points in each trajectory point combination, wherein the center corresponding to the circumference is each trajectory point; With a preset smoothness corresponding to each trajectory point combination, a preset number of reference points are obtained on the circumference corresponding to the trajectory point combination to obtain a reference point corresponding to each trajectory point combination; Based on the reference point corresponding to each trajectory point combination and the trajectory points included in each trajectory point combination, generating three-dimensional model data corresponding to the trajectory to be rendered; Based on the three-dimensional model data and the material capture map, a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered is generated.

2. The method according to claim 1, characterized in that The step of obtaining the trajectory to be rendered includes: Capturing the image of the touch object in the three-dimensional space through an image acquisition device; Identifying a plurality of track points in the image drawn by the touch object; The plurality of trajectory points are smoothed to obtain trajectories to be rendered corresponding to the plurality of trajectory points.

3. The method according to claim 1, characterized in that The generating three-dimensional model data based on the trajectory to be rendered includes: Obtain preset three-dimensional model data; Obtaining the sum of distances between every two adjacent trajectory points in the trajectory to be rendered; Scaling the length of the preset three-dimensional model data to be equal to the sum of the distances to obtain scaled preset three-dimensional model data; Based on the ratio of the distance between every two adjacent trajectory points to the sum of the distances, the scaled preset three-dimensional model data is divided longitudinally to obtain reference three-dimensional model data corresponding to every two adjacent trajectory points; Based on the coordinates of every two adjacent trajectory points and the slope of the vector formed by every two adjacent trajectory points, the vertices of the reference three-dimensional model data corresponding to every two adjacent trajectory points are calculated to obtain the three-dimensional model data.

4. The method according to claim 1, characterized in that: Before generating the three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map, the method further includes: Get the preset brush material, preset lighting information and preset sphere model, and generate a material capture map.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The style of the three-dimensional graffiti rendering result is changed.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The three-dimensional graffiti rendering result is shared and displayed through the network.

7. A trajectory rendering device, characterized in that: The device comprises: A trajectory acquisition unit, used to acquire a trajectory to be rendered, where the trajectory to be rendered is a sliding trajectory of the touch object in the three-dimensional space; A data generating unit, configured to generate three-dimensional model data based on the trajectory to be rendered; the generating three-dimensional model data based on the trajectory to be rendered includes: establishing a trajectory point combination based on every two adjacent trajectory points in the trajectory to be rendered to obtain a plurality of trajectory point combinations; respectively obtaining the circumferences corresponding to the two trajectory points in each trajectory point combination, wherein the center corresponding to the circumference is each trajectory point; obtaining a preset number of reference points on the circumference corresponding to the trajectory point combination with a preset smoothness corresponding to each trajectory point combination to obtain reference points corresponding to each trajectory point combination; generating three-dimensional model data corresponding to the trajectory to be rendered based on the reference points corresponding to each trajectory point combination and the trajectory points included in each trajectory point combination; A rendering unit is used to generate a three-dimensional graffiti rendering result corresponding to the trajectory to be rendered based on the three-dimensional model data and the material capture map.

8. An electronic device, characterized in that: The method comprises one or more processors; one or more programs are stored in a memory and configured to execute the method according to any one of claims 1 to 6 by the one or more processors.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 6 is executed.

Images