Method and device for generating stroke track of virtual writing brush and electronic equipment

Abstract

The invention provides a method and a device for generating a stroke track of a virtual writing brush and electronic equipment. The method comprises the following steps: acquiring motion parameter information of an operation object; generating a continuous grid of a stroke track of a virtual writing brush in a virtual coordinate system according to at least part of parameters in the motion parameter information; and according to at least part of parameters in the motion parameter information, determining the color shade value of each pixel point in each grid in the continuous grids, and obtaining a first image of the stroke track. The method is not limited to writing of a virtual writing brush based on a two-dimensional plane any more, and the stroke track of the virtual writing brush with the stereoscopic impression can be presented according to the real-time movement process of the operation object.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to a method, apparatus, and electronic device for generating a stroke trajectory of a virtual writing brush. Background Art

[0002] With the country's increasing emphasis on carrying forward traditional culture, the promotion of brush calligraphy has become more popular. Among them, various virtual brush calligraphy writing systems based on different hardware carriers have continuously improved their application value due to their flexible use and novel forms.

[0003] In most current virtual brush writing systems, efforts are made to simulate the virtual brush writing effects to be similar to the actual brush writing, so as to make the user's writing experience closer to the actual brush writing experience. However, a common problem in existing virtual brush writing systems is that they overly focus on matching the actual paper writing effect and more on being "similar" to the actual calligraphy writing effect, such as simulating the ink bleeding effect on rice paper or the characteristics of the brush tip when writing on a flat paper surface.

[0004] It can be seen that the current virtual brush writing solutions are limited to two-dimensional plane writing, and thus cannot present three-dimensional brush writing effects. Summary of the Invention

[0005] In view of the above problems, embodiments of this application provide a method, apparatus, and electronic device for generating a stroke trajectory of a virtual writing brush that overcome or at least partially solve the above problems.

[0006] In a first aspect, embodiments of this application provide a method for generating a stroke trajectory of a virtual writing brush, including:

[0007] Obtaining motion parameter information of an operation object;

[0008] Generating a continuous grid of the stroke trajectory of the virtual writing brush in a virtual coordinate system according to at least some of the parameters in the motion parameter information;

[0009] Determining the color density values of each pixel point in each grid of the continuous grid according to at least some of the parameters in the motion parameter information to obtain a first image of the stroke trajectory.

[0010] In a second aspect, embodiments of this application provide a device for generating a stroke trajectory of a virtual writing brush, including:

[0011] A motion parameter acquisition module, configured to obtain motion parameter information of an operation object;

[0012] A grid generation module, configured to generate a continuous grid of the stroke trajectory of the virtual writing brush in a virtual coordinate system according to at least some of the motion parameter information;

[0013] A color shade determination module, configured to determine the color shade values of each pixel point in each grid of the continuous grid according to at least some of the motion parameter information, so as to obtain a first image of the stroke trajectory.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method for generating the stroke trajectory of the virtual writing brush as described in the first aspect above are implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for generating the stroke trajectory of the virtual writing brush as described in the first aspect above are implemented.

[0016] The technical solution of the embodiment of the present application can obtain the motion parameter information of the operation object, so as to generate a continuous grid of the stroke trajectory of the virtual writing brush in a virtual coordinate system according to at least some of the motion parameter information, and then determine the color shade values of each pixel point in each grid of the continuous grid according to at least some of the motion parameter information, so as to obtain a first image of the stroke trajectory.

[0017] It can be seen that in the embodiment of the present application, a continuous grid of the stroke trajectory of the virtual writing brush can be constructed in a virtual coordinate system according to the motion process of the operation object, and the color shade values of the pixel points in each grid can be determined; wherein, the construction of the continuous grid and the color shade values of the pixel points of the grid are both related to the motion parameters of the operation object. In this way, when the motion parameters of the operation object change, the construction result of the grid and the color shade values of the grid pixel points will change, thereby realizing the real-time simulation of the writing of the virtual writing brush based on the motion of the operation object. Moreover, the comparison of the color shades of the continuous grid can make the stroke trajectory of the virtual writing brush present a three-dimensional sense.

[0018] It can be known from this that the embodiment of the present application is no longer limited to the writing of the virtual writing brush based on a two-dimensional plane, but can present a three-dimensional stroke trajectory of the virtual writing brush according to the real-time motion process of the operation object. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram showing the method for generating the stroke trajectory of the virtual writing brush provided by the embodiment of the present application;

[0020] Figure 2 One of the schematic diagrams of the continuous grid in the embodiments of the present application;

[0021] Figure 3 Another schematic diagram of the continuous grid in the embodiments of the present application;

[0022] Figure 4 Schematic diagram of the mapping relationship between the movement speed and the stroke width in the embodiments of the present application;

[0023] Figure 5 Schematic flowchart of determining the color shade value of the pixel points of the continuous grid in the embodiments of the present application;

[0024] Figure 6 Schematic diagram of the mapping relationship between the movement speed and the transparency in the embodiments of the present application;

[0025] Figure 7 Schematic diagram of the process of simulating fluid emission in the embodiments of the present application;

[0026] Figure 8 Schematic diagram of the cleaning process in the embodiments of the present application;

[0027] Figure 9 Schematic diagram of the specific implementation of the method for generating the stroke trajectory of the virtual writing brush in the embodiments of the present application;

[0028] Figure 10 Schematic flowchart of the specific implementation of step 2 in the specific implementation manner of the present application;

[0029] Figure 11 Schematic flowchart of the specific implementation of step 3 in the specific implementation manner of the present application;

[0030] Figure 12 Schematic flowchart of the specific implementation of step 5 in the specific implementation manner of the present application;

[0031] Figure 13 Schematic diagram of the stroke trajectory of the virtual writing brush obtained when the continuous grid in the embodiments of the present application is a 3D grid;

[0032] Figure 14 Schematic diagram of the three-dimensional stroke trajectory with simulated fluid effect in the embodiments of the present application;

[0033] Figure 15 Schematic diagram of the device for generating the stroke trajectory of the virtual writing brush provided in the embodiments of the present application;

[0034] Figure 16 Schematic diagram of the structure of the electronic device provided in the embodiments of the present application. Specific implementation manner

[0035] Next, in combination with the accompanying drawings in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present application.

[0036] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present application. Therefore, the "in one embodiment" or "in an embodiment" that appears throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics can be combined in one or more embodiments in any suitable manner. Multiple in the embodiments of the present application can include two and more than two.

[0037] In various embodiments of the present application, it should be understood that the magnitude of the sequence numbers of the following processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0038] See Figure 1 , the embodiments of the present application provide a method for generating a stroke trajectory of a virtual writing brush. The method may include the following steps 101 to 103:

[0039] Step 101: Obtain the motion parameter information of the operation object.

[0040] Among them, the operation object can be a human body part (such as a hand), or a control device (such as a handle).

[0041] In some embodiments, when the operation object is a hand, the motion parameter information of the hand can be collected by a depth camera. Among them, the depth camera can be, for example, a Kinect (that is, a 3D body sensor camera) or a RealSense camera.

[0042] Alternatively, in some embodiments, when the operation object is a control device, the motion parameter information of the control device can be obtained by a tracker (Valvetracker) or visual simultaneous localization and mapping (V-SLAM) technology.

[0043] Optionally, the motion parameter information includes at least one of the position, motion speed, and motion direction of the operation object.

[0044] Optionally, in step 101, the obtaining of the motion parameter information of the operation object includes the following steps A-1 or A-2:

[0045] Step A-1: When it is detected that the operation object presents a first state, start obtaining the motion parameter information of the operation object, and stop obtaining the motion parameter information of the operation object until it is detected that the operation object presents a second state;

[0046] Step A-2: When a first control instruction is received, start obtaining the motion parameter information of the operation object, and stop obtaining the motion parameter information of the operation object until a second control instruction is received.

[0047] As can be seen from step A-1, the start and end of obtaining the motion parameter information of the operation object (i.e., the start and end of controlling the stroke trajectory of the virtual writing brush) can be controlled by different states of the operation object. In some embodiments, when the operation object is a hand, the first state may be a fist state, and the second state may be a finger extension state; it can be understood that when the operation object is a hand, the first state and the second state may also be two other different states of the hand respectively, which will not be listed one by one here.

[0048] As can be seen from step A-2, the start and end of obtaining the motion parameter information of the operation object (i.e., the start and end of controlling the stroke trajectory of the virtual writing brush) can be controlled by instructions. In some embodiments, when the operation object is a control device, a first operation button and a second operation button may be set on the control device. Among them, when the first operation button is pressed, the control device can send a first control instruction. In this way, the device that receives the first control instruction can start obtaining the motion parameter information of the control device; when the second operation button is pressed, the control device can send a second control instruction. In this way, the device that receives the second control instruction can stop obtaining the motion parameter information of the control device.

[0049] It can be understood that other methods can also be used to control the start and end of obtaining the motion parameter information of the operation object (i.e., the start and end of controlling the stroke trajectory of the virtual writing brush), such as voice control, etc., which will not be listed one by one here.

[0050] Step 102: Generate a continuous grid of the stroke trajectory of the virtual writing brush in the virtual coordinate system according to at least some of the parameters in the motion parameter information.

[0051] Among them, the continuous grid is a geometric representation method in computer graphics. In the embodiments of the present application, the stroke trajectory of the virtual writing brush is represented by the continuous grid, or it can also be understood that: the continuous grid can present the outline of the stroke trajectory of the virtual writing brush. Optionally, the continuous grid may include a plurality of triangular grids.

[0052] It should be noted that in the embodiments of the present application, the continuous grid is constructed according to at least some of the motion parameter information of the operation object. Therefore, the specific presentation state of the continuous grid depends on the motion process of the operation object.

[0053] Step 103: Determine the color shade values of each pixel point in each grid of the continuous grid according to at least some of the motion parameter information, and obtain the first image of the stroke trajectory.

[0054] Among them, the color shade value can be represented by a gray value. Usually, the range of the gray value is from 0 to 255, where white is 255 and black is 0; the higher the gray value, the lighter the color; the lower the gray value, the darker the color.

[0055] In addition, the shade of the color can present the advancing and retreating relationship of the color, with dark colors retreating and light colors advancing; high-purity colors can stimulate the sense of space more than low-purity colors; therefore, in the embodiments of the present application, the contrast of the color shade of the continuous grid can make the stroke trajectory of the virtual writing brush present a certain three-dimensional sense.

[0056] It can be seen from the above steps 101 to 103 that in the embodiments of the present application, the motion parameter information of the operation object can be obtained, so that according to at least some of the motion parameter information, a continuous grid of the stroke trajectory of the virtual writing brush in the virtual coordinate system is generated, and then according to at least some of the motion parameter information, the color shade values of each pixel point in each grid of the continuous grid are determined, and the first image of the stroke trajectory is obtained.

[0057] It can be seen that in the embodiments of the present application, a continuous grid of the stroke trajectory of the virtual writing brush can be constructed in the virtual coordinate system according to the motion process of the operation object, and the color shade values of the pixel points in each grid are determined; among them, the construction of the continuous grid and the color shade values of the pixel points of the grid are both related to the motion parameters of the operation object. In this way, when the motion parameters of the operation object change, the construction result of the grid and the color shade values of the grid pixel points will change, thus realizing the real-time simulation of the writing of the virtual writing brush based on the motion of the operation object. And the contrast of the color shade of the continuous grid can make the stroke trajectory of the virtual writing brush present a three-dimensional sense.

[0058] It can be seen from this that the embodiments of the present application are no longer limited to the writing of the virtual writing brush based on a two-dimensional plane, but can present the stroke trajectory of the virtual writing brush with a three-dimensional sense according to the real-time motion process of the operation object.

[0059] Optionally, the virtual coordinate system is a 3D coordinate system, and at least some of the meshes in the continuous mesh are in different planes, that is, the continuous mesh is a 3D mesh.

[0060] Wherein, when the continuous mesh is in the virtual 3D coordinate system, a three-dimensional stroke trajectory can be presented through the continuous mesh. For example, when the continuous mesh includes a plurality of triangular meshes, if at least some of the plurality of triangular meshes are in different planes, a three-dimensional stroke trajectory can be presented.

[0061] It can be seen that in some embodiments of the application, based on the movement process of the operating object, a continuous 3D mesh can be constructed in the virtual 3D coordinate system, thereby further enhancing the three-dimensional effect of the stroke trajectory of the virtual writing brush.

[0062] It can be understood that the virtual coordinate system can also be a 2D coordinate system, and the continuous mesh can also be in the same plane.

[0063] The construction process of the continuous mesh in step 102 is specifically introduced below:

[0064] Optionally, in the above step 102, generating a continuous mesh of the stroke trajectory of the virtual writing brush in the virtual coordinate system according to at least some of the parameters in the motion parameter information includes the following steps B-1 to B-2:

[0065] Step B-1: Determine the coordinates of the i-th trajectory key point of the stroke trajectory in the virtual coordinate system according to at least some of the motion parameters in the motion parameter information of the operating object when collecting the i-th frame image of the operating object, and based on the coordinates of the i-th trajectory key point, determine the coordinates of two stroke contour points corresponding to the i-th trajectory key point, where i is an integer from 1 to n;

[0066] Step B-2: When the stroke contour points corresponding to n trajectory key points are obtained, connect every three adjacent stroke contour points to obtain the continuous mesh, wherein the areas of the meshes in the continuous mesh do not overlap.

[0067] It should be noted that the above trajectory key point can be understood as: a point on the first trajectory line, where the first trajectory line is: the motion trajectory of the operating object in the space coordinate system, mapped to the trajectory in the virtual coordinate system. The stroke contour point can be understood as: a point on the contour of the stroke trajectory.

[0068] Among them, the image of the operation object can be collected at a preset first frequency. In this way, multiple frames of images of the operation object can be collected within a period of time. For each frame of image, based on at least some of the motion parameters in the motion operation information of the operation object when collecting this frame of image, a trajectory key point and two stroke contour points can be obtained in the virtual coordinate system.

[0069] It can be seen from this that based on multiple frames of images of the operation object, multiple trajectory key points and the stroke contour lines corresponding to each trajectory key point can be obtained. In this way, by connecting every three adjacent stroke contour lines, a continuous grid including triangular meshes can be obtained.

[0070] For example, in the first example, as Figure 2 shown, P0 is a trajectory key point obtained based on the first frame of image, a01 and a02 are two stroke contour points corresponding to P0, P1 is a trajectory key point obtained based on the second frame of image, a11 and a12 are two stroke contour points corresponding to P1, P2 is a trajectory key point obtained based on the third frame of image, a21 and a22 are two stroke contour points corresponding to P2, P3 is a trajectory key point obtained based on the fourth frame of image, a31 and a32 are two stroke contour points corresponding to P3. In this way, by connecting every three adjacent stroke contour points and ensuring that the areas of the obtained meshes do not overlap, the following triangular meshes can be obtained:

[0071] The triangular mesh with a01, a02, and a11 as vertices, the triangular mesh with a02, a12, and a11 as vertices, the triangular mesh with a11, a12, and a21 as vertices, the triangular mesh with a12, a21, and a22 as vertices, the triangular mesh with a21, a22, and a31 as vertices, the triangular mesh with a22, a31, and a32 as vertices.

[0072] It should be noted that the various stroke contour points in the first example can also be connected into a triangular mesh as Figure 3 shown.

[0073] Optionally, the motion parameter information includes position; in the above step B-1, the step of determining the coordinates of the i-th trajectory key point of the stroke trajectory in the virtual coordinate system according to at least some of the motion parameters in the motion parameter information of the operation object when collecting the i-th frame of image of the operation object includes the following step B-1.1:

[0074] Step B-1.1: Determine the coordinates of the i-th trajectory key point of the stroke trajectory in the virtual coordinate system according to the spatial coordinates corresponding to the position of the operation object when collecting the i-th frame of image and the first mapping relationship;

[0075] Among them, the first mapping relationship is the mapping relationship between the space coordinate system and the virtual coordinate system.

[0076] It can be seen from this that the mapping relationship between the space coordinate system of the space where the operation object is located and the virtual coordinate system can be established in advance. In this way, the coordinates of the operation object in the space coordinate system can be mapped to the virtual coordinate system; that is, the position of the operation object in the space coordinate system determines the position of the trajectory key point in the virtual coordinate system.

[0077] Optionally, the motion parameter information includes motion speed and motion direction; in the above step B-1, determining the coordinates of two stroke profile points corresponding to the i-th trajectory key point based on the coordinates of the i-th trajectory key point includes the following steps B-1.2 to B-1.3:

[0078] Step B-1.2: Determine the stroke width corresponding to the i-th trajectory key point according to the motion speed of the operation object when collecting the i-th frame of image.

[0079] Step B-1.3: Determine the coordinates of two stroke profile points corresponding to the i-th trajectory key point according to the coordinates of the i-th trajectory key point, the stroke width, and the motion direction of the operation object when collecting the i-th frame of image.

[0080] It can be seen from this that the stroke width of the stroke trajectory is related to the motion speed of the operation object. For example, the mapping curve between the motion speed of the operation object and the stroke width can be determined in advance (such as Figure 4 shown). In this way, based on the motion speed of the operation object, the stroke width of the stroke trajectory can be obtained; furthermore, based on the trajectory key point, the stroke width, and the motion direction of the operation object, the stroke profile points located on the contour of the stroke trajectory can be determined.

[0081] Optionally, the absolute value of the difference between the angle between the first straight line and the second straight line and 90° is less than or equal to the first threshold, where the first straight line is: the straight line passing through the two stroke profile points corresponding to the i-th trajectory key point, and the second straight line is: the straight line parallel to the motion direction of the operation object when collecting the i-th frame of image and passing through the i-th trajectory key point;

[0082] The absolute value of the difference between the first distance and the second distance is less than or equal to the second threshold, where the first distance is: the distance between the i-th trajectory key point and one of the stroke profile points corresponding to the i-th trajectory key point, and the second distance is: the distance between the i-th trajectory key point and the other stroke profile point corresponding to the i-th trajectory key point.

[0083] In some embodiments, the first straight line is perpendicular to the second straight line, and the i-th trajectory key point is located at the midpoint of the first line segment (i.e., the above-mentioned first distance is equal to the second distance); the first line segment is: a line segment with two stroke contour points corresponding to the i-th trajectory key point as endpoints.

[0084] For example Figure 2 As shown, after obtaining the trajectory key point P0, a second straight line 202 passing through P0 and parallel to the movement direction can be drawn, and then a first straight line 201 perpendicular to the second straight line 202 and passing through P0 can be drawn. In this way, two points (i.e., a01 and a02) on the first straight line 201 at a distance of 1 / 2 * (the stroke width corresponding to P0) from P0 are the two stroke contour points corresponding to P0.

[0085] Similarly Figure 2 The determination processes of the two stroke contour points corresponding to P1 (i.e., a11 and a12), the two stroke contour points corresponding to P2 (i.e., a21 and a22), and the two stroke contour points corresponding to P3 (i.e., a31 and a32) in [reference] are the same as the determination process of the two stroke contour points corresponding to P0 above, and will not be elaborated here.

[0086] Optionally, the method further includes the following steps C-1 to C-3:

[0087] Step C-1: When i is greater than 2, after obtaining the coordinates of the i-th trajectory key point, obtain the third distance between the i-th trajectory key point and the (i - 1)-th trajectory key point, and the first angle by which the second movement direction deviates from the first movement direction, where the first movement direction is: the movement direction of the operation object when collecting the (i - 1)-th frame image, and the second movement direction is: the movement direction of the operation object when collecting the i-th frame image;

[0088] Step C-2: When the third distance is greater than or equal to the third threshold and the first angle is greater than or equal to the fourth threshold, retain the i-th trajectory key point;

[0089] Step C-3: When the third distance is less than the third threshold or the first angle is less than the fourth threshold, delete the i-th trajectory key point.

[0090] It can be seen from steps C-1 to C-3 that in order to avoid the trajectory key points being too dense and causing unnecessary jitter in the stroke trajectory, after obtaining 2 trajectory key points, then for each subsequent obtained trajectory key point, it is possible to determine whether to retain Pi based on the third distance between Pi and P i-1 and the first angle by which the second movement direction deviates from the first movement direction, where Pi represents the obtained i-th trajectory key point.

[0091] Wherein, if the third distance is less than the third threshold or the first angle is less than the fourth threshold, it indicates that the distance between the i-th trajectory key point and the obtained trajectory key points is relatively close, or the deflection angle is relatively small. If the i-th trajectory key point is retained, the trajectory key points will be too dense. In this case, the i-th trajectory point can be deleted.

[0092] Optionally, the method further includes the following step D-1:

[0093] Step D-1: When m trajectory key points are obtained, smooth the m trajectory key points, where m is an integer greater than 1.

[0094] As can be seen from step D-1, after obtaining continuous m trajectory key points, in order to further optimize the stroke trajectory, the existing m trajectory key points can be further smoothed, such as avoiding overly sharp and steep turns.

[0095] The following specifically introduces the process of determining the color shade value of the continuous grid in step 103:

[0096] Optionally, the motion parameter information includes the motion speed; in the above step 103, determining the color shade value of each pixel point in each grid of the continuous grid according to at least some of the parameters in the motion parameter information includes the following steps E-1.1 to E-1.3:

[0097] Step E-1.1: Map the j-th grid of the continuous grid in the virtual coordinate system to the texture image in the image coordinate system to obtain the j-th mapped grid, where j is an integer from 1 to k, and k is the number of grids included in the continuous grid;

[0098] Step E-1.2: Determine the transparency of each pixel point in the j-th grid according to the motion speed corresponding to the stroke contour points of the j-th grid, where the motion speed corresponding to the stroke contour points is the same as the motion speed corresponding to the trajectory key points corresponding to the stroke contour points, and the motion speed corresponding to the i-th trajectory key point is: the motion speed of the operating object when the i-th frame of image is collected;

[0099] Step E-1.3: Determine the color shade value of each pixel point in the j-th grid according to the color shade value of each pixel point in the j-th mapped grid and the transparency of each pixel point in the j-th grid.

[0100] It should be noted that the stroke contour points of any grid in the continuous grid refer to: the stroke contour points forming the grid, that is, the vertices of the grid.

[0101] It should also be noted that the texture image is preset.

[0102] For example Figure 2 As shown, 6 triangular meshes are currently obtained. Among them, for the triangular mesh with vertices a01, a02, and a11: This triangular mesh can be mapped onto the texture image in the image coordinate system. In this way, in the texture image in the image coordinate system, there are pixel points corresponding to each pixel point in this triangular mesh. Then, the initial color shade value of each pixel point in this triangular mesh is the color shade value of the pixel point corresponding to it in the texture image. Additionally, the speed of movement of the operation object affects the shade of the color of the stroke trajectory of the virtual brush. Therefore, the movement speed corresponding to the trajectory key point can be determined according to the movement speed of the operation object, so as to determine the movement speed corresponding to the stroke contour point corresponding to the trajectory key point. Furthermore, according to the movement speed corresponding to the stroke contour points forming the triangular mesh, the transparency of each pixel point in this triangular mesh is determined. Then, based on the initial color shade value and transparency of the same pixel point in this triangular mesh, the final color shade value of this pixel point is obtained.

[0103] It can be understood that Figure 2 The process of determining the color shade values of the pixel points of other triangular meshes in [] is the same as the above process and will not be elaborated here.

[0104] As can be seen from the above, based on the pre-set texture image, after mapping according to the continuous mesh shown in Figure 2 strokes with different color shades can be obtained, as shown in Figure 5 shown.

[0105] It can be seen that in the embodiments of the present application, the color shade values of the pixel points of the continuous mesh of the stroke trajectory of the virtual brush in the virtual coordinate system are determined according to the color shade values of the pixel points of the pre-set texture image and the movement speed of the operation object, so that the image of the obtained stroke trajectory can have a color shade that matches the movement process of the operation object.

[0106] Optionally, after the above step 103, after determining the color shade values of each pixel point of each mesh in the continuous mesh and obtaining the above first image, the first image can be further rendered to obtain a second image as shown in Figure 5 so that the stroke trajectory of the virtual brush has a trailing effect.

[0107] Optionally, the mapping of the j-th mesh of the continuous mesh in the virtual coordinate system to the texture image in the image coordinate system to obtain the j-th mapped mesh in the above step E-1.1 includes the following steps E-1.1.1 to E-1.1.2:

[0108] Step E-1.1.1: Determine the mapped points of the stroke contour points of the j-th grid in the image coordinate system according to the coordinates of the stroke contour points of the j-th grid in the virtual coordinate system and the second mapping relationship, where the second mapping relationship is the mapping relationship between the virtual coordinate system and the image coordinate system;

[0109] Step E-1.1.2: Connect the mapped points of the stroke contour points of the j-th grid in the image coordinate system to obtain the j-th mapped grid.

[0110] Among them, the j-th grid is formed by connecting stroke contour points. Therefore, when mapping the j-th grid from the virtual coordinate system to the image coordinate system, the stroke contour points of the j-th grid can be first mapped from the virtual coordinate system to the image coordinate system to obtain the corresponding mapped points, and then the mapped points of the stroke contour points of the j-th grid in the image coordinate system are connected, so that the j-th mapped grid corresponding to the j-th grid in the continuous grid can be obtained. In this way, it is not necessary to map each pixel point in the j-th grid to the image coordinate system one by one to obtain the mapped grid corresponding to the j-th grid, thereby simplifying the algorithm complexity of the mapping process and saving the processing time.

[0111] In addition, the above second mapping relationship can be determined in advance.

[0112] Optionally, in the above step E-1.1, the determining the transparency of each pixel point in the j-th grid according to the motion speed corresponding to the stroke contour points of the j-th grid includes the following steps E-1.2.1 to E-1.2.2:

[0113] Step E-1.2.1: Determine the transparency of the stroke contour points of the j-th grid according to the motion speed corresponding to the stroke contour points of the j-th grid and the third mapping relationship, where the third mapping relationship is the mapping relationship between the motion speed and the transparency;

[0114] Step E-1.2.2: Perform interpolation processing based on the transparency of the stroke contour points of the j-th grid to obtain the transparency of each pixel point in the j-th grid.

[0115] It can be seen from this that the third mapping relationship between the motion speed and the transparency can be determined in advance (for example Figure 6 as shown). In this way, when determining the transparency of each pixel point in the j-th grid, based on the motion speed corresponding to the stroke contour points of the j-th grid, the transparency of each stroke contour point in the j-th grid can be obtained, and then based on the transparency of each stroke contour point in the j-th grid, the transparency of each pixel point in the j-th grid can be interpolated.

[0116] Optionally, in the above step E-1.1, determining the transparency of each pixel point in the j-th grid according to the motion speed corresponding to the stroke contour points of the j-th grid includes the following steps E-1.2.3 to E-1.2.4:

[0117] Step E-1.2.3: Based on the motion speed corresponding to the stroke contour points of the j-th grid, perform interpolation processing to obtain the motion speed corresponding to each pixel point in the j-th grid;

[0118] Step E-1.2.4: Determine the transparency of each pixel point in the j-th grid according to the motion speed corresponding to each pixel point in the j-th grid and the third mapping relationship, where the third mapping relationship is the mapping relationship between the motion speed and the transparency.

[0119] It can be seen from this that when determining the transparency of each pixel point in the j-th grid, it is also possible to first interpolate the motion speed corresponding to each pixel point in the j-th grid based on the motion speed corresponding to each stroke contour point in the j-th grid, and then according to the pre-determined third mapping relationship between the motion speed and the transparency (for example Figure 6 as shown), obtain the transparency of each pixel point in the j-th grid.

[0120] Optionally, the method further includes the following steps F-1 to F-3:

[0121] Step F-1: Sample the first image to obtain a plurality of sampling points;

[0122] Step F-2: Using the sampling points as emission points, perform fluid emission according to pre-determined emission parameters to obtain a fluid simulation image;

[0123] Step F-3: Superimpose and display the fluid simulation image and the first image. (It should be noted that the so-called superimposed display described in the text may not be just a simple combination of the two images, but a combination of the two in a weighted summation manner according to a certain ratio.)

[0124] Among them, in step F-1, the first image of the stroke trajectory can be sampled according to a pre-determined sampling density.

[0125] In addition, the emission parameters may include at least one of liquid density, gravity, and perturbation force. After setting the emission parameters for each emission point, continuous calculation of the "emission" of fluid from the emission point and the continuous change effect of each frame of the emitted fluid in the virtual coordinate system can be started.

[0126] In addition, in the embodiments of the present application, a high degree of physical simulation effect is not pursued. Therefore, the emission parameter settings of each emission point can adopt corresponding setting strategies according to requirements such as computational complexity, richness of stroke trajectory changes, and change persistence. For example, if a higher richness of handwriting changes is desired, the perturbation force parameter can be appropriately increased.

[0127] It should be noted that the fluid simulation image obtained by fluid emission based on the emission point according to the emission parameters is a two-dimensional image. Therefore, if it is required that the image of the stroke trajectory of the virtual writing brush in the embodiments of the present application has a simulated fluid effect, the fluid simulation image can be superimposed and displayed with the first image of the stroke trajectory, so that a stroke trajectory with a three-dimensional effect and a simulated fluid effect can be obtained.

[0128] In some embodiments, after step 103, when a second image with a trailing effect is obtained by rendering the first image, the method further includes: sampling the second image to obtain a plurality of sampling points; using the sampling points as emission points, performing fluid emission according to predetermined emission parameters to obtain a fluid simulation image; and superimposing and displaying the fluid simulation image and the second image.

[0129] It should also be noted that if the stroke trajectory obtained before simulating fluid emission mainly appears white and the background is black, then the image of the stroke trajectory obtained before simulating fluid emission can be inverted so that the background is white and the stroke trajectory mainly appears black.

[0130] For example Figure 7 As shown, after further rendering the first image obtained through the aforementioned steps 101 to 103 to obtain a second image with a trailing effect, the second image can be inverted to obtain a third image; and based on the second image, fluid emission is simulated and inverted to obtain a fluid simulation image; and then the third image and the fluid simulation image are weighted and summed (i.e., superimposed or combined), so as to obtain a three-dimensional stroke trajectory with a simulated fluid effect.

[0131] It can be understood that in some embodiments, fluid emission can also be simulated based on the third image to obtain a fluid simulation image, and then the fluid simulation image and the third image are weighted and summed (i.e., superimposed or combined), and a three-dimensional stroke trajectory with a simulated fluid effect can also be obtained.

[0132] It can be seen that through simulating fluid emission, on the one hand, the smudging and diffusion characteristics during brush writing can be moderately simulated, so as to better conform to the overall visual perception of brush writing. On the other hand, the three-dimensional effect of the brushstrokes can be further highlighted, even including richer effects such as spatial flow.

[0133] Optionally, the method further includes:

[0134] When the clearing instruction is obtained, the displayed first image is cleared, and the fluid emission of the emission point is stopped.

[0135] It can be seen from this that when the first image and the fluid simulation image are superimposed and displayed, if the clearing instruction is obtained, then the first image can be cleared first, and the fluid emission of the emission point can be stopped; however, it should be noted that although the fluid emission of the emission point is stopped, the calculation of the emission point will continue for a period of time. Therefore, during the clearing process, the stroke trajectory of the writing brush will gradually disappear in a flowing state.

[0136] It can be understood that when the fluid simulation image and the second image are superimposed and displayed, if the clearing instruction is received, then the displayed second image can be cleared, and the fluid emission of the emission point can be stopped; similarly, the process in which the stroke trajectory of the writing brush gradually disappears in a flowing state can be displayed, as Figure 8 shown.

[0137] Among them, a first operation control can be set in the display interface of the stroke trajectory of the virtual writing brush. When the click operation of the first operation control is detected, the above clearing instruction can be generated; alternatively, when the operation object is detected to be in the third state, the above clearing instruction can be generated; or, the above clearing instruction can also be sent by the control device.

[0138] In summary, the specific implementation of the method for generating the stroke trajectory of the virtual writing brush according to the embodiments of the present application, as Figure 9 shown, mainly includes the following four core steps:

[0139] Step 1: Collect the motion parameters of the hand or the motion parameters of the control device (such as spatial coordinates, motion direction, motion speed);

[0140] That is, when the "start writing" instruction issued by the user through the hand or the control device is obtained, the rendering process of a stroke is started, and the spatial coordinates, motion direction, and motion speed of the hand or the control device in each frame are recorded.

[0141] Among them, for the input mechanism of barehanded air writing, depth camera devices such as Kinect and Realsense can be used to obtain the motion parameters and gesture information of the user's (for writing) hand in the three-dimensional space in each frame. The change of the gesture is mainly used to control states such as "start writing / stop writing".

[0142] For input mechanisms of control devices such as handles, motion parameters of the control device in each frame can be obtained based on methods such as Valve tracker or V-SLAM. At the same time, the user's key information can also be obtained for controlling states such as "start writing / stop writing".

[0143] Step 2: Generate a continuous grid of the stroke trajectory of the virtual writing brush in the virtual coordinate system according to the motion parameters of the hand or the control device;

[0144] That is, according to the three-dimensional motion information of the user waving the hand or the control device during writing, a grid of each stroke trajectory in the virtual space can be generated, so as to obtain the shape of each stroke.

[0145] Specifically, the specific implementation process of Step 2 can be as Figure 10 shown:

[0146] First, through the mapping relationship between the space coordinate system and the virtual coordinate system, the three-dimensional coordinates of the hand / control device in each frame are converted into the coordinates of a trajectory key point on the stroke trajectory of the virtual writing brush, such as Figure 2 a certain point P in.

[0147] Secondly, it is judged whether the currently obtained trajectory key point Pi needs to be retained. That is, in order to avoid the trajectory points being too dense and causing unnecessary jitter in the handwriting trajectory, it is possible to check whether the distance between two adjacent key points Pi and Pi-1, and the angle between the line segment "Pi—Pi-1" and the line segment "Pi-1—Pi-2" meet the preset distance and angle thresholds, so as to decide whether to retain the latest trajectory key point Pi.

[0148] Thirdly, smoothing processing is performed. That is, after obtaining a continuous number of trajectory key points P, in order to further optimize the writing trajectory, further smoothing processing can be performed on the existing number of trajectory key points, such as avoiding overly sharp and steep turns.

[0149] Thirdly, determine the stroke contour points corresponding to each trajectory key point. That is, after determining the trajectory key points, the motion speed of the hand / control device calculated in each frame can be combined. According to the pre-set mapping curve of the motion speed and the stroke width (such as Figure 4 shown), the stroke width corresponding to each trajectory key point is obtained. Then, according to the stroke width and the trajectory direction, the coordinates of two "stroke contour points" (that is, Figure 2 a series of points a in) can be determined on both sides of each trajectory key point.

[0150] Thirdly, obtain a continuous grid based on the stroke contour points. That is, after determining each stroke contour point, then every three adjacent contour points (for example Figure 2Connect a01, a02, and a11) in it to obtain multiple consecutive triangular meshes, thereby obtaining the continuous mesh of the entire stroke trajectory.

[0151] Step 3: Determine the color shade values of the pixel points of each mesh in the continuous mesh of the stroke trajectory to obtain the first image of the stroke trajectory;

[0152] Specifically, the specific implementation process of Step 3 can be as Figure 5 and Figure 11 shown:

[0153] First, based on the pre-determined texture image, determine the initial color shade values of each pixel point of each mesh in the continuous mesh, that is, for each mesh in the continuous mesh, map the stroke contour points of each mesh to the texture image in the image coordinate system to obtain the mapped points of the stroke contour points of each mesh. Then, connect the mapped points of the stroke contour points of the same mesh to obtain the mapped mesh of the mesh in the image coordinate system. In this way, the pixel points in a mapped mesh correspond one-to-one with the pixel points in its corresponding mesh, so the color shade values of the pixel points in the mapped mesh can be used as the initial shade values of the corresponding pixel points in its corresponding mesh;

[0154] Secondly, determine the transparency of each pixel point in each mesh according to the motion speed of the hand / control device corresponding to the stroke contour points of each mesh, that is, the transparency of the stroke contour points of each mesh can be determined according to the pre-determined mapping relationship between the motion speed and the transparency, and then, based on the transparency of the stroke contour points of the same mesh, interpolate the transparency of the remaining pixel points in the mesh;

[0155] Thirdly, multiply the initial color shade value and the transparency of the same pixel point in the continuous mesh to obtain the color shade values of the pixel points of each mesh in the continuous mesh.

[0156] It should be noted that in the case where the above virtual coordinate system is a 3D coordinate system and at least some meshes in the continuous mesh are in different planes (i.e., the continuous mesh is a 3D mesh), the three-dimensional effect of the stroke trajectory can be further enhanced. For example, a complete "sword" character obtained through the above Step 1 to Step 3 is as Figure 13 shown.

[0157] Step 4: Render the first image to form a second image with a trailing effect;

[0158] Step 5: Simulate fluid emission on the second image to obtain the complete stroke trajectory.

[0159] Among them, by adding the fluid simulation effect, the 3D effect of the handwriting and the consistency with the overall visual perception of brush writing can be further highlighted.

[0160] Specifically, the specific implementation process of step 5 can be as follows Figure 7 and Figure 12 shown as follows:

[0161] First, invert the second image to obtain a third image;

[0162] Second, simulate fluid emission based on the second image to obtain a fluid simulation image, that is, in the second image, sample the white trajectory points at a certain density, and the specific density value can be adjusted according to the actual effect; among them, the obtained sampling points will be used as the fluid emission points during fluid simulation, and each emission point contains a series of emission parameters, such as liquid density, gravity, perturbation force, etc. After setting the emission parameters of each emission point, the continuous calculation of "emitting" fluid from the emission point and the continuous change effect of each frame of the emitted fluid in three-dimensional space can be started, and then an inversion process is performed to obtain a fluid simulation image;

[0163] Third, superimpose and display the fluid simulation image and the third image, that is, based on each frame obtained, the fluid simulation image and the third image as shown above Figure 7 are weighted and summed to obtain the final mixed and fully rendered stroke trajectory of each virtual brush calligraphy (for example Figure 7 the three-dimensional stroke trajectory with simulated fluid effect as shown). Exemplarily, the three-dimensional stroke trajectory with simulated fluid effect of a complete character "big" can be as Figure 14 shown.

[0164] It can be seen that based on the above complete process of rendering each stroke, the rendered stroke trajectory combines three-dimensional shape, three-dimensional shading change, and three-dimensional fluid effect, thus jointly ensuring that the final result has both a three-dimensional (dynamic) visual effect and maintains the proper consistency with the style of calligraphy writing itself. It can be seen that the 3D effects of basically all strokes are relatively obvious, and at the same time, its overall visual effect does not conflict with the calligraphy writing style.

[0165] Therefore, it can be known that the method for generating the stroke trajectory of the virtual brush in the embodiment of the present application can make the virtual brush handwriting written present a 3D visual effect that is significantly different from the actual two-dimensional plane writing handwriting. At the same time, its 3D rendering effect still fits well with the overall style of calligraphy writing, does not conflict with the general visual effect of calligraphy writing, and at the same time, the writing process operation of the user also maintains good naturalness and freedom.

[0166] Referring to Figure 15 , the embodiment of the present application provides a device for generating the stroke trajectory of a virtual brush, as Figure 15 shown, the device may include the following modules:

[0167] A motion parameter acquisition module 1501, configured to acquire motion parameter information of an operating object;

[0168] A grid generation module 1502, configured to generate a continuous grid of the stroke trajectory of a virtual writing brush in a virtual coordinate system according to at least some of the parameters in the motion parameter information;

[0169] A color shade determination module 1503, configured to determine color shade values of each pixel point in each grid in the continuous grid according to at least some of the parameters in the motion parameter information, so as to obtain a first image of the stroke trajectory.

[0170] Optionally, the virtual coordinate system is a 3D coordinate system, and at least some of the grids in the continuous grid are in different planes.

[0171] Optionally, the grid generation module 1502 includes:

[0172] A coordinate determination sub-module, configured to determine coordinates of an i-th trajectory key point of the stroke trajectory in the virtual coordinate system according to at least some motion parameters in the motion parameter information of the operating object when the i-th frame image of the operating object is acquired, and determine coordinates of two stroke contour points corresponding to the i-th trajectory key point based on the coordinates of the i-th trajectory key point, where i is an integer from 1 to n;

[0173] A connection sub-module, configured to connect every three adjacent stroke contour points when stroke contour points corresponding to n trajectory key points are obtained, so as to obtain the continuous grid, where regions of the grids in the continuous grid do not overlap.

[0174] Optionally, the motion parameter information includes position;

[0175] The coordinate determination sub-module determines coordinates of the i-th trajectory key point of the stroke trajectory in the virtual coordinate system according to at least some motion parameters in the motion parameter information of the operating object when the i-th frame image of the operating object is acquired, including:

[0176] Determining coordinates of the i-th trajectory key point of the stroke trajectory in the virtual coordinate system according to spatial coordinates corresponding to the position of the operating object when the i-th frame image is acquired and a first mapping relationship;

[0177] Wherein, the first mapping relationship is a mapping relationship between a spatial coordinate system and the virtual coordinate system.

[0178] Optionally, the motion parameter information includes motion speed and motion direction;

[0179] The coordinate determination sub-module determines the coordinates of two stroke contour points corresponding to the $i$-th trajectory key point based on the coordinates of the $i$-th trajectory key point, including:

[0180] Determine the stroke width corresponding to the $i$-th trajectory key point according to the movement speed of the operation object when the $i$-th frame of image is collected;

[0181] Determine the coordinates of two stroke contour points corresponding to the $i$-th trajectory key point according to the coordinates of the $i$-th trajectory key point, the stroke width, and the movement direction of the operation object when the $i$-th frame of image is collected.

[0182] Optionally, the absolute value of the difference between the angle between the first straight line and the second straight line and 90° is less than or equal to the first threshold, where the first straight line is: the straight line passing through the two stroke contour points corresponding to the $i$-th trajectory key point, and the second straight line is: parallel to the movement direction of the operation object when the $i$-th frame of image is collected and passing through the $i$-th trajectory key point;

[0183] The absolute value of the difference between the first distance and the second distance is less than or equal to the second threshold, where the first distance is: the distance between the $i$-th trajectory key point and one of the stroke contour points corresponding to the $i$-th trajectory key point, and the second distance is: the distance between the $i$-th trajectory key point and the other stroke contour point corresponding to the $i$-th trajectory key point.

[0184] Optionally, the device further includes: a trajectory key point processing module, configured to:

[0185] When $i$ is greater than 2, after obtaining the coordinates of the $i$-th trajectory key point, obtain the third distance between the $i$-th trajectory key point and the $(i - 1)$-th trajectory key point, and the first angle by which the second movement direction deviates from the first movement direction, where the first movement direction is: the movement direction of the operation object when the $(i - 1)$-th frame of image is collected, and the second movement direction is: the movement direction of the operation object when the $i$-th frame of image is collected;

[0186] When the third distance is greater than or equal to the third threshold and the first angle is greater than or equal to the fourth threshold, retain the $i$-th trajectory key point;

[0187] When the third distance is less than the third threshold or the first angle is less than the fourth threshold, delete the $i$-th trajectory key point.

[0188] Optionally, the device further includes:

[0189] A smoothing module, configured to perform smoothing processing on the m trajectory key points when the m trajectory key points are obtained, where m is an integer greater than 1.

[0190] Optionally, the motion parameter information includes a motion speed;

[0191] The color shade determination module 1503 includes:

[0192] A grid mapping sub-module, configured to map the j-th grid of the continuous grid in the virtual coordinate system to a mapped image in the image coordinate system, to obtain the j-th mapped grid, where j is an integer from 1 to k, and k is the number of grids included in the continuous grid;

[0193] A transparency determination sub-module, configured to determine the transparency of each pixel point in the j-th grid according to the motion speed corresponding to the stroke contour points of the j-th grid, where the motion speed corresponding to the stroke contour points is the same as the motion speed corresponding to the trajectory key points corresponding to the stroke contour points, and the motion speed corresponding to the i-th trajectory key point is: the motion speed of the operation object when the i-th frame of image is collected;

[0194] A color shade value determination sub-module, configured to determine the color shade value of each pixel point in the j-th grid according to the color shade value of each pixel point in the j-th mapped grid and the transparency of each pixel point in the j-th grid.

[0195] Optionally, the grid mapping sub-module is specifically configured to:

[0196] According to the coordinates of the stroke contour points of the j-th grid in the virtual coordinate system and a second mapping relationship, determine the mapped points of the stroke contour points of the j-th grid in the image coordinate system, where the second mapping relationship is the mapping relationship between the virtual coordinate system and the image coordinate system;

[0197] Connect the mapped points of the stroke contour points of the j-th grid in the image coordinate system to obtain the j-th mapped grid.

[0198] Optionally, the transparency determination sub-module is specifically configured to:

[0199] According to the motion speed corresponding to the stroke contour points of the j-th grid and a third mapping relationship, determine the transparency of the stroke contour points of the j-th grid, where the third mapping relationship is the mapping relationship between the motion speed and the transparency;

[0200] Based on the transparency of the stroke contour points of the j-th grid, perform interpolation processing to obtain the transparency of each pixel point in the j-th grid.

[0201] Optionally, the device further includes: a fluid emission module, configured to:

[0202] Sample the first image to obtain a plurality of sampling points;

[0203] Using the sampling points as emission points, perform fluid emission according to pre-determined emission parameters to obtain a fluid simulation image;

[0204] Overlay and display the fluid simulation image with the first image.

[0205] Optionally, the device further includes:

[0206] A clearing module, configured to: when a clearing instruction is obtained, clear the displayed first image and stop the fluid emission at the emission points.

[0207] Optionally, the motion parameter acquisition module 1501 is specifically configured to:

[0208] When it is detected that the operation object presents a first state, start acquiring the motion parameter information of the operation object until it is detected that the operation object presents a second state, and then stop acquiring the motion parameter information of the operation object;

[0209] Or,

[0210] When a first control instruction is received, start acquiring the motion parameter information of the operation object until a second control instruction is received, and then stop acquiring the motion parameter information of the operation object.

[0211] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For related parts, refer to the partial description of the method embodiment.

[0212] An embodiment of the present application further provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, it implements each process of the method embodiment for generating the stroke trajectory of the virtual writing brush as described above, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0213] For example, Figure 16 The schematic physical structure diagram of an electronic device is shown. As Figure 16As shown in the figure, the electronic device may include: a processor 1610, a communications interface 1620, a memory 1630, and a communication bus 1640. Among them, the processor 1610, the communications interface 1620, and the memory 1630 communicate with each other through the communication bus 1640. The processor 1610 may call the logic instructions in the memory 1630. The processor 1610 is used to execute each process of the method for generating the stroke trajectory of the virtual writing brush in the embodiments of the present application, which will not be further elaborated here.

[0214] In addition, when the logic instructions in the above-mentioned memory 1630 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0215] The embodiments of the present application also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, it implements each process of the above-mentioned embodiment of the method for generating the stroke trajectory of the virtual writing brush and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0216] It should be noted that in this article, the term "including", "comprising", or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or device including a series of elements includes not only those elements but also other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or device. Without further limitations, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article, or device including that element.

[0217] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, disk, optical disc), and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0218] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims. All of them fall within the protection scope of the present application.

[0219] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present application.

[0220] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0221] In the embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling, direct coupling, or communication connection can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0222] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0223] In addition, each functional unit in various embodiments of the present application may be integrated into a processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit.

[0224] If the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0225] As described above, the above are only the specific implementation manners of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for generating a stroke trajectory of a virtual writing brush, characterized in that, The method includes: Obtaining motion parameter information of an operation object; Generating a continuous grid of the stroke trajectory of a virtual writing brush in a virtual coordinate system according to at least some of the parameters in the motion parameter information; Determining the color shade values of each pixel point in each grid in the continuous grid according to at least some of the parameters in the motion parameter information, to obtain a first image of the stroke trajectory.

2. The method according to claim 1, wherein The virtual coordinate system is a 3D coordinate system, and at least some of the grids in the continuous grid are in different planes.

3. The method according to claim 1 or 2, characterized in that, The generating a continuous grid of the stroke trajectory of a virtual writing brush in a virtual coordinate system according to at least some of the parameters in the motion parameter information includes: Determining the coordinates of the i-th trajectory key point of the stroke trajectory in the virtual coordinate system according to at least some of the motion parameters in the motion parameter information of the operation object when collecting the i-th frame image of the operation object, and based on the coordinates of the i-th trajectory key point, determining the coordinates of two stroke contour points corresponding to the i-th trajectory key point, where i is an integer from 1 to n; When obtaining the stroke contour points corresponding to n trajectory key points, connecting every three adjacent stroke contour points to obtain the continuous grid, where the regions of the grids in the continuous grid do not overlap.

4. The method according to claim 3, characterized in that, The motion parameter information includes position; The determining the coordinates of the i-th trajectory key point of the stroke trajectory in the virtual coordinate system according to at least some of the motion parameters in the motion parameter information of the operation object when collecting the i-th frame image of the operation object includes: Determining the coordinates of the i-th trajectory key point of the stroke trajectory in the virtual coordinate system according to the spatial coordinates corresponding to the position of the operation object when collecting the i-th frame image and a first mapping relationship; Wherein, the first mapping relationship is the mapping relationship between the spatial coordinate system and the virtual coordinate system.

5. The method according to claim 3, characterized in that, The motion parameter information includes motion speed and motion direction; The determining the coordinates of two stroke contour points corresponding to the i-th trajectory key point based on the coordinates of the i-th trajectory key point includes: Determining the stroke width corresponding to the i-th trajectory key point according to the motion speed of the operation object when collecting the i-th frame image; Determining the coordinates of two stroke contour points corresponding to the i-th trajectory key point according to the coordinates of the i-th trajectory key point, the stroke width, and the motion direction of the operation object when collecting the i-th frame image.

6. The method according to claim 5, characterized in that, The absolute value of the difference between the angle between the first straight line and the second straight line and 90° is less than or equal to a first threshold, where the first straight line is: the straight line passing through the two stroke contour points corresponding to the i-th trajectory key point, and the second straight line is: the straight line parallel to the motion direction of the operation object when collecting the i-th frame image and passing through the i-th trajectory key point; The absolute value of the difference between the first distance and the second distance is less than or equal to a second threshold, where the first distance is the distance between the i-th trajectory key point and one of the stroke contour points corresponding to the i-th trajectory key point, and the second distance is the distance between the i-th trajectory key point and the other stroke contour point corresponding to the i-th trajectory key point.

7. The method according to claim 3, wherein The method further includes: When i is greater than 2, after obtaining the coordinates of the i-th trajectory key point, acquiring a third distance between the i-th trajectory key point and the (i - 1)-th trajectory key point, and a first angle by which the second movement direction deviates from the first movement direction, where the first movement direction is the movement direction of the operating object when the (i - 1)-th frame of image is collected, and the second movement direction is the movement direction of the operating object when the i-th frame of image is collected; When the third distance is greater than or equal to a third threshold and the first angle is greater than or equal to a fourth threshold, retaining the i-th trajectory key point; When the third distance is less than the third threshold or the first angle is less than the fourth threshold, deleting the i-th trajectory key point.

8. The method according to claim 3, wherein The method further includes: When m trajectory key points are obtained, performing smoothing processing on the m trajectory key points, where m is an integer greater than 1.

9. The method according to claim 3, wherein The motion parameter information includes a motion speed; The determining the color shade value of each pixel point in each grid of the continuous grid according to at least some of the parameters in the motion parameter information includes: Mapping the j-th grid of the continuous grid in the virtual coordinate system to a texture image in the image coordinate system to obtain a j-th mapped grid, where j is an integer from 1 to k, and k is the number of grids included in the continuous grid; Determining the transparency of each pixel point in the j-th grid according to the motion speed corresponding to the stroke contour point of the j-th grid, where the motion speed corresponding to the stroke contour point is the same as the motion speed corresponding to the trajectory key point corresponding to the stroke contour point, and the motion speed corresponding to the i-th trajectory key point is the motion speed of the operating object when the i-th frame of image is collected; Determining the color shade value of each pixel point in the j-th grid according to the color shade value of each pixel point in the j-th mapped grid and the transparency of each pixel point in the j-th grid.

10. The method according to claim 9, characterized in that The mapping the j-th grid of the continuous grid in the virtual coordinate system to a texture image in the image coordinate system to obtain a j-th mapped grid includes: Determining the mapped point of the stroke contour point of the j-th grid in the image coordinate system according to the coordinates of the stroke contour point of the j-th grid in the virtual coordinate system and a second mapping relationship, where the second mapping relationship is the mapping relationship between the virtual coordinate system and the image coordinate system; Connecting the mapped points of the stroke contour points of the j-th grid in the image coordinate system to obtain the j-th mapped grid.

11. The method according to claim 9, wherein Determining the transparency of each pixel point in the j-th grid according to the motion speed corresponding to the stroke contour points of the j-th grid includes: Determining the transparency of the stroke contour points of the j-th grid according to the motion speed corresponding to the stroke contour points of the j-th grid and a third mapping relationship, where the third mapping relationship is a mapping relationship between motion speed and transparency; Performing interpolation processing based on the transparency of the stroke contour points of the j-th grid to obtain the transparency of each pixel point in the j-th grid.

12. The method according to claim 1 or 2, characterized in that, The method further includes: Sampling the first image to obtain a plurality of sampling points; Using the sampling points as emission points to perform fluid emission according to pre-determined emission parameters to obtain a fluid simulation image; Overlaying and displaying the fluid simulation image and the first image.

13. The method according to claim 12, characterized in that, The method further includes: When a clearing instruction is obtained, clearing the displayed first image and stopping the fluid emission of the emission points.

14. The method according to claim 1 or 2, characterized in that, The obtaining of the motion parameter information of the operation object includes: When it is detected that the operation object presents a first state, starting to obtain the motion parameter information of the operation object until it is detected that the operation object presents a second state, and then stopping to obtain the motion parameter information of the operation object; Or, When a first control instruction is received, starting to obtain the motion parameter information of the operation object until a second control instruction is received, and then stopping to obtain the motion parameter information of the operation object.

15. An apparatus for generating a stroke trajectory of a virtual writing brush, characterized in that, The device includes: A motion parameter acquisition module for acquiring the motion parameter information of an operation object; A grid generation module for generating a continuous grid of the stroke trajectory of a virtual writing brush in a virtual coordinate system according to at least some of the parameters in the motion parameter information; A color shade determination module for determining the color shade values of each pixel point in each grid of the continuous grid according to at least some of the parameters in the motion parameter information to obtain a first image of the stroke trajectory.

16. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the method for generating the stroke trajectory of the virtual writing brush according to any one of claims 1 to 14.

Images