Virtual element generation method and device, drawing system, computer equipment and medium

Abstract

The invention relates to a virtual element generation method and device, a drawing system, computer equipment and a medium. The method comprises the following steps: in response to a target operation of a target object, triggering a virtual element drawing request corresponding to the target operation in a virtual space; obtaining a target action of the target object based on the virtual element drawing request, wherein the target action is used for representing execution of a virtual element drawing operation corresponding to the virtual element drawing request in a virtual space; and obtaining a motion trail of the target object, and executing a virtual element drawing operation according to the motion trail of the target object to generate a virtual element. Different trigger operations corresponding to different virtual element drawing requests are set, and different trigger actions for executing the virtual element drawing operations corresponding to the virtual element drawing requests are set, so that a user draws the virtual elements in the virtual space based on the different trigger operations and the different trigger actions; the drawing operation in the virtual space is provided for the user, and the experience feeling of virtual-real interaction is improved.

Description

Technical Field

[0001] The present application relates to the field of virtual scene technology, and in particular to a virtual element generation method, device, rendering system, computer equipment and medium. Background Art

[0002] With the development of virtual reality or augmented reality technology, users in the real environment have the demand to add virtual elements to the virtual environment or virtual model.

[0003] Users can add virtual elements to a virtual scene by drawing. Traditionally, users need to use a real electronic brush to draw on a drawing board, and then map the drawing content drawn on the drawing board to the virtual scene, thereby achieving the addition of virtual elements to the virtual scene.

[0004] However, the traditional way of drawing in virtual scenes has the problem of poor virtual-reality interaction experience. Summary of the invention

[0005] Based on this, it is necessary to provide a virtual element generation method, device, rendering system, computer equipment, computer-readable storage medium and computer program product that can improve the virtual-reality interaction experience in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for generating a virtual element. The method comprises:

[0007] In response to a target operation of a target object, triggering a virtual element drawing request corresponding to the target operation in a virtual space;

[0008] Acquire a target action of a target object based on a virtual element drawing request; the target action is used to represent a virtual element drawing operation corresponding to the virtual element drawing request being performed in the virtual space;

[0009] The motion trajectory of the target object is acquired, and the virtual element drawing operation is performed according to the motion trajectory of the target object to generate a virtual element.

[0010] In one embodiment, virtual keys corresponding to different virtual element drawing requests are displayed in a virtual space, and the target operation includes a triggering operation on the virtual key; in response to the target operation of the target object, triggering a virtual element drawing request corresponding to the target operation in the virtual space includes:

[0011] In response to a triggering operation of a target virtual key by a target object, a virtual element drawing request corresponding to the triggering operation of the target virtual key is triggered in a virtual space; the target virtual key includes any one of the virtual keys corresponding to different virtual element drawing requests; the virtual element drawing request corresponding to the triggering operation of the target virtual key carries drawing attribute information corresponding to the target virtual key.

[0012] In one embodiment, virtual keys corresponding to different virtual element drawing requests are displayed in a virtual space, and the target operation includes a triggering operation on the virtual key; in response to the target operation of the target object, triggering a virtual element drawing request corresponding to the target operation in the virtual space includes:

[0013] In response to a triggering operation of the target object on the target virtual key, displaying at least one virtual sub-key corresponding to the target virtual key in the virtual space;

[0014] In response to a trigger operation on a target virtual sub-key, a virtual element drawing request corresponding to the trigger operation of the target virtual sub-key is triggered in a virtual space; the target virtual sub-key includes at least one virtual sub-key corresponding to the target virtual key; the virtual element drawing request corresponding to the trigger operation of the target virtual sub-key carries drawing attribute information corresponding to the target virtual sub-key.

[0015] In one embodiment, obtaining a target action of a target object based on a virtual element drawing request includes:

[0016] Based on the virtual element drawing request, the target object is photographed by an image acquisition device to generate a first image;

[0017] Performing image recognition on the first image to obtain an image recognition result, and acquiring a target action of the target object from the image recognition result.

[0018] In one embodiment, if the virtual element drawing request includes a graphics drawing request, the virtual element drawing operation includes a graphics drawing operation; obtaining a motion trajectory of the target object, performing the virtual element drawing operation according to the motion trajectory of the target object, and generating a virtual element includes:

[0019] The target object is photographed by an image acquisition device to generate multiple frames of second images of the target object;

[0020] Acquire a motion trajectory of a target object from a plurality of frames of second images;

[0021] The first drawing attribute information is obtained from the graphics drawing request, and the graphics drawing operation is performed according to the first drawing attribute information and the motion trajectory of the target object to generate a virtual element.

[0022] In one embodiment, performing a graphics drawing operation according to the first drawing attribute information and the motion trajectory of the target object to generate a virtual element includes:

[0023] Performing a graphics drawing operation according to the first drawing attribute information and the motion trajectory of the target object to generate an initial virtual element;

[0024] The initial virtual element is smoothed to generate a virtual element, and the virtual element is displayed in a virtual space.

[0025] In one embodiment, smoothing the initial virtual element to generate the virtual element includes:

[0026] Determine a plurality of initial drawing points and coordinate information of each initial drawing point from the initial virtual element;

[0027] According to the coordinate information of each initial drawing point, a Bezier interpolation method is used to determine the coordinate information of multiple target drawing points and each target drawing point;

[0028] Generate virtual elements based on the coordinate information of each target drawing point.

[0029] In one embodiment, based on the coordinate information of each initial drawing point, a Bezier interpolation method is used to determine the coordinate information of multiple target drawing points and each target drawing point, including:

[0030] For each initial drawing point, the coordinate information of the initial drawing point is updated by using the first Bezier interpolation method according to the coordinate information of the initial drawing point, the coordinate information of the previous drawing point of the initial drawing point, and the coordinate information of the next drawing point of the initial drawing point, so as to obtain a new initial drawing point corresponding to the initial drawing point and the coordinate information of the new initial drawing point;

[0031] According to the coordinate information of the new initial drawing point, the coordinate information of the previous drawing point and the coordinate information of the next drawing point, the second Bezier interpolation method and the golden section value are used to determine the coordinate information of the first intermediate drawing point and the first intermediate drawing point between the previous drawing point and the new initial drawing point, and the coordinate information of the second intermediate drawing point and the second intermediate drawing point between the new initial drawing point and the next drawing point;

[0032] The new initial drawing point, the first intermediate drawing point and the second intermediate drawing point are used as target drawing points.

[0033] In one embodiment, generating a virtual element according to the coordinate information of each target drawing point includes:

[0034] Generate candidate virtual elements according to the coordinate information of each target drawing point;

[0035] For each target drawing point in the candidate virtual element, determine the connecting pipeline corresponding to the target drawing point and the pipeline information of the connecting pipeline according to the pipeline information of the first pipeline and the pipeline information of the second pipeline adjacent to the target drawing point; the first pipeline is a pipeline formed based on the previous drawing point of the target drawing point and the target drawing point; the second pipeline is a pipeline formed based on the target drawing point and the next drawing point of the target drawing point;

[0036] Based on the pipeline information of the connecting pipeline corresponding to each target drawing point, the position corresponding to each target drawing point in the candidate virtual element is filled to generate a virtual element.

[0037] In one embodiment, if the virtual element drawing request includes an erase request, the virtual element drawing operation includes an erase operation; obtaining a motion trajectory of the target object, performing the virtual element drawing operation according to the motion trajectory of the target object, and generating the virtual element includes:

[0038] The target object is photographed by an image acquisition device to generate multiple frames of third images of the target object;

[0039] Acquire a motion trajectory of a target object from a plurality of frames of third images;

[0040] The second drawing attribute information is obtained from the erasing request, and an erasing operation is performed according to the second drawing attribute information and the motion trajectory of the target object to generate a virtual element.

[0041] In one of the embodiments, if the target virtual key is an erase virtual key, the at least one virtual sub-key corresponding to the target virtual key includes a partial erase sub-key and a full erase sub-key;

[0042] If the second drawing attribute information includes partial erasing attribute information, performing an erasing operation according to the second drawing attribute information and the motion trajectory of the target object to generate a virtual element includes:

[0043] According to the motion trajectory of the target object, if it is determined that the target object collides with the virtual element, a partial erasing operation is performed on the collided portion of the virtual element to generate an erased virtual element;

[0044] If the second drawing attribute information includes overall erasing attribute information, performing an erasing operation according to the second drawing attribute information and the motion trajectory of the target object to generate a virtual element includes:

[0045] According to the motion trajectory of the target object, if it is determined that the target object collides with the virtual element, an overall erasing operation is performed on the virtual element.

[0046] In one embodiment, if the virtual element drawing request includes a color brush request, the virtual element drawing operation includes a color brush operation; obtaining a motion trajectory of the target object, performing the virtual element drawing operation according to the motion trajectory of the target object, and generating the virtual element includes:

[0047] The target object is photographed by an image acquisition device to generate multiple frames of fourth images of the target object;

[0048] Acquire a motion trajectory of a target object from a plurality of fourth image frames;

[0049] The third drawing attribute information is obtained from the color brush request, and a color brush operation is performed according to the third drawing attribute information and the motion trajectory of the target object to generate a modified virtual element.

[0050] In one of the embodiments, if the target virtual key is a color brush virtual key, the at least one virtual sub-key corresponding to the target virtual key includes a partial color brush key and a whole color brush key;

[0051] If the third drawing attribute information includes part of the color brush attribute information, a color brush operation is performed according to the third drawing attribute information and the motion trajectory of the target object to generate a modified virtual element, including:

[0052] According to the motion trajectory of the target object, if it is determined that the target object collides with the virtual element, a partial color brush operation is performed on the collision portion of the virtual element to generate a modified virtual element;

[0053] If the third drawing attribute information includes overall color brush attribute information, a color brush operation is performed according to the third drawing attribute information and the motion trajectory of the target object to generate a modified virtual element, including:

[0054] According to the motion trajectory of the target object, if it is determined that the target object collides with the virtual element, an overall color brushing operation is performed on the virtual element to generate a modified virtual element.

[0055] In a second aspect, the present application also provides a virtual element generation device. The device includes:

[0056] A trigger module, configured to trigger, in response to a target operation of a target object, a virtual element drawing request corresponding to the target operation in a virtual space;

[0057] A display module, used for acquiring a target action of a target object based on a virtual element drawing request; the target action is used for representing a virtual element drawing operation corresponding to the virtual element drawing request executed in a virtual space;

[0058] The drawing module is used to obtain the motion trajectory of the target object, perform a virtual element drawing operation according to the motion trajectory of the target object, and generate a virtual element.

[0059] In a third aspect, the present application further provides a virtual element drawing system, characterized in that it includes a mixed reality device; the mixed reality device is used to execute the steps of any one of the methods in the above-mentioned first aspect.

[0060] In a fourth aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the virtual element generation method in the first aspect when executing the computer program.

[0061] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the virtual element generation method in the first aspect are implemented.

[0062] In a sixth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the virtual element generation method in the first aspect are implemented.

[0063] The above-mentioned virtual element generation method, device, drawing system, computer device, storage medium and computer program product trigger a virtual element drawing request corresponding to the target operation in the virtual space in response to the target operation of the target object; and obtain the target action of the target object based on the virtual element drawing request, and the target action is used to characterize the execution of the virtual element drawing operation corresponding to the virtual element drawing request in the virtual space; then, the motion trajectory of the target object is obtained, so as to execute the virtual element drawing operation according to the motion trajectory of the target object, thereby generating a virtual element. That is, the virtual element generation method in the embodiment of the present application sets different trigger operations corresponding to different virtual element drawing requests, and sets different trigger actions for executing the virtual element drawing operation corresponding to each virtual element drawing request, so that the user can trigger different virtual element drawing requests in the virtual space and trigger the virtual element drawing operation corresponding to each virtual element drawing request based on different trigger operations and different trigger actions, thereby providing the user with the function of drawing virtual elements in the virtual space; because in the method of the present application, virtual element drawing is performed based on different operations and different actions of the user, without relying on a real brush, which can not only improve the experience of virtual-reality interaction, but also improve the convenience and flexibility of virtual element drawing. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a schematic diagram of the relationship between the virtual and real interaction mode and the metaverse;

[0065] Figure 2 A diagram of an application environment of a method for generating a virtual element in an embodiment;

[0066] Figure 3 A schematic diagram of a flow chart of a method for generating a virtual element in one embodiment;

[0067] Figure 4 is a flow chart of a method for generating a virtual element in another embodiment;

[0068] Figure 5 is a flow chart of a method for generating a virtual element in another embodiment;

[0069] Figure 6 is a flow chart of a method for generating a virtual element in another embodiment;

[0070] Figure 7 is a flow chart of a method for generating a virtual element in another embodiment;

[0071] Figure 8 is a flow chart of a method for generating a virtual element in another embodiment;

[0072] Fig. 9 A schematic diagram of two Bezier interpolation smoothing in one embodiment;

[0073] Fig.10 A schematic diagram of optimizing the connection of brush lines in one embodiment;

[0074] Fig.11 is a flow chart of a method for generating a virtual element in another embodiment;

[0075] Fig.12 is a flow chart of a method for generating a virtual element in another embodiment;

[0076] Fig.13 A schematic diagram of a brush drawing process in an embodiment;

[0077] Fig.14 A schematic diagram of an erasing process in an embodiment;

[0078] Fig.15 A schematic diagram of a color brush process in one embodiment;

[0079] Fig.16 FIG. 4 is a structural block diagram of a virtual element generating device in an embodiment. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0081] With the development of virtual reality or augmented reality technology, metaverse space technology has emerged. Metaverse is a virtual world constructed by humans using digital technology, which is a reflection of or transcends the real world and can interact with the real world. It is a digital living space with a new social system. Figure 1 As shown in FIG. 1 , it shows the relationship between the virtual-reality interaction mode and the metaverse, that is, the virtual-reality interaction is the main way to enhance the immersive experience of the combination of virtual and real in the metaverse space. The virtual-reality interaction includes drawing, clicking, and voice.

[0082] At present, drawing in the metaverse space to add virtual elements in the metaverse space, such as virtual drawing lines, graphics, etc., can be done through drawing tools based on augmented reality / virtual reality. Traditionally, drawing in the metaverse space is done based on a real drawing unit, such as a real pen with an electromagnetic induction device; this method changes the induced magnetic field of the real pen by pressing, and converts the magnetic field into a corresponding force and synchronizes it to the virtual pen in the virtual space of the metaverse, thereby changing the width of the line drawn by the virtual brush.

[0083] This solution is strongly dependent on a real brush. Once the brush is lost or out of power, this drawing solution will not work. At the same time, the way of changing the line width by pressing the button on the real pen is too different from the way of changing the line in the real world. It takes time for users to adapt, resulting in a poor user experience of virtual-reality interaction.

[0084] Based on this, this application proposes a method for generating virtual elements to provide an implementation method of a three-dimensional free brush drawing tool in the metaverse space. This method mainly uses specific gestures as the triggering action of a specific drawing tool to draw realistic three-dimensional lines in the metaverse space. The metaverse space can be jointly constructed by a mixed reality headset (such as Hololens2) and Unity software (a real-time 3D interactive content creation and operation platform), and relies on the MRTK toolkit (Unity's third-generation Microsoft Mixed Reality Toolkit) for gesture recognition.

[0085] The virtual element generation method provided in the embodiment of the present application can be applied to Figure 2In the application environment shown. Among them, the virtual reality device 102 includes a processor, a memory and a communication interface connected through a system bus, the processor is used to provide computing and control capabilities; the memory includes a non-volatile storage medium and an internal memory; the non-volatile storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a virtual element generation method is implemented.

[0086] Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the virtual reality device to which the solution of the present application is applied. The specific virtual reality device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0087] Optionally, the virtual reality device 102 can be a head-mounted virtual reality device, such as Hololens 2. A metaverse space can be constructed through the head-mounted virtual reality device Hololens 2 and Unity. When the user draws three-dimensional space lines in real time, he can immersively experience the interactive process of the fusion of reality and virtuality, thereby improving the virtual-reality interactive experience.

[0088] It should be noted that the virtual element generation method can be applied to a virtual reality device, to a server connected to the virtual reality device for communication, or to a virtual reality interactive system composed of a virtual reality device and a server; the embodiments of the present application do not specifically limit this.

[0089] In one embodiment, Figure 3 As shown, a virtual element generation method is provided, and the method is applied to Figure 2 The virtual reality device in the example is used to illustrate, including the following steps:

[0090] Step 301 : In response to a target operation of a target object, triggering a virtual element drawing request corresponding to the target operation in a virtual space.

[0091] Optionally, the target object here may be a user, a limb of the user, or a smart device worn by the user, such as a data glove, etc. The target operation of the target object may be a posture operation, a gesture operation, a click operation, a collision operation, etc.

[0092] Exemplarily, taking the target object as the user's hand as an example, the user can trigger the drawing function in the virtual space by simulating different gestures; for example, different gesture actions corresponding to different virtual element drawing requests can be set, and the user can trigger different virtual element drawing requests by simulating different gesture actions; wherein the virtual element drawing request may include but is not limited to at least one of a graphic drawing request, an erase request and a color brush request.

[0093] The user can customize gesture actions corresponding to different virtual element drawing requests, so that the user can trigger different virtual element drawing requests by simulating different gesture actions, thereby realizing the function of drawing virtual elements in the virtual space. The virtual reality device monitors the user's hand information to obtain the user's gesture action, and in response to the user's gesture operation, triggers a virtual element drawing request corresponding to the user's gesture operation in the virtual space when it is determined that the user's gesture action is a preset gesture action corresponding to the virtual element drawing request.

[0094] Exemplarily, virtual buttons corresponding to different virtual element drawing requests can also be set and displayed in the virtual space, and the target operation can be a click or trigger operation of the user on the virtual button; that is, the user can trigger the drawing function in the three-dimensional space by clicking or triggering the virtual button; at this time, the virtual reality device can monitor the click or trigger events of each virtual button, and when the user clicks a target virtual button, in response to the target object, that is, the user's trigger operation on the target virtual button, a virtual element drawing request corresponding to the trigger operation of the target virtual button is triggered in the virtual space.

[0095] Different virtual keys may correspond to different virtual element drawing requests, for example, a drawing virtual key corresponds to a graphics drawing request, an eraser virtual key corresponds to an erase request, a color brush virtual key corresponds to a color brush request, etc. The target virtual key may include any one of the virtual keys corresponding to different virtual element drawing requests.

[0096] In addition, the virtual element drawing request corresponding to the triggering operation of the target virtual key may also carry drawing attribute information corresponding to the target virtual key; wherein the drawing attribute information includes at least one of the color attribute, line attribute, and erasing attribute involved in the virtual element drawing process; the color attribute may include a specific drawing color, the line attribute may include a drawing line type, a drawing line thickness, etc., and the erasing attribute may include attribute information such as eraser size, partial erasing, and full erasing. Optionally, the user may customize the drawing attribute information in the virtual element drawing request, and the virtual reality device may also draw according to the default attribute information when the user does not set the relevant drawing attribute information.

[0097] Step 302 : acquiring a target action of a target object based on a virtual element drawing request; the target action is used to represent a virtual element drawing operation corresponding to the virtual element drawing request executed in a virtual space.

[0098] Among them, different virtual element drawing requests correspond to different virtual element drawing operations, such as: a graphic drawing request can correspond to a graphic drawing operation, an erase request can correspond to an erase operation, a color brush request can correspond to a color brush operation, etc. Optionally, for different virtual element drawing operations, during the execution of the virtual element drawing operation, the corresponding virtual drawing tool model can also be synchronously displayed in the virtual space to provide the user with a visual drawing experience; illustratively, for different virtual element drawing requests, different virtual drawing tool models can also be corresponding; for example: a graphic drawing request can correspond to a virtual brush model, an erase request can correspond to a virtual eraser model, and a color brush request can correspond to a virtual color brush model. Of course, for different virtual element drawing requests, the same virtual drawing tool model can also be set to prompt the user to start or execute the virtual element drawing operation corresponding to the virtual element drawing request; this application does not make specific limitations on this.

[0099] When the virtual element drawing request corresponding to the target operation is triggered based on the target operation of the target object, the virtual reality device can further obtain the target action of the target object, which can be a user-defined setting or a system default setting, and is used to execute the trigger action of the virtual element drawing operation corresponding to the virtual element drawing request. At the same time, it can also be used as a trigger action for outputting and displaying the drawing tool model corresponding to the virtual element drawing request. Optionally, the target action of the target object can be a gesture action, a click action, or other operation actions. The execution of virtual element drawing operations corresponding to different virtual element drawing requests can correspond to the same or different target actions. Since the virtual element drawing request and the virtual element drawing operation are one-to-one corresponding, the trigger action of the virtual element drawing operation corresponding to different virtual element drawing requests can be set to the same trigger action, and will not affect the execution of different virtual element drawing requests.

[0100] Similarly, different drawing tool models may correspond to the same or different target actions. When the target action of the target object is monitored, the virtual reality device may output and display the drawing tool model corresponding to the virtual element drawing request. After the target action of the target object disappears, the drawing tool model may also disappear to prompt the user that when making the target action, the virtual element drawing operation corresponding to the virtual element drawing request can be executed.

[0101] Exemplarily, the virtual reality device may capture the user's hand movements through an image capture device, or may capture the user's hand movements through a data glove, and determine whether the captured user's hand movements are triggering actions for the virtual element drawing operation corresponding to the virtual element drawing request; for example: when it is determined that the similarity between the captured user's hand movements and the triggering action for the virtual element drawing operation corresponding to the virtual element drawing request exceeds a preset similarity threshold, it may be determined that the captured user's hand movements are the triggering action for the virtual element drawing operation corresponding to the virtual element drawing request; at this time, the virtual element drawing operation corresponding to the virtual element drawing request may be started.

[0102] For the drawing tool model corresponding to the virtual element drawing request, when it is determined that the collected user's hand movement is the triggering action of the virtual element drawing operation corresponding to the virtual element drawing request, it can obtain the drawing tool model corresponding to the virtual element drawing request from the drawing tool model library according to the virtual element drawing request or the virtual element drawing operation corresponding to the virtual element drawing request; it can also obtain the drawing tool model corresponding to the virtual element drawing request from the drawing tool model library according to the user's hand movement, and display the drawing tool model corresponding to the virtual element drawing request in the virtual space to remind the user to start drawing.

[0103] Step 303: Acquire the motion trajectory of the target object, perform a virtual element drawing operation according to the motion trajectory of the target object, and generate a virtual element.

[0104] That is to say, when the user makes a target action corresponding to a virtual element drawing operation corresponding to a request to execute a virtual element drawing, it means that the user has started the drawing operation. Then, the virtual reality device can track the motion trajectory of the target object and perform the virtual element drawing operation according to the motion trajectory of the target object, thereby generating a virtual element.

[0105] Exemplarily, when the user makes a target action and a drawing tool model corresponding to a virtual element drawing request is displayed in the virtual space, the virtual reality device can track the motion trajectory of the target object and control the drawing tool model accordingly to move along the motion trajectory of the target object, so that when the drawing tool model follows the movement of the target object, the virtual reality device performs a virtual element drawing operation according to the motion trajectory of the target object, thereby drawing and generating a corresponding virtual element, thereby realizing the function of the user drawing a virtual element in the virtual space.

[0106] Furthermore, in the process of generating the virtual element, the virtual reality device may also generate the virtual element with drawing attributes based on the drawing attribute information carried in the virtual element drawing request.

[0107] In addition, during the execution of a virtual element drawing operation, if the similarity between the user's hand movement and the target movement corresponding to the virtual element drawing operation is monitored to be less than a preset similarity threshold, the drawing operation can be paused or stopped, and accordingly, the drawing tool model will disappear, indicating that the user's current drawing is finished; then, when the drawing tool model in the virtual space disappears, that is, when the current movement of the target object is a non-target movement, the virtual reality device can pause the drawing; then, after the target movement of the target object is monitored again, the virtual element drawing operation can be continued, the drawing tool model can be redisplayed in the virtual space, and the next drawing can be started; so as to generate a virtual element drawn by the user in the virtual space.

[0108] It should be noted that if the interval time for the user to re-make the target action is greater than the preset interval time threshold, that is, the interval time between the previous drawing operation and the next drawing operation is too long, then the virtual element drawing request corresponding to the previous drawing operation can be terminated; in this case, the user can re-trigger the virtual element drawing request; of course, such a setting may not be adopted, that is, when the user triggers a virtual element drawing request, the virtual reality device can always maintain the virtual element drawing request until the user triggers the next virtual element drawing request, or the user cancels the virtual element drawing request.

[0109] In the above virtual element generation method, the virtual reality device triggers a virtual element drawing request corresponding to the target operation in the virtual space in response to the target operation of the target object; and obtains the target action of the target object based on the virtual element drawing request, and the target action is used to characterize the execution of the virtual element drawing operation corresponding to the virtual element drawing request in the virtual space; then, the motion trajectory of the target object is obtained, so as to execute the virtual element drawing operation according to the motion trajectory of the target object, thereby generating a virtual element. That is, the virtual element generation method in the embodiment of the present application sets different trigger operations corresponding to different virtual element drawing requests, and sets different trigger actions for executing the virtual element drawing operation corresponding to each virtual element drawing request, so that the user can trigger different virtual element drawing requests in the virtual space and trigger the virtual element drawing operation corresponding to each virtual element drawing request based on different trigger operations and different trigger actions, thereby providing the user with the function of drawing virtual elements in the virtual space; because in the method of the present application, virtual element drawing is performed based on different operations and different actions of the user, without relying on a real brush, which can not only improve the experience of virtual-reality interaction, but also improve the convenience and flexibility of virtual element drawing.

[0110] In addition, since the present application implements graphics drawing in a virtual space, by adopting a method of starting drawing by executing a corresponding virtual element drawing operation based on the target action of the target object after triggering a virtual element drawing request, it is not only possible to avoid the misrecognition caused by directly drawing graphics based on the motion trajectory of the target object after the virtual element drawing request is triggered, that is, treating the non-drawing motion trajectory of the target object in the virtual space as a drawing motion trajectory, and drawing virtual lines that the user does not want in the virtual space; it is also possible to achieve intermittent drawing of complex graphics. For example, when drawing multiple lines, the user does not need to trigger a virtual element drawing request before each drawing. It is only necessary to cancel the target action after the drawing is completed, and to make the target action again at the next drawing; thereby achieving efficient and accurate drawing in the virtual space.

[0111] In one embodiment of the present application, when virtual keys corresponding to different virtual element drawing requests are displayed in a virtual space, since different virtual element drawing requests can carry drawing attribute information corresponding to the virtual keys, and the drawing attribute information can be customized by the user; therefore, for virtual keys corresponding to different virtual element drawing requests, sub-keys corresponding to the virtual keys can also be set, and the sub-keys can be used for users to customize different drawing attribute information. Based on this, Figure 4 As shown, the above virtual element generation method may further include:

[0112] Step 401: In response to a triggering operation of a target object on a target virtual key, displaying at least one virtual sub-key corresponding to the target virtual key in a virtual space.

[0113] Among them, different virtual keys can correspond to different virtual sub-keys; for example: the drawing virtual key can correspond to a color sub-key, a line type sub-key, a line thickness sub-key, etc.; the eraser virtual key can correspond to an eraser size sub-key, an erase mode sub-key, etc., wherein the erase mode sub-key can include a partial erase sub-key and an overall erase sub-key, etc.; the color brush virtual key can include a color sub-key, a color modification mode sub-key, etc., wherein the color modification mode sub-key can include a partial line color modification sub-key and an overall line color modification sub-key, etc.

[0114] Optionally, when the virtual reality device detects a triggering event of a target virtual key by a target object, the virtual reality device obtains a virtual sub-key corresponding to the target virtual key in response to the triggering operation of the target virtual key by the target object, and displays each virtual sub-key corresponding to the target virtual key at a preset position in the virtual space, wherein the preset position can be a position within a preset range of the target virtual key or any position in the virtual space, and the embodiments of the present application do not specifically limit this.

[0115] Step 402 : In response to a triggering operation on a target virtual sub-key, triggering a virtual element drawing request corresponding to the triggering operation on the target virtual sub-key in a virtual space.

[0116] The target virtual sub-key includes at least one virtual sub-key among the virtual sub-keys; and the virtual element drawing request corresponding to the triggering operation of the target virtual sub-key carries drawing attribute information corresponding to the target virtual sub-key.

[0117] When a virtual sub-button corresponding to a target virtual button is displayed in a virtual space, the virtual reality device can monitor triggering events of the target object on each virtual sub-button. If a triggering operation of the target object on the target virtual sub-button is detected, the drawing attribute information corresponding to the target virtual sub-button is obtained; then, a virtual element drawing request is generated based on the drawing attribute information corresponding to each target virtual sub-button triggered by the target object; the virtual element drawing request carries user-defined drawing attribute information.

[0118] In this embodiment, the virtual reality device displays at least one virtual sub-key corresponding to the target virtual key in the virtual space in response to the triggering operation of the target virtual key by the target object; and in response to the triggering operation of the target virtual sub-key, triggers a virtual element drawing request corresponding to the triggering operation of the target virtual sub-key in the virtual space; wherein the target virtual sub-key includes at least one virtual sub-key among the virtual sub-keys; and the virtual element drawing request corresponding to the triggering operation of the target virtual sub-key carries the drawing attribute information corresponding to the target virtual sub-key. That is, in this embodiment, the user can trigger the virtual key and the sub-key corresponding to the virtual key, thereby triggering the virtual element drawing request carrying the user-defined drawing attribute information, so that the virtual reality device can meet the different drawing requirements of the user to draw virtual elements, thereby increasing the virtual-reality interaction experience and improving the flexibility and diversity of virtual element drawing in the virtual space.

[0119] Figure 5 FIG. 1 is a flow chart of a virtual element generation method in another embodiment. This embodiment relates to an optional implementation process of a virtual reality device obtaining a target action of a target object based on a virtual element drawing request. Based on the above embodiment, Figure 5 As shown, the above step 302 includes:

[0120] Step 501: Based on a virtual element drawing request, a target object is photographed by an image acquisition device to generate a first image.

[0121] The image acquisition device may be integrated into the virtual reality device, or may be independently arranged outside the virtual reality device and communicated with the virtual reality device. The image acquisition device may include at least one of a camera, a video camera, a camera, a scanner, and other devices with a photographing function. The target object may be photographed by the image acquisition device, thereby obtaining a real-time image of the target object.

[0122] It should be noted that the image acquisition device can acquire images of the target object in real time, and can also start working when triggered by the function of the virtual reality device, so as to reduce the power consumption of the image acquisition device and avoid waste of resources.

[0123] Exemplarily, when the virtual reality device obtains a virtual element drawing request, it can control the image acquisition device to open based on the virtual element drawing request, and shoot the target object through the image acquisition device, thereby generating a first image of the target object. The image acquisition device sends the captured first image of the target object to the virtual reality device in real time, so that the virtual reality device obtains the current image of the target object.

[0124] Step 502: Perform image recognition on the first image to obtain an image recognition result, and obtain a target action of the target object from the image recognition result.

[0125] The target action of the target object may be a preset trigger action of the virtual element drawing operation corresponding to the virtual element drawing request, which is used to trigger and start executing the virtual element drawing operation corresponding to the virtual element drawing request.

[0126] Exemplarily, the virtual reality device may use a preset image processing model to perform image recognition on the first image of the target object, obtain an image recognition result, and obtain the current action of the target object from the image recognition result; wherein the image recognition result may include the action characteristics of the target object, or the action information of the target object. Then, the virtual reality device may determine whether the current action of the identified target object is consistent with the preset trigger action of the virtual element drawing operation corresponding to the virtual element drawing request. If the similarity between the current action of the target object and the preset trigger action of the virtual element drawing operation is greater than or equal to the preset similarity threshold, it may be determined that the current action of the target object is consistent with the preset trigger action; at this time, it may be determined that the target action of the target object has been collected.

[0127] Optionally, when it is determined that the target action of the target object is captured, a drawing tool model corresponding to the virtual element drawing request can also be displayed in the virtual space according to the target action of the target object; exemplarily, the target position of the target action of the target object in the virtual space can be determined, and the drawing tool model corresponding to the virtual element drawing request can be displayed at the target position.

[0128] Optionally, the virtual reality device can also determine the position information of the target action of the target object in the virtual space based on the first image of the target object, thereby determining the target position of the target action in the virtual space, and then displaying the drawing tool model corresponding to the virtual element drawing request at the target position. Exemplarily, the first image can be a two-dimensional image or a three-dimensional image. When the first image is a two-dimensional image, the two-dimensional position information of the target action can be determined from the first image, and then the two-dimensional position information is converted into three-dimensional spatial position information in the virtual space to determine the three-dimensional spatial position of the target action in the virtual space.

[0129] In this embodiment, the virtual reality device, based on the virtual element drawing request, shoots the target object through the image acquisition device to generate a first image; and performs image recognition on the first image to obtain an image recognition result, and obtains the target action of the target object from the image recognition result. That is, in this embodiment, with the help of the image acquisition device on the virtual reality device, the image recognition technology is used to capture and recognize the action of the target object, so that the recognition efficiency of the action can be improved, and then the drawing efficiency of virtual element drawing in the virtual space is improved.

[0130] Regarding the virtual element drawing request in the embodiment of the present application, which includes but is not limited to a graphics drawing request, an erase request and a color brush request, different virtual element drawing requests are described in detail below.

[0131] First, when the virtual element drawing request includes a graphics drawing request, the virtual element drawing operation corresponding to the virtual element drawing request may include a graphics drawing operation, and accordingly, the drawing tool model corresponding to the virtual element drawing request may include a graphics drawing tool model; for example, the graphics drawing tool may be a virtual brush, and the graphics drawing tool model may be a virtual brush model; refer to Figure 6 As shown, in the above step 303, "obtaining the motion trajectory of the target object, performing a virtual element drawing operation according to the motion trajectory of the target object, and generating a virtual element" may include:

[0132] Step 601: photograph a target object by using an image acquisition device to generate multiple frames of second images of the target object.

[0133] Step 602: Acquire the motion trajectory of the target object from multiple frames of second images.

[0134] Step 603: Obtain first drawing attribute information from the graphics drawing request, perform a graphics drawing operation according to the first drawing attribute information and the motion trajectory of the target object, and generate a virtual element.

[0135] That is, in this embodiment, when the user triggers a graphics drawing request and when the user makes a target action corresponding to the image drawing request, it indicates that the user wants to perform an image drawing operation at this time, and accordingly instructs the virtual reality device to perform the graphics drawing operation; at the same time, a graphics drawing tool model corresponding to the graphics drawing request can also be displayed in the virtual space; at this time, the motion video image of the target object is acquired in real time by the image acquisition device, that is, multiple frames of second images of the target object are obtained, and the multiple frames of second images are analyzed to obtain the motion trajectory of the target object; based on the motion trajectory of the target object, the graphics drawing tool model can be controlled to move in the virtual space following the movement of the target object, and at the same time, based on the motion trajectory of the target object, corresponding virtual elements can be drawn in the virtual space and output for display; and in the process of drawing virtual elements, graphics drawing can be performed based on the first drawing attribute information carried in the graphics drawing request to generate virtual elements corresponding to the first drawing attribute information and the motion trajectory of the target object; for example, the virtual element can be a virtual line.

[0136] In one implementation, when a user draws graphics in a virtual space, the line smoothness may be poor due to jitter or other factors, which may cause the drawn curve to have bumps or peaks. The virtual reality device can smooth the initial line drawn by the user to obtain a smoothed virtual line, thereby improving the accuracy of virtual space drawing.

[0137] For example, when the virtual reality device performs a graphic drawing operation based on the first drawing attribute information and the motion trajectory of the target object to generate a virtual element, it can also perform real-time smoothing on the virtual element drawn by the user, especially on the line at the tip position with a more obvious bend, so as to obtain a smooth drawing curve. Figure 7 As shown, step 603 may also include:

[0138] Step 701: Perform a graphic drawing operation according to first drawing attribute information and a motion trajectory of a target object to generate an initial virtual element.

[0139] Step 702: Smoothing the initial virtual element to generate a virtual element, and displaying the virtual element in a virtual space.

[0140] That is to say, when the virtual reality device draws graphics in the virtual space based on the motion trajectory of the target object, it can smooth the initial virtual object drawn in real time; for example: at the current frame moment, the virtual reality device can smooth part of the initial lines drawn before the current frame moment, thereby obtaining a virtual element drawn from the start moment to the current frame moment and smoothed; then, the virtual element is displayed in the virtual reality space.

[0141] For example, reference Figure 8 As shown, the process of the virtual reality device smoothing the initial virtual element to generate the virtual element may include:

[0142] Step 801: determine a plurality of initial drawing points and coordinate information of each initial drawing point from a virtual element.

[0143] Optionally, when the virtual reality device determines the motion trajectory of the target object based on multiple frames of second images of the target object, and then generates a virtual element based on the motion trajectory of the target object, it does so by determining the position information of the target object in each frame of the second image, and then determining multiple motion points of the target object during the motion process based on the position information of the target object in each frame of the image, and then generating the virtual element based on the multiple motion points of the target object during the motion process; therefore, when determining the initial drawing point from the virtual element, the motion point of the target object in each frame of the second image can be used as the initial drawing point, and the position information of the motion point of the target object in each frame of the second image can be used as the coordinate information of each initial drawing point.

[0144] Step 802: According to the coordinate information of each initial drawing point, a Bezier interpolation method is used to determine a plurality of target drawing points and the coordinate information of each target drawing point.

[0145] Exemplarily, the coordinate information of each initial drawing point can be updated by using a first-order or multi-order Bezier interpolation method according to the coordinate information of each initial drawing point, so as to obtain at least one target drawing point corresponding to each initial drawing point, and the coordinate information of each target drawing point corresponding to each initial drawing point. That is, for each initial drawing point, the target drawing point corresponding to the initial drawing point can be determined, and multiple adjacent target drawing points corresponding to the initial drawing point can also be determined, so as to change the curvature of the drawing curve and obtain a smoother drawing curve.

[0146] Step 803: Generate virtual elements according to the coordinate information of each target drawing point.

[0147] In one implementation, the following method may be used to update the coordinates of each initial drawing point, obtain multiple target drawing points and coordinate information of each target drawing point, and then obtain a drawing curve. Therefore, the above step 802 may include the following steps:

[0148] Step a: for each initial drawing point, the coordinate information of the initial drawing point, the coordinate information of the previous drawing point of the initial drawing point and the coordinate information of the next drawing point of the initial drawing point are updated by using the first Bezier interpolation method to obtain a new initial drawing point corresponding to the initial drawing point and the coordinate information of the new initial drawing point.

[0149] Step b, based on the coordinate information of the new initial drawing point, the coordinate information of the previous drawing point and the coordinate information of the next drawing point, the second Bezier interpolation method and the golden section value are used to determine the coordinate information of the first intermediate drawing point and the first intermediate drawing point between the previous drawing point and the new initial drawing point, and the coordinate information of the second intermediate drawing point and the second intermediate drawing point between the new initial drawing point and the next drawing point.

[0150] Step c: taking the new initial drawing point, the first intermediate drawing point and the second intermediate drawing point as target drawing points.

[0151] It should be noted that the initial drawing point here may not include the first drawing point and the last drawing point on the virtual element.

[0152] For example, reference Fig. 9 As shown, for the initial drawn point P i-1 , P i , P i+1 , P i+2 The resulting plotted curve 1, where P i+2 It can indicate the location of the gesture in the current frame, P i-1 , P i and P i+1 Respectively represent the spatial positions of the gestures in the third frame, the second frame, and the first frame before the current frame. i+2 For example, since there is no next frame for the current frame, the previous frame of the current frame can be smoothed at the current frame; for example, the first frame P before the current frame can be smoothed. i+1 Smoothing can also be performed on the second frame P before the current frame i Smoothing can also be performed on the third frame P before the current frame. i-1Smoothing is performed, etc. This application does not make specific restrictions on this. In view of the visual discontinuity between the interpolation part and the drawing part caused by the smoothing process, it is possible to consider smoothing optimization processing by staggered interpolation. For example, the points drawn in the i-th frame will be interpolated and optimized in the i+2 frame. After optimization, all lines associated with the points in the i-th frame (i.e., L i-1→i With L i→i+1 ) will be redrawn (i.e., L' i-1→i With L' i→i+1 ).

[0153] Next, we will draw the initial point P i Take correction as an example to illustrate.

[0154] First, according to the initial drawing point P i-1 , P i and P i+1 The coordinate information of the three points determines the initial drawing point P i-1 With P i Draw a curve between the units L i-1→i , and the initial plot point P i With P i+1 Draw a curve between the units L i→i+1 Is the angle α between them less than the preset angle threshold? If the angle is greater than or equal to the preset angle threshold, then it can be said that the initial drawing point P i The curve segment at is relatively smooth and does not need to be smoothed; that is, there is no need to i In this case, the initial drawing point is directly used as the target drawing point corresponding to the initial drawing point, and the coordinate information of the target drawing point is the coordinate information of the initial drawing point; if the angle is less than the preset angle threshold, it means that the initial drawing point P i The smoothness of the curve segment at is poor. At this time, the initial drawing point P needs to be i The coordinates of are optimized and updated.

[0155] Next, the first Bezier interpolation is used to draw the initial point P i The coordinate information of the target drawing point P' is updated to obtain the updated target drawing point i And the target plot point P' i The coordinate information of is calculated as shown in formula (1).

[0156] P i ′=(1-t) 2 P i-1 +2t(1-t)P i +t 2 P i+1 (1)

[0157] Where, t = |P i -P i-1 | / |P i+1 -P i-1 |, t∈[0,1]; || means taking the modulus of the coordinates.

[0158] After determining the updated target drawing point P' i After obtaining the coordinate information of the initial drawing point P i-1 、Target drawing point P' i And the initial drawing point P i+1 The three points generate the updated drawing curve L' i-1→i+1 ,Right now Figure 8 The plotted curve 2 in includes the curve unit L' i-1→i and L' i→i+1 .

[0159] Furthermore, at the target point P' i The left and right golden section points are located at the second Bezier interpolation, and the target drawing points P' are added respectively. i-left and P' i-right , where the target plot point P' i-left The coordinate information calculation method of the target drawing point P' is shown in formula (2). i-right The coordinate information calculation method is shown in formula (3).

[0160] p i ' -left =(1-t l ) 2 p i-1 +2t(1-t l ) i ′+t l 2 p i+1 (2)

[0161] p i ' -right =(1-t r ) 2 p i-1 +2t(1-t r ) i ′+t r 2 p i+1 (3)

[0162] Among them, t l and t r They are:

[0163] t l =(1-β)|pi ′-p i+1 | / |p i-1 -p i+1 | (4)

[0164] t r =(|p i ′-p i-1 |+β|p i-1 -p i+1 |) / |p i-1 -p i+1 | (5)

[0165] Among them, β is the golden section point, that is, β = 0.618.

[0166] Next, after determining the target drawing point P' i-left The coordinate information of the target drawing point P' i-right After obtaining the coordinate information, we can draw the initial point P based on i-1 、Target drawing point P' i-left 、Target drawing point P' i 、Target drawing point P' i-right And the initial drawing point P i+1 , five points to generate the updated drawing curve L" i-1→i+1 ,Right now Figure 8 Draw curve 3 in .

[0167] Using the above initial drawing point P i In the correction method, each drawing point in the virtual element drawing process can be smoothed. In the real-time smoothing process, the coordinate information of the initial drawing point (that is, the drawing point obtained according to the user's motion trajectory) may be updated or not updated; then, when the initial drawing point P corresponding to the i+1th frame is i+1 When making corrections, if the initial drawing point P corresponding to the i-th frame i The coordinate information of the initial drawing point P is updated. i+1 The previous drawing point should be the initial drawing point P i The corresponding target drawing point P' i-right , the initial drawing point P i+1 The next drawing point is the initial drawing point P corresponding to the i+2th frame i+2 .

[0168] It should be noted that the two Bessel interpolations used in this embodiment can be second-order Bessel interpolations. Of course, the Bessel interpolations can also be other multi-order Bessel interpolations. For the two Bessel interpolations, the Bessel interpolation orders used can be the same or different, and this embodiment does not specifically limit this.

[0169] Furthermore, when a columnar preform is used to draw a curve, there may be discontinuities at the joints of the columnar preforms. Based on this, the joints of each curve drawing unit can be repaired using a minimum drawing unit of adaptive size. It should be noted that in this embodiment, a spherical preform is not used to optimize the joints. The main reason is to meet the conditions of dynamic batch processing and avoid freezes and delays during drawing. In addition, in this embodiment, a pure color fluorescent shader is used to weaken the lines of the minimum drawing unit itself and the lines at the overlap of the minimum drawing units to improve the smooth processing effect of the curve.

[0170] Based on this, the above step 803 of “generating virtual elements according to the coordinate information of each target drawing point” may include the following steps:

[0171] Step a: Generate candidate virtual elements according to the coordinate information of each target drawing point.

[0172] Step b: for each target drawing point in the candidate virtual element, determine the connecting pipeline corresponding to the target drawing point and the pipeline information of the connecting pipeline according to the pipeline information of the first pipeline and the pipeline information of the second pipeline adjacent to the target drawing point.

[0173] The first pipeline is a pipeline formed based on the previous drawing point of the target drawing point and the target drawing point; the second pipeline is a pipeline formed based on the target drawing point and the next drawing point of the target drawing point. The pipeline information of the connecting pipeline may include information such as the direction information of the pipeline, the coordinate information of the center point of the pipeline, the width of the pipeline, and the radius of the pipeline.

[0174] It should be noted that the target drawing point here may not include the first target drawing point and the last target drawing point of the candidate virtual element.

[0175] Step c: Based on the pipeline information of the connecting pipeline corresponding to each target drawing point, the position corresponding to each target drawing point in the candidate virtual element is filled to generate a virtual element.

[0176] refer to Fig.10 As shown, for each target drawing point, the pipeline information of the first pipeline (ie, pipeline A, CylinderA) adjacent to the target drawing point and the pipeline information of the second pipeline (ie, pipeline B, CylinderB) adjacent to the target drawing point are determined.

[0177] Exemplarily, the orientation information of the first pipeline A can be determined based on the direction vector between the coordinate information of the previous drawing point of the target drawing point and the coordinate information of the target drawing point, and the coordinate information of the pipeline center point of the first pipeline A can be determined based on the coordinate information of the previous drawing point of the target drawing point and the coordinate information of the target drawing point. The orientation information of the second pipeline B can be determined based on the direction vector between the coordinate information of the target drawing point and the coordinate information of the next drawing point of the target drawing point, and the coordinate information of the pipeline center point of the second pipeline B can be determined based on the coordinate information of the target drawing point and the coordinate information of the next drawing point of the target drawing point.

[0178] Furthermore, when the pipeline information of the first pipeline A and the pipeline information of the second pipeline B are determined, the connecting pipeline corresponding to the target drawing point and the pipeline information of the connecting pipeline can be obtained by the following calculation method.

[0179] First, the radius of the connecting pipeline can be determined by formula (6):

[0180] CylinderInsert.localscale.y=CylinderA.localscale.y×cos(0.5γ)(6)

[0181] Among them, CylinderInsert.localscale.y represents the radius of the connecting pipeline, that is, the radius in the y-plane direction perpendicular to the direction of the connecting pipeline, CylinderA.localscale.y represents the pipeline radius of the first pipeline A, that is, the radius in the y-plane direction perpendicular to the direction of the first pipeline A, and γ represents the angle determined according to the direction information CylinderA.up of the first pipeline A and the direction information CylinderB.up of the second pipeline B.

[0182] Second, the width of the connecting pipeline can be determined by formula (7):

[0183] CylinderInsert.localwidth.y=CylinderA.localscale.y×sin(0.5γ)(7)

[0184] Among them, CylinderInsert.localwidth.y represents the width of the connecting pipeline, that is, the length of the connecting pipeline.

[0185] Third, the coordinate information of the center point of the connecting pipeline can be determined by formula (8):

[0186] CylinderInsert.localposition.y=0.5×CylinderA.localscale.y×(1-sin(0.5γ)) (8)

[0188] Among them, CylinderInsert.localposition.y represents the coordinate information of the center point of the connecting pipeline.

[0189] Fourth, the orientation information of the connecting pipeline can be determined by formula (9):

[0190] CylinderInsert.up=CylinderA.up-CylinderB.up (9)

[0191] Among them, CylinderInsert.up represents the orientation information of the connecting pipeline, CylinderA.up represents the orientation information of the first pipeline A, and CylinderB.up represents the orientation information of the second pipeline B.

[0192] Next, for each target drawing point, after determining the pipeline information of the connecting pipeline corresponding to each target drawing point, the position corresponding to each target drawing point in the candidate virtual element can be filled based on the pipeline information of the connecting pipeline corresponding to each target drawing point to generate a virtual element.

[0193] Second, when the virtual element drawing request is an erase request, the virtual element drawing operation corresponding to the virtual element drawing request may include an erase operation, and accordingly, the drawing tool model corresponding to the virtual element drawing request may include an erase tool model, such as a virtual eraser; Fig.11 As shown, in the above step 303, "obtaining the motion trajectory of the target object, performing a virtual element drawing operation according to the motion trajectory of the target object, and generating a virtual element" may include:

[0194] Step 1101: photograph a target object by using an image acquisition device to generate multiple frames of third images of the target object.

[0195] Step 1102: Acquire the motion trajectory of the target object from multiple frames of third images.

[0196] Step 1103 , obtaining second drawing attribute information from the erasing request, performing an erasing operation according to the second drawing attribute information and the motion trajectory of the target object, and generating a virtual element.

[0197] That is, in this embodiment, when the user triggers an erasing request and when the user makes a target action corresponding to the erasing request, it indicates that the user wants to perform a graphic erasing operation at this time, and accordingly instructs the virtual reality device to perform the graphic erasing operation; at the same time, an erasing tool model corresponding to the erasing request can also be displayed in the virtual space; at this time, the motion video image of the target object is acquired in real time by the image acquisition device, that is, a plurality of third frames of the target object are obtained, and the plurality of third frames are analyzed to obtain the motion trajectory of the target object; based on the motion trajectory of the target object, the erasing tool model can be controlled to move in the virtual space following the movement of the target object; at the same time, during the movement, the collision body properties of the virtual element are monitored, and if the erasing tool model collides with the virtual element, it can be indicated that the target object collides with the virtual element, then, the virtual element is erased to obtain the erased virtual element.

[0198] Optionally, when erasing a graphic, in order to take into account the user's needs for modifying details of the drawn graphic and erasing the graphic as a whole, this embodiment provides a partial erasing mode and a full erasing mode for the user to choose from. Exemplarily, when the target virtual key is an erase virtual key, the at least one virtual sub-key corresponding to the target virtual key may include a partial erase sub-key and a full erase sub-key.

[0199] When the second drawing attribute information includes partial erasing attribute information, that is, the user selects the partial erasing sub-button, the "performing an erasing operation according to the second drawing attribute information and the motion trajectory of the target object to generate a virtual element" in the above step 1103 may include:

[0200] According to the motion trajectory of the target object, if it is determined that the target object collides with the virtual element, a partial erasing operation is performed on the collision portion of the virtual element to generate an erased virtual element.

[0201] When the second drawing attribute information includes overall erase attribute information, that is, the user selects the overall erase sub-button, the "performing an erase operation according to the drawing attribute information and the motion trajectory of the target object to generate a virtual element" in the above step 1103 may include:

[0202] According to the motion trajectory of the target object, if it is determined that the target object collides with the virtual element, an overall erasing operation is performed on the virtual element.

[0203] It should be noted that the virtual element here is the line curve drawn by the user through a gesture (i.e., the target action), that is, from the appearance of the gesture to the disappearance of the gesture, which is considered to be the same line curve, i.e., a virtual element. After the user draws a virtual element, the virtual reality device names and identifies the virtual element to obtain the identification information of the virtual element; when the overall erasing solution is adopted, the virtual reality device will find the line where the drawing unit is located according to the identification information of the first drawing unit that the virtual eraser collides with, and delete all the drawing units in the line, that is, delete the entire virtual element. Among them, a virtual element includes multiple drawing units, and the drawing curve formed by two drawing points in two adjacent frames of images is a drawing unit.

[0204] In addition, when a partial erasing solution is adopted, the virtual reality device will erase the drawing unit that the virtual eraser collides with.

[0205] Third, in the case where the virtual element drawing request is a color brush request, the virtual element drawing operation corresponding to the virtual element drawing request includes a color brush operation, and accordingly, the drawing tool model corresponding to the virtual element drawing request may include a color brush tool model, such as a virtual color brush; Fig.12 As shown, in the above step 303, "obtaining the motion trajectory of the target object, performing a virtual element drawing operation according to the motion trajectory of the target object, and generating a virtual element" may include:

[0206] Step 1201: photograph a target object by using an image acquisition device to generate multiple frames of fourth images of the target object.

[0207] Step 1202: Acquire the motion trajectory of the target object from multiple frames of fourth images.

[0208] Step 1203: Acquire third drawing attribute information from the color brush request, perform a color brush operation according to the third drawing attribute information and the motion trajectory of the target object, and generate a modified virtual element.

[0209] That is, in this embodiment, when the user triggers a color brush request and when the user makes a target action corresponding to the color brush request, it means that the user wants to perform a color attachment or modification operation on the graphic at this time, and accordingly instructs the virtual reality device to perform the graphic color brush operation; at the same time, a color brush tool model corresponding to the color brush request can also be displayed in the virtual space; at this time, the motion video image of the target object is captured in real time by the image acquisition device, that is, a plurality of fourth frames of the target object are obtained, and the plurality of fourth frames are analyzed to obtain the motion trajectory of the target object; based on the motion trajectory of the target object, the color brush tool model can be controlled to follow the movement of the target object in the virtual space; at the same time, during the movement, the collision body properties of the virtual element are monitored, and if the color brush tool model collides with the virtual element, it can be indicated that the target object collides with the virtual element, then the color of the virtual element is modified to obtain the virtual element with modified color.

[0210] Optionally, when performing color modification, partial color modification and overall color modification may also be included. Exemplarily, when the target virtual key is a color brush virtual key, at least one virtual sub-key corresponding to the target virtual key may include a partial color brush key and an overall color brush key.

[0211] When the third drawing attribute information includes partial color brush attribute information, that is, the user selects a partial color brush button, the “performing a color brush operation according to the third drawing attribute information and the motion trajectory of the target object to generate a virtual element” in the above step 1203 may include:

[0212] According to the motion trajectory of the target object, if it is determined that the target object collides with the virtual element, a partial color brush operation is performed on the collision portion of the virtual element to generate a modified virtual element.

[0213] When the third drawing attribute information includes overall color brush attribute information, that is, the user selects the overall color brush button, the “performing a color brush operation according to the third drawing attribute information and the motion trajectory of the target object to generate a virtual element” in the above step 1203 may include:

[0214] According to the motion trajectory of the target object, if it is determined that the target object collides with the virtual element, an overall color brushing operation is performed on the virtual element to generate a modified virtual element.

[0215] That is to say, when performing partial color modification, the virtual reality device modifies the color of the drawing unit of the virtual element that collides with the color brush tool model to the color carried in the third drawing attribute information; when performing overall color modification, the virtual reality device modifies the color of the entire virtual element to which the drawing unit that collides with the color brush tool model belongs to to the color carried in the third drawing attribute information; thereby obtaining a virtual element with modified color.

[0216] In one embodiment, a virtual element drawing system is provided, the system comprising a mixed reality device, the mixed reality device being used to execute the steps of the virtual element generating method in any of the above embodiments.

[0217] In one implementation, the drawing function module in the virtual element drawing system may include the following parts: (1) a preset grid model, including a preset virtual brush model, a preset virtual eraser model, and a preset virtual color brush model.

[0218] (2) Virtual keys, including a drawing virtual key, an erasing virtual key, a color brush virtual key, and virtual sub-keys corresponding to each virtual key; wherein the virtual sub-keys corresponding to the drawing virtual key may include but are not limited to a color selection sub-key, a brush type sub-key, etc.; the virtual sub-keys corresponding to the erasing virtual key may include but are not limited to a partial erasing sub-key, a full erasing sub-key, etc.; the virtual sub-keys corresponding to the color brush virtual key may include but are not limited to a color selection sub-key, a partial color brush key, a full color brush key, etc.

[0219] All virtual buttons have the function of listening to click events. After clicking the drawing button, the color selection sub-button and brush type sub-button will pop up; after clicking the erase button, the erase mode, namely the partial erase sub-button and the overall erase sub-button, will pop up; after clicking the color brush button, the color selection sub-button and color brush mode, namely the partial color brush button and the overall color brush button, will pop up.

[0220] (3) Custom gesture and gesture recognition module. After clicking the color selection sub-button under the drawing button, the custom gesture recognition module will be enabled. The custom gesture recognition module compares the hand gesture model captured by the camera module in the virtual reality device HoloLens2 with the preset drawing gesture. If the hand gesture is consistent with the preset drawing gesture, a virtual brush will appear at the real gesture. After clicking the erase mode sub-button under the erase button, the custom gesture recognition module will compare the hand gesture model captured by the camera with the preset erase gesture. If the hand gesture is consistent with the preset erase gesture, a virtual eraser will appear at the real gesture. After clicking the color brush button and its related sub-buttons, the camera will capture the gesture in real time and compare it with the preset color brush gesture. If the captured gesture is consistent with the preset gesture, a color brush model will appear at the gesture and collision judgment will begin, starting the color brush function.

[0221] (4) Line drawing module: the smallest line drawing unit is a cylinder model.

[0222] After the virtual brush appears, a virtual line will be generated where the gesture passes. The direction and length of the smallest drawing unit of the virtual line are determined by the drawing points collected by the camera for each frame of the gesture. The definition of a complete line in this application can be: drawing starts when the gesture appears, and drawing ends when the gesture disappears. And a specific naming rule is used to distinguish each complete line. Optionally, in order to take into account the smoothness of the system, the line preform does not have any collision body in the line drawing stage, and the collision body will be re-added to the line preform in the erasing stage and the color brush stage.

[0223] (5) Line interpolation smoothing module: For lines with obvious bending directions, that is, the angle α between the two minimum drawing units is less than or equal to the preset angle threshold, two Bezier interpolations can be used to re-determine the target drawing point, thereby smoothing the initial virtual line and generating a smoothed virtual line.

[0224] (6) Drawing unit connection beautification module: To ensure the continuity of the drawn lines, the space between any two minimum drawing units is beautified using an additional minimum drawing combined with a specific size algorithm (such as the above formula 6-9); that is, a new cylinder is inserted between the minimum drawing units, and its rotation angle and scaling are modulated to ensure the visual integrity of the lines.

[0225] (7) Line erasing module, which relies on the collision detection function of the virtual reality device Hololens2, and provides two erasing modes: partial erasing and full erasing, for users to choose.

[0226] (8) The color brush function module relies on the collision detection function of the virtual reality device Hololens2 and provides two color brush modes: partial color brush and full color brush for users to choose.

[0227] The following is a detailed description of the brush drawing process, erasing process and color brush process.

[0228] First, as Fig.13 As shown, a specific process of brush drawing is provided, including the following steps: the virtual reality device detects a drawing button click event, and if the drawing button is detected to be clicked, the drawing sub-button is displayed; if the color selection sub-button click event is detected, the corresponding color prefabricated body is loaded; if the color selection sub-button click event is not detected, the default color prefabricated body is selected; the preset gesture is detected by a custom gesture recognition module, and the user's drawing gesture is judged, if it is consistent with the preset drawing gesture, the virtual brush is displayed at the gesture position; and the user's hand image is collected, and when it is detected that the hand movement distance in two adjacent images is greater than the preset distance threshold, the virtual brush is controlled to move and draw a virtual line; then, for each drawing point in the virtual line, the angle corresponding to the drawing point is judged; if the angle is less than or equal to the preset angle threshold, the drawing unit adjacent to the drawing point is interpolated and smoothed to generate a smoothed virtual line.

[0229] Second, if Fig.14 As shown, a specific process of erasing is provided, including the following steps: the virtual reality device detects the click event of the erase button, and if the erase button is detected to be clicked, the erase mode selection sub-button pops up; then, the click event of the erase mode selection sub-button is detected, if the erase mode selection sub-button is clicked, the preset gesture is detected by the custom gesture recognition module, and if the preset gesture is detected, collision detection is performed; if the erase mode is partial erase, the prefabricated body that collides is deleted; if the erase mode is overall erase, the same type of prefabricated body with the same identification information as the prefabricated body that collides is obtained, and the same type of prefabricated body is deleted.

[0230] Third, if Fig.15 As shown, a specific process of color brush is provided, including the following steps: the virtual reality device detects the color brush button click event, if it is detected that the color brush button is clicked, then the color selection sub-button of the color brush is popped up; then, the click event of the color selection sub-button is detected, if the color selection sub-button is clicked, the preset gesture is detected by the custom gesture recognition module, if the preset gesture is detected, collision detection is performed; the color of the line drawing unit that collides is changed to the selected color.

[0231] For the virtual lines drawn above, the OpenGL module (OpenGraphics Library) provided by the Unity software can be used to render the virtual lines into the metaverse space in real time.

[0232] In this embodiment, the head-mounted virtual reality device Hololens2 and Unity are used to construct the metaverse space. Users can immersively experience the interactive process of reality-virtual fusion when drawing three-dimensional space lines in real time; three tools such as brushes, erasers and color brushes are provided to meet the objective needs of users in the metaverse to mark, draw and modify real and virtual materials. At the same time, this application uses specific gestures to activate the virtual brush for drawing, which is completely independent of the brush in reality, and is convenient to use whenever you want. In addition, in order to avoid jamming due to too many types of line units during the drawing process, this application adopts a unified cylindrical line as the minimum drawing unit to meet the requirements of the rendering pipeline in the Unity software for dynamic batch processing of materials in the scene, thereby reducing the rendering workload of OpenGL and reducing jams during the drawing process.

[0233] Furthermore, due to the geometric characteristics of the drawing unit itself, the present application uses Bezier interpolation to smooth the drawn lines, and proposes a unique way to eliminate the discontinuous visual perception at the junction of the minimum drawing unit. During the interpolation process, in order to reduce the visual discontinuity caused by the Bezier interpolation process, a staggered interpolation method is proposed to make the drawing process visually smoother. Not only that, the definition of the minimum drawing unit allows users to make local modifications to the drawn lines. Therefore, in terms of function, this application proposes an eraser tool and a color brush tool for the brush function, which greatly improves the practicality of the three-dimensional free brush tool.

[0234] In terms of experience: the camera on the HoloLens (mixed reality helmet) monitors the custom gestures in real time to determine whether the current state is drawing or erasing, preventing accidental drawing and erasing, solving the strong dependence on the real pen while enhancing the interactive experience of combining virtual and real; the brush model and the drawn lines appear in real time at the location of the gesture, giving users a real drawing experience.

[0235] In terms of performance: the minimum drawing unit (hereinafter referred to as the drawing unit) is defined, so that users can make local modifications to the drawn lines, avoiding redrawing while improving the rendering performance of OpenGL; the erasing function and color brush are triggered by collision detection (depending on the collision body properties). In order to avoid the performance loss of the drawing unit due to the collision body properties, the collision body properties of the drawing unit will be shielded during the drawing process. When the eraser or color brush gesture is recognized, the collision body properties of the minimum drawing unit are activated.

[0236] In terms of effect: two second-order Bezier interpolations are used to smooth the drawn lines; the interpolation in the drawing process is staggered and delayed, taking into account both visual continuity and real-time drawing; two line erasing modes, partial erasing and overall erasing, are provided, taking into account both erasing accuracy and efficiency; unique compensation processing is performed on the joints of the smallest drawing units of the lines to reduce the jaggedness of the lines; the color brush function is used for local color modification.

[0237] Furthermore, in one implementation, if the user needs to restore the erased line, a custom interactive gesture may be provided. When the virtual reality device recognizes the custom gesture, the most recently deleted line may be redrawn.

[0238] In one implementation, the drawing virtual key may also include a brush type selection sub-key, such as pencil, pen, brush, etc.; by providing different types of virtual brush models and waking them up through different gestures, different drawing needs of users can be met.

[0239] In one implementation, if the user needs to make the color lines look more colorful visually, multiple special effect processing modules can also be provided; multiple special effect processing modules can be implemented through a combination of shader and material. Multiple special effect processing modules can be called out through specific buttons to achieve line special effects by adding specific maps. For example, effects can be added to lines by means of texture maps to enrich the colors of the lines. Multiple line special effects can also be preset, and users can select different line special effects before drawing to draw virtual lines with special effects.

[0240] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0241] Based on the same inventive concept, the embodiment of the present application also provides a virtual element generation device for implementing the virtual element generation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more virtual element generation device embodiments provided below can refer to the limitations of the virtual element generation method above, and will not be repeated here.

[0242] In one embodiment, Fig.16 As shown, a virtual element generation device is provided, including: a trigger module 1601, a display module 1602 and a drawing module 1603, wherein:

[0243] The trigger module 1601 is configured to trigger a virtual element drawing request corresponding to the target operation in the virtual space in response to the target operation of the target object.

[0244] The display module 1602 is used to obtain a target action of a target object based on a virtual element drawing request; the target action is used to represent a virtual element drawing operation corresponding to the virtual element drawing request being performed in the virtual space.

[0245] The drawing module 1603 is used to obtain the motion trajectory of the target object, perform the virtual element drawing operation according to the motion trajectory of the target object, and generate a virtual element.

[0246] In one of the embodiments, virtual keys corresponding to different virtual element drawing requests are displayed in the virtual space, and the target operation includes a trigger operation on the virtual key; a trigger module 1601 is used to trigger a virtual element drawing request corresponding to the trigger operation of the target virtual key in the virtual space in response to the trigger operation of the target object on the target virtual key; the target virtual key includes any one of the virtual keys corresponding to different virtual element drawing requests; the virtual element drawing request corresponding to the trigger operation of the target virtual key carries the drawing attribute information corresponding to the target virtual key.

[0247] In one of the embodiments, virtual keys corresponding to different virtual element drawing requests are displayed in the virtual space, and the target operation includes a trigger operation on the virtual key; the above-mentioned trigger module 1601 includes a first trigger sub-module and a second trigger sub-module; wherein the first trigger sub-module is used to display at least one virtual sub-key corresponding to the target virtual key in the virtual space in response to the trigger operation of the target object on the target virtual key; the second trigger sub-module is used to trigger a virtual element drawing request corresponding to the trigger operation of the target virtual sub-key in the virtual space in response to the trigger operation on the target virtual sub-key; the target virtual sub-key includes at least one virtual sub-key corresponding to the target virtual key; the virtual element drawing request corresponding to the trigger operation of the target virtual sub-key carries the drawing attribute information corresponding to the target virtual sub-key.

[0248] In one embodiment, the display module 1602 includes a first shooting submodule and a first acquisition submodule; wherein the first shooting submodule is used to shoot the target object through an image acquisition device based on a virtual element drawing request to generate a first image; the first acquisition submodule is used to perform image recognition on the first image to obtain an image recognition result, and obtain the target action of the target object from the image recognition result.

[0249] In one embodiment, if the virtual element drawing request includes a graphics drawing request, the virtual element drawing operation includes a graphics drawing operation; the drawing module 1603 includes a second shooting submodule, a second acquisition submodule and a generation submodule; wherein the second shooting submodule is used to shoot the target object through an image acquisition device to generate multiple frames of second images of the target object; the second acquisition submodule is used to obtain the motion trajectory of the target object from the multiple frames of the second image; the generation submodule is used to obtain the first drawing attribute information from the graphics drawing request, perform the graphics drawing operation according to the first drawing attribute information and the motion trajectory of the target object, and generate a virtual element.

[0250] In one embodiment, the generation submodule includes a generation unit and a smoothing unit; wherein the generation unit is used to perform a graphics drawing operation according to the first drawing attribute information and the motion trajectory of the target object to generate an initial virtual element; the smoothing unit is used to smooth the initial virtual element to generate a virtual element and display the virtual element in a virtual space.

[0251] In one embodiment, the smoothing unit includes a first determining subunit, a second determining subunit and a generating subunit; wherein the first determining subunit is used to determine multiple initial drawing points and coordinate information of each initial drawing point from the initial virtual element; the second determining subunit is used to determine multiple target drawing points and coordinate information of each target drawing point by using Bezier interpolation method according to the coordinate information of each initial drawing point; and the generating subunit is used to generate the virtual element according to the coordinate information of each target drawing point.

[0252] In one of the embodiments, the second determination subunit is used to update the coordinate information of each initial drawing point by using the first Bezier interpolation method according to the coordinate information of the initial drawing point, the coordinate information of the previous drawing point of the initial drawing point, and the coordinate information of the next drawing point of the initial drawing point, so as to obtain a new initial drawing point corresponding to the initial drawing point and the coordinate information of the new initial drawing point; determine the coordinate information of the first intermediate drawing point and the first intermediate drawing point between the previous drawing point and the new initial drawing point, and the coordinate information of the second intermediate drawing point and the second intermediate drawing point between the new initial drawing point and the next drawing point according to the coordinate information of the new initial drawing point, the coordinate information of the previous drawing point, and the coordinate information of the next drawing point by using the second Bezier interpolation method and the golden section value; and use the new initial drawing point, the first intermediate drawing point, and the second intermediate drawing point as the target drawing point.

[0253] In one of the embodiments, a generating subunit is used to generate candidate virtual elements according to coordinate information of each target drawing point; for each target drawing point in the candidate virtual elements, a connecting pipeline corresponding to the target drawing point and the pipeline information of the connecting pipeline are determined according to pipeline information of a first pipeline and pipeline information of a second pipeline adjacent to the target drawing point; the first pipeline is a pipeline formed based on a previous drawing point of the target drawing point and the target drawing point; the second pipeline is a pipeline formed based on the target drawing point and a next drawing point of the target drawing point; based on the pipeline information of the connecting pipeline corresponding to each target drawing point, the position corresponding to each target drawing point in the candidate virtual element is filled to generate a virtual element.

[0254] In one of the embodiments, if the virtual element drawing request includes an erase request, the virtual element drawing operation includes an erase operation; the second shooting submodule is further used to shoot the target object through an image acquisition device to generate multiple frames of third images of the target object; the second acquisition submodule is further used to obtain the motion trajectory of the target object from the multiple frames of the third image; the generation submodule is further used to obtain the second drawing attribute information from the erase request, perform the erase operation according to the second drawing attribute information and the motion trajectory of the target object, and generate the virtual element.

[0255] In one of the embodiments, if the target virtual key is an erase virtual key, then at least one virtual sub-key corresponding to the target virtual key includes a partial erase sub-key and a whole erase sub-key; if the second drawing attribute information includes partial erase attribute information, then a sub-module is generated for performing a partial erase operation on the collided portion of the virtual element to generate an erased virtual element if it is determined that the target object collides with the virtual element based on the motion trajectory of the target object; if the second drawing attribute information includes whole erase attribute information, then a sub-module is generated for performing a whole erase operation on the virtual element if it is determined that the target object collides with the virtual element based on the motion trajectory of the target object.

[0256] In one of the embodiments, if the virtual element drawing request includes a color brush request, the virtual element drawing operation includes a color brush operation; the second shooting submodule is further used to shoot the target object through an image acquisition device to generate multiple frames of fourth images of the target object; the second acquisition submodule is further used to obtain the motion trajectory of the target object from the multiple frames of fourth images; the generation submodule is further used to obtain third drawing attribute information from the color brush request, perform the color brush operation according to the third drawing attribute information and the motion trajectory of the target object, and generate a modified virtual element.

[0257] In one of the embodiments, if the target virtual key is a color brush virtual key, the at least one virtual sub-key corresponding to the target virtual key includes a partial color brush key and an overall color brush key; if the third drawing attribute information includes partial color brush attribute information, a sub-module is generated for performing a partial color brush operation on the collision part of the virtual element to generate a modified virtual element if it is determined that the target object collides with the virtual element based on the motion trajectory of the target object; if the third drawing attribute information includes overall color brush attribute information, a sub-module is generated for performing an overall color brush operation on the virtual element to generate a modified virtual element if it is determined that the target object collides with the virtual element based on the motion trajectory of the target object.

[0258] Each module in the above virtual element generation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0259] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the virtual element generation method in the above-mentioned various embodiments when executing the computer program.

[0260] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the virtual element generation method in the above-mentioned various embodiments are implemented.

[0261] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps of the virtual element generation method in the above-mentioned various embodiments are implemented.

[0262] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0263] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0264] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0265] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for generating a virtual element, characterized in that: The method comprises: In response to a target operation of a target object, triggering a virtual element drawing request corresponding to the target operation in a virtual space; Acquire a target action of the target object based on the virtual element drawing request; the target action is used to represent the execution of a virtual element drawing operation corresponding to the virtual element drawing request in the virtual space; The motion trajectory of the target object is acquired, and the virtual element drawing operation is performed according to the motion trajectory of the target object to generate a virtual element.

2. The method according to claim 1, characterized in that The virtual space displays virtual keys corresponding to different virtual element drawing requests, and the target operation includes a triggering operation on the virtual keys; The step of triggering a virtual element drawing request corresponding to the target operation in the virtual space in response to the target operation of the target object includes: In response to a triggering operation of a target virtual key by a target object, a virtual element drawing request corresponding to the triggering operation of the target virtual key is triggered in a virtual space; the target virtual key includes any one of the virtual keys corresponding to the different virtual element drawing requests; the virtual element drawing request corresponding to the triggering operation of the target virtual key carries drawing attribute information corresponding to the target virtual key.

3. The method according to claim 1, characterized in that The virtual space displays virtual keys corresponding to different virtual element drawing requests, and the target operation includes a triggering operation on the virtual keys; The step of triggering a virtual element drawing request corresponding to the target operation in the virtual space in response to the target operation of the target object includes: In response to a triggering operation of a target object on a target virtual key, displaying at least one virtual sub-key corresponding to the target virtual key in a virtual space; In response to a triggering operation on a target virtual sub-key, a virtual element drawing request corresponding to the triggering operation of the target virtual sub-key is triggered in the virtual space; the target virtual sub-key includes at least one virtual sub-key corresponding to the target virtual key; the virtual element drawing request corresponding to the triggering operation of the target virtual sub-key carries drawing attribute information corresponding to the target virtual sub-key.

4. The method according to claim 2 or 3, characterized in that: The acquiring a target action of the target object based on the virtual element drawing request includes: Based on the virtual element drawing request, photographing the target object through an image acquisition device to generate a first image; Performing image recognition on the first image to obtain an image recognition result, and acquiring a target action of the target object from the image recognition result.

5. The method according to claim 2 or 3, characterized in that: If the virtual element drawing request includes a graphics drawing request, the virtual element drawing operation includes a graphics drawing operation; the acquiring the motion trajectory of the target object, performing the virtual element drawing operation according to the motion trajectory of the target object, and generating a virtual element includes: Photographing the target object by an image acquisition device to generate multiple frames of second images of the target object; Acquire a motion trajectory of the target object from the plurality of frames of second images; The first drawing attribute information is obtained from the graphic drawing request, and the graphic drawing operation is performed according to the first drawing attribute information and the motion trajectory of the target object to generate a virtual element.

6. The method according to claim 5, characterized in that The performing the graphic drawing operation according to the first drawing attribute information and the motion trajectory of the target object to generate a virtual element includes: Execute the graphic drawing operation according to the first drawing attribute information and the motion trajectory of the target object to generate an initial virtual element; The initial virtual element is smoothed to generate the virtual element, and the virtual element is displayed in the virtual space.

7. The method according to claim 6, characterized in that The step of smoothing the initial virtual element to generate the virtual element includes: Determine a plurality of initial drawing points and coordinate information of each of the initial drawing points from the initial virtual element; According to the coordinate information of each of the initial drawing points, a Bezier interpolation method is used to determine a plurality of target drawing points and the coordinate information of each of the target drawing points; The virtual element is generated according to the coordinate information of each target drawing point.

8. The method according to claim 7, characterized in that The method of determining a plurality of target drawing points and the coordinate information of each target drawing point by using the Bezier interpolation method according to the coordinate information of each initial drawing point includes: For each of the initial drawing points, the coordinate information of the initial drawing point is updated by using a first Bezier interpolation method according to the coordinate information of the initial drawing point, the coordinate information of a previous drawing point of the initial drawing point, and the coordinate information of a next drawing point of the initial drawing point, so as to obtain a new initial drawing point corresponding to the initial drawing point and the coordinate information of the new initial drawing point; According to the coordinate information of the new initial drawing point, the coordinate information of the previous drawing point and the coordinate information of the next drawing point, the second Bezier interpolation method and the golden section value are used to determine the coordinate information of the first intermediate drawing point between the previous drawing point and the new initial drawing point and the first intermediate drawing point, and the coordinate information of the second intermediate drawing point between the new initial drawing point and the next drawing point and the second intermediate drawing point; The new initial drawing point, the first intermediate drawing point and the second intermediate drawing point are used as target drawing points.

9. The method according to claim 7 or 8, characterized in that: Generating the virtual element according to the coordinate information of each target drawing point includes: Generate candidate virtual elements according to the coordinate information of each target drawing point; For each of the target drawing points in the candidate virtual elements, determine a connecting pipeline corresponding to the target drawing point and the pipeline information of the connecting pipeline according to the pipeline information of a first pipeline and a second pipeline adjacent to the target drawing point; the first pipeline is a pipeline formed based on a previous drawing point of the target drawing point and the target drawing point; the second pipeline is a pipeline formed based on the target drawing point and a next drawing point of the target drawing point; Based on the pipeline information of the connecting pipeline corresponding to each of the target drawing points, the position corresponding to each of the target drawing points in the candidate virtual elements is filled to generate the virtual element.

10. The method according to claim 3, characterized in that: If the virtual element drawing request includes an erase request, the virtual element drawing operation includes an erase operation; the acquiring the motion trajectory of the target object, performing the virtual element drawing operation according to the motion trajectory of the target object, and generating a virtual element includes: photographing the target object by using an image acquisition device to generate multiple frames of third images of the target object; Acquire a motion trajectory of the target object from the plurality of frames of third images; The second drawing attribute information is obtained from the erasing request, and the erasing operation is performed according to the second drawing attribute information and the motion trajectory of the target object to generate a virtual element.

11. The method according to claim 10, characterized in that If the target virtual key is an erase virtual key, the at least one virtual sub-key corresponding to the target virtual key includes a partial erase sub-key and a full erase sub-key; If the second drawing attribute information includes partial erasing attribute information, performing the erasing operation according to the second drawing attribute information and the motion trajectory of the target object to generate a virtual element includes: If it is determined according to the motion trajectory of the target object that the target object collides with the virtual element, a partial erasing operation is performed on the collided portion of the virtual element to generate an erased virtual element; If the second drawing attribute information includes overall erasing attribute information, performing the erasing operation according to the second drawing attribute information and the motion trajectory of the target object to generate a virtual element includes: If it is determined according to the motion trajectory of the target object that the target object collides with the virtual element, an overall erasing operation is performed on the virtual element.

12. The method according to claim 3, characterized in that If the virtual element drawing request includes a color brush request, the virtual element drawing operation includes a color brush operation; the acquiring the motion trajectory of the target object, performing the virtual element drawing operation according to the motion trajectory of the target object, and generating the virtual element includes: photographing the target object by using an image acquisition device to generate multiple frames of fourth images of the target object; Acquire a motion trajectory of the target object from the plurality of fourth image frames; The third drawing attribute information is obtained from the color brush request, and the color brush operation is performed according to the third drawing attribute information and the motion trajectory of the target object to generate a modified virtual element.

13. The method according to claim 12, characterized in that If the target virtual key is a color brush virtual key, the at least one virtual sub-key corresponding to the target virtual key includes a partial color brush key and a whole color brush key; If the third drawing attribute information includes partial color brush attribute information, performing the color brush operation according to the third drawing attribute information and the motion trajectory of the target object to generate a modified virtual element includes: If it is determined according to the motion trajectory of the target object that the target object collides with the virtual element, a partial color brush operation is performed on the collision portion of the virtual element to generate a modified virtual element; If the third drawing attribute information includes overall color brush attribute information, performing the color brush operation according to the third drawing attribute information and the motion trajectory of the target object to generate a modified virtual element includes: If it is determined according to the motion trajectory of the target object that the target object collides with the virtual element, an overall color brushing operation is performed on the virtual element to generate a modified virtual element.

14. A virtual element generating device, characterized in that: The device comprises: A trigger module, configured to trigger, in response to a target operation of a target object, a virtual element drawing request corresponding to the target operation in a virtual space; A display module, configured to obtain a target action of the target object based on the virtual element drawing request; the target action is used to represent a virtual element drawing operation corresponding to the virtual element drawing request being performed in the virtual space; The drawing module is used to obtain the motion trajectory of the target object, perform the virtual element drawing operation according to the motion trajectory of the target object, and generate a virtual element.

15. A virtual element rendering system, characterized in that: The invention comprises a mixed reality device; the mixed reality device is used to execute the steps of the method described in any one of claims 1 to 13 above.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 13 are implemented.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images