Dragging method and device

Abstract

The invention provides a dragging method and device.The method comprises the steps that the dragging direction of a to-be-dragged object dragged by a user from a first position and the direction indicated by posture information of the user are detected; in response to the fact that the dragging direction tends to be consistent with the direction indicated by the posture information of the user, determining a motion track of dragging the to-be-dragged object based on the dragging direction and the second position; the second position is determined based on the posture information of the user; and dragging the to-be-dragged object from the first position to a second position based on the motion trail. According to the embodiment, the accuracy and efficiency when the user drags the to-be-dragged object can be improved.

Description

Technical Field

[0001] The present invention relates to the field of extended reality technology, and in particular to a dragging method and device. Background Art

[0002] Extended reality is a general term for virtual reality (VR), augmented reality (AR) and mixed reality (MR), and is used to describe all immersive technologies that combine physical reality with the virtual world. In the related technologies in the field of extended reality, users can interact with virtual objects through body movements and other instructions, which greatly improves the user's immersive experience. However, in the existing technology, the accuracy of user interaction with virtual objects is low, and it is difficult to drag the virtual object to the target position at one time. The user needs to adjust the placement of the virtual object multiple times to place it in the ideal position, resulting in low efficiency in dragging virtual objects. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure at least provide a dragging method, device, electronic device and storage medium, which can improve the accuracy and efficiency of users when dragging virtual objects in an extended reality scene.

[0004] In a first aspect, an embodiment of the present disclosure provides a dragging method, including:

[0005] Detecting a dragging direction of the object to be dragged by the user from the first position and a direction indicated by the user's gesture information;

[0006] In response to the dragging direction being consistent with the direction indicated by the user's gesture information, determining a motion trajectory of the object to be dragged based on the dragging direction and a second position; the second position is determined based on the user's gesture information;

[0007] Based on the motion trajectory, the object to be dragged is dragged from the first position to the second position.

[0008] Optionally, the second position is determined based on the following steps:

[0009] Determine the user's orientation and the plane of objects in the user's environment;

[0010] The intersection point of a ray starting from the user position and with the user's orientation as the direction and the object plane is taken as the second position.

[0011] Optionally, determining a motion trajectory of dragging the object to be dragged based on the dragging direction and the second position includes:

[0012] Determine a vector having the first position as a starting point and a dragging direction of the object to be dragged by the user from the first position as a first direction vector;

[0013] Determine a vector with the first position as a starting point and the second position as an end point as a second direction vector; the first direction vector and the second direction vector have different end points;

[0014] The plane where the first direction vector and the second direction vector are located is used as the dragging motion plane;

[0015] The motion trajectory of the object to be dragged is determined based on the drag motion plane.

[0016] Optionally, determining a motion trajectory of the object to be dragged based on the drag motion plane includes:

[0017] A target circle is constructed in the dragging motion plane; the target circle takes the first direction vector as an external tangent vector, takes the first position as a tangent point and passes through the second position;

[0018] Determining the center of the target circle according to the coordinate data of the first position and the second position to construct the target circle in the dragging motion plane;

[0019] The arc between the first position and the second position on the target circle is used as the motion trajectory of the object to be dragged.

[0020] Optionally, based on the motion trajectory, dragging the object to be dragged from the first position to the second position includes:

[0021] Mapping the motion trajectory to the environment space where the user is located to obtain a dragging path between the first position and the second position;

[0022] Drag the object to be dragged from the first position to the second position according to the dragging path.

[0023] Optionally, based on the motion trajectory, dragging the object to be dragged from the first position to the second position includes:

[0024] Determine, according to a preset time interval, a third position of the object to be dragged from the first position to the second position by the user; the third position is between the first position and the second position;

[0025] Monitoring the user's posture information when the object to be dragged reaches the third position;

[0026] Based on the user's posture information and motion trajectory, determining a presentation position corresponding to when the object to be dragged reaches a third position;

[0027] The object to be dragged is presented to the user based on the presentation position in the process of dragging the object to be dragged from the first position to the second position.

[0028] In a second aspect, an embodiment of the present disclosure provides a towing device, including:

[0029] A detection module, used to detect a dragging direction of the object to be dragged by the user from the first position and a direction indicated by the user's gesture information;

[0030] a determination module, configured to determine a motion trajectory of the object to be dragged based on the drag direction and a second position in response to the drag direction being consistent with the direction indicated by the user's gesture information; the second position being determined based on the user's gesture information;

[0031] The dragging module is used to drag the object to be dragged from a first position to a second position based on a motion trajectory.

[0032] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps in any optional implementation of the first aspect are performed.

[0033] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect, or any optional implementation of the first aspect are executed.

[0034] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program, which implements the method of any of the above embodiments when executed by a processor.

[0035] The effects of the above-mentioned dragging device, electronic device and storage medium refer to the description of the above-mentioned dragging method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0037] Figure 1 A flowchart of a dragging method provided by an embodiment of the present disclosure is shown;

[0038] Figure 2A schematic flow chart of a dragging method provided by an embodiment of the present disclosure is shown;

[0039] Figure 3 A schematic diagram of a drag operation provided by an embodiment of the present disclosure is shown;

[0040] Figure 4 A schematic diagram of a dragging motion plane provided by an embodiment of the present disclosure is shown;

[0041] Figure 5 A schematic diagram showing coordinates of an environment space where a user is located provided by an embodiment of the present disclosure;

[0042] Figure 6 A schematic diagram of a towing device provided by an embodiment of the present disclosure is shown;

[0043] Figure 7 An exemplary system architecture is shown in which embodiments of the present disclosure may be applied;

[0044] Figure 8 A schematic diagram of the structure of a computer system of a terminal device or a server used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0046] It should be noted that in the technical solution of the present invention, the collection, use, storage, sharing and transfer of user personal information involved are in compliance with the provisions of relevant laws and regulations, and it is necessary to inform the user and obtain the user's consent or authorization. When applicable, the user's personal information is de-identified and / or anonymized and / or encrypted.

[0047] The raising of the above problems and the solutions are the results obtained by the inventor after practice and careful research. The discovery process of the above problems and the solutions proposed for the above problems should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0048] The technical solutions in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0049] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0050] To facilitate understanding of the present embodiment, a drag method disclosed in the present embodiment is first introduced in detail. The execution subject of the drag method provided in the present embodiment is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the drag method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0051] See also Figure 1 FIG. 1 is a flowchart of a dragging method provided by an embodiment of the present disclosure, and the method includes S101 to S103, wherein:

[0052] S101: Detecting a dragging direction in which a user drags an object to be dragged from a first position and a direction indicated by the user's gesture information.

[0053] In the embodiments of the present disclosure, the user can drag the object to be dragged through a device that supports the extended reality technology, such as a wearable extended reality device, a projected extended reality device, an interactive desktop system, a holographic display device, and a mobile terminal device. The embodiments of the present disclosure do not limit the specific device used to execute the embodiments of the present disclosure. In actual applications, the device can be set according to the actual situation and requirements, so as to achieve its functions.

[0054] In the embodiment of the present disclosure, the first position may represent the starting position of the object to be dragged, or any position before reaching the end point during the dragging process. When the user starts to drag the object to be dragged, a direction vector is generated, which indicates the direction in which the user is currently performing the dragging operation on the object to be dragged. At the same time, the execution body of the embodiment of the present disclosure may also obtain the user's current posture information through relevant equipment, and determine the direction indicated by the user's current posture information. In a specific implementation, when the user starts to drag the object to be dragged through a gesture, a mouse, a touch screen or other input device, the execution body of the embodiment of the present disclosure may determine the direction in which the user is dragging the object to be dragged at this time based on the user's input information, and compare the direction with the direction indicated by the user's current posture information. The subsequent dragging process is carried out in this way, and the subsequent steps are specifically described in S102, which will not be repeated here.

[0055] S102: In response to the dragging direction being consistent with the direction indicated by the user's gesture information, determining a motion trajectory of the object to be dragged based on the dragging direction and a second position; the second position is determined based on the user's gesture information.

[0056] In this step, when the dragging direction is consistent with the direction indicated by the user's gesture information, it can be considered that the user intends to drag the object to be dragged to the second position. The consistency here can be defined as the dragging direction is completely consistent with the direction indicated by the user's gesture information, or it can be defined as the deviation between the dragging direction and the direction indicated by the user's gesture information does not exceed a preset angle or value, which is not specifically limited in the embodiments of the present disclosure.

[0057] The execution body of the embodiment of the present disclosure can determine the direction, speed, acceleration and other data of the object to be dragged by the user at this time according to the user's input information, and then can calculate and predict the position that the object to be dragged may reach in the next frame based on these data. This position can be called the middle point of the drag. The vector starting from the first position, along the dragging direction and ending at the middle point of the drag can be used as the current direction vector for indicating that the user drags the object to be dragged, that is, the first direction vector mentioned in the subsequent steps. It should be noted that the direction of the first direction vector can be pointing to the second position. The "pointing" here can be understood as the first direction vector being able to indicate that the object to be dragged is moving in the direction toward the second position. As for whether it is in the same straight line as dragging the second position, the embodiment of the present disclosure does not make specific restrictions.

[0058] In the embodiments of the present disclosure, Figure 2As shown, the second position can be determined according to the user's posture and the object plane in the space environment; the second position here can be the end point of the dragging, or any target point in the dragging process, and the disclosed embodiment does not specifically limit this. Specifically, the second position is determined based on the following steps: determining the user's orientation and the object plane in the user's environment space; taking the intersection of a ray with the user's position as the starting point and the user's orientation as the direction and the object plane as the second position.

[0059] Taking the case where the user is wearing a head-mounted device as an example, since the user's head tends to lean toward the target position to be reached when dragging the object to be dragged, the user's head posture can be used as a reference factor for determining the second position. Alternatively, when the device used by the user is a non-wearable device, the direction of the second position can also be determined by capturing information such as the user's body orientation and gaze orientation. The object plane in the environmental space can be a plane in the real world, that is, a plane of a real object detected by a camera or sensor, such as a table, floor, wall, etc. These planes of real objects can interact with virtual objects or serve as the placement position of virtual objects. In addition, the object plane in the environmental space can also be a plane in the virtual space. For example, in a VR or MR scene, the walls in a virtual room, the floor of a virtual building, etc. can also be defined as an object plane in the environmental space, and these planes in the virtual space can also interact with virtual objects.

[0060] In a specific implementation, assuming that the user is wearing a head-mounted device, the user's orientation can be determined based on the user's head posture, and then a ray is made from the center point of the user's head or between the two eyes in the same direction as the user's orientation, and then the intersection of this ray and the object plane in the ambient space is taken as the second position.

[0061] In another possible implementation, the second position may also be calibrated in advance. For example, the user can calibrate a second position by himself before starting to drag, and then the subsequent dragging process can be performed with this point as the target. For another example, the second position may also be predetermined by the execution subject of the embodiment of the present disclosure according to the actual scene. For example, when the user is playing a VR game, a virtual object needs to be placed at the target position to complete the game task. At this time, the second position may be pre-calibrated by the execution subject of the embodiment of the present disclosure. The above method is only a feasible method for determining the second position in the embodiment of the present disclosure, and does not constitute an improper limitation on the present invention. In actual applications, the second position can be determined according to the actual situation. The embodiment of the present disclosure does not make specific limitations on this, and its function can be realized.

[0062] like Figure 2As shown in FIG. 1 , after determining that the second position exists, it can be determined whether the user intends to drag the virtual object to be dragged to the second position. Specifically, as Figure 3 As shown, a virtual object enclosing volume can be constructed at the second position, and the shape of the enclosing volume can be selected according to actual needs, and the embodiments of the present disclosure do not make specific restrictions on this. When it is identified that the extension line of the above-mentioned first direction vector and the enclosing sphere have an intersection, it can be proved that the user has the intention to move the object to be dragged to the second position at this time. After it is determined that the intention exists, the subsequent steps of determining the motion trajectory between the first position and the second position can be performed; as for the case where there is no second position or no clear dragging intention, the dragging position of the virtual object can be determined in real time according to the user's dragging action, without calculating the corresponding motion trajectory. The enclosing sphere here is a minimized sphere that surrounds all vertices of the virtual object, which is used to simplify the complex geometric shape of the object to be dragged; in addition to being a sphere, it can also be any other three-dimensional shape, and the embodiments of the present disclosure do not make specific restrictions on this.

[0063] In the embodiment of the present disclosure, after the first direction vector and the second position are determined, the drag motion plane can be determined based on the first direction vector and the second position. Specifically, the motion trajectory of the object to be dragged is determined based on the drag direction and the second position, including: determining a vector with the first position as the starting point and the drag direction of the object to be dragged from the first position as the direction as the first direction vector; determining a vector with the first position as the starting point and the second position as the end point as the second direction vector; the end points of the first direction vector and the second direction vector are different; using the plane where the first direction vector and the second direction vector are located as the drag motion plane; and determining the motion trajectory of the object to be dragged based on the drag motion plane.

[0064] In a specific implementation, when there is a clear second position and a dragging intention toward the target point, a dragging motion plane where the motion process of dragging the object to be dragged from the first position to the second position is located can be constructed. Specifically, Figure 4 As shown, let point A be the first position, point B be the determined dragging middle point, and vector is the first direction vector, point D is the second position, and vector is the second direction vector. Then the dragging motion plane can be directly constructed according to the first direction vector and the second direction vector.

[0065] In another possible implementation, a triangle consisting of point A, point B and point D may be constructed first. And vector , and then the vector With vector And according to the right-hand coordinate system rule, direction, that is Normal vector of the plane . Through the first position A and the determined normal vector You can construct And the dragging motion plane where the object to be dragged moves from point A to point D.

[0066] In another possible implementation, if the current dragging scene is known to be performed on the surface of an object, such as dragging a virtual object located on the left side of the desktop to the right side of the desktop; at this time, the normal vector of the plane can be directly used as the normal vector of the dragging motion plane, thereby determining the dragging motion plane. Alternatively, the dragging motion plane can also be determined in combination with the user's camera perspective; specifically, the forward vector and lateral vector of the camera direction can be obtained first, and then a dragging motion plane based on the perspective can be derived based on the starting point of the camera direction and the user interaction. It should be noted that the dragging motion plane in the embodiment of the present disclosure is the plane where the object to be dragged moves from the first position to the second position. On the basis of ensuring that the starting point of the first direction vector and the second position are both located on the plane, the method for determining the dragging motion plane can be set and selected according to actual needs and circumstances, and the embodiment of the present disclosure does not specifically limit this.

[0067] In the embodiment of the present disclosure, after the first direction vector, the second position and the dragging motion plane are determined, the motion trajectory of the object to be dragged on the plane can be determined in the determined dragging motion plane.

[0068] In the disclosed embodiment, the motion trajectory of the object to be dragged on the dragging motion plane can be determined by constructing a target circle. Specifically, determining the motion trajectory of the object to be dragged based on the dragging motion plane includes: constructing a target circle in the dragging motion plane; the target circle uses a first direction vector as an external tangent vector, a first position as a tangent point and passes through a second position; determining the center of the target circle based on the coordinate data of the first position and the second position to construct the target circle in the dragging motion plane; and using the arc between the first position and the second position on the target circle as the motion trajectory of the object to be dragged.

[0069] In a specific implementation, after obtaining the drag motion plane, as Figure 4 As shown, we can construct a vector The circle with the tangent point A and passing through the second position D. The coordinate O of the center of the target circle in the dragging motion plane can be calculated through points A and D, thus obtaining the following: Figure 4 Then the arc between the first position A and the first position D can be As a movement trajectory.

[0070] For example, assume that in the drag motion plane, the coordinates of the first position A are (10, 20), the coordinates of the dragging middle point B are (12, 24), and the coordinates of the second position D are (15, 22). First, the vector The direction vector is , since the target circle and the vector The tangent point is A, so the radius of the target circle is at point A. Vertical. After determining the vector After the direction vector is determined, the The direction vector can be (-4,2), which means that the center of the target circle must be on a straight line passing through point A (10,20) and with a direction of (-4,2). In addition, the target circle needs to pass through the second position D. Assuming that the coordinates of the center O are (x,y), the distance from the center to point A is equal to the distance to point D and the center is located on a straight line passing through point A (10,20) and with a direction of (-4,2) can be used to construct an equation group. Finally, the coordinates of the center of the circle are solved as ( ). After obtaining the coordinates of the center of the circle, the above conditions can be combined to determine a unique target circle in the dragging motion plane, and then the major arc between point A and point D can be used as the motion trajectory of the object to be dragged.

[0071] In another possible implementation, in addition to constructing a target circle to use the arc between the first position and the second position as the motion path, the motion trajectory can also be determined by constructing, for example, a sector shape, an ellipse shape, etc. Alternatively, a path planning algorithm can be used to determine each candidate path node between the first position and the second position, and then each candidate path node is connected and smoothed to obtain the final motion trajectory; when using a path planning algorithm to determine the motion trajectory of the object to be dragged, its cost function can be set according to demand, for example, factors such as the shortest path and the smallest turning angle can be taken into account in the cost function, so that the final generated motion trajectory is as short and stable as possible. It should be noted that the above is only used as an example of a feasible implementation method in the embodiment of the present disclosure, and does not constitute an improper limitation on the present invention. In actual application, it is based on the ability to realize its functions, and the embodiment of the present disclosure does not make specific limitations on this.

[0072] S103: dragging the object to be dragged from the first position to the second position based on the motion trajectory.

[0073] In this step, since the dragging motion plane is a two-dimensional plane, and the user drags the object to be dragged in three-dimensional space, the motion trajectory on the dragging motion plane cannot accurately reflect the dragging path in the actual three-dimensional space. Therefore, the dragging path of the object to be dragged in the three-dimensional space can be determined based on the two-dimensional motion trajectory determined in the previous article.

[0074] In an embodiment of the present disclosure, based on the motion trajectory, the object to be dragged is dragged from a first position to a second position, including: mapping the motion trajectory to the environment space where the user is located to obtain a dragging path between the first position and the second position; and dragging the object to be dragged from the first position to the second position according to the dragging path.

[0075] In a specific implementation, after determining the target circle, a Cartesian coordinate system can be constructed with the center O as the origin; wherein, the vector The direction is the horizontal axis direction, and the normal vector of the dragging motion plane The direction of is the longitudinal axis, and the vertical axis direction is determined by the right-hand rule. Figure 5 As shown, the coordinate system of the user's space environment is different from the coordinate system determined based on the target circle on the drag motion plane. Therefore, the transformation relationship between the two can be determined based on the second coordinate system corresponding to the user's environment space and the first coordinate system determined based on the target circle. The change relationship here can be specifically reflected in the form of a matrix. For example, the transformation relationship between the first coordinate system and the second coordinate system is as follows:

[0076] M = ;

[0077] Where R is a rotation matrix for controlling the rotation of the coordinate system; T is a translation vector for controlling the position change of the coordinate system; 0 is a zero vector for ensuring the data integrity of the matrix, and 1 is a scale factor for homogeneous coordinate transformation. It should be noted that the disclosed embodiment does not specifically limit the method for determining the transformation relationship between the first coordinate system and the second coordinate system. In practical applications, after determining the change relationship between the first coordinate system and the second coordinate system in a specific implementation, each point in the second coordinate system can be photographed in the first coordinate system for calculation. After the calculation is completed, the points in the first coordinate system can be mapped back to the second coordinate system, thereby reducing the difficulty of calculation and obtaining each point in the three-dimensional coordinate system. After mapping each point in the motion trajectory in the drag motion plane to the coordinate system corresponding to the user's environment space, the drag path of the object to be dragged can be obtained. This ensures that when the user drags the virtual object to be dragged, it is easier to drag it to the user's expected target position, thereby improving the accuracy and efficiency of dragging.

[0078] In another possible implementation, it is also possible not to rely on a specific coordinate system conversion, but to project the two-dimensional motion trajectory into three-dimensional space through a predefined surface. Specifically, a surface can be selected, such as a spherical surface, a cylindrical surface, or any other surface, and then each point in the two-dimensional motion trajectory can be mapped to a corresponding point on the surface. For example, the two-dimensional point can be projected onto a spherical surface, and the three-dimensional coordinates can be obtained using a spherical coordinate system, thereby generating a drag path in the user's environment space. Alternatively, a corresponding three-dimensional point can be generated for each point on the two-dimensional motion trajectory, and then its height or other parameters can be defined according to the interpolation rules, thereby generating a smooth drag path in three-dimensional space, and continuous motion can be achieved through interpolation points.

[0079] It should be noted that the above method of obtaining a three-dimensional dragging path based on a two-dimensional motion trajectory is only used as an example of a possible implementation method of the present disclosure, and does not constitute an improper limitation on the present invention. In practical applications, a suitable method can be selected according to actual needs and circumstances, and the present disclosure does not make specific limitations on this.

[0080] In the disclosed embodiment, in order to improve the user's visual experience when the user drags the virtual object to be dragged, after determining the dragging path, the display position of the object to be dragged can also be determined according to the user's posture. Specifically, based on the motion trajectory, the object to be dragged is dragged from the first position to the second position, including: determining the third position of the user in the process of dragging the object to be dragged from the first position to the second position according to a preset time interval; the third position is between the first position and the second position; monitoring the user's posture information when the object to be dragged reaches the third position; based on the user's posture information and the motion trajectory, determining the corresponding presentation position when the object to be dragged reaches the third position; presenting the object to be dragged in the process of dragging the object to be dragged from the first position to the second position to the user based on the presentation position.

[0081] In a specific implementation, when the user needs to drag the virtual object to the target position, in order to maintain visual consistency between the dragging path of the virtual object and the user's operation, the user's position can be projected onto the dragging motion plane to obtain a projection point C. Then, the intersection of the line segment CB formed by the projection point C and the dragging middle point B and the motion trajectory in the dragging motion plane is used as the display position of the current object to be dragged. It should be noted that the dragging middle point here changes with the user's dragging operation, and the position of this point may be different in different frames or at different times, so that the real-time position of the object to be dragged can be displayed to the user when the object to be dragged is dragged, which not only improves the user's visual experience, but also further improves the real-time and accuracy of the dragging operation.

[0082] In another possible implementation, since the user's perspective is constantly changing, the display position of the object to be dragged can be adjusted by updating the projection of the object to be dragged in the user's perspective in real time. Specifically, when the user drags the object to be dragged through an AR or VR device, the display position of the object on the screen can be corrected based on the perspective projection, so as to ensure that the visual effect of the dragging process is consistent with the actual position change.

[0083] It should be noted that the above method for ensuring visual and operational consistency is only used as an example of a feasible implementation method in the embodiment of the present disclosure, and does not constitute an improper limitation of the present invention. In practical applications, it can be set and selected according to actual conditions and needs, and the embodiment of the present disclosure does not make specific limitations on this.

[0084] According to a second aspect of the embodiment of the present disclosure, Figure 6 As shown, a towing device 600 is provided, comprising:

[0085] A detection module 601, configured to detect a dragging direction in which a user drags an object to be dragged from a first position and a direction indicated by the user's gesture information;

[0086] A determination module 602, configured to determine a motion trajectory of the object to be dragged based on the drag direction and a second position in response to the drag direction being consistent with the direction indicated by the user's gesture information; the second position is determined based on the user's gesture information;

[0087] The dragging module 603 is used to drag the object to be dragged from a first position to a second position based on the motion trajectory.

[0088] Optionally, the determination module 602 is specifically configured to:

[0089] Determine the user's orientation and the plane of objects in the user's environment;

[0090] The intersection point of a ray starting from the user position and with the user's orientation as the direction and the object plane is taken as the second position.

[0091] Optionally, the determination module 602 is specifically configured to:

[0092] Determine a vector having the first position as a starting point and a dragging direction of the object to be dragged by the user from the first position as a first direction vector;

[0093] Determine a vector with the first position as a starting point and the second position as an end point as a second direction vector; the first direction vector and the second direction vector have different end points;

[0094] The plane where the first direction vector and the second direction vector are located is used as the dragging motion plane;

[0095] The motion trajectory of the object to be dragged is determined based on the drag motion plane.

[0096] Optionally, the determination module 602 is specifically configured to:

[0097] A target circle is constructed in the dragging motion plane; the target circle takes the first direction vector as an external tangent vector, takes the first position as a tangent point and passes through the second position;

[0098] Determining the center of the target circle according to the coordinate data of the first position and the second position to construct the target circle in the dragging motion plane;

[0099] The arc between the first position and the second position on the target circle is used as the motion trajectory of the object to be dragged.

[0100] Optionally, the drag module 603 is specifically used for:

[0101] Mapping the motion trajectory to the environment space where the user is located to obtain a dragging path between the first position and the second position;

[0102] Drag the object to be dragged from the first position to the second position according to the dragging path.

[0103] Optionally, the drag module 603 is specifically used for:

[0104] Determine, according to a preset time interval, a third position of the object to be dragged from the first position to the second position by the user; the third position is between the first position and the second position;

[0105] Monitoring the user's posture information when the object to be dragged reaches the third position;

[0106] Based on the user's posture information and motion trajectory, determining a presentation position corresponding to when the object to be dragged reaches a third position;

[0107] The object to be dragged is presented to the user based on the presentation position in the process of dragging the object to be dragged from the first position to the second position.

[0108] According to the third aspect of an embodiment of the present disclosure, a dragging electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by one or more processors, the one or more processors implement the method provided by the first aspect of the embodiment of the present invention.

[0109] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present disclosure is implemented.

[0110] According to a fifth aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and the computer program implements the method of any of the above embodiments when executed by a processor.

[0111] Figure 7 An exemplary system architecture 700 is shown to which the dragging method or the dragging apparatus implemented by the present disclosure can be applied.

[0112] like Figure 7 As shown, system architecture 700 may include terminal devices 701, 702, 703, network 704 and server 705. Network 704 is used to provide a medium for communication links between terminal devices 701, 702, 703 and server 705. Network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0113] The user can use the terminal devices 701, 702, 703 to interact with the server 705 through the network 704 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 701, 702, 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0114] The terminal devices 701 , 702 , and 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0115] The server 705 may be a server that provides various services, such as a background management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 701, 702, and 703. The background management server may process the received drag request and feed back the processing result (only an example) to the terminal device.

[0116] It should be noted that the dragging method provided in the embodiment of the present invention is generally executed by the server 705, and accordingly, the dragging device is generally set in the server 705. The dragging method provided in the embodiment of the present invention can also be executed by the terminal devices 701, 702, and 703, and accordingly, the dragging device can be set in the terminal devices 701, 702, and 703.

[0117] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0118] Reference below Figure 8, which shows a schematic diagram of the structure of a computer system 800 of a terminal device suitable for implementing an embodiment of the present invention. Figure 8 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0119] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0120] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0121] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are executed.

[0122] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0123] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor. For example, a processor includes a plane determination module, a trajectory determination module, and a path determination module, wherein the names of these modules do not constitute limitations on the modules themselves in certain cases. For example, the detection module may also be described as "detecting the dragging direction of the object to be dragged by the user from the first position and the direction indicated by the user's posture information".

[0125] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device implements the following method: detecting the dragging direction of the object to be dragged from the first position by the user and the direction indicated by the user's posture information; in response to the trend of the dragging direction and the direction indicated by the user's posture information being consistent, determining the motion trajectory of the object to be dragged based on the dragging direction and the second position; the second position is determined based on the user's posture information; based on the motion trajectory, dragging the object to be dragged from the first position to the second position.

[0126] Finally, it should be noted that the above embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A dragging method, characterized in that: include: Detecting a dragging direction of the object to be dragged by a user from a first position and a direction indicated by the user's gesture information; In response to the dragging direction being consistent with the direction indicated by the user's gesture information, determining a motion trajectory of dragging the object to be dragged based on the dragging direction and a second position; the second position is determined based on the user's gesture information; Based on the motion trajectory, the object to be dragged is dragged from the first position to the second position.

2. The method according to claim 1, characterized in that The second position is determined based on the following steps: Determining the orientation of a user and the plane of objects in the environment space where the user is located; The intersection point of a ray that takes the user position as a starting point and the user's orientation as a direction and the object plane is taken as the second position.

3. The method according to claim 1, characterized in that Determining a motion trajectory of dragging the object to be dragged based on the dragging direction and the second position includes: Determine a vector having the first position as a starting point and a direction in which a user drags the object to be dragged from the first position as a first direction vector; Determine a vector with the first position as a starting point and the second position as an end point as a second direction vector; the first direction vector and the second direction vector have different end points; The plane where the first direction vector and the second direction vector are located is used as the dragging motion plane; The motion trajectory of the object to be dragged is determined based on the drag motion plane.

4. The method according to claim 3, characterized in that Determining the motion trajectory of the object to be dragged based on the drag motion plane includes: Constructing a target circle in the dragging motion plane; the target circle takes the first direction vector as an external tangent vector, takes the first position as a tangent point and passes through the second position; Determining the center of the target circle according to the coordinate data of the first position and the second position to construct the target circle in the dragging motion plane; An arc between the first position and the second position on the target circle is used as a motion trajectory of the object to be dragged.

5. The method according to claim 1, characterized in that Based on the motion trajectory, dragging the object to be dragged from the first position to the second position includes: Mapping the motion trajectory to the environment space where the user is located to obtain a dragging path between the first position and the second position; According to the dragging path, the object to be dragged is dragged from the first position to the second position.

6. The method according to any one of claims 1 to 5, characterized in that: Based on the motion trajectory, dragging the object to be dragged from the first position to the second position includes: Determining, according to a preset time interval, a third position of the object to be dragged from the first position to the second position by the user; the third position is between the first position and the second position; Monitoring the user's posture information when the object to be dragged reaches the third position; Based on the user's posture information and the motion trajectory, determining a presentation position corresponding to when the object to be dragged reaches the third position; The object to be dragged is presented to the user based on the presentation position in a process of dragging the object to be dragged from the first position to the second position.

7. A towing device, characterized in that: include: A detection module, used to detect a dragging direction of the object to be dragged by the user from the first position and a direction indicated by the user's gesture information; a determination module, configured to determine a motion trajectory of dragging the object to be dragged based on the dragging direction and a second position in response to the dragging direction being consistent with the direction indicated by the user's gesture information; the second position being determined based on the user's gesture information; The dragging module is used to drag the object to be dragged from the first position to the second position based on the motion trajectory.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images