Virtual-real fitting positioning method and device

Abstract

The invention discloses a virtual-real fitting positioning method and device. The method is applied to mixed reality equipment and comprises the steps that a motion track of a target gesture moving on a real object is obtained, the real object comprises a first identification point and a first identification line with the first identification point as an end point, the first identification point is a point on a ridge line of the real object, the ridge line comprises the first identification line, and the first identification point is a point on the ridge line of the real object; the motion trail comprises a motion trail starting from the first identification point and moving along the first identification line; determining spatial position data of the real object according to the motion trail; according to the spatial position data, the position and posture of a virtual object displayed in a virtual scene of the mixed reality equipment are changed, and the shape contour of the virtual object and the shape contour of the real object meet preset similar conditions. According to the virtual-real fitting positioning method provided by the embodiment of the invention, the efficiency of virtual-real fitting positioning can be improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of virtual-real fusion, and particularly relates to a virtual-real fusion positioning method and device. BACKGROUND

[0002] With the development of science and technology, the virtual-real fusion technology is applied more and more widely. The application process of the virtual-real fusion technology usually includes the following steps: obtaining a virtual object that is almost completely consistent with a real object through three-dimensional scanning or other technologies, and then performing accurate position fitting and attitude fitting on the virtual object and the real object.

[0003] At present, the position fitting and attitude fitting of the virtual object and the real object include a manual interaction mode, which usually includes the following steps: displaying the real object, the virtual object and a plurality of interaction controls for adjusting the position and attitude of the virtual object, receiving multiple and repeated operations of a user on the plurality of interaction controls to transform the position and attitude of the virtual object, and making the virtual object completely coincide with the real object.

[0004] However, the above-mentioned manual interaction mode is complicated to operate, which results in low efficiency of virtual-real fusion positioning. SUMMARY

[0005] Embodiments of the application provide a virtual-real fusion positioning method, device, equipment, computer readable storage medium and computer program product, which can simplify user operation and improve the efficiency of virtual-real fusion positioning.

[0006] In a first aspect, embodiments of the application provide a virtual-real fusion positioning method applied to a mixed reality device, and the method includes the following steps.

[0007] Obtaining a motion trajectory of a target gesture moving on a real object, the real object including a first identification point and a first identification line with the first identification point as an end point, the first identification point being a point on a ridge line of the real object, the ridge line including the first identification line, and the motion trajectory including a motion trajectory of moving along the first identification line from the first identification point;

[0008] Determining spatial position data of the real object according to the motion trajectory;

[0009] Transforming a position and an attitude of a virtual object displayed in a virtual scene of the mixed reality device according to the spatial position data, the shape contour of the virtual object and the real object satisfying a preset similarity condition.

[0010] In a possible implementation manner, the obtaining of the motion trajectory of the target gesture moving on the real object includes the following steps.

[0011] Identifying a user gesture;

[0012] In a case where the user gesture is a target gesture, a motion trajectory of the target gesture moving on the real object is acquired.

[0013] In a possible implementation, the acquiring of the motion trajectory of the target gesture moving on the real object in a case where the user gesture is a target gesture comprises:

[0014] In a case where the user gesture is a target gesture, a target position of the target gesture is determined.

[0015] A position of the target position is tracked to obtain the motion trajectory of the target gesture moving on the real object.

[0016] In a possible implementation, the determining of the target position of the target gesture in a case where the user gesture is a target gesture comprises:

[0017] In a case where the user gesture is a target gesture, a target gesture image is acquired.

[0018] The target gesture image is recognized to obtain the target position of the target gesture.

[0019] In a possible implementation, the spatial position data comprises first position data of the first identification point and a first spatial position vector corresponding to the first identification line, and the determining of the spatial position data of the real object according to the motion trajectory comprises:

[0020] First position data of the target position is acquired.

[0021] The first position data of the target position is determined as the first position data of the first identification point.

[0022] A first spatial position vector corresponding to the first identification line is determined according to the first position data and the motion trajectory.

[0023] In a possible implementation, the virtual object comprises a second identification point and a second identification line, a position of the second identification point in the virtual object corresponds to a position of the first identification point in the real object, and a position of the second identification line in the virtual object corresponds to a position of the first identification line in the real object; and the determining of the spatial position data of the real object according to the motion trajectory comprises:

[0024] A target spatial rectangular coordinate system is constructed with the second identification point as an origin, the second identification line as a first axis, and two lines respectively perpendicular to the second identification line and perpendicular to each other as a second axis and a third axis.

[0025] According to the motion trajectory, spatial position data of the real object in the target space rectangular coordinate system is determined;

[0026] The method further comprises:

[0027] According to the spatial position data, a position and a pose of a virtual object displayed in a virtual scene of the mixed reality device are transformed.

[0028] According to the first position data and the second position data, a second identification point in the virtual object is moved to a position where the first identification point is located.

[0029] According to the first spatial position vector and the second spatial position vector, a second identification line in the virtual object is transformed to a pose where the first identification line is located.

[0030] In a possible implementation, the real object comprises a polyhedron composed of a plurality of edge lines, the first identification point is a vertex of the polyhedron, and the first identification line is an edge line with the first identification point as an end point.

[0031] In a possible implementation, the real object is horizontally placed, and the first identification line is parallel to a horizontal line.

[0032] In a possible implementation, before the position and the pose of the virtual object displayed in the virtual scene of the mixed reality device are transformed according to the spatial position data, the method further comprises:

[0033] The shape contour of the real object is scanned to obtain shape contour information.

[0034] The shape contour information is three-dimensionally reconstructed to obtain the virtual object.

[0035] In a second aspect, an embodiment of the present application provides a virtual-real fitting positioning device applied to a mixed reality device, which comprises:

[0036] An acquisition module is configured to acquire a motion trajectory of a target gesture moving on a real object, the real object comprising a first identification point and a first identification line with the first identification point as an end point, the first identification point being a point on an edge line of the real object, the edge line comprising the first identification line, and the motion trajectory comprising a motion trajectory of moving along the first identification line from the first identification point.

[0037] A determination module is configured to determine spatial position data of the real object according to the motion trajectory.

[0038] a transformation module configured to transform a position and a pose of a virtual object displayed in a virtual scene of the mixed reality device according to the spatial position data, the virtual object and the real object satisfying a preset similarity condition in shape profile.

[0039] In a third aspect, an electronic device is provided. The device includes a processor and a memory storing computer program instructions.

[0040] The processor, when executing the computer program instructions, implements the method in any possible implementation method of the first aspect.

[0041] In a fourth aspect, a computer readable storage medium is provided. The computer readable storage medium stores computer program instructions. The computer program instructions, when executed by a processor, implement the method in any possible implementation method of the first aspect.

[0042] In a fifth aspect, a computer program product is provided. The instructions in the computer program product, when executed by a processor of an electronic device, cause the electronic device to perform the method in any possible implementation method of the first aspect.

[0043] In the virtual-real fitting positioning method and device, if a first identification point of the real object and a first identification line with the first identification point as an end point are determined, the position and the pose of the real object are determined, and then in the case that the virtual object and the real object satisfy a preset similarity condition in shape profile, the position and the pose of the virtual object to be transformed are determined through the first identification point and the first identification line. In this way, by obtaining a movement track of the target gesture moving from the first identification point along the first identification line on the real object, determining the spatial position data of the real object according to the movement track, and transforming the position and the pose of the virtual object displayed in the virtual scene of the mixed reality device according to the spatial position data, the virtual-real fitting positioning is realized through the movement track of the target gesture based on the first identification point and the first identification line, without the need for the user to perform multiple and repeated operations on multiple interactive controls, so that the user operation is simplified and the virtual-real fitting positioning efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. For those skilled in the art, without paying creative labor, other drawings can also be obtained according to these drawings.

[0045] Figure 1 is a flowchart of a virtual-real fitting positioning method provided by the embodiments of the present application.

[0046] Figure 2 is a schematic diagram of a real object provided by an embodiment of the present application;

[0047] Figure 3 is a schematic diagram of a real object provided by an embodiment of the present application;

[0048] Figure 4 is a schematic diagram of a user interaction operation provided by an embodiment of the present application;

[0049] Figure 5 is a schematic diagram of a display screen of an MR device provided by an embodiment of the present application;

[0050] Figure 6 is a schematic diagram of a target space rectangular coordinate system provided by an embodiment of the present application;

[0051] Figure 7 is a schematic diagram of a virtual-real fitting positioning process provided by an embodiment of the present application;

[0052] Figure 8 is a schematic diagram of a virtual-real fitting positioning result provided by an embodiment of the present application;

[0053] Figure 9 is a schematic diagram of a virtual-real fitting positioning device provided by an embodiment of the present application;

[0054] Figure 10 is a schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0055] The features and exemplary embodiments of various aspects of the present application will be described below in detail, in order to make the purposes, technical solutions and advantages of the present application more clear and apparent, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, but not to limit the present application. The present application can be implemented without some of these specific details by those skilled in the art. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0056] It is to be noted that, in this document, the terms such as first and second, etc. are used merely to distinguish one entity or action from another, and are not necessarily required by the disclosure. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, so that a process, method, article, or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, the elements defined by the statement "comprise" do not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the elements.

[0057] The virtual-real fusion technology is mainly applied in a virtual-real combined simulation scene, and the purpose is to match the virtual object and the real object in the spatial coordinates. Through the use of the technology, high-precision and high-efficiency matching positioning of the virtual object and the real object can be realized, so as to assist users to carry out more accurate virtual simulation activities and more efficient and accurate scientific research activities.

[0058] As described in the background section, with the development of science and technology, the application of virtual-real fusion technology is becoming more and more extensive. The development of virtual-real fusion technology benefits from the support of artificial intelligence, big data, the Internet of Things, mobile communication, edge computing, and extended reality, etc. These technologies help break the boundaries between virtual and reality, and promote the development of mixed experience from reality perception to virtual-real combination. In recent years, the application of virtual-real fusion technology is becoming more and more extensive, and virtual-real fusion technology has been applied in many scenes and industries, including virtual reality (VR) / augmented reality (AR) industrial empowerment, VR / AR immersive tourism experience, VR / AR public fitness, VR / AR online broadcasting, and VR / AR smart business district, etc. These application scenarios show the broad application potential of virtual-real fusion technology. Researchers can now use VR / AR devices to realize the mixing of virtual and real scenes. Researchers can use computer technology to create a three-dimensional virtual environment and update the scene with data-driven changes, and then compare with the real physical state to achieve the purpose of guidance, training, and design. Virtual-real fusion technology aims to achieve seamless integration between people, real environment, and virtual environment, and provides a new technical approach for high-end equipment development, complex task planning and training, innovative digital entertainment and education, etc.

[0059] In the application process of virtual-real fusion technology, although a virtual object that is almost completely consistent with a real object can be obtained, precise position matching and posture matching of the virtual object and the real object according to requirements is a complex process.

[0060] From the subordination of position fitting, it is generally divided into two ways, which are real object fitting to virtual object and virtual object fitting to real object. For the type of real object which is more portable and convenient to move, the way of real object fitting to virtual object is more convenient, but when such components interact with users, the displacement of components often occurs due to the convenient moving feature, thereby causing the fitting degree to be reduced.

[0061] Virtual object fitting to real object can avoid the above problems. However, the common fitting methods also have problems. The common methods include automatic matching and manual matching. Automatic matching refers to completing the spatial positioning of the virtual object through image recognition or multi-view positioning technology, but this often brings high technical cost and has the problem of insufficient matching accuracy or stability.

[0062] Manual matching refers to adjusting the position of the virtual object or the real object through manual interaction, but it also has the problems of low efficiency and poor stability. For example, manual matching can be to display the real object, the virtual object and a plurality of interactive controls for adjusting the position and attitude of the virtual object, and then receive multiple and repeated operations of the user on the plurality of interactive controls to transform the position and attitude of the virtual object, so that the virtual object and the real object completely coincide. However, the above manual interaction method is complicated to operate, resulting in low efficiency and poor stability of virtual-real fitting positioning. The poor stability can be that the virtual and real seamless docking cannot be continuously and stably maintained. The poor stability can be caused by the movement or change of the user's perspective. Due to the movement or change of the user's perspective and other reasons, the finally adjusted virtual object and real object may not completely coincide, resulting in low accuracy of virtual-real fitting positioning.

[0063] Based on this, the inventive concept of the present application is as follows:

[0064] Assuming that the spatial position of the real object is fixed, if the spatial position of the virtual object is to be fitted to the real object, the spatial position data of the real object often needs to be obtained and applied to the virtual object. The spatial position data can include six sets of data in the spatial rectangular coordinate system, i.e. the coordinate values of the position on the three axes of the spatial rectangular coordinate system, and the rotation angles of the attitude on the three axes. In the case where the shape contour of the real object is unchanged and the spatial position is fixed, if the position coordinates of a point (such as a first identification point) in the real object and the spatial position vector of a line (such as a first identification line) with the point as an end point are obtained, the spatial position of the real object can be determined, so there is no need to obtain six sets of data, and only two sets of data can realize virtual-real fitting positioning.

[0065] Therefore, the virtual-real fitting positioning can be realized based on the first identification point and the first identification line without the user performing multiple and repeated operations on the multiple interactive controls, so that the user operation is simplified and the virtual-real fitting positioning efficiency is improved. In addition, since the first identification point and the first identification line are structures that actually exist in the real object, the virtual-real fitting positioning based on the first identification point and the first identification line instead of the virtual-real fitting positioning based on the user control on the interactive controls can avoid errors caused by the movement or the change of the visual angle of the user, and ensure the accuracy of the virtual-real fitting positioning.

[0066] Therefore, in order to solve the problems in the prior art, the embodiment of the present application provides a virtual-real fitting positioning method, device, equipment, computer readable storage medium and computer program product. The virtual-real fitting positioning method can be applied to a mixed reality (MR) scene.

[0067] As can be known from the above description, the virtual-real fitting positioning method in the embodiment of the present application is an interactive spatial positioning method. The interactive mode can be manual interaction.

[0068] The virtual-real fitting positioning method provided by the embodiment of the present application is described below.

[0069] Figure 1 A flowchart of a virtual-real fitting positioning method provided by the embodiment of the present application is shown. The virtual-real fitting positioning method can be executed by an MR device. The MR device can include a head-mounted MR device, a glasses-mounted MR device, a tethered MR device, a flip-down MR device, etc.

[0070] As shown in Figure 1 The virtual-real fitting positioning method provided by the embodiment of the present application includes steps S110 to S130.

[0071] S110, a motion trajectory of a target gesture moving on a real object is acquired, the real object includes a first identification point and a first identification line with the first identification point as an end point, the first identification point is a point on an edge line of the real object, the edge line includes the first identification line, and the motion trajectory includes a motion trajectory from the first identification point to the first identification line.

[0072] The real object can be a physical object existing in a real scene. The real object can include at least one edge line. The real object includes a polyhedron composed of multiple edge lines. The real object can be, for example, a mobile phone, a computer, a table, a chair, a cabinet, an operation table, etc. A schematic diagram of a real object provided by the embodiment of the present application can be as shown in Figure 2

[0073] ​Specifically, the real object can include a first identification point and a first identification line with the first identification point as an end point. The first identification point can be a point on an edge line of the real object. The first identification point can be an end point of the edge line or an arbitrary point in the middle of the edge line, which is not limited herein.

[0074] In the case that the real object is a polyhedron, the first identification point can be a vertex of the polyhedron. If the side of the polyhedron facing the user is determined as the outer side of the polyhedron, and the side of the polyhedron away from the user is determined as the inner side of the polyhedron, the vertex can include a vertex protruding outward and a vertex protruding inward. Since the user can move on the real object based on the first identification point through the target gesture in the subsequent process of virtual-real fitting positioning, in some embodiments, in order to avoid the operation failure caused by the collision between the user's hand and the polyhedron, the first identification point can be determined as the vertex of the polyhedron protruding outward. The vertex of the polyhedron protruding outward is usually not obviously occluded around. Therefore, by determining the first identification point as the vertex of the polyhedron protruding outward, the operation failure caused by the collision between the user's hand and the polyhedron can be avoided, the smoothness and effectiveness of the movement trajectory are ensured, and thus the accuracy of the virtual-real fitting positioning is ensured.

[0075] In addition, one end point of the first identification line can be the first identification point. Since the first identification point is located on the edge line, the edge line can include the first identification line. That is, the first identification line can be a part of the edge line or the entire edge line, which is not limited herein. Since the user can move on the real object based on the first identification line through the target gesture, in some embodiments, in order to facilitate the target gesture movement and ensure the smoothness and effectiveness of the movement trajectory, in the case that the real object is a polyhedron, the first identification line can be a complete edge line in the polyhedron with the first identification point as one end point. By determining the first identification line as the edge line with the first identification point as an end point, both end points of the first identification point and the direction of the line can be determined, thereby facilitating the target gesture movement, ensuring the smoothness and effectiveness of the movement trajectory, and thus ensuring the accuracy of the virtual-real fitting positioning.

[0076] The schematic diagram of the real object provided by the embodiments of the present application, which includes the first identification point and the first identification line, can be as shown in Figure 3

[0077] In addition, the target gesture can be a gesture that can be recognized to the movement trajectory. The target gesture can have the characteristics of being easy to implement and can be maintained for a long time. The target gesture can be any one of the gestures such as pinching of the index finger and the thumb, extending the index finger, extending the palm, etc.

[0078] ​As an example, after a user wears the MR device, a real object in a real scene and the user's hands can be observed through the MR device. The user can also clearly observe the first identification point and the first identification line on the real object through the MR device. Based on this, if the target gesture is that the user pinches the index finger and the thumb, the user can pinch the index finger and the thumb at the first identification point of the real object, keep the pinching state of the fingers unchanged, and slide the hand along the first identification line to obtain a motion trajectory.

[0079] A schematic diagram of a user interaction operation provided by an embodiment of the present application can be as shown in Figure 4

[0080] Based on this, in order to further simplify the interaction process and improve the virtual-real fitting positioning efficiency, in some embodiments, the real object can be placed horizontally, and the first identification line can be parallel to the horizontal line.

[0081] By adding a horizontal state constraint to the virtual-real fitting positioning process, only a set of first spatial position vectors need to be given when performing pose fitting, thereby simplifying the interaction process. In addition, if the first identification line is parallel to the horizontal line, the user gesture can be moved horizontally, further simplifying the interaction process and improving the virtual-real fitting positioning efficiency.

[0082] Generally, the MR device will have a built-in gesture recognition function to support the user to interact with the real object and the virtual object through both hands.

[0083] Based on this, in order to ensure the accuracy of the motion trajectory and thus ensure the accuracy of the subsequent virtual-real fitting positioning, in some embodiments, S110 can specifically include:

[0084] Recognize the user gesture;

[0085] In the case where the user gesture is the target gesture, obtain a motion trajectory of the target gesture moving on the real object.

[0086] Here, the user gesture can include hand clapping, hand waving, pinching the index finger and the thumb, extending the index finger, extending the palm, etc. In the case where the user wears the MR device and the MR device is running normally, the user gesture can be recognized through the built-in gesture recognition function, and in the case where the user gesture is recognized as the target gesture, the motion trajectory of the target gesture moving on the real object is automatically triggered. The motion trajectory of the target gesture can be the motion trajectory of the entire hand corresponding to the target gesture, or the motion trajectory of a certain hand position corresponding to the target gesture, which is not limited here.

[0087] ​The embodiment of the present application can ensure the accuracy of the motion track and the precision of subsequent virtual-real fitting positioning by acquiring the motion track of the target gesture on the real object when the user gesture is determined as the target gesture.

[0088] Therefore, in some embodiments, the acquiring of the motion track of the target gesture on the real object when the user gesture is the target gesture can specifically include:

[0089] determining a target position of the target gesture when the user gesture is the target gesture;

[0090] tracking the position of the target position to obtain the motion track of the target gesture on the real object.

[0091] Here, the target position can be a hand position used to acquire the motion track. The target positions of different target gestures can be different. For example, if the target gesture is pinching of the index finger and the thumb, the target position can be the position of the pinch of the index finger and the thumb. If the target gesture is extending the index finger, the target position can be the position of the tip of the index finger. If the target gesture is extending the palm, the target position can be the position of the palm or the position of the tip of the index finger.

[0092] As an example, when the MR device identifies that the user gesture is the target gesture, the target position can be found in the target gesture based on the built-in gesture recognition function, and the position of the target position can be tracked when the user keeps the target gesture unchanged and moves the target gesture to obtain the motion track of the target gesture on the real object. That is, the motion track of the target gesture on the real object can be the motion track of the target position on the real object.

[0093] The embodiment of the present application can further improve the accuracy of the motion track by tracking the position of the target position (i.e., a certain point) to determine the motion track, compared with tracking the position of the entire hand to determine the motion track.

[0094] In addition, in order to ensure the accuracy of the target position, in some embodiments, the determining of the target position of the target gesture when the user gesture is the target gesture can specifically include:

[0095] acquiring a target gesture image when the user gesture is the target gesture;

[0096] recognizing the target gesture image to obtain the target position of the target gesture.

[0097] Here, the target gesture image can be a two-dimensional image or a three-dimensional image. The MR device can take a photo of the target gesture to obtain a two-dimensional target gesture image or perform three-dimensional reconstruction on the target gesture to obtain a three-dimensional target gesture image when the user gesture is identified as the target gesture. In addition, the MR device can include an image recognition module. The image recognition module can include a target neural network model implemented based on an image recognition algorithm. The target gesture image can be recognized by the target neural network model to obtain the target position of the target gesture. The target neural network model can be trained using gesture image samples and corresponding target position samples.

[0098] In this way, by first obtaining the target gesture image of the target gesture and then recognizing the target gesture image to obtain the target position of the target gesture, the accuracy of the target position can be ensured.

[0099] In S120, the spatial position data of the real object is determined according to the motion trajectory.

[0100] After obtaining the accurate motion trajectory, the spatial position data of the real object can be determined according to the motion trajectory to locate the real object. The spatial position data can include first position data of the first identification point and a first spatial position vector corresponding to the first identification line. The first position data can be the position coordinates of the first identification point in the world coordinate system. The first spatial position vector can be a spatial position vector determined in the world coordinate system. It should be noted that in the embodiments of the present application, the first identification line can only have a size without a direction, but the first spatial position vector corresponding to the first identification line can include a size and a direction.

[0101] In addition, the mainstream MR device currently has a stable spatial positioning function. The built-in processing system of the MR device can effectively identify environmental features, thereby establishing a stable spatial rectangular coordinate system, providing a basis for the data of the first identification point and the first identification line. The spatial rectangular coordinate systems established by different MR devices can be different, resulting in slight differences in the spatial position data of the real object, but it does not affect the virtual-real fitting positioning effect in the embodiments of the present application. The reason can be seen in the description of the virtual object below.

[0102] It should be noted that, whether it is a spatial positioning function or a gesture recognition function, the developer of the MR device only provides an API interface that can be used for development, and does not open source all the function codes, but this does not affect the use of the related black box function as a component of the method involved in the invention.

[0103] Based on this, in order to ensure the accuracy of the spatial position data, in some embodiments, S120 can specifically include:

[0104] acquire first position data of the target position;

[0105] determine the first position data of the target position as first position data of the first identification point;

[0106] determine a first spatial position vector corresponding to the first identification line according to the first position data and the motion trajectory.

[0107] Here, since the gesture recognition is to automatically fit the virtual gesture with the real gesture by the image recognition, the spatial coordinates of the first identification point can be obtained through the position of the virtual gesture in the spatial coordinate system. That is, after acquiring the first position data of the target position, the first position data of the target position can be determined as the first position data of the first identification point.

[0108] In addition, the motion trajectory can include the starting point, the ending point and the motion direction of the motion. Since the first identification point is the end point of the first identification line, after determining the first position data, the spatial coordinates of the first end point of the first identification line (i.e. the first position data) can be determined. During the motion of the target gesture along the first identification line, the motion direction of the target gesture can be acquired in real time. In the case of the end of the motion, the spatial coordinates of the second end point of the first identification line can be acquired. In this way, according to the starting point coordinates of the motion (i.e. the first position data) and the ending point coordinates of the motion (i.e. the spatial coordinates of the second end point of the first identification line), the spatial coordinates of the motion trajectory can be determined, and the spatial coordinates of the motion trajectory are determined as the first spatial position vector corresponding to the first identification line.

[0109] Of course, since the first spatial position vector can be determined according to the starting point coordinates of the motion and the ending point coordinates of the motion, during the motion of the target gesture along the first identification line, the motion direction of the target gesture can also not be acquired in real time, so as to save the computing resources of the MR device.

[0110] As an example, after a user wears an MR device, a real object in a real scene and the user's own hands can be observed through the MR device. The user can also clearly observe a first identification point and a first identification line on the real object through the MR device. Based on this, if the target gesture is that the user pinches the index finger and the thumb, the user can pinch the index finger and the thumb at the first identification point of the real object. In a case where the MR device detects that the user pinches the index finger and the thumb, the spatial coordinates of the pinched position of the index finger and the thumb can be obtained, and first position data of the first identification point is obtained. In a process in which the user keeps the pinched state of the fingers unchanged and slides the hand along the first identification line, the sliding direction of the hand can be obtained in real time. In a case where the duration of the hand stopping sliding is greater than a preset duration, it can be determined that the hand movement is ended, and the position where the hand movement is ended is determined as a second end point of the first identification line, the spatial coordinates of the second end point are obtained, and then the first spatial position vector can be determined according to the first position data and the spatial coordinates of the second end point of the first identification line. The preset duration may, for example, be 0.5 s, 1 s, etc.

[0111] It should be noted that, since the first identification line is all or part of the edge line of the real object, when the hand keeps the pinched posture and slides on the edge line, the straight structure of the edge line can give the hand stable path dependence, thereby calibrating an accurate first spatial position vector.

[0112] Embodiments of the present application can ensure the accuracy of the spatial position data by determining the first position data of the target position as the first position data of the first identification point, and determining the first spatial position vector corresponding to the first identification line according to the first position data and the motion trajectory.

[0113] After the spatial position data of the real object is obtained, the spatial position data can be temporarily stored in the MR device.

[0114] S130, according to the spatial position data, transforming the position and posture of a virtual object displayed in a virtual scene of the mixed reality device, the shape outlines of the virtual object and the real object satisfying a preset similarity condition.

[0115] Here, the virtual object can be an object ready to be virtually fitted and positioned with the real object. The shape outlines of the virtual object and the real object can be substantially the same. That is, the shape outlines of the virtual object and the real object can satisfy the preset similarity condition. Wherein, the shape outlines of the virtual object and the real object satisfying the preset similarity condition can be that the difference rate between the shape outline information of the virtual object and the shape outline information of the real object is less than a preset threshold.

[0116] As an example, when a user wears an MR device, on one hand, a real object in a real scene can be observed through the MR device; on the other hand, a virtual object corresponding to the real object can be observed from a virtual scene of the MR device.

[0117] A schematic diagram of a display screen of an MR device provided by an embodiment of the present application can be as shown in Figure 5 As shown in Figure 5 The left side can be a real object, and the right side can be a virtual object that meets a preset similarity condition with a shape contour of the real object.

[0118] As an example, after the spatial position data of the real object is acquired, the accurate position of the real object can be determined according to the control position data, and then the virtual object can be transformed to the position of the real object and coincide with the real object by transforming the position and posture of the virtual object according to the spatial position data, so as to complete the real-virtual fitting positioning process and realize the spatial position fitting of the real object and the virtual object.

[0119] Therefore, in order to obtain a virtual object that meets a preset similarity condition with a shape contour of a real object and ensure the real-virtual fitting positioning effect, in some embodiments, before S130, the method further includes:

[0120] scanning the shape contour of the real object to obtain shape contour information;

[0121] performing three-dimensional reconstruction on the shape contour information to obtain the virtual object.

[0122] By scanning the shape contour of the real object in a laser scanning manner, an embodiment of the present application can obtain shape contour information of the real object, and then perform three-dimensional reconstruction on the shape contour information to obtain a virtual object that meets a preset similarity condition with a shape contour of the real object, thereby ensuring the real-virtual fitting positioning effect.

[0123] For example, a virtual object corresponding to a real object can be made by using a three-dimensional making tool such as 3Ds Max, Blender, Maya, Rhino, and the like.

[0124] After the first mark point and the first mark line are determined on the real object, a mark processing can be performed on a corresponding position of the virtual object to obtain a second mark point and a second mark line. That is, the virtual object can include the second mark point and the second mark line. The position of the second mark point in the virtual object corresponds to the position of the first mark point in the real object, and the position of the second mark line in the virtual object corresponds to the position of the first mark line in the real object.

[0125] It should be noted that the virtual object and the real object can be in the same space rectangular coordinate system. Therefore, the relative spatial position data between the virtual object and the real object is unchanged regardless of the establishment of the space rectangular coordinate system, that is, regardless of the position coordinates of the first identification point, the position coordinates of the second identification point, the spatial position vector of the first identification line, and the spatial position vector of the second identification line.

[0126] On this basis, in order to further improve the virtual-real fitting positioning efficiency, in some embodiments, the above S120 can specifically include:

[0127] establishing a target space rectangular coordinate system with the second identification point as the origin, the second identification line as the first axis, and two lines perpendicular to the second identification line and perpendicular to each other as the second axis and the third axis;

[0128] According to the motion trajectory, the spatial position data of the real object in the target space rectangular coordinate system is determined.

[0129] Based on this, the above S130 can specifically include:

[0130] According to the target space rectangular coordinate system, the second position data of the second identification point and the second spatial position vector corresponding to the second identification line are determined;

[0131] According to the first position data and the second position data, the second identification point in the virtual object is moved to the position where the first identification point is located;

[0132] According to the first spatial position vector and the second spatial position vector, the second identification line in the virtual object is transformed to the attitude where the first identification line is located.

[0133] Here, the first axis can be the x-axis, the y-axis, or the z-axis, which is not limited here. In the case where the first axis is the x-axis, the second axis and the third axis can be the y-axis and the z-axis, respectively. In addition, in the case where the first axis is the x-axis, the positive direction of the x-axis can be the direction from the second identification point to the second identification line. Since the second identification point corresponds to the first identification point, and the second identification line corresponds to the first identification line, the positive direction of the x-axis can be the same as the direction of the first spatial position vector.

[0134] The MR device can establish a target space rectangular coordinate system with the second identification point as the origin, the second identification line as the first axis, and two lines perpendicular to the second identification line and perpendicular to each other as the second axis and the third axis according to its own space recognition function.

[0135] Based on this, in order to further improve the virtual-real fitting positioning efficiency, in some embodiments, the virtual object can be constrained to be placed vertically. At this time, the second identification line can be a horizontal line. In the case where the virtual object is constrained to be placed vertically, the virtual object provides a plumb line that can be used as a coordinate axis, that is, the vertical direction perpendicular to the horizontal plane can be determined as a coordinate axis. Thus, in the case where the coordinate origin and two coordinate axes are known in the three-dimensional coordinate system, the third axis can be determined.

[0136] A schematic diagram of a target space rectangular coordinate system provided by an embodiment of the present application can be as shown in Figure 6 .

[0137] After determining the target space rectangular coordinate system, on the one hand, the spatial position data of the real object can be determined in the target space rectangular coordinate system according to the motion trajectory; on the other hand, the second position data of the second identification point can be determined as (0, 0, 0), and if the first axis is the x-axis, the second spatial position vector corresponding to the second identification line can be (x, 0, 0).

[0138] By determining the second position data as (0, 0, 0) and the second spatial position vector as (x, 0, 0), the embodiment of the present application can improve the efficiency of determining the first difference between the first position data and the second position data and the second difference between the first spatial position vector and the second spatial position vector. In this way, by moving the second identification point in the virtual object to the position where the first identification point is located according to the first position data and the second position data, and transforming the second identification line in the virtual object to the attitude where the first identification line is located according to the first spatial position vector and the second spatial position vector, the virtual-real fitting positioning efficiency can be further improved.

[0139] The software can complete the virtual-real fitting positioning by implementing the above logic through code.

[0140] Based on the above embodiments, a specific example is given.

[0141] In the implementation of the virtual-real combined flight simulation cabin, the above method can be used to complete the complete fusion of the digital twin simulation cabin and the physical simulation cabin, thereby helping the test personnel or users to complete the work.

[0142] A schematic diagram of a virtual-real fitting positioning process provided by an embodiment of the present application can be as shown in Figure 7 . As Figure 7 indicated, the virtual-real fitting positioning process can include the virtual three-dimensional model of the simulation cockpit and the interactive action in the virtual-real fitting positioning process.

[0143] A schematic diagram of a virtual-real fitting positioning result provided by an embodiment of the present application can be as shown in Figure 8 . In Figure 7In this way, the virtual three-dimensional model is above the seat at this time, which is consistent with the relative position relationship between the user and the flight simulation cockpit in the real scene.

[0144] In the virtual-real fitting positioning method in the embodiments of the present application, if a first identification point of the real object and a first identification line with the first identification point as an end point are determined, the position and the pose of the real object can be determined, and then in the case that the shape outlines of the virtual object and the real object meet the preset similarity condition, the position and the pose to be transformed of the virtual object can be determined through the first identification point and the first identification line. In this way, by acquiring the motion track of the target gesture moving from the first identification point along the first identification line on the real object, the spatial position data of the real object is determined according to the motion track, and the position and the pose of the virtual object displayed in the virtual scene of the mixed reality device are transformed according to the spatial position data, that is, the virtual-real fitting positioning can be realized through the motion track of the target gesture based on the first identification point and the first identification line, without the need for the user to perform multiple and repeated operations on multiple interactive controls, so as to simplify the user operation and improve the virtual-real fitting positioning efficiency.

[0145] In addition, the virtual-real fitting positioning based on the first identification point and the first identification line instead of the virtual-real fitting positioning based on the control of the interactive controls by the user can avoid the error caused by the movement or the change of the visual angle of the user, and ensure the accuracy of the virtual-real fitting positioning.

[0146] In addition, the virtual-real fitting positioning can be realized by only using the original function of the MR device itself in combination with the virtual-real fitting positioning method, without additional investment, so that the implementation cost of the virtual-real fitting positioning is relatively low.

[0147] To sum up, the embodiments of the present application use the structural features of the real object and the flexible and stable interactive features of the hand, and innovatively propose an interactive virtual-real fitting positioning method. As known from the foregoing description, the virtual-real fitting positioning method has the characteristics of high execution efficiency, low production cost, easy operation and high precision, and is an implementation manner of the virtual-real fitting positioning function with low cost, good effect and simple operation.

[0148] Based on the virtual-real fitting positioning method provided in the above embodiments, correspondingly, the present application also provides a specific implementation manner of a virtual-real fitting positioning device. The virtual-real fitting positioning device can be applied to a mixed reality device. For details, please refer to the following embodiments.

[0149] As shown in Figure 9 The virtual-real fitting positioning device 900 provided by the embodiments of the present application includes the following modules:

[0150] The acquisition module 910 is configured to acquire a motion track of a target gesture moving on a real object, the real object including a first mark point and a first mark line with the first mark point as an end point, the first mark point being a point on a ridge line of the real object, the ridge line including the first mark line, and the motion track including a motion track of the target gesture moving along the first mark line from the first mark point.

[0151] The determination module 920 is configured to determine spatial position data of the real object according to the motion track.

[0152] The transformation module 930 is configured to transform a position and a posture of a virtual object displayed in a virtual scene of the mixed reality device according to the spatial position data, the virtual object and the real object satisfying a preset similarity condition in shape contour.

[0153] The virtual-real fitting positioning apparatus 900 is described below in detail as follows.

[0154] In some embodiments, the acquisition module 910 can specifically include:

[0155] The identification sub-module is configured to identify the user gesture.

[0156] The first acquisition sub-module is configured to acquire the motion track of the target gesture moving on the real object in a case where the user gesture is the target gesture.

[0157] In some embodiments, the first acquisition sub-module can specifically include:

[0158] The determination unit is configured to determine a target position of the target gesture in a case where the user gesture is the target gesture.

[0159] The tracking unit is configured to track a position of the target position to obtain the motion track of the target gesture moving on the real object.

[0160] In some embodiments, the determination unit can specifically include:

[0161] The acquisition sub-unit is configured to acquire a target gesture image of the target gesture in a case where the user gesture is the target gesture.

[0162] The identification sub-unit is configured to identify the target gesture image to obtain the target position of the target gesture.

[0163] In some embodiments, the spatial position data includes first position data of the first mark point and a first spatial position vector corresponding to the first mark line. Based on this, the determination module 920 can specifically include:

[0164] The second acquisition sub-module is configured to acquire the first position data of the target position.

[0165] The first determining sub-module is configured to determine first position data of the target position as first position data of the first identification point.

[0166] The second determining sub-module is configured to determine a first spatial position vector corresponding to the first identification line according to the first position data and the motion trajectory.

[0167] In some embodiments, the virtual object includes a second identification point and a second identification line, a position of the second identification point in the virtual object corresponds to a position of the first identification point in the real object, and a position of the second identification line in the virtual object corresponds to a position of the first identification line in the real object.

[0168] Based on this, the determining module 920 can specifically include:

[0169] The constructing sub-module is configured to construct a target spatial rectangular coordinate system with the second identification point as an origin, the second identification line as a first axis, and two lines perpendicular to the second identification line and to each other as a second axis and a third axis.

[0170] The third determining sub-module is configured to determine spatial position data of the real object in the target spatial rectangular coordinate system according to the motion trajectory.

[0171] Based on this, the transforming module 930 can specifically include:

[0172] The fourth determining sub-module is configured to determine second position data of the second identification point and a second spatial position vector corresponding to the second identification line according to the target spatial rectangular coordinate system.

[0173] The moving sub-module is configured to move the second identification point in the virtual object to a position where the first identification point is located according to the first position data and the second position data.

[0174] The transforming sub-module is configured to transform the second identification line in the virtual object to a posture where the first identification line is located according to the first spatial position vector and the second spatial position vector.

[0175] In some embodiments, the real object includes at least a polyhedron composed of a plurality of edge lines, the first identification point is a vertex of the polyhedron, and the first identification line is an edge line with the first identification point as an end point.

[0176] In some embodiments, the real object is horizontally placed, and the first identification line is parallel to a horizontal line.

[0177] In some embodiments, the virtual-real fitting positioning device 900 can further include:

[0178] The scanning module is configured to scan a shape contour of the real object to obtain shape contour information before transforming a position and a pose of a virtual object displayed in a virtual scene of the mixed reality device according to the spatial position data.

[0179] The constructing module is configured to perform three-dimensional reconstruction on the shape contour information to obtain the virtual object.

[0180] In the virtual-real fitting positioning apparatus provided in the embodiments of the present application, if a first identification point of the real object and a first identification line with the first identification point as an end point are determined, the position and the pose of the real object can be determined, and then in the case that the shape contour of the virtual object and the shape contour of the real object satisfy a preset similarity condition, the position and the pose to be transformed of the virtual object can be determined through the first identification point and the first identification line. In this way, by obtaining a movement track of the target gesture moving from the first identification point along the first identification line on the real object, determining the spatial position data of the real object according to the movement track, and transforming the position and the pose of the virtual object displayed in the virtual scene of the mixed reality device according to the spatial position data, the virtual-real fitting positioning can be realized through the movement track of the target gesture based on the first identification point and the first identification line, without the need of the user to perform multiple and repeated operations on multiple interactive controls, so that the user operation can be simplified and the virtual-real fitting positioning efficiency can be improved.

[0181] Based on the virtual-real fitting positioning method provided in the above embodiments, the embodiments of the present application further provide a specific implementation of an electronic device. Figure 10 A schematic diagram of an electronic device 1000 provided in the embodiments of the present application is shown.

[0182] The electronic device 1000 can include a processor 1010 and a memory 1020 storing computer program instructions.

[0183] Specifically, the processor 1010 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or can be configured as one or more integrated circuits implementing the embodiments of the present application.

[0184] The memory 1020 can include a mass storage, for data or instructions. For example, without limitation, the memory 1020 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. The memory 1020 can be removable and / or non-removable (or fixed) as appropriate. The memory 1020 can be internal or external as appropriate. In certain embodiments, the memory 1020 is a non-volatile solid-state memory.

[0185] The memory can include read-only memory (ROM), random-access memory (RAM), magnetic disk storage mediums, optical storage mediums, flash memory devices, electrical, optical, or other physically tangible / memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software that, when executed (e.g., by the one or more processors), is operable to perform operations described with reference to the method according to the first aspect of the present application.

[0186] The processor 1010 implements any one of the virtual-real positioning methods in the above-described embodiments by reading and executing computer program instructions stored in the memory 1020.

[0187] In one example, the electronic device 1000 can further include a communication interface 1030 and a bus 1040. As shown, the processor 1010, the memory 1020, and the communication interface 1030 are connected through the bus 1040 and complete communication therebetween. Figure 10

[0188] The communication interface 1030 is mainly used to realize the communication between various modules, devices, units, and / or equipment in the embodiments of the present application.

[0189] ​Bus 1040 includes hardware, software, or both, to couple electronic devices to each other in a manner that allows information to be passed between or among the coupled devices. By way of example, and not limitation, a bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 1040 can include one or more buses. Although the example embodiments described and illustrated herein relate to a particular bus, the application contemplates any suitable bus or interconnect.

[0190] By way of example, electronic device 1000 can be a cellular telephone, a tablet computer, a laptop computer, a handheld computer, a vehicle-mounted electronic device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), among other examples.

[0191] The electronic device can perform the virtual-real fitting positioning method in the embodiments of the application, thereby realizing the virtual-real fitting positioning method and apparatus described in the embodiments of the application. Figure 1 and Figure 9 the virtual-real fitting positioning method and apparatus described in the embodiments of the application.

[0192] In addition, in combination with the virtual-real fitting positioning method in the embodiments described above, the embodiments of the application can provide a computer-readable storage medium to implement. The computer-readable storage medium has computer program instructions stored thereon; the computer program instructions are executed by a processor to implement any one of the virtual-real fitting positioning methods in the embodiments described above.

[0193] In combination with the virtual-real fitting positioning method in the embodiments described above, the embodiments of the application can provide a computer program product to implement. The instructions in the computer program product are executed by the processor of the electronic device to implement any one of the virtual-real fitting positioning methods in the embodiments described above.

[0194] It needs to be made clear that the application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted here. In the embodiments described above, several specific steps are described and shown as examples. However, the method processes of the application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the application.

[0195] The functions noted in the block diagrams of the above-described structures can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, functional cards, and the like. When implemented in software, the elements of the present application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. The "machine-readable medium" can include any medium that can store or transfer information. Examples of the machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, and the like. The code segments can be downloaded via a computer network such as the Internet, an intranet, and the like.

[0196] It is also noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in an order different from the embodiments, or several steps can be performed simultaneously.

[0197] The above-described aspects of the present application can be described with reference to flow charts and / or block diagrams illustrating the architecture, functionality, and operation of implementations of methods, apparatus (systems) and computer program products according to the present application. It will be understood that each block of the flow chart and / or block diagrams, and combinations of blocks in the flow chart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / acts specified in the flow chart and / or block diagram block or blocks. Such processors can be, but not limited to, general purpose processors, special purpose processors, special purpose application specific processors, or field programmable logic arrays (FPLAs). It should also be understood that each block of the flow chart and / or block diagrams and combinations of blocks in the flow chart and / or block diagrams can be implemented by special purpose hardware-based computer systems which perform some or all of the steps depending on the circumstances. Or, combinations of special purpose hardware and computer instructions can be utilized.

[0198] The above merely describes a specific implementation of the present application. Those skilled in the art can clearly understand the specific working processes of the system, modules and units described above for the convenience and brevity of description, and can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein again. It should be understood that the protection scope of the present application is not limited to this, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A virtual-real fitting positioning method, characterized in that: Applied to a mixed reality device, the method includes: Obtaining a motion trajectory of a target gesture on a real object, where the real object includes a first identification point and a first identification line with the first identification point as an endpoint, the first identification point being a point on an edge line of the real object, the edge line including the first identification line, and the motion trajectory including a motion trajectory starting from the first identification point and moving along the first identification line; Determining spatial position data of the real object according to the motion trajectory; According to the spatial position data, the position and posture of the virtual object displayed in the virtual scene of the mixed reality device are transformed, and the shape contours of the virtual object and the real object meet a preset similarity condition.

2. The method according to claim 1, characterized in that The obtaining of the motion trajectory of the target gesture on the real object includes: Recognize user gestures; In a case where the user gesture is a target gesture, a motion trajectory of the target gesture on the real object is obtained.

3. The method according to claim 2, characterized in that When the user gesture is a target gesture, obtaining a motion trajectory of the target gesture on the real object includes: In a case where the user gesture is a target gesture, determining a target position of the target gesture; The position of the target position is tracked to obtain a motion trajectory of the target gesture moving on the real object.

4. The method according to claim 3, characterized in that When the user gesture is a target gesture, determining a target position of the target gesture includes: When the user gesture is a target gesture, obtaining a target gesture image; The target gesture image is recognized to obtain a target position of the target gesture.

5. The method according to claim 3, characterized in that The spatial position data includes first position data of the first identification point and a first spatial position vector corresponding to the first identification line. Determining the spatial position data of the real object according to the motion trajectory includes: Acquiring first position data of the target position; Determining the first position data of the target position as the first position data of the first identification point; A first spatial position vector corresponding to the first identification line is determined according to the first position data and the motion trajectory.

6. The method according to claim 5, characterized in that The virtual object includes a second identification point and a second identification line, the position of the second identification point in the virtual object corresponds to the position of the first identification point in the real object, and the position of the second identification line in the virtual object corresponds to the position of the first identification line in the real object; Determining the spatial position data of the real object according to the motion trajectory includes: Constructing a target space rectangular coordinate system with the second marking point as the origin, the second marking line as the first axis, and two lines perpendicular to the second marking line and to each other as the second axis and the third axis; Determining spatial position data of the real object in the target space rectangular coordinate system according to the motion trajectory; The step of transforming the position and posture of a virtual object displayed in a virtual scene of the mixed reality device according to the spatial position data includes: Determine, according to the target space rectangular coordinate system, second position data of the second marking point and a second spatial position vector corresponding to the second marking line; Moving the second identification point in the virtual object to the position where the first identification point is located according to the first position data and the second position data; The second marking line in the virtual object is transformed to the posture of the first marking line according to the first spatial position vector and the second spatial position vector.

7. The method according to any one of claims 1 to 6, characterized in that The real object at least includes a polyhedron composed of multiple edge lines, the first identification point is a vertex of the polyhedron, and the first identification line is an edge line with the first identification point as an endpoint.

8. The method according to any one of claims 1 to 6, characterized in that The real object is placed horizontally, and the first identification line is parallel to the horizontal line.

9. The method according to any one of claims 1 to 6, characterized in that Before transforming the position and posture of the virtual object displayed in the virtual scene of the mixed reality device according to the spatial position data, the method further includes: Scanning the shape contour of the real object to obtain shape contour information; The shape contour information is three-dimensionally reconstructed to obtain the virtual object.

10. A virtual-real fitting positioning device, characterized in that: Applied to a mixed reality device, the apparatus comprises: an acquisition module, configured to acquire a motion trajectory of a target gesture on a real object, wherein the real object includes a first identification point and a first identification line with the first identification point as an endpoint, the first identification point being a point on an edge line of the real object, the edge line including the first identification line, and the motion trajectory including a motion trajectory starting from the first identification point and moving along the first identification line; a determination module, configured to determine spatial position data of the real object according to the motion trajectory; A transformation module is used to transform the position and posture of a virtual object displayed in a virtual scene of the mixed reality device according to the spatial position data, and the shape contours of the virtual object and the real object meet a preset similarity condition.

Images