Positioning processing method and device

Abstract

The invention discloses a positioning processing method and device, and the method comprises the steps: obtaining a first image of a first object through an image collection module, and the first image comprises the first object and a target object having a binding relation with the first object; determining a first spatial position of the first object based on the first image; determining a target spatial position of the target object based on physical appearance data and image appearance data of the target object and internal and external parameters of the image acquisition module, the physical appearance data being preset data, and the image appearance data being determined based on image analysis of the target object in the first image; and correcting the first spatial position of the first object based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position.

Description

Technical Field

[0001] The present application relates to virtual interaction technology, and more specifically, to a positioning processing method and device. Background Art

[0002] Hand-object interaction in scenes such as naked-eye 3D requires obtaining the accurate position of the operator's hand relative to the camera origin. Traditional solutions often use cameras to locate the user's hand position, which has certain positioning errors. Summary of the invention

[0003] In view of this, this application provides the following technical solutions:

[0004] A positioning processing method, the method comprising:

[0005] Obtaining a first image of a first object through an image acquisition module, wherein the first image includes the first object and a target object having a binding relationship with the first object;

[0006] determining a first spatial position of the first object based on the first image;

[0007] Determine the target spatial position of the target object based on physical appearance data, image appearance data of the target object, and internal and external parameters of the image acquisition module, wherein the physical appearance data is preset data and the image appearance data is determined based on image analysis of the target object in the first image;

[0008] The first spatial position of the first object is corrected based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position.

[0009] In an optional implementation, after obtaining the first image of the first object through the image acquisition module, the method further includes:

[0010] Verifying the target object includes:

[0011] Determine the position and posture information of the target object based on the first image;

[0012] Obtaining inertial measurement data of the target object;

[0013] If the posture information and the inertial measurement data match, a first spatial position of the first object is determined based on the first image, wherein the match indicates that an error between the posture represented by the inertial measurement data and the posture information is lower than a set value.

[0014] In an optional implementation, determining the target spatial position of the target object based on the physical appearance data, the image appearance data of the target object and the intrinsic and extrinsic parameters of the image acquisition module includes:

[0015] Determine first coordinate data of the target object in a spatial coordinate system corresponding to the image acquisition module based on the image appearance data of the target object in the first image and the internal parameters of the image acquisition module;

[0016] Determine second coordinate data of the target object in a world coordinate system based on the first coordinate data and an external parameter of the image acquisition module;

[0017] A target spatial position of the target object is determined based on the second coordinate data and the physical appearance data of the target object.

[0018] In an optional implementation, the correcting the first spatial position of the first object based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position includes:

[0019] Determine the to-be-compared spatial position of the first object based on the target spatial position and the binding position of the target object on the first object;

[0020] The first spatial position is corrected based on an error between the spatial position to be compared and the first spatial position.

[0021] In an optional implementation, the correcting the first spatial position based on the error between the spatial position to be compared and the first space includes:

[0022] If the error between the spatial position to be compared and the first spatial position is less than a first threshold, determining the spatial position to be compared as a first corrected position;

[0023] If the error between the spatial position to be compared and the first spatial position is greater than or equal to the first threshold and less than a second threshold, linear summing the spatial position to be compared and the first spatial position is performed, and the processing result is determined as a first corrected position;

[0024] If the error between the spatial position to be compared and the first spatial position is greater than or equal to the second threshold, the spatial position to be compared and the first spatial position are deleted.

[0025] In an optional implementation, the first image also includes a second object, and after determining the target spatial position, the method further includes:

[0026] determining a second spatial position of the second object based on the first image;

[0027] The second spatial position of the second object is corrected based on the image appearance data of the target object and the first corrected position to obtain a second corrected position.

[0028] In an optional implementation, the correcting the second spatial position of the second object based on the image appearance data of the target object and the first corrected position to obtain a second corrected position includes:

[0029] determining a rotation parameter and a translation parameter between the first object and the second object based on the first image;

[0030] Controlling the target object to perform a rotation and translation operation based on the rotation parameter and the translation parameter, so that the posture of the target object after the rotation and translation is the same as the posture of the second object and is located at the binding position of the second object;

[0031] Scaling the rotated and translated target object so that the scaled target object matches the second object, wherein the matching indicates that the target object and the second object present a binding relationship between the target object and the first object before the rotation and translation;

[0032] A second corrected position of the target object is obtained based on the scaling ratio of the scaling process, the first corrected position, and the second spatial position.

[0033] In an optional implementation, it also includes:

[0034] determining an appearance deviation coefficient between the first object and the second object based on the first image;

[0035] The obtaining the second corrected position of the target object based on the scaling ratio of the scaling process, the first corrected position and the second spatial position includes:

[0036] A second corrected position of the target object is obtained based on a scaling ratio of the scaling process, the first corrected position, the second spatial position, and the appearance deviation coefficient.

[0037] In an optional implementation, determining a rotation parameter and a translation parameter between the first object and the second object based on the first image includes:

[0038] determining a first pose of the first object and a second pose of the second object based on the first image;

[0039] determining a rotation parameter for transitioning from the first posture to the second posture;

[0040] A translation parameter from the first spatial position to the second spatial position is determined.

[0041] The present application also discloses a positioning processing device, comprising:

[0042] An image acquisition module 601 is configured to acquire a first image of a first object through an image acquisition module, wherein the first image includes the first object and a target object having a binding relationship with the first object;

[0043] A first position obtaining module 602, configured to determine a first spatial position of the first object based on the first image;

[0044] A target position determination module 603 is used to determine the target spatial position of the target object based on physical appearance data of the target object, image appearance data, and internal and external parameters of the image acquisition module, wherein the physical appearance data is preset data and the image appearance data is determined based on image analysis of the target object in the first image;

[0045] The first position correction module 604 is configured to correct the first spatial position of the first object based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position.

[0046] Furthermore, the present application also discloses a positioning processing device, comprising an image acquisition device and at least one processor, wherein:

[0047] The image acquisition device is used to acquire images;

[0048] The processor is connected to the image acquisition device and is configured to:

[0049] Obtaining a first image of a first object through an image acquisition module, wherein the first image includes the first object and a target object having a binding relationship with the first object;

[0050] determining a first spatial position of the first object based on the first image;

[0051] Determine the target spatial position of the target object based on physical appearance data, image appearance data of the target object, and internal and external parameters of the image acquisition module, wherein the physical appearance data is preset data and the image appearance data is determined based on image analysis of the target object in the first image;

[0052] The first spatial position of the first object is corrected based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0054] Figure 1 A flowchart of a positioning processing method disclosed in an embodiment of the present application;

[0055] Figure 2 This is an example diagram of a hand wearing a ring disclosed in an embodiment of the present application;

[0056] Figure 3 A flowchart for determining a target spatial position disclosed in an embodiment of the present application;

[0057] Figure 4 A flowchart of determining a second corrected position disclosed in an embodiment of the present application;

[0058] Figure 5 An exemplary diagram of a ring moving from an operating hand to a non-operating hand disclosed in an embodiment of the present application;

[0059] Figure 6 A schematic diagram of the structure of a positioning processing device disclosed in an embodiment of the present application;

[0060] Figure 7 This is a schematic diagram of the structure of another positioning processing device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] The embodiments of the present application can be applied to electronic devices. The present application does not limit the product form of the electronic device, which can include but is not limited to smart phones, tablet computers, wearable devices, netbooks, etc., and can be selected according to application requirements.

[0063] Figure 1 This is a flow chart of a positioning processing method disclosed in an embodiment of the present application. Figure 1 As shown, the positioning processing method may include:

[0064] Step 101: obtaining a first image of a first object through an image acquisition module, wherein the first image includes the first object and a target object having a binding relationship with the first object.

[0065] The positioning processing method disclosed in the embodiment of the present application can be applied to scenes such as virtual reality VR and naked-eye 3D, and the image acquisition module can be an image acquisition device built into the VR device or the naked-eye 3D device. The first object can be an object in space for interacting with the VR device, the naked-eye 3D device, etc., such as the user's hand, which can perform relevant operations based on the virtual image in the space to achieve virtual interaction.

[0066] The first image includes an image of the user's operating hand (first object), and of course may also include images of other hands and other objects within the viewing range, such as walls, tables, etc. It should be noted that in the present application, the user's operating hand also wears a ring for auxiliary positioning (target object), and the implementation of its auxiliary positioning will be described in detail in the embodiments below, and will not be described in detail here. The ring can be worn on any finger of the user's operating hand. Generally, since users are accustomed to using their index fingers for virtual interactive operations, in order to ensure the accuracy of the positioning results, the ring for auxiliary positioning can be worn on the index finger or on the middle finger that is closer to the index finger. There is a binding relationship between the first object and the target object, that is, the ring is worn at a fixed position on the user's finger. Figure 2 This is an example diagram of a hand wearing a ring disclosed in an embodiment of the present application.

[0067] Step 102: Determine a first spatial position of the first object based on the first image.

[0068] Based on the first image, a depth analysis can be performed, and an image depth algorithm can be used to determine the first spatial position of the first object. The implementation of using an image depth algorithm to determine the spatial position of an object in an image is an existing technology and will not be described in detail here. The accuracy of the first spatial position is relatively low, so the first spatial position can be corrected based on the relevant analysis and processing of the target object to obtain a more accurate spatial position of the first object, so as to meet the high-precision requirements for the position of the operating body in the virtual interaction scene.

[0069] Step 103: Determine the target spatial position of the target object based on the physical appearance data, image appearance data of the target object, and the internal and external parameters of the image acquisition module, wherein the physical appearance data is preset data and the image appearance data is determined based on image analysis of the target object in the first image.

[0070] The physical appearance data of the target object is known, that is, the actual diameter, width, thickness, etc. of the ring are known, and can be directly read from the storage address when needed. The image appearance data can be determined by analyzing the image of the target object in the first image, that is, the width, thickness, diameter or radius, etc. of the ring in the image.

[0071] In implementation, the position of the target object in space can be determined based on the image appearance data of the target object and the parameters of the image acquisition module for obtaining the first image, and then the exact spatial position of the target object, that is, the target spatial position, can be determined in combination with the position of the target object in space and its actual physical appearance data.

[0072] Step 104: Correct the first spatial position of the first object based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position.

[0073] After the accurate target space position of the target object is determined, since the target object has The binding relationship is that the ring is worn on a fixed knuckle of the user's hand, so the position of the first object (user's hand) can be corrected based on the accurate position of the target object (ring).

[0074] In the positioning processing method described in this embodiment, a target object with accurate physical appearance data is bound to the first object. Therefore, the accurate spatial position of the target object can be obtained based on a comprehensive analysis of the physical appearance data of the target object and its image data. Then, based on the binding relationship between the target object and the first object, the accurate spatial position of the target object is used to correct the first spatial position of the first object, thereby obtaining a more accurate first corrected position, thereby meeting the high-precision requirements for positioning the operating body in the virtual interaction scene.

[0075] In the above embodiment, after obtaining the first image of the first object through the image acquisition module, the method may further include: verifying the target object. In this embodiment, the target object may be configured with a sensor for detecting inertial measurement data of the target object, such as an acceleration sensor, an inertial sensor, etc., and when verifying the target object, the target object may be verified based on the inertial measurement data acquired by the above sensor.

[0076] The specific verification process may include: determining the posture information of the target object based on the first image; obtaining inertial measurement data of the target object; if the posture information and the inertial measurement data match, determining the first spatial position of the first object based on the first image, wherein the match indicates that the error between the posture represented by the inertial measurement data and the posture information is lower than a set value.

[0077] In implementation, the position and posture information of the target object can be first determined based on the image analysis of the target object in the first image, and then the inertial measurement data detected by the sensor in the target object can be obtained through a wireless connection with the target object. The inertial measurement data can characterize the rotation angle of the target object in each dimension, and thus the position and posture of the target object can be indirectly determined based on the inertial measurement data; and then the position and posture determined based on the inertial measurement data is compared with the position information determined based on the image analysis of the target object. If the error is less than a set threshold, it is considered that the two match, and the relevant data determined based on the image of the target object is credible, and the subsequent operation of determining the first spatial position of the first object can continue.

[0078] In this embodiment, before using the image of the target object for corresponding analysis and processing, the target object is first verified. Only when the verification passes and it is determined that the corresponding analysis and processing based on the image of the target object are effective, will the subsequent steps of the positioning processing be continued, thereby ensuring the effectiveness of the execution of the positioning processing flow.

[0079] Figure 3 This is a flow chart for determining the target spatial position disclosed in the embodiment of the present application. Figure 3 As shown, determining the target spatial position of the target object based on the physical appearance data, image appearance data of the target object and the internal and external parameters of the image acquisition module may include:

[0080] Step 301: Based on the image appearance data of the target object in the first image and the internal parameters of the image acquisition module, determine the first coordinate data of the target object in the space coordinate system corresponding to the image acquisition module.

[0081] After obtaining the first image, we can first analyze and determine the image appearance data of the target object in the first image, such as the diameter and width of the ring in the image; then combine the internal parameters of the image acquisition module, such as focal length, principal point (optical center) coordinates, distortion coefficient, etc., to determine the coordinate data of the target object in the spatial coordinate system of the image acquisition module, that is, the first coordinate data.

[0082] Step 302: Determine second coordinate data of the target object in a world coordinate system based on the first coordinate data and external parameters of the image acquisition module.

[0083] Since the first coordinate data in the internal coordinate system of the image acquisition module cannot reflect the position in the real physical world, it is necessary to further convert the first coordinate data into second coordinate data in the real physical world based on the external parameters of the image acquisition module, wherein the external parameters include a rotation matrix and a translation vector.

[0084] Step 303: Determine the second spatial coordinates of the target object based on the second coordinate data and the physical appearance data of the target object.

[0085] After determining the second coordinate data of the target object in the real physical world, the second spatial coordinate of the target object can be determined based on the second coordinate data and the physical appearance data of the target object. That is, the size of the object mapped to the real physical world by the second coordinate data may not be the same as the physical appearance data of the target object, so the object mapped to the real physical world by the second coordinate data needs to be enlarged or reduced to make its size consistent with the physical appearance data of the target object, thereby determining the target spatial position of the target object based on the scaling factor.

[0086] The above content introduces in detail the specific implementation of determining the target spatial position of the target object, which is convenient for technical personnel in the field to better understand and implement the technical solution of the present application.

[0087] In one implementation, the correcting the first spatial position of the first object based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position may include: determining the to-be-compared spatial position of the first object based on the target spatial position and the binding position of the target object on the first object; and correcting the first spatial position based on an error between the to-be-compared spatial position and the first spatial position.

[0088] It is understandable that, since there is a binding relationship between the target object and the first object, after determining the accurate spatial position of the target object, the first spatial position of the first object can be corrected based on the relative position relationship between the target object and the first object. The spatial position to be compared is the spatial position of the first object determined based on the relative position relationship between the target object and the first object and the target spatial position of the target object. Subsequently, the position correction can be performed based on the relationship between the spatial position to be compared and the first spatial position.

[0089] Specifically, the correction of the first spatial position based on the error between the spatial position to be compared and the first space may include: if the error between the spatial position to be compared and the first spatial position is less than a first threshold, determining the spatial position to be compared as the first corrected position; if the error between the spatial position to be compared and the first spatial position is greater than or equal to the first threshold and less than a second threshold, performing linear summation processing on the spatial position to be compared and the first spatial position, and determining the processing result as the first corrected position; if the error between the spatial position to be compared and the first spatial position is greater than or equal to the second threshold, deleting the spatial position to be compared and the first spatial position.

[0090] If the error between the spatial position to be compared and the first spatial position is less than the first threshold, it means that the two are close, and the spatial position to be compared with higher accuracy can be determined as the first correction position. Since the error between the spatial position to be compared and the first spatial position is very small, the spatial position to be compared is directly used as the final first correction position, and from the user's viewing experience, there will be no discomfort.

[0091] If the error between the spatial position to be compared and the first spatial position is greater than or equal to the first threshold and less than the second threshold, it indicates that there is a certain error between the two. If the spatial position to be compared is directly determined as the first correction position, it may also cause the result to deviate too much from the actual position. Therefore, the first correction position can be determined by combining the spatial position to be compared and the first spatial position. For example, the spatial position to be compared and the first spatial position are linearly fitted to obtain the first correction position. It can be understood that if the error between the spatial position to be compared and the first spatial position is very small during the processing of the previous frame of the image, the spatial position to be compared is directly used as the first correction position; and in the processing of this frame of the image, the error between the spatial position to be compared and the first spatial position is between the first threshold and the second threshold, and the confidence of the spatial position to be compared is lower than the confidence of the spatial position to be compared in the previous frame. If the spatial position to be compared is directly used as the first correction position, the virtual display screen may give the user the perception of a slight jump in the position of the first object. Therefore, in this case, the spatial position to be compared and the first spatial position are linearly fitted to obtain the first correction position.

[0092] If the error between the spatial position to be compared and the first spatial position is greater than or equal to the second threshold, it indicates that there may be a recognition error between the two, for example, the first spatial position actually recognizes the position of other interfering hands (other hands other than the operating hand) in the space; in this case, if the spatial position to be compared and / or the first spatial position are continued to be used to determine the first correction position, the user may experience a relatively obvious jump in the position of the first object in the virtual display screen. Therefore, in this case, the spatial position to be compared and the first spatial position can be deleted, that is, any data obtained from the processing and analysis of the current frame image is not used and is directly discarded.

[0093] This embodiment performs different processing according to the size of the error between the spatial position to be compared and the first spatial position, and effectively avoids the situation where erroneous data is used to trigger interaction when the first object is recognized incorrectly, and also avoids the situation where the virtual reality screen may be stuck or jump due to directly using one of the spatial positions to be compared as the first correction position when there is a certain error between the spatial position to be compared and the first spatial position.

[0094] In other implementations, the first image also includes a second object, that is, the first image includes other non-operating hands in addition to the operator. After determining the target spatial position, the positioning processing method may further include: determining a second spatial position of the second object based on the first image; and correcting the second spatial position of the second object based on the image appearance data of the target object and the first corrected position to obtain a second corrected position.

[0095] The overall process of determining the second corrected position of the second object is the same as the overall process of determining the first corrected position of the first object, that is, firstly, based on the first image, the second spatial position of the second object is determined based on the image depth algorithm, and then the second spatial position of the second object is corrected based on the spatial position of the target object to obtain the second corrected position. However, since the target object is only bound to the first object but not to the second object, when determining the second corrected position, it is necessary to correct the second spatial position of the second object based on the image appearance data of the target object and the first corrected position.

[0096] Specifically, Figure 4 A flow chart for determining the second corrected position is shown. Combined with the example shown in step 4, the second spatial position of the second object is corrected based on the image appearance data of the target object and the first corrected position to obtain the second corrected position, which may include:

[0097] Step 401: Determine a rotation parameter and a translation parameter between the first object and the second object based on the first image.

[0098] In the first image, the first object and the second object have different postures and positions, and firstly, the rotation parameter and the translation parameter between the first object and the second object can be determined, wherein the rotation parameter represents the change in posture between the first object and the second object, and the translation parameter represents the change in position between the first object and the second object.

[0099] Specifically, a first posture of the first object and a second posture of the second object may be determined based on the first image, and then a rotation parameter for converting from the first posture to the second posture may be determined, and a translation parameter from the first spatial position to the second spatial position may be determined.

[0100] Step 402: Control the target object to perform a rotation and translation operation based on the rotation parameter and the translation parameter, so that the posture of the target object after the rotation and translation is the same as the posture of the second object and is located at the binding position of the second object.

[0101] That is, the posture of the target object is adjusted to correspond to the second object, and the target object is moved from the binding position of the first object to the binding position of the second object, such as from the first joint of the index finger of the manipulating hand to the first joint of the index finger of the non-manipulating hand (the second object). This process can be understood as moving the ring from the manipulating hand to the corresponding finger position of the non-manipulating hand.

[0102] Step 403: scaling the rotated and translated target object so that the scaled target object matches the second object, wherein the matching represents that the target object and the second object present a binding relationship between the target object and the first object before rotation and translation.

[0103] It is understandable that, since the depths of the first object and the second object in the first image may be different, even if the target object is moved to the binding position of the second object, it may not match the second object, such as the diameter of the ring is obviously smaller than the diameter of the finger of the non-operating hand, or the diameter of the ring is obviously larger than the diameter of the finger of the non-operating hand, so the ring needs to be scaled so that it can be properly "worn" on the non-operating hand. Of course, there may be a certain occlusion relationship between the ring and the non-operating hand, so in the implementation, corresponding processing can be performed according to the perspective camera principle to make the target object "match" with the second object.

[0104] Figure 5 This is an example diagram of a ring moving from an operating hand to a non-operating hand disclosed in an embodiment of the present application. The upper to middle diagrams correspond to step 402, which involves the content of rotating and translating the ring to the binding position of the non-operating hand; the middle to lower diagrams correspond to step 403, which involves the content of scaling the ring to match the non-operating hand. Figure 5 Understand the foregoing.

[0105] Step 404: Obtain a second corrected position of the target object based on the scaling ratio of the scaling process, the first corrected position, and the second spatial position.

[0106] After determining the scaling ratio of the scaling process, since the positional relationship between the first object and the second object can also be determined based on the first image, the second spatial position can be corrected based on the accurate first corrected position of the first object and the scaling ratio to obtain the second corrected position of the second object.

[0107] This embodiment introduces the specific implementation of correcting the spatial positions of other objects based on the first image and target object-related data when other objects exist in the first image, so as to accurately identify other interfering objects other than the first object and perform subsequent interference elimination operations, thereby ensuring the recognition accuracy of the first object.

[0108] In another implementation, based on the aforementioned disclosure, the positioning processing method may further include: determining an appearance deviation coefficient between the first object and the second object based on the first image. Then, obtaining a second corrected position of the target object based on the scaling ratio of the scaling processing, the first corrected position, and the second spatial position includes: obtaining the second corrected position of the target object based on the scaling ratio of the scaling processing, the first corrected position, the second spatial position, and the appearance deviation coefficient.

[0109] The first object and the second object in the first image may have different physical appearances. For example, if the operator and the operator's hand belong to different users and the thickness of the fingers of the two users is very different, then when determining the second correction position, the physical appearance difference between the two objects, that is, the appearance deviation coefficient, also needs to be considered. In this way, when determining the second correction position, the appearance deviation coefficient is also added to the analysis process, which can avoid errors caused by the difference in physical appearance of the two objects.

[0110] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0111] The method is described in detail in the embodiments disclosed in the above-mentioned application. The method of the application can be implemented by various forms of devices. Therefore, the application also discloses a device, and a specific embodiment is given below for detailed description.

[0112] Figure 6 This is a schematic diagram of the structure of a positioning processing device disclosed in an embodiment of the present application. Figure 6 As shown, the positioning processing device 60 may include:

[0113] The image acquisition module 601 is configured to acquire a first image of a first object through an image acquisition module, wherein the first image includes the first object and a target object having a binding relationship with the first object.

[0114] The first position obtaining module 602 is configured to determine a first spatial position of the first object based on the first image.

[0115] The target position determination module 603 is used to determine the target spatial position of the target object based on the physical appearance data, image appearance data of the target object and the internal and external parameters of the image acquisition module, wherein the physical appearance data is preset data and the image appearance data is determined based on the image analysis of the target object in the first image.

[0116] The first position correction module 604 is configured to correct the first spatial position of the first object based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position.

[0117] In the positioning processing device described in this embodiment, a target object with accurate physical appearance data is bound to the first object. Therefore, the accurate spatial position of the target object can be obtained based on a comprehensive analysis of the physical appearance data of the target object and its image data. Then, based on the binding relationship between the target object and the first object, the accurate spatial position of the target object is used to correct the first spatial position of the first object, thereby obtaining a more accurate first corrected position, thereby meeting the high-precision requirements for positioning the operating body in the virtual interactive scene.

[0118] The specific implementation of the positioning processing device and the modules it contains can be found in the corresponding parts of the method embodiment, which will not be repeated here.

[0119] Any one of the positioning processing devices described in the above embodiments includes a processor and a memory. The image acquisition module, the first position acquisition module, the target position determination module, the first position correction module, etc. in the above embodiments are all stored in the memory as program modules, and the processor executes the above program modules stored in the memory to implement corresponding functions.

[0120] The processor includes a kernel, which retrieves the corresponding program module from the memory. One or more kernels can be set, and the processing of the access data can be realized by adjusting the kernel parameters.

[0121] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0122] In an exemplary embodiment, a computer-readable storage medium is also provided, which can be directly loaded into the internal memory of a computer and contains software code. After being loaded and executed by a computer, the computer program can implement the steps shown in any embodiment of the above-mentioned positioning processing method.

[0123] In an exemplary embodiment, a computer program product is also provided, which can be directly loaded into the internal memory of a computer and contains software codes. After being loaded and executed by a computer, the computer program can implement the steps shown in any embodiment of the positioning processing method described above.

[0124] Furthermore, the present application also discloses a positioning processing device, such as Figure 7 As shown, the positioning processing device 70 includes an image acquisition device 701, at least one processor 702 and a memory 703 connected to the processor, wherein: the image acquisition device is used to acquire images; the memory is used to store computer programs; and the processor is used to execute the computer program so that the electronic device can implement any one of the positioning processing methods in the aforementioned embodiments.

[0125] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0126] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0127] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0128] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A positioning processing method, the method comprising: Obtaining a first image of a first object through an image acquisition module, wherein the first image includes the first object and a target object having a binding relationship with the first object; determining a first spatial position of the first object based on the first image; Determine the target spatial position of the target object based on physical appearance data, image appearance data of the target object, and internal and external parameters of the image acquisition module, wherein the physical appearance data is preset data and the image appearance data is determined based on image analysis of the target object in the first image; The first spatial position of the first object is corrected based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position.

2. The positioning processing method according to claim 1, after obtaining the first image of the first object through the image acquisition module, further comprising: Verifying the target object includes: Determine the position and posture information of the target object based on the first image; Obtaining inertial measurement data of the target object; If the posture information and the inertial measurement data match, a first spatial position of the first object is determined based on the first image, wherein the match indicates that an error between the posture represented by the inertial measurement data and the posture information is lower than a set value.

3. The positioning processing method according to claim 1, wherein determining the target spatial position of the target object based on the physical appearance data, image appearance data of the target object and the internal and external parameters of the image acquisition module comprises: Determine first coordinate data of the target object in a spatial coordinate system corresponding to the image acquisition module based on the image appearance data of the target object in the first image and the internal parameters of the image acquisition module; Determine second coordinate data of the target object in a world coordinate system based on the first coordinate data and an external parameter of the image acquisition module; A target spatial position of the target object is determined based on the second coordinate data and the physical appearance data of the target object.

4. The positioning processing method according to claim 1, wherein the first spatial position of the first object is corrected based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position, comprising: Determine the to-be-compared spatial position of the first object based on the target spatial position and the binding position of the target object on the first object; The first spatial position is corrected based on an error between the spatial position to be compared and the first spatial position.

5. The positioning processing method according to claim 4, wherein the step of correcting the first spatial position based on the error between the spatial position to be compared and the first spatial position comprises: If the error between the spatial position to be compared and the first spatial position is less than a first threshold, determining the spatial position to be compared as a first corrected position; If the error between the spatial position to be compared and the first spatial position is greater than or equal to the first threshold and less than a second threshold, linear summation is performed on the spatial position to be compared and the first spatial position, and the processing result is determined as a first corrected position; If the error between the spatial position to be compared and the first spatial position is greater than or equal to the second threshold, the spatial position to be compared and the first spatial position are deleted.

6. The positioning processing method according to claim 1, wherein the first image also includes a second object, and after determining the target spatial position, the method further includes: determining a second spatial position of the second object based on the first image; The second spatial position of the second object is corrected based on the image appearance data of the target object and the first corrected position to obtain a second corrected position.

7. The positioning processing method according to claim 6, wherein the step of correcting the second spatial position of the second object based on the image appearance data of the target object and the first corrected position to obtain a second corrected position comprises: determining a rotation parameter and a translation parameter between the first object and the second object based on the first image; Controlling the target object to perform a rotation and translation operation based on the rotation parameter and the translation parameter, so that the posture of the target object after the rotation and translation is the same as the posture of the second object and is located at the binding position of the second object; Scaling the rotated and translated target object so that the scaled target object matches the second object, wherein the matching indicates that the target object and the second object present a binding relationship between the target object and the first object before the rotation and translation; A second corrected position of the target object is obtained based on the scaling ratio of the scaling process, the first corrected position, and the second spatial position.

8. The positioning processing method according to claim 7, further comprising: determining an appearance deviation coefficient between the first object and the second object based on the first image; The obtaining the second corrected position of the target object based on the scaling ratio of the scaling process, the first corrected position and the second spatial position includes: A second corrected position of the target object is obtained based on a scaling ratio of the scaling process, the first corrected position, the second spatial position, and the appearance deviation coefficient.

9. The positioning processing method according to claim 7, determining the rotation parameter and the translation parameter between the first object and the second object based on the first image, comprising: determining a first pose of the first object and a second pose of the second object based on the first image; determining a rotation parameter for transitioning from the first posture to the second posture; A translation parameter from the first spatial position to the second spatial position is determined.

10. A positioning processing device, comprising an image acquisition device and at least one processor, wherein: The image acquisition device is used to acquire images; The processor is connected to the image acquisition device and is configured to: Obtaining a first image of a first object through an image acquisition module, wherein the first image includes the first object and a target object having a binding relationship with the first object; determining a first spatial position of the first object based on the first image; Determine the target spatial position of the target object based on physical appearance data, image appearance data of the target object, and internal and external parameters of the image acquisition module, wherein the physical appearance data is preset data and the image appearance data is determined based on image analysis of the target object in the first image; The first spatial position of the first object is corrected based on the target spatial position and the binding position of the target object on the first object to obtain a first corrected position.

Images