Position determination method, device and electronic device

Abstract

The present application discloses a method, device and electronic device for determining a posture, the method comprising: obtaining a first image of a first electronic device, the first electronic device being capable of controlling a virtual scene output by a second electronic device, the first electronic device being provided with a plurality of target objects; determining the first posture information of the first electronic device relative to the second electronic device at least based on the relative position relationship between the target objects in the first image and a position relationship model created based on the three-dimensional coordinate information of each target object of the first electronic device. The scheme of the present application can determine the posture information of the first electronic device as the control end relative to the second electronic device as the controlled end.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and more specifically, to a posture determination method, device and electronic device. Background Art

[0002] At present, it is very common to use one electronic device to control another electronic device. For example, in virtual reality (VR) and augmented reality (AR), a handle is needed to control a virtual scene output by a virtual reality device.

[0003] In order to improve the convenience of control, an electronic device can determine the control command based on the posture (position and posture information) of another electronic device. For example, in a game scene based on AR or VR, players can control the game by changing the position and posture of the handle. Based on this, it can be seen that how to determine the posture of the electronic device as the control end is a prerequisite for achieving precise control of the electronic device as the controlled end, and it is also a technical problem that technicians in this field need to solve. Summary of the invention

[0004] The present application provides a posture determination method, device and electronic device.

[0005] A method for determining a position and posture includes:

[0006] Obtaining a current first image of a first electronic device, the first electronic device being capable of controlling a virtual scene output by a second electronic device, and a plurality of target objects being arranged on the first electronic device;

[0007] The first position information of the first electronic device relative to the second electronic device is determined based at least on the relative position relationship between the target objects in the first image and a position relationship model created based on the three-dimensional coordinate information of each target object of the first electronic device.

[0008] Preferably, the determining the current first position information of the first electronic device relative to the second electronic device based on at least the relative position relationship between the target objects in the first image and the position relationship model created based on the three-dimensional coordinate information of each target object of the first electronic device includes:

[0009] Determine a target object group included in the first image based on the relative position relationship between the target objects in the first image and the position relationship model, wherein the target objects in the target object group have a first relative position relationship, and the position relationship model can at least characterize the three-dimensional distribution of the target objects in the first electronic device;

[0010] Based at least on the first relative position relationship and the three-dimensional coordinate information of each target object in the first image, the first position information of the first electronic device relative to the second electronic device is determined.

[0011] Preferably, it also includes:

[0012] The projection position points of the remaining target objects in the first electronic device on the first image are determined at least based on the position relationship model and the first posture information, and the second posture information of the first electronic device relative to the second electronic device is determined according to the projection position points and the three-dimensional coordinate information corresponding to the remaining target objects.

[0013] Preferably, the first image includes at least four target objects;

[0014] The determining, based on the relative positional relationship between the target objects in the first image and the positional relationship model, a target object group included in the first image includes:

[0015] Determining at least three first target objects for pose calculation and at least one second target object for pose verification from the at least four target objects in the first image;

[0016] Based on the positional relationship model and the relative positional relationship between the target objects in the first image, determining a first target object group and a second target object group included in the first image, wherein the first target object group includes at least three target objects in the first electronic device that match the at least three first target objects, the second target object group includes at least one target object in the first electronic device that matches the at least one second target object, and the first target object group has a first relative positional relationship;

[0017] The determining, based at least on the first relative position relationship and the three-dimensional coordinate information of each target object in the first image, the first posture information of the first electronic device relative to the second electronic device includes:

[0018] Determine at least one set of first posture information of the first electronic device relative to the second electronic device based at least on a first relative position relationship of each target object in the first target object group and three-dimensional coordinate information of each target object in the first image;

[0019] The method further comprises:

[0020] Projection verification is performed on the at least one set of first pose information based on each target object in the second target object group and the position relationship model, and correct pose information in the at least one set of first pose information is determined according to the projection verification result.

[0021] Preferably, it also includes:

[0022] If it is determined according to the projection verification result that correct posture information does not exist in the at least one group of first posture information, the operation of determining the first target object group and the second target object group contained in the first image is returned to re-determine the first target object group and the second target object group contained in the first image from the multiple target objects of the first electronic device.

[0023] Preferably, determining the first position information of the first electronic device relative to the second electronic device at least based on the first relative position relationship and the three-dimensional coordinate information of each target object in the first image includes:

[0024] Based at least on the first relative position relationship and the three-dimensional coordinate information of each target object in the first image, and using a perspective projection algorithm, the first position information of the first electronic device relative to the second electronic device is determined.

[0025] Preferably, it also includes:

[0026] If the number of target objects in the first image is less than the set number, third posture information determined by the inertial measurement device in the first electronic device is obtained, and the third posture information is determined as the posture information of the first electronic device relative to the second electronic device.

[0027] Preferably, it also includes:

[0028] Obtaining a current second image of the first electronic device;

[0029] Determining fourth position information of the first electronic device relative to the second electronic device based at least on a second relative position relationship between target objects in the second image and the position relationship model;

[0030] Based on the first posture information and the fourth posture information, input control data for the second electronic device is determined.

[0031] Among them, a posture determination device includes:

[0032] An image obtaining unit, used to obtain a current first image of a first electronic device, the first electronic device can control a virtual scene output by a second electronic device, and a plurality of target objects are arranged on the first electronic device;

[0033] A posture determination unit is used to determine the first posture information of the first electronic device relative to the second electronic device based on at least the relative position relationship between the target objects in the first image and a position relationship model created based on the three-dimensional coordinate information of each target object of the first electronic device.

[0034] An electronic device includes:

[0035] Memory and processor;

[0036] The processor is used to execute the posture determination method as described in any one of the above items;

[0037] The memory is used to store programs required for the processor to perform operations.

[0038] It can be seen from the above scheme that in this application, multiple target objects are set on the first electronic device serving as the control end, and the second electronic device can obtain the current first image of the first electronic device. Since the relative position relationship between the target objects in the first image can reflect the two-dimensional coordinate information of at least some target objects in the first electronic device, therefore, based on the relative position relationship between the target objects in the first image and the position relationship model that can reflect the three-dimensional coordinate information of each target object of the first electronic device, the posture information of the first electronic device relative to the second electronic device can be determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A schematic diagram of a flow chart of a method for determining a posture provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of an arrangement of multiple target objects on a first electronic device provided in an embodiment of the present application;

[0042] Figure 3 for Figure 2 A plan view schematic diagram of multiple target objects on the first electronic device shown;

[0043] Figure 4 A schematic diagram of another flow chart of a method for determining a posture provided in an embodiment of the present application;

[0044] Figure 5A planar schematic diagram of the distribution of target objects on the first electronic device;

[0045] Figure 6 for Figure 5 A schematic diagram of target objects in a first image of a first electronic device is shown;

[0046] Figure 7 A schematic diagram of another flow chart of a method for determining a posture provided in an embodiment of the present application;

[0047] Figure 8 for Figure 2 A schematic diagram of a target object in a first image of a first electronic device is shown;

[0048] Fig. 9 A schematic diagram of the structure of a posture determination device provided in an embodiment of the present application;

[0049] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0050] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims and the above drawings are used to distinguish similar parts, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated herein. DETAILED DESCRIPTION

[0051] The solution of the present application is applicable to determining the position information of a first electronic device as a control terminal relative to a second electronic device as a controlled terminal. For example, the first electronic device may be a handle in a virtual reality scene, and the second electronic device may be a smart helmet that outputs a virtual display scene.

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] See also Figure 1 , Figure 1 This is a schematic diagram of an implementation flow of a method for determining a posture provided in an embodiment of the present application. The method of this embodiment can be applied to an electronic device as a controlled end, that is, a second electronic device. The method of this embodiment may include:

[0054] S101, obtaining a current first image of a first electronic device.

[0055] The first electronic device is capable of controlling the virtual scene output by the second electronic device.

[0056] In the present application, the first electronic device is provided with a plurality of target objects, which are identification marks on the first electronic device, such as icons, LED lights or other identifiers provided on the housing of the first electronic device.

[0057] It is understandable that, since the first electronic device has multiple target objects as identification marks, after obtaining the current image of the first electronic device, the image will include at least part of the target objects on the first electronic device. For the sake of distinction, the image of the first electronic device obtained in step S101 is referred to as the first image.

[0058] The first image may include an image of all or part of the target object in the first electronic device.

[0059] Among them, the multiple target objects on the first electronic device can be arranged as needed.

[0060] Optionally, in order to more accurately and conveniently identify the target objects in the first image as target objects in the first electronic device, in the present application, multiple target objects on the first electronic device are arranged according to a set arrangement rule.

[0061] Among them, the arrangement rule can have multiple possibilities. For example, in one possible implementation, the multiple target objects of the first electronic device are the same, and on this basis, the arrangement of the multiple target objects on the first electronic device can conform to the set pattern rule. For example, according to the arrangement rule of multiple patterns such as rings, triangles, rhombuses, and five-pointed stars being arranged in sequence, multiple target objects (such as multiple round lights or black dots, etc.) can be used on the first electronic device to sequentially construct multiple patterns such as rings, triangles, rhombuses, and five-pointed stars arranged in sequence.

[0062] In another possible scenario, the first electronic device is provided with a plurality of target objects of at least two different types, and the plurality of target objects of the at least two different types are arranged with a set arrangement rule.

[0063] For example, the first electronic device may include multiple different target objects, and the multiple different target objects are arranged in a set order. For example, the multiple target objects are multiple markers with different patterns, and the multiple patterns on the multiple markers can be arranged in a set order by arranging the multiple markers.

[0064] For another example, the multiple target objects of the first electronic device can be divided into at least two types, and each type of target object includes at least one. On this basis, the arrangement rules of different types of target objects in the first electronic device can be set. Figure 2 , which shows a schematic diagram of the arrangement of multiple target objects on a first electronic device.

[0065] exist Figure 2 In the example, taking the first electronic device as a ring-shaped control handle 200, a plurality of LED lights 201 may be provided on the outer side of the handle, such as Figure 2 As shown in the various dots on the handle 200 shown in the figure. Among them, the multiple LED lights are divided into two types, one is a smaller LED light, and the other is a relatively large LED, etc. Figure 2 Of course, LED lights of the same size can also be set, but the first electronic device controls the LED lights to present different brightness to achieve Figure 2 The effect of Figure 2 It can be seen that each large LED light is spaced in sequence on the upper and lower edges of the handle, and each large LED light is surrounded by four small LED lights. Figure 3 , Figure 3 for Figure 2 The schematic diagram of the distribution of LED lights on the handle shown in FIG. Figure 3 It can be seen that the arrangement of large LED lights and small LED lights has a regular arrangement.

[0066] S102: Determine first position information of the first electronic device relative to the second electronic device based at least on the relative position relationship between the target objects in the first image and the position relationship model of the target objects of the first electronic device.

[0067] The position relationship model is created based on the three-dimensional coordinate information of each target object of the first electronic device. For example, the three-dimensional coordinate information of each target object of the first electronic device may be the three-dimensional coordinates of each target object relative to the first electronic device, for example, the center point of the first electronic device is used as the origin of the coordinate axis and a spatial coordinate system is constructed, and then the three-dimensional coordinate information of each target object in the first electronic device is determined.

[0068] Among them, the position relationship model can be stored on the second electronic device side. Correspondingly, the second electronic device side can restore the three-dimensional coordinate information of each target object contained in the first electronic device based on the position relationship model. At the same time, the three-dimensional coordinate information of each target object on the first electronic device can also reflect the arrangement mode of each target object on the first electronic device, so that the arrangement rule of each target object in the first electronic device can be obtained.

[0069] It can be understood that since the position relationship model can reflect the three-dimensional coordinate information of each target object on the first electronic device, and the relative position relationship of the target objects in the first image actually reflects the two-dimensional coordinate information of at least some of the target objects on the first electronic device in the coordinate system of the first image, based on this, the two-dimensional coordinate and three-dimensional coordinate information corresponding to at least some of the target objects on the first electronic device can be obtained.

[0070] Among them, the two-dimensional coordinates of at least part of the target object in the first electronic device refer to the two-dimensional coordinates of at least part of the target object in the first electronic device on the camera coordinate system of the second electronic device. Therefore, based on the two-dimensional coordinates of at least part of the target object and its three-dimensional coordinate information, the posture information of the first electronic device relative to the second electronic device is determined.

[0071] Among them, the position and posture can be determined by combining the mapping relationship between the two-dimensional coordinates and the three-dimensional coordinates of at least part of the target object in the first electronic device and combining any three-dimensional reconstruction method, without limitation.

[0072] For example, in an optional manner, the first position information of the first electronic device relative to the second electronic device can be determined based on at least the first relative position relationship and the three-dimensional coordinate information of each target object in the first image, and using a perspective projection algorithm. The perspective projection algorithm can also be called a perspective-n-point (PNP) algorithm.

[0073] The position information includes the position information and attitude information of the first electronic device relative to the second electronic device. For example, the attitude information may include a pitch angle, a heading angle, and a roll angle.

[0074] For ease of distinction, this application refers to the position and posture information of the first electronic device relative to the second electronic device determined based on the first image as the first position and posture information.

[0075] It can be seen that in the present application, a plurality of target objects are arranged on the first electronic device serving as the control terminal, and the second electronic device can obtain the current first image of the first electronic device. Since the relative position relationship between the target objects in the first image can reflect the two-dimensional coordinate information of at least some target objects in the first electronic device, the position relationship model based on the relative position relationship between the target objects in the first image and the three-dimensional coordinate information of each target object of the first electronic device can be used to determine the posture information of the first electronic device relative to the second electronic device.

[0076] At the same time, compared to the current positioning of the control end device based on ultrasound, which can only determine the position information of the first electronic device relative to the second electronic device, the present application can also obtain attitude information, which is conducive to more accurately controlling the virtual scene output by the second electronic device based on the first electronic device; it can also avoid the situation where the accuracy is too low due to the jitter of ultrasonic positioning. Moreover, compared with combining ultrasound and the inertial measurement device on the first electronic device side to determine the position information of the first electronic device, the present application can effectively avoid the drift caused by the long-term integration of the gyroscope in the inertial measurement device. The yaw angle cannot be effectively corrected, resulting in the instability of the determined attitude.

[0077] In addition, since the present application can determine the posture information of the first electronic device based only on the first image currently obtained, without combining the previous frame image before the first image to determine the posture information, it can avoid the situation where the target object as a landmark cannot be correctly identified due to mismatching due to the rapid movement of the first electronic device, and then the target object tracking abnormality and ultimately the posture positioning abnormality.

[0078] like Figure 4 As shown, it shows a flow chart of another embodiment of a method for determining a posture of the present application. The method of this embodiment may include:

[0079] S401: Obtain a current first image of a first electronic device.

[0080] The first electronic device is capable of controlling the virtual scene output by the second electronic device.

[0081] In the present application, a plurality of target objects are arranged on the first electronic device.

[0082] S402: Determine a target object group included in the first image based on the relative position relationship between the target objects in the first image and a position relationship model.

[0083] The position relationship model can at least characterize the three-dimensional distribution of target objects in the first electronic device. The position relationship model can be specifically described in the previous embodiment, and will not be described in detail here.

[0084] There is a first relative position relationship between the target objects in the target object group.

[0085] The target object group includes: at least a set number of target objects in the first image that match the target object of the first electronic device, determined based on the relative position relationship between the target objects in the first image and the position relationship between the target objects in the electronic device. The set number can be set as needed, such as the set number can be four or five, and the target object group can include at least four or five target objects.

[0086] It is understandable that, in practical applications, the number of target objects required to be included in the target object group can be set as needed. At the same time, the first relative positional relationship between the target objects included in the target object group will also be different. For example, assuming that the first image actually contains images of target objects 1-target objects 10 in the first electronic device, then if the target object group is set to include 4 target objects, the determined target object group may be composed of target object 1, target object 2, target object 3 and target object 4, or may be composed of target object 3, target object 4, target object 5 and target object 6, and in these two cases, the relative positional relationship between the target objects in the target object group may be different.

[0087] It is understandable that, since the target object in the first image is actually an image of the target object on the first electronic device side, determining the target object group included in the first image actually means determining which target objects on the first electronic device side the target object in the first image belongs to, and finally determining at least a set number of target objects on the first electronic device included in the first image. Correspondingly, determining the target object group included in the first image can also be seen as determining the target object group on the first electronic device side, and the target object group includes: target objects whose relative position relationship matches the relative position relationship between the target objects in the first image.

[0088] It can be understood that since the relative position relationship of each target object in the first image can reflect the relative position relationship of each target object in the first image on the first electronic device, it is possible to determine the target objects in the first electronic device that match the relative position relationship of each target object in the first image based on the position relationship model corresponding to each target object in the first electronic device, thereby determining the target object group in the first image corresponding to the first electronic device side.

[0089] For example, the relative position relationship between the target objects in the first image can reflect the arrangement of the target objects in the first image. Accordingly, based on the arrangement, target objects that conform to the arrangement can be searched from the first electronic device side to determine which target objects contained in the first image belong to the first electronic device.

[0090] Combination Figure 5and Figure 6 As shown, Figure 5 is a plan view schematically showing the distribution of various target objects on the first electronic device, Figure 5 In the example, the target object of the first electronic device is Figure 5 As shown in the middle dot 501, these target objects form different images. Figure 6 for Figure 5 Schematic diagram of each target object in the first image of the first electronic device shown. Figure 5 and Figure 6 It can be seen that Figure 6 The first image in includes Figure 5 The target objects that make up the circular pattern and the square pattern are composed of Figure 5 Based on this, it can be determined that the first image contains the first electronic device side. Figure 5 The target objects within the dotted box shown in the figure are composed of a target object group.

[0091] As an optional method, the position relationship model can also be associated with the attribute characteristics of each target object on the first electronic device, such as the attribute characteristics can include: the shape of the target object, the pattern characteristics it has, and the image of the target object, etc., which can reflect one or more external characteristics presented by the target object. On this basis, since the first image contains the images of each target object, the attribute information of each target object in the first image can be determined. Based on this, it is also possible to combine the images of each target object in the first image and the relative position relationship to determine which target objects in the first electronic device the target object group contained in the first image is.

[0092] S403: Determine a first position information of the first electronic device relative to the second electronic device based at least on the first relative position relationship and the three-dimensional coordinate information of each target object in the first image.

[0093] The three-dimensional coordinate information of each target object in the first image can be obtained based on the target object group included in the first image and the position relationship model of each target object on the first electronic device. For example, based on the target object group in the first image, the corresponding relationship between the target object group in the first image and the target object on the first electronic device can be determined, so that the three-dimensional coordinate information of each target object in the first image can be determined based on the position relationship model.

[0094] It can be understood that the first relative position relationship between each target object in the target object group is the two-dimensional coordinate information between each target object in the target object group, and the three-dimensional coordinate information of each target object in the first image includes the three-dimensional coordinate information of each target object in the target object group. Therefore, based on algorithms such as PNP, the first position information of the first electronic device relative to the second electronic device can be determined.

[0095] Optionally, the first pose information can be determined based on the first relative position relationship and the three-dimensional coordinate information of each target object group in the target object group. It can be understood that the first relative position relationship is the two-dimensional coordinate information of each target object in the target object group in the coordinate system of the first image, and the three-dimensional coordinate information of each target object in the target object group can be directly obtained from the position relationship model, so that the three-dimensional coordinate information of each target object group in the target object group can be obtained more efficiently, which is conducive to more efficient determination of the first pose information.

[0096] In an embodiment of the present application, based on the relative position relationship of each target object in the first image and the relative position relationship model of each target object of the first electronic device, the target object group contained in the first image is first determined, and the corresponding relationship between the target object in the first image and the target object on the first electronic device is determined. The relative position relationship of the target object group on the first image is the two-dimensional coordinate information on the camera coordinate system of the second electronic device. Therefore, based on the two-dimensional coordinate information of each target object in the target object group and the three-dimensional coordinate information of each target object in the target object group, the relative spatial posture between the first electronic device and the second electronic device can be restored, and then the first posture information can be obtained.

[0097] S404, determining the projection position points of the remaining target objects in the first electronic device on the first image at least based on the position relationship model and the first pose information, and determining the second pose information of the first electronic device relative to the second electronic device according to the projection position points and the three-dimensional coordinate information corresponding to the remaining target objects.

[0098] The remaining target objects in the first electronic device refer to target objects in the first electronic device that do not belong to the target object group.

[0099] There are multiple ways to determine the projection position of the target object on the first image:

[0100] For example, according to the position relationship model and the first pose information, each target object outside the target object group in the first electronic device is projected onto the camera coordinate system of the second electronic device (the plane coordinate system where the first image is located) to obtain the projection position points of the remaining target objects in the first electronic device on the first image.

[0101] For another example, the projection position points of the remaining target objects in the first electronic device on the first image may be determined according to the mapping relationship between each target object in the target object group in the first electronic device and the target object in the first image, the position relationship model, and the first pose information. The second pose information may be determined based on the projection position points of the remaining target objects in the first electronic device on the first image and the three-dimensional coordinate information of each remaining target object in the first electronic device in combination with algorithms such as PNP.

[0102] For example, the mapping relationship between the projection position points of each remaining target object and each remaining target object in the electronic device can be combined, based on the projection position points of the remaining target objects in the first electronic device on the first image and the three-dimensional coordinate information of each remaining target object in the first electronic device, and using algorithms such as PNP to determine the second position information. Among them, the implementation principle of determining the second posture information is similar to that of determining the first posture information, which will not be repeated here. It should be noted that step S404 is an optional step, and its purpose is to be able to use the projection position points of the first electronic device that are not projected onto the first image to recalculate the second posture information of the first electronic device relative to the second electronic device after determining the first posture information, so as to improve the accuracy of the determined posture information. Depending on the requirements for the accuracy of the posture information, step S404 may be executed or not.

[0103] It is understandable that, in order to improve accuracy, step S404 can also be to determine the projection position points of each target object in the first electronic device in the first image based on at least the position relationship model and the first pose information. Then, based on the position relationship model and the mapping relationship between the projection position points of each target object in the first electronic device in the first image, determine the second pose information of the first electronic device relative to the second electronic device.

[0104] Among them, after the projection position points of each target object of the first electronic device on the first image are determined, the mapping relationship between each projection position point and each target object in the first electronic device is also determined. Therefore, based on the mapping relationship, the two-dimensional coordinate information (projection position point) of each target object in the first image and the position relationship model, the second posture information can be determined using algorithms such as PNP.

[0105] For ease of understanding, the process of determining the first pose information based on the target object group is described below using an implementation method as an example.

[0106] like Figure 7 As shown, it shows another implementation flow diagram of a method for determining a posture in an embodiment of the present application. The method of this embodiment may include:

[0107] S701: Obtain a current first image of a first electronic device.

[0108] The first electronic device is capable of controlling the virtual scene output by the second electronic device.

[0109] A plurality of target objects are arranged on the first electronic device.

[0110] In the present application, the first image includes at least four target objects.

[0111] S702: Determine at least three first target objects for pose calculation and at least one second target object for pose verification from at least four target objects in the first image.

[0112] For example, at least three target objects can be randomly selected from multiple target objects in the first image as first target objects for calculating posture. Correspondingly, all or part of the target objects other than the first target object in the first image can be used as second target objects for posture verification to obtain at least one second target object.

[0113] Optionally, at least three adjacent first target objects may be selected according to the relative position relationship of the at least four target objects in the first image, so as to reduce the difficulty of subsequent matching of the first target object group.

[0114] S703: Determine a first target object group and a second target object group included in the first image based on the position relationship model and the relative position relationship between the target objects in the first image.

[0115] The position relationship model can at least characterize the three-dimensional distribution of each target object in the first electronic device, and details can be found in the above related introduction.

[0116] The first target object group includes: at least three target objects in the first electronic device that match the at least three first target objects. That is, the first target object group included in the first image is each target object in the first electronic device that matches the at least three first target objects. The number of target objects in the first target object group is the same as the number of the at least three first target objects.

[0117] For example, taking obtaining three first target objects as an example, based on the relative position relationship of the three target objects in the first image, it is determined that the arrangement pattern of the three target objects in the first image is consistent with the arrangement pattern of target object 1, target object 2 and target object 3 in the first electronic device, then it can be determined that the target object group contained in the first image is target object 1, target object 2 and target object 3 in the first electronic device.

[0118] Similarly, the second target object group includes at least one target object in the first electronic device that matches the at least one second target object.

[0119] Among them, the first target object group has a first relative position relationship.

[0120] It can be understood that after determining the first target object group and the second target object group included in the first image, the three-dimensional coordinate information of each target object in the first target object group and the second target object group can be obtained from the position relationship model.

[0121] S704: Determine at least one set of first posture information of the first electronic device relative to the second electronic device based at least on the first relative position relationship of each target object in the first target object group and the three-dimensional coordinate information of each target object in the first image.

[0122] For example, based on the first relative position relationship of each target object in the first target object group and the three-dimensional coordinate information corresponding to each target object in the first target object group contained in the first image, at least one set of first pose information is determined.

[0123] The method of calculating the first pose information in this embodiment is similar to the previous one.

[0124] It is understandable that when determining the pose information based on the two-dimensional coordinates and three-dimensional coordinates of multiple object points, if the number of object points is only three, then multiple sets of pose information will be determined. For example, when calculating the pose information based on the PNP algorithm, if there are three feature points (corresponding to the first target object in the first target object group in this embodiment), the value of N is 3, that is, using the P3P algorithm, 4 sets of pose information can be determined; and when there are more than three feature points, a set of pose information can be uniquely determined. Therefore, when the first target object group includes at least three target objects, at least one set of pose information can be determined.

[0125] S705, performing projection verification on at least one set of first pose information based on each target object in the second target object group and the position relationship model, and determining correct pose information in the at least one set of first pose information according to the projection verification result.

[0126] During projection verification, for each first pose information, the projection position points of each target object belonging to the second target object group in the first electronic device can be determined on the first image based on the first pose information of the first electronic device and the position relationship model, and the projection position points of each target object in the first target object group in the first electronic device can be compared with their actual positions in the first image to see whether they are consistent.

[0127] It can be understood that, for a certain first pose information, if the projection position points of each target object in the second target object group in the first electronic device are consistent with the position points of each target object in the second target object group contained in the first image, then it is indicated that the first pose information is correct; otherwise, the first pose information is considered to be wrong. Based on this, the correct pose information can be determined from at least one set of calculated first pose information, and the verified correct pose information can be determined as the pose information of the first electronic device relative to the second electronic device.

[0128] It can be seen that in this embodiment, in addition to determining the first target object group for calculating posture contained in the first electronic device, a second target object group for posture verification will also be determined. On this basis, after multiple groups of first posture information are determined based on the first target object group, the second target object group will also be used to perform projection verification on multiple groups of first posture information.

[0129] It can be understood that if the relative positional relationships between any three target objects in the first electronic device are not repeated with the relative positional relationships between at least three other target objects, then when the first image contains at least four target objects, the first target object group in the first electronic device that matches the at least three first target objects in the first image can be uniquely determined based on the at least three first target objects in the first image.

[0130] For example, the relative positional relationship between any target object in the first electronic device and other electronic devices is different. Therefore, based on the first relative positional relationship of at least three first target objects in the first image, the first target object group matching the first relative positional relationship of the at least three first target objects in the first electronic device can be directly and accurately determined. Figure 5 and Figure 6 ,based on Figure 6 The first image shown can uniquely determine which target objects in the first electronic device are contained in the first image.

[0131] However, if there is duplication in the relative positional relationships between different target objects in the first electronic device, then based on the relative positional relationships of at least three first target objects in the first image, multiple matching first target object groups may be determined on the first electronic device.

[0132] For example, the first electronic device is a ring handle and the target object set on it is as follows Figure 2 and 3 As shown. Figure 2 and 3It can be seen that the first electronic device is provided with a plurality of large LED lights (or fully lit LEDs) and a plurality of small LEDs (or half-lit LEDs), and each large LED light is surrounded by four small LED lights. Therefore, the arrangement of LEDs around any large LED is the same. Therefore, based on the first image, it is impossible to determine with certainty and uniqueness which LED lights on the electronic device constitute the first target object group contained in the first image. Figure 8 , which shows Figure 2 The first image of an electronic device. Figure 1 , 2 and Figure 8 It can be seen that the first electronic device meets the Figure 8 There are many possibilities for the LED lamp groups with relative positional relationships between the individual LED lamps shown.

[0133] In the case where there may be multiple first target object groups in the first electronic device, each time after determining the first target object group that may be included in the first image, the above steps S704 and S705 can be performed. It can be understood that if the first target object group included in the first image currently determined is wrong, then it means that at least one first target object actually included in the first image does not correspond to the target object in the first target object group in the electronic device. On this basis, through the projection verification of the second target object group, it can be obtained that at least one set of first pose information obtained based on the first target object group is incorrect, so that the first target object group and the second target object group can be re-determined.

[0134] Correspondingly, if it is determined according to the projection verification result that the correct posture information does not exist in the at least one group of first posture information, the process returns to step S704 to re-determine the first target object group and the second target object group actually contained in the first image from the multiple target objects of the first electronic device.

[0135] In a possible implementation, in any one of the above embodiments of the present application, if the number of target objects in the first image is less than a set number, third posture information determined by the inertial measurement device in the first electronic device is obtained, and the third posture information is determined as the posture information of the first electronic device relative to the second electronic device.

[0136] The inertial measurement device may include a device such as a gyroscope that can determine the position and posture.

[0137] For example, the set number may be four. If the number of target objects in the first image is less than four, it may be impossible to use algorithms such as PNP for calculating two-dimensional to three-dimensional posture information to determine the posture information of the first electronic device relative to the second electronic device.

[0138] It can be understood that in any of the above embodiments of the present application, the present application can also obtain two images at the same time and determine the posture information based on the two images respectively.

[0139] Specifically, the present application can also obtain the second image currently displayed by the first electronic device. For example, the second electronic device can be provided with two image acquisition devices, and the two image acquisition devices respectively acquire the first image and the second image. Of course, the second image and the first image can also be acquired by the same image acquisition device, and the second image can be a frame of image adjacent to the first image.

[0140] On this basis, the fourth position information of the first electronic device relative to the second electronic device can be determined based at least on the second relative position relationship between the target objects in the second image and the position relationship model. In order to facilitate distinction, the position information of the first electronic device relative to the second electronic device determined based on the second image is referred to as the fourth position information.

[0141] Correspondingly, input control data for the second electronic device can be determined based on the first posture information and the fourth posture information to improve the accuracy of input control.

[0142] Corresponding to a posture determination method of the present application, the present application also provides a posture determination device. Fig. 9 As shown, it shows another schematic diagram of the composition structure of a posture determination device of the present application. The device of this embodiment may include:

[0143] An image obtaining unit 901 is used to obtain a current first image of a first electronic device, the first electronic device can control a virtual scene output by a second electronic device, and a plurality of target objects are arranged on the first electronic device;

[0144] The first posture determination unit 902 is used to determine the first posture information of the first electronic device relative to the second electronic device based on at least the relative position relationship between the target objects in the first image and a position relationship model created based on the three-dimensional coordinate information of each target object of the first electronic device.

[0145] In a possible implementation, the first pose determination unit includes:

[0146] a group positioning subunit, configured to determine a target object group included in the first image based on a relative position relationship between target objects in the first image and the position relationship model, wherein target objects in the target object group have a first relative position relationship with each other, and wherein the position relationship model is capable of at least characterizing a three-dimensional distribution of target objects in the first electronic device;

[0147] The first posture determination subunit is used to determine the first posture information of the first electronic device relative to the second electronic device based on at least the first relative position relationship and the three-dimensional coordinate information of each target object in the first image.

[0148] Optionally, the first posture determination subunit is specifically used to determine the first posture information of the first electronic device relative to the second electronic device based at least on the first relative position relationship and the three-dimensional coordinate information of each target object in the first image, and using a perspective projection algorithm.

[0149] Optionally, the device may further include:

[0150] A second posture determination unit is used to determine the projection position points of the remaining target objects in the first electronic device on the first image based at least on the position relationship model and the first posture information, and to determine the second posture information of the first electronic device relative to the second electronic device according to the projection position points and the three-dimensional coordinate information corresponding to the remaining target objects.

[0151] In yet another possible implementation manner, the first image obtained by the image obtaining unit includes at least four target objects;

[0152] The assembly positioning subunit comprises:

[0153] an object classification subunit, configured to determine at least three first target objects for pose calculation and at least one second target object for pose verification from the at least four target objects in the first image;

[0154] a group determination subunit, configured to determine, based on the position relationship model and the relative position relationship between the target objects in the first image, a first target object group and a second target object group included in the first image, wherein the first target object group includes at least three target objects in the first electronic device that match the at least three first target objects, the second target object group includes at least one target object in the first electronic device that matches the at least one second target object, and the first target object group has a first relative position relationship;

[0155] The first pose determination subunit is specifically configured to determine at least one set of first pose information of the first electronic device relative to the second electronic device based at least on the first relative position relationship of each target object in the first target object group and the three-dimensional coordinate information of each target object in the first image;

[0156] The device may also include:

[0157] A posture verification unit is used to perform projection verification on the at least one set of first posture information based on each target object in the second target object group and the position relationship model, and determine the correct posture information in the at least one set of first posture information according to the projection verification result.

[0158] Optionally, the device may further include:

[0159] A group re-determination unit is used to return to execute the group determination sub-unit operation to re-determine the first target object group and the second target object group included in the first image from the multiple target objects of the first electronic device if it is determined based on the projection verification result that correct posture information does not exist in the at least one group of first posture information.

[0160] In another possible implementation, in any one of the above device embodiments, the device may further include:

[0161] A third posture determination unit is used to obtain third posture information determined by the inertial measurement device in the first electronic device if the number of target objects in the first image is less than a set number, and determine the third posture information as the posture information of the first electronic device relative to the second electronic device.

[0162] In another possible implementation, in any one of the above device embodiments, the device may further include:

[0163] an auxiliary image obtaining unit, configured to obtain a current second image of the first electronic device;

[0164] a fourth position determination unit, configured to determine fourth position information of the first electronic device relative to the second electronic device based at least on a second relative position relationship between target objects in the second image and the position relationship model;

[0165] An input control unit is used to determine input control data for the second electronic device based on the first posture information and the fourth posture information.

[0166] On the other hand, the present application also provides an electronic device, for example, the electronic device may be the specific structure of the second electronic device mentioned above. Fig.10 As shown, it shows a schematic diagram of the composition structure of an electronic device of the present application. The electronic device of this embodiment at least includes: a processor 1001 and a memory 1002.

[0167] The processor is used to execute the posture determination method of any one of the above embodiments.

[0168] The memory is also used to store programs that the processor needs to perform operations.

[0169] It is understandable that the electronic device may also include other components, such as Fig.10 As shown, the electronic device may further include a display 1003, an input device 1004 connected to the electronic device, and a communication bus 1005. The processor, the memory, the display, and the input device may be connected via a communication bus.

[0170] Of course, electronic equipment may also include, for example Fig.10 There may be more or fewer components, without limitation.

[0171] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining a posture, comprising: Obtaining a current first image of a first electronic device, the first electronic device being capable of controlling a virtual scene output by a second electronic device, and a plurality of target objects being arranged on the first electronic device; Based at least on the relative position relationship between the target objects in the first image and a position relationship model created based on the three-dimensional coordinate information of each target object of the first electronic device, determine the current first position information of the first electronic device relative to the second electronic device, wherein the relative position relationship between the target objects in the first image reflects the arrangement of the target objects in the first image; and the position relationship model is created based on the three-dimensional coordinate information of each target object relative to the first electronic device.

2. The method according to claim 1, wherein determining the first position information of the first electronic device relative to the second electronic device based on at least the relative position relationship between the target objects in the first image and the position relationship model created based on the three-dimensional coordinate information of each target object of the first electronic device comprises: Based on the relative positional relationship between the target objects in the first image and the positional relationship model, a target object group included in the first image, target objects in the target object group having a first relative positional relationship, and the positional relationship model being capable of at least characterizing a three-dimensional distribution of the target objects in the first electronic device; Based at least on the first relative position relationship and the three-dimensional coordinate information of each target object in the first image, the first position information of the first electronic device relative to the second electronic device is determined.

3. The method according to claim 2, further comprising: The projection position points of the remaining target objects in the first electronic device on the first image are determined at least based on the position relationship model and the first posture information, and the second posture information of the first electronic device relative to the second electronic device is determined according to the projection position points and the three-dimensional coordinate information corresponding to the remaining target objects.

4. The method according to claim 2 or 3, wherein the first image includes at least four target objects; The determining, based on the relative positional relationship between the target objects in the first image and the positional relationship model, a target object group included in the first image includes: Determining at least three first target objects for pose calculation and at least one second target object for pose verification from the at least four target objects in the first image; Based on the positional relationship model and the relative positional relationship between the target objects in the first image, determining a first target object group and a second target object group included in the first image, wherein the first target object group includes at least three target objects in the first electronic device that match the at least three first target objects, the second target object group includes at least one target object in the first electronic device that matches the at least one second target object, and the first target object group has a first relative positional relationship; The determining, based at least on the first relative position relationship and the three-dimensional coordinate information of each target object in the first image, the first posture information of the first electronic device relative to the second electronic device includes: Determine at least one set of first posture information of the first electronic device relative to the second electronic device based at least on a first relative position relationship of each target object in the first target object group and three-dimensional coordinate information of each target object in the first image; The method further comprises: Projection verification is performed on the at least one set of first pose information based on each target object in the second target object group and the position relationship model, and correct pose information in the at least one set of first pose information is determined according to the projection verification result.

5. The method according to claim 4, further comprising: If it is determined according to the projection verification result that correct posture information does not exist in the at least one group of first posture information, the operation of determining the first target object group and the second target object group contained in the first image is returned to re-determine the first target object group and the second target object group contained in the first image from the multiple target objects of the first electronic device.

6. The method according to claim 2, wherein determining the first position information of the first electronic device relative to the second electronic device at least based on the first relative position relationship and the three-dimensional coordinate information of each target object in the first image comprises: Based at least on the first relative position relationship and the three-dimensional coordinate information of each target object in the first image, and using a perspective projection algorithm, the first position information of the first electronic device relative to the second electronic device is determined.

7. The method according to claim 1, further comprising: If the number of target objects in the first image is less than the set number, third posture information determined by the inertial measurement device in the first electronic device is obtained, and the third posture information is determined as the posture information of the first electronic device relative to the second electronic device.

8. The method according to claim 1, further comprising: Obtaining a current second image of the first electronic device; Determining fourth position information of the first electronic device relative to the second electronic device based at least on a second relative position relationship between target objects in the second image and the position relationship model; Based on the first posture information and the fourth posture information, input control data for the second electronic device is determined.

9. A posture determination device, comprising: An image obtaining unit, used to obtain a current first image of a first electronic device, the first electronic device can control a virtual scene output by a second electronic device, and a plurality of target objects are arranged on the first electronic device; A posture determination unit is used to determine the current first posture information of the first electronic device relative to the second electronic device based on at least the relative position relationship between the target objects in the first image and a position relationship model created based on the three-dimensional coordinate information of each target object of the first electronic device, wherein the relative position relationship between the target objects in the first image reflects the arrangement of the target objects in the first image; and the position relationship model is created based on the three-dimensional coordinate information of each target object relative to the first electronic device.

10. An electronic device comprising: Memory and processor; The processor is used to execute the posture determination method as described in any one of claims 1 to 8 above; The memory is used to store programs required for the processor to perform operations.

Images