The method for positioning the handle and the method and device for controlling the handle indicator lights.

Abstract

This application discloses a method for positioning a handle and a method and device for controlling handle indicator lights, belonging to the field of XR technology. The method for positioning the handle includes: acquiring a panel image of the handle captured by a camera; acquiring the identifiers of multiple target indicator lights currently illuminated on the handle panel, wherein the handle panel has multiple indicator lights arranged in a predetermined shape on the panel, and the multiple indicator lights are evenly distributed according to the predetermined shape, and the target indicator light is one of the multiple indicator lights; determining the pose of the handle based on first position information of the multiple target indicator lights in the panel image and the identifiers of the multiple target indicator lights.

Description

Technical Field

[0001] This application belongs to the field of XR technology, specifically relating to a method for positioning a handle and a method and device for controlling the handle indicator light. Background Technology

[0002] As XR technology gradually develops, its controller tracking technology is also constantly being updated to meet people's needs.

[0003] Among the currently popular gamepads, the ring-type gamepad is the most mainstream solution, but its structural design is bulky. Although the active tracking gamepad has a compact structure, its production cost is too high. Summary of the Invention

[0004] The purpose of this application is to provide a method for positioning a handle and a method and apparatus for controlling the handle indicator light, which can at least solve one of the problems of lower production cost and more compact structure than existing handles, while maintaining the original positioning and tracking performance as much as possible.

[0005] In a first aspect, embodiments of this application provide a method for positioning a handle, the method comprising:

[0006] The system acquires an image of the controller's panel captured by a camera; it acquires the identifiers of multiple target indicator lights currently illuminated on the controller's panel, wherein the controller's panel has multiple indicator lights arranged in a predetermined shape and uniformly distributed according to the predetermined shape, and the target indicator light is one of the multiple indicator lights; based on the first position information of the multiple target indicator lights in the panel image and the identifiers of the multiple target indicator lights, the system determines the pose of the controller.

[0007] Secondly, embodiments of this application provide a handle positioning device, the device comprising:

[0008] The first acquisition module is used to acquire a panel image of the handle captured by a camera; the second acquisition module is used to acquire the identifiers of multiple target indicator lights currently illuminated on the panel of the handle, wherein multiple indicator lights are provided on the panel of the handle, the multiple indicator lights form a predetermined shape on the panel, and the multiple indicator lights are evenly distributed according to the predetermined shape, and the target indicator light is one of the multiple indicator lights; the determination module is used to determine the pose of the handle based on the first position information of the multiple target indicator lights in the panel image and the identifiers of the multiple target indicator lights.

[0009] Thirdly, this application provides a control method for a handle indicator light, including: predicting multiple target indicator lights to be lit next based on the current position of the handle panel, wherein multiple indicator lights are provided on the handle panel, the multiple indicator lights form a predetermined shape on the panel, and the multiple indicator lights are evenly arranged according to the predetermined shape, and the target indicator light is one of the multiple indicator lights;

[0010] The controller illuminates the plurality of target indicator lights on the control panel and turns off the other indicator lights.

[0011] Fourthly, embodiments of this application provide a control device for a handle indicator light, comprising: a prediction module, configured to predict, based on the current position of the handle's panel, a plurality of target indicator lights to be illuminated next, wherein the handle's panel is provided with a plurality of indicator lights, the plurality of indicator lights forming a predetermined shape on the panel, and the plurality of indicator lights being evenly distributed according to the predetermined shape, and the target indicator light being one of the plurality of indicator lights; and a control module, configured to control the illumination of the plurality of target indicator lights on the handle's panel, and to control the extinguishing of the indicator lights other than the plurality of target indicator lights.

[0012] Fifthly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions being executed by the processor to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the third aspect.

[0013] In a sixth aspect, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect, or implement the steps of the method described in the third aspect.

[0014] In a seventh aspect, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect, or to implement the steps of the method as described in the third aspect.

[0015] Eighthly, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect, or to implement the steps of the method described in the third aspect.

[0016] In this embodiment, an image of the handle's panel captured by a camera can be acquired, and the identifiers of multiple target indicator lights currently illuminated on the handle's panel can be obtained. The handle's panel has multiple indicator lights arranged in a predetermined shape, and these lights are evenly distributed according to the predetermined shape. The target indicator light is one of these indicator lights. Furthermore, the handle's pose can be determined based on the first position information of the multiple target indicator lights in the panel image and their identifiers. Because the indicator lights in this embodiment are evenly distributed, the manufacturing process is simplified, thus reducing production costs. Additionally, geometric constraints are introduced during pose positioning, which compensates for the accuracy degradation caused by the evenly distributed structure, ensuring excellent positioning and tracking capabilities for the handle.

[0017] In addition, the control method for the handle indicator light provided in this application embodiment reduces the overall power consumption of the LED and extends the battery life by introducing a dynamic lighting control mechanism. Attached Figure Description

[0018] Figure 1 This is a flowchart of a handle positioning method provided in an exemplary embodiment of this application;

[0019] Figure 2a This is a schematic diagram of an exemplary embodiment of this application showing the indicator light illuminated;

[0020] Figure 2b This is a schematic diagram illustrating another indicator light illumination provided in an exemplary embodiment of this application;

[0021] Figure 3 This is another flowchart of a handle positioning method provided in an exemplary embodiment of this application;

[0022] Figure 4a This is a schematic diagram illustrating the definition of the P3P problem provided in an exemplary embodiment of this application;

[0023] Figure 4b This is a schematic diagram of planar geometric constraints under spatial projection provided in an exemplary embodiment of this application;

[0024] Figure 5 This is a schematic diagram of a handle positioning and tracking process according to an exemplary embodiment of this application;

[0025] Figure 6a This is a schematic diagram of an exemplary embodiment of the present application providing an indicator light layout;

[0026] Figure 6b This is a schematic diagram of planar geometric constraints under spatial projection provided in another exemplary embodiment of this application;

[0027] Figure 7a Another exemplary embodiment of this application provides a schematic diagram of an indicator light layout;

[0028] Figure 7b This is a schematic diagram of planar geometric constraints under spatial projection provided in another exemplary embodiment of this application;

[0029] Figure 8 Another exemplary embodiment of this application provides a schematic diagram of an indicator light layout.

[0030] Figure 9 This is a flowchart of a control method for a handle indicator light provided in an exemplary embodiment of this application;

[0031] Figure 10 This is a flowchart of another control method for the handle indicator light provided in an exemplary embodiment of this application;

[0032] Figure 11 This is a schematic diagram of a handle positioning device provided in an exemplary embodiment of this application;

[0033] Figure 12 This is a schematic diagram of a control device for a handle indicator light provided in an exemplary embodiment of this application;

[0034] Figure 13 This is a schematic diagram of an electronic device provided in an exemplary embodiment of this application;

[0035] Figure 14 This is a schematic diagram of the hardware structure of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0037] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0038] The positioning method for the handle and the control method for the indicator light provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0039] Figure 1 This application illustrates an exemplary embodiment of a controller positioning method 100, which can be executed by a terminal device. The terminal device can be a virtual reality device, such as a VR headset, or it can be a personal terminal device such as a mobile phone or computer connected to a virtual reality device. Specific embodiments of this application are not limited to this method. Figure 1 As shown, the method mainly includes the following steps:

[0040] S101: Acquire the panel image of the handle captured by the camera.

[0041] In practical applications, the camera on the helmet can capture an image of the handle's panel. This image contains crucial information for locating the handle, and the handle information can be obtained from the content displayed on the panel image to perform further calculations.

[0042] S102: Obtain the identifiers of the multiple target indicator lights currently illuminated on the panel of the handle.

[0043] In this embodiment of the application, the handle panel is provided with a plurality of indicator lights, the plurality of indicator lights are arranged in a predetermined shape on the panel, and the plurality of indicator lights are evenly distributed according to the predetermined shape, and the target indicator light is one of the plurality of indicator lights.

[0044] Specifically, the multiple target indicator lights are asymmetrically distributed. In a specific implementation, to avoid the pattern formed by the illuminated target indicator lights being symmetrical, such as... Figure 2a As shown, this layout results in the illuminated target indicator lights forming a symmetrical shape. No matter how the panel rotates, it cannot differ from its initial state, leading to ambiguous solutions in the next pose calculation and ultimately inaccurate handle positioning. And as... Figure 2bAs shown, an asymmetrical distribution of target indicator lights avoids this problem. For example, the total number of indicator lights is 13, and the number of target indicator lights is 4. In practice, this can be changed according to specific circumstances. This asymmetrical layout ensures that the identification of the target indicator lights is unambiguous, leading to the correct positioning result.

[0045] Optionally, the predetermined shape includes a circle, and the plurality of indicator lights are evenly distributed along the periphery of the circle. In this embodiment, a common circle is selected as the predetermined shape, such as... Figure 2b As shown, the indicator lights form a circle, with 13 lights evenly distributed around its perimeter. In practical applications, they are activated in turn in groups to provide the light spot signal needed for visual positioning by an external infrared camera. However, this is not a limitation; in practical applications, the predetermined shape can also be other shapes, such as ellipses.

[0046] S103: Determine the pose of the handle based on the first position information of the plurality of target indicator lights in the panel image and the identifiers of the plurality of target indicator lights.

[0047] The first position information of the target indicator light is the projection relationship between two coordinate systems. The identifier of the target indicator light is the positional relationship between the target indicator lights. With the addition of planar geometric constraints, a circle is used here, so the formula for a circle is added. The PnP series algorithm used for non-divisional design is modified accordingly, and the pose algorithm of the divisional design is calculated to obtain the rotation and translation of the handle relative to the camera, that is, the pose of the handle.

[0048] In this embodiment, the handle is positioned by acquiring an image of the handle's panel captured by a camera, and then obtaining the identifiers of multiple target indicator lights currently illuminated on the handle's panel. The handle's panel has multiple indicator lights arranged in a predetermined shape, and these indicator lights are uniformly distributed according to the predetermined shape. The target indicator light is one of these indicator lights. This uniform distribution is easier to manufacture than a non-uniform distribution, reducing production costs and improving yield. Finally, the handle's pose is determined based on the first position information of the multiple target indicator lights in the panel image and the identifiers of these target indicator lights. Incorporating the position information of the target indicator lights during pose calculation adds additional geometric constraints, which to some extent compensates for the accuracy degradation caused by the uniform layout structure of the handle, ensuring the handle's positioning and tracking function as much as possible.

[0049] Figure 3A flowchart illustrating another handheld controller positioning method according to an exemplary embodiment of this application is shown. This method 300 can be executed by a terminal device, which can be a virtual reality device, such as a VR headset, or a personal terminal device such as a mobile phone or computer connected to a virtual reality device. Specific embodiments of this application are not limited to this method. Figure 3 As shown, the method mainly includes the following steps:

[0050] S301: Acquire the panel image of the handle captured by the camera.

[0051] This step is the same as Figure 1 For details regarding S101, please refer to the relevant descriptions above, which will not be repeated here.

[0052] S302: Obtain the identifiers of the multiple target indicator lights currently illuminated on the panel of the handle.

[0053] The handle has multiple indicator lights on its panel, which are arranged in a predetermined shape and are evenly distributed according to the predetermined shape. The target indicator light is one of the multiple indicator lights.

[0054] This step is the same as Figure 1 For details regarding S102, please refer to the relevant descriptions above, which will not be repeated here.

[0055] In this embodiment of the application, a circle is used as an example for illustration.

[0056] S303: Based on the identifiers of the plurality of target indicator lights, obtain second position information of the plurality of target indicator lights relative to the center of the circle;

[0057] By obtaining the identifier of the target indicator light in the previous step, the position information of the target indicator light relative to the center of the circle is obtained. In this embodiment of the application, during the movement of the handle, the second position information of the multiple target indicator lights relative to the center of the circle remains unchanged. Therefore, the pose of the handle can be located based on the second position information and the first position information of the multiple target indicator lights on the panel image.

[0058] S304: Constructing the objective function O i ′ (θ):

[0059]

[0060] Among them, u i and v i For the position information of the i-th target indicator light in the panel image, π θ (p i) represents the first position information of the i-th target indicator light, e θ (·) is the equation representing the ellipse formed by the projection of the circle onto the panel image.

[0061] In this embodiment, in order to adapt to the structure of the handle panel, geometric constraints were introduced and corresponding modifications were made based on the original non-divided PnP algorithm.

[0062] Taking P3P as an example, a typical PnP problem can be summarized as follows: Given the relative positional relationship between three 3D spatial coordinate points, and the 2D coordinates of the projection of these three points onto the camera's imaging plane, find the rotation and translation of the spatial coordinate system about the camera. Figure 4a This demonstrates a typical application scenario for P3P (n=3) pose calculation.

[0063] In the application scenario of lamp ring handle positioning, the world coordinate system in the figure is generally aligned with the handle's IMU (Inertial Measurement Unit), that is, the local coordinate system of the IMU device is used as the world coordinate system. The calculated pose is the rotation and translation of the handle relative to the helmet camera coordinate system.

[0064] The general solution process can be summarized as follows: construct the projection equation using the correspondence between 2D image coordinates and 3D world coordinates; construct the objective function based on the projection equation; and solve for the pose parameters.

[0065] refer to Figure 4a World coordinates can be represented as Image coordinates can be represented as The projection equation from 3D to 2D can then be written as:

[0066]

[0067] Where, λ i Let [R|t] be the scale and [R|t] be the pose matrix. Since R and t can each be described by three parameters, the projection relationship can be further simplified to π. θ (p i ) = R θ ·p i +t θ Where θ is the pose parameter, the objective function can be constructed as follows:

[0068]

[0069] The objective function can be solved by optimization or by direct linear transformation.

[0070] The equally divided design in this embodiment introduces a special geometric shape, allowing the projection relationship of the light spot to satisfy additional constraints. By adding extra constraint terms to the original PnP cost function to construct the objective function, both the stability and accuracy of the solution are improved.

[0071] Since the intended shape is circular, all indicator lights fall on a circle of fixed radius; therefore, their coordinates must satisfy the equation of a plane circle:

[0072]

[0073] Without loss of generality, we can assume that the center of the panel circle overlaps with the camera coordinate system, such as Figure 4b p i_ref The defined circle is shown. (From [R]) θ |t θ Define the pose of the handle, i.e., the pose of the spatial circular panel. In this configuration, the projection of the spatial circle onto the normalized plane π can be considered as the oblique tangency of the triangular pyramid op1p2p3 to plane π, which can be described using the conic section equation, i.e. Figure 4b The ellipse e in θ The equation. Correspondingly, b i The image coordinates represented by the ellipse must satisfy the equation of the ellipse, i.e., e θ ([u i v i ] T = 0. Combining the expression for the objective function in the P3P problem of non-equilibrium design, a new objective function can be defined as:

[0074]

[0075] e θ The expression can be derived from the formula for a plane geometrically constrained circle under spatial projection via [R] θ |t θ The result is obtained by normalization after spatial transformation.

[0076] S305: Minimize the objective function to obtain the target value of the pose parameter.

[0077] The objective function O obtained in the previous step i i By minimizing (θ), the target value of the pose parameter θ can be obtained through calculation.

[0078] S306: Based on the target value, obtain the pose of the handle.

[0079] The pose may include: [R] θ |t θ ], the R θThe t parameter is used to represent the rotation of the panel of the handle relative to the camera. θ The panel used to indicate the displacement of the handle relative to the camera.

[0080] By using the target value of θ calculated in the previous step, we can obtain the rotation and displacement of the handle's panel relative to the camera, that is, obtain the handle's pose.

[0081] By obtaining the pose of the controller, controller positioning can be achieved; for example, it can be done according to... Figure 5 The visual-inertial fusion algorithm shown is used for the positioning and tracking of the handle. For example... Figure 5 As shown, the state of the handle can be predicted based on the handle's acceleration, angular velocity, geomagnetic field strength, and state variables (e.g., position, velocity, attitude, error offset, etc.). (For example, the prediction can be output to a prediction model for prediction.) Then, the prediction results and the handle's structural geometric parameters are combined to obtain the reprojection of the handle on the imaging plane. The reprojected image is then corrected based on image observations (e.g., two-dimensional coordinate points) to obtain the prediction / observation error. The covariance is then updated based on the obtained prediction / observation error to obtain a correction estimate. The state variables of the handle are adjusted through the correction estimate. This process is repeated to achieve the positioning and tracking of the handle.

[0082] Although the above embodiments use a circle as an example for illustration, it is not limited to this. In specific applications, the predetermined shape can also be other shapes.

[0083] For example, the predetermined shape can be an ellipse, and the setting of each indicator light can weaken the constraints to the form of equal division of the central angle of the ellipse, such as... Figure 6a As shown, the projection of the spatial ellipse onto the imaging plane is as follows: Figure 6b As shown, the projected ellipse can also be denoted as e. θ In addition, the placement of each indicator light also takes into account angular constraints, such as... Figure 7a As shown, the projection of the spatial ellipse onto the imaging plane is as follows: Figure 7b As shown.

[0084] In this embodiment, the elliptical panel is divided into K = 4 segments; the central angle corresponding to the arc length of each segment is an integer multiple of the bisector angle r, for example, Figure 8 In the middle, ∠p i Op i+1 =2r=α i Assume the projection relationship caused by the current controller pose is as follows: Then we have:

[0085]

[0086] For Δp i ′ O′ p i ′ +1 have: Right now

[0087]

[0088] Abbreviated as In equation (7), β i For the central angle α i In projection relationship The following provides geometric constraints. Equation (7) can be added after equation (5) and denoted as:

[0089]

[0090] Equation (8) is the objective function after introducing the included angle constraint.

[0091] By minimizing the objective function, the target value of θ can be calculated, and thus the pose of the handle can be obtained [R]. θ |t θ ].

[0092] For example, the predetermined shape can be a non-bounded polygon, such as... Figure 8 As shown, the coordinates of each indicator light no longer satisfy the equation of a certain conic section, then e θ The constraint no longer holds, and The included angle constraint still exists, so e in equation (8) can be... θ After removing the terms, the general objective function for the non-circumscribed polygon is:

[0093]

[0094] Using the objective function described above, the pose reference θ can be obtained, and thus the pose of the handle can be determined.

[0095] Figure 9 A flowchart illustrating a control method for a controller indicator light according to an exemplary embodiment of this application is shown. This method 900 can be executed by an electronic device, which can be a controller for controlling a VR device. Figure 9 As shown, the method mainly includes the following steps:

[0096] S901: Based on the current position of the handle's panel, predict the next target indicator light to be illuminated. The handle's panel has multiple indicator lights arranged in a predetermined shape, and these lights are evenly distributed according to the predetermined shape. The target indicator light is one of the multiple indicator lights.

[0097] In this embodiment, not all indicator lights will be captured by the camera during actual use. Based on the current position of the handle and the main light emission direction of the indicator lights, it can be predicted which indicator lights are more easily observed, and then corresponding adjustments can be made to the indicator light illumination control. Specifically, based on the predicted result, a subset is selected from the set of all indicator lights, and then the individual indicator lights that should be illuminated in the next frame are selected from the subset.

[0098] By adopting the above technical solution, the number of observed indicator lights, i.e., the target indicator lights, can be more controllable, thereby improving the stability of the solution. Under normal circumstances, some areas of the handle's panel may not be effectively captured by the camera. When the lighting control sequence does not perform predictive selection processing, it is very likely that some selected indicator lights will be located in unobservable areas, resulting in missing observations and posing a significant challenge to the calculation of visual pose. For example, if a lighting control sequence is preset to illuminate four indicator lights, but two indicator lights are located in unobservable areas due to shell obstruction, the "four-point correspondence" degenerates into "two-point correspondence," causing ambiguous solutions and a decrease in result accuracy. After introducing the predictive control mechanism, even with a certain deviation, the "four-point correspondence" will at most degenerate into "three-point correspondence," and the stability of the solution can still be effectively guaranteed.

[0099] Specifically, the multiple target indicator lights are asymmetrically distributed. The predetermined shape includes a circle, with the multiple indicator lights evenly distributed along the periphery of the circle. These two requirements echo the aforementioned handle positioning method, avoiding ambiguous interpretations and reducing manufacturing complexity. In particular, they ensure coordination between the handle positioning function and the indicator light control function, reducing power consumption and extending battery life.

[0100] S902: Control the multiple target indicator lights on the control handle panel to light up, and control the indicator lights other than the multiple target indicator lights to turn off.

[0101] Based on the prediction result, the target indicator light on the panel is illuminated, while all other indicator lights on the panel are turned off. This operation ensures that the panel ultimately displays the predicted target indicator light result, allowing precise control over the number of target indicator lights and ultimately avoiding ambiguous solutions and a decrease in the accuracy of the handle positioning result.

[0102] In this embodiment, based on the current position of the handle's panel, multiple target indicator lights are predicted to be illuminated next. Then, the multiple target indicator lights are controlled to illuminate, while the other indicator lights are controlled to turn off. This improves the controllability of the final result, avoids ambiguous solutions, and reduces resource waste. Furthermore, it complements the handle's positioning method, making the overall handle structure more complete.

[0103] Figure 10 A flowchart illustrating another control method for a handle indicator light, as shown in an exemplary embodiment of this application, is illustrated. The method mainly includes the following steps:

[0104] S1002: Based on the current position of the panel of the handle, predict the target position of the panel's next movement.

[0105] By combining the current position of the handle's panel with the handle's rotation and displacement, the target position of the panel's next movement can be predicted.

[0106] S1003: Based on the target location, determine multiple candidate indicator lights that can be captured by the camera of the terminal device from among the multiple indicator lights set on the panel.

[0107] By considering the target location, the main light emission direction of the indicator light, or the location of the indicator light captured by the camera, suitable candidate indicator lights are selected from all indicator lights.

[0108] S1004: Select the target indicator light to be lit next from the plurality of candidate indicator lights.

[0109] Based on the results of the candidate indicator lights determined in the previous step, the individual indicator lights that should be lit next are selected as the target indicator lights.

[0110] S1005: Send the identifiers of the multiple target indicator lights to the terminal device.

[0111] The target indicator light identifier obtained in the previous step is sent to the terminal device, so that the terminal device can know the identifier of the target indicator light to be lit next, thereby performing the pose positioning of the controller. The terminal device can be a VR device connected to the controller, such as a VR headset.

[0112] S1006: Control the multiple target indicator lights on the control handle panel to light up, and control the indicator lights other than the multiple target indicator lights to turn off.

[0113] This step is the same as Figure 9 For details regarding S902, please refer to the relevant descriptions above, which will not be repeated here.

[0114] The handle positioning method provided in this application can be executed by a handle positioning device. This application uses the handle positioning device executing the handle positioning method as an example to illustrate the handle positioning device provided in this application.

[0115] Figure 11 This invention provides a schematic diagram of a handle positioning device according to an embodiment of the present application. Figure 11 As shown, the device 1100 may include: a first acquisition module 1101, a second acquisition module 1102, and a determination module 1103.

[0116] In this embodiment, the first acquisition module 1101 is used to acquire a panel image of the handle captured by a camera. The second acquisition module 1102 is used to acquire the identifiers of multiple target indicator lights currently illuminated on the handle panel, wherein multiple indicator lights are provided on the handle panel, the multiple indicator lights are arranged in a predetermined shape on the panel, and the multiple indicator lights are evenly distributed according to the predetermined shape, and the target indicator light is one of the multiple indicator lights. The determining module 1103 is used to determine the pose of the handle based on the first position information of the multiple target indicator lights in the panel image and the identifiers of the multiple target indicator lights.

[0117] In this embodiment, the uniform layout of the indicator lights on the handle panel reduces manufacturing difficulty, lowers production costs, and improves yield. The handle's pose is then determined by the determining module 1103. The position information of the target indicator lights is incorporated into the pose calculation, adding additional geometric constraints. This mitigates the accuracy degradation caused by the uniform layout structure of the handle to some extent, ensuring the handle's positioning and tracking function as much as possible.

[0118] The control method for the handle indicator light provided in this application can be executed by a control device for the handle indicator light. This application uses an example of a control device for the handle indicator light executing the control method to illustrate the control device for the handle indicator light provided in this application.

[0119] Figure 12 This application provides a schematic diagram of the control device for a handle indicator light according to an embodiment of the present application. Figure 12 As shown, the device 1200 mainly includes a prediction module 1201 and a control module 1202.

[0120] In this embodiment of the application, the prediction module 1201 is used to predict the next target indicator lights to be lit based on the current position of the handle panel, wherein the handle panel is provided with multiple indicator lights, the multiple indicator lights are arranged in a predetermined shape on the panel, and the multiple indicator lights are evenly distributed according to the predetermined shape, and the target indicator light is one of the multiple indicator lights; the control module 1202 is used to control the multiple target indicator lights on the handle panel to light up, and control the indicator lights other than the multiple target indicator lights to turn off.

[0121] In this embodiment, the indicator light's illumination and extinguishing are controlled by the prediction module 1201 and the control module 1202. This improves the controllability of the final result, avoids ambiguous solutions, and reduces resource waste. Furthermore, it complements the handle's positioning method, making the overall handle structure more complete.

[0122] The positioning device for the handle and the control device for the handle indicator light in this embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or any other device besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This embodiment does not specifically limit the functionality of the electronic device.

[0123] The positioning device for the handle and the control device for the handle indicator light in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit the specific operating system.

[0124] The handle positioning device provided in this application embodiment can achieve Figure 1 and Figure 3 The various processes implemented in the method embodiments enable the control device for the handle indicator light to achieve... Figure 9 and Figure 10 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0125] Optionally, such as Figure 13As shown, this application embodiment also provides an electronic device 1300, including a processor 1301 and a memory 1302. The memory 1302 stores a program or instructions that can run on the processor 1301. When the program or instructions are executed by the processor 1301, they implement the various steps of the above-mentioned handle positioning method or handle indicator light control method embodiment, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0126] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0127] Figure 14 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0128] The electronic device 1400 includes, but is not limited to, components such as: radio frequency unit 1401, network module 1402, audio output unit 1403, input unit 1404, sensor 1405, display unit 1406, user input unit 1407, interface unit 1408, memory 1409, and processor 1410.

[0129] Those skilled in the art will understand that the electronic device 1400 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1410 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 14 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0130] The processor 1410 is configured to acquire a panel image of the handle captured by a camera; and to acquire the identifiers of multiple target indicator lights currently illuminated on the handle panel, wherein the handle panel is provided with multiple indicator lights, the multiple indicator lights are arranged in a predetermined shape on the panel, and the multiple indicator lights are evenly distributed according to the predetermined shape, and the target indicator light is one of the multiple indicator lights; and to determine the pose of the handle based on the first position information of the multiple target indicator lights in the panel image and the identifiers of the multiple target indicator lights.

[0131] Optionally, the processor 1410 is further configured to obtain second position information of the plurality of target indicator lights relative to the center of the circle based on the identifiers of the plurality of target indicator lights, and to determine the pose of the handle based on the second position information and the first position information of the plurality of target indicator lights, including: constructing a target function O′ i (θ):

[0132] Among them, u i and v i For the position information of the i-th target indicator light in the panel image, π θ (p i ) represents the first position information of the i-th target indicator light, e θ (.) is the equation representing the ellipse formed by the projection of the circle onto the panel image; the objective function O′ i (θ) is minimized to obtain a target value of θ; based on the target value, the pose of the handle is obtained, wherein the pose includes: [R θ |t θ ], the R θ The t parameter is used to represent the rotation of the panel of the handle relative to the camera. θ The panel used to indicate the displacement of the handle relative to the camera.

[0133] Alternatively, the processor 1410 is configured to predict, based on the current position of the handle's panel, a plurality of target indicator lights to be illuminated next, wherein the handle's panel has a plurality of indicator lights arranged in a predetermined shape on the panel and the plurality of indicator lights are evenly distributed according to the predetermined shape, and the target indicator light is one of the plurality of indicator lights; control the plurality of target indicator lights on the handle's panel to illuminate, and control the indicator lights other than the plurality of target indicator lights to turn off.

[0134] Optionally, the processor 1410 is further configured to: predict the target position of the next movement of the panel based on the current position of the panel on the handle; determine, based on the target position, a plurality of candidate indicator lights among a plurality of indicator lights provided on the panel that can be captured by the camera of the terminal device; select the plurality of target indicator lights to be lit next from the plurality of candidate indicator lights; and send the identifiers of the plurality of target indicator lights to the terminal device.

[0135] It should be understood that, in this embodiment, the input unit 1404 may include a graphics processing unit (GPU) 14041 and a microphone 14042. The GPU 14041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1406 may include a display panel 14061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1407 includes at least one of a touch panel 14071 and other input devices 14072. The touch panel 14071 is also called a touch screen. The touch panel 14071 may include a touch detection device and a touch controller. Other input devices 14072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0136] The memory 1409 can be used to store software programs and various data. The memory 1409 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1409 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1409 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0137] Processor 1410 may include one or more processing units; optionally, processor 1410 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1410.

[0138] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described methods for positioning the handle or controlling the handle indicator light, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0139] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0140] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described embodiments of the handle positioning method or handle indicator light control method, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0141] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0142] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described handle positioning method or handle indicator light control method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0143] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0145] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for positioning a handle, characterized in that, include: Acquire the panel image of the controller captured by the camera; The identifiers of multiple target indicator lights currently illuminated on the panel of the handle are obtained. The panel of the handle is provided with multiple indicator lights, which form a predetermined shape on the panel and are evenly distributed according to the predetermined shape. The target indicator light is one of the multiple indicator lights. Based on the first position information of the plurality of target indicator lights in the panel image and the identifiers of the plurality of target indicator lights, the pose of the handle is determined; The predetermined shape includes a circle, and the plurality of indicator lights are evenly distributed along the periphery of the circle; Based on the first position information of the plurality of target indicator lights in the panel image and the identifiers of the plurality of target indicator lights, the pose of the handle is determined, including: Based on the identifiers of the plurality of target indicator lights, obtain second position information of the plurality of target indicator lights relative to the center of the circle; Construct the objective function : ;in, and This refers to the position information of the i-th target indicator light in the panel image. Let be the coordinates of the i-th target indicator light in the world coordinate system. The projection relationship between the camera imaging plane coordinate system and the world coordinate system is as follows: = · + , The equation representing the ellipse formed by the projection of the circle onto the panel image; The objective function Minimize to obtain Target value; Based on the target value, the pose of the handle is obtained, wherein the pose includes: The The parameters used to indicate the rotation of the panel of the handle relative to the camera, The panel used to indicate the displacement of the handle relative to the camera.

2. The method according to claim 1, characterized in that, The multiple target indicator lights are asymmetrically distributed.

3. A method for controlling a handle indicator light, characterized in that, include: Based on the current position of the handle's panel, predict the next target indicator lights to be lit, wherein the current position is determined by the positioning method according to any one of claims 1-2, the handle's panel is provided with multiple indicator lights, the multiple indicator lights form a predetermined shape on the panel, and the multiple indicator lights are evenly distributed according to the predetermined shape, and the target indicator light is one of the multiple indicator lights; The controller illuminates the plurality of target indicator lights on the control panel and turns off the other indicator lights. Based on the current position of the controller's panel, predict the target indicator light to be illuminated next, including: Based on the current position of the panel of the handle, predict the target position of the panel's next movement; Based on the target location, determine multiple candidate indicator lights that can be captured by the camera of the terminal device from among the multiple indicator lights set on the panel; Select the target indicator light to be lit next from the plurality of candidate indicator lights.

4. The method according to claim 3, characterized in that, After predicting the multiple target indicator lights to be lit next, the method further includes sending the identifiers of the multiple target indicator lights to a terminal device.

5. A positioning device for a handle, characterized in that, include, The first acquisition module is used to acquire the panel image of the handle captured by the camera; The second acquisition module is used to acquire the identifiers of multiple target indicator lights currently lit on the panel of the handle, wherein the panel of the handle is provided with multiple indicator lights, the multiple indicator lights are arranged in a predetermined shape on the panel, and the multiple indicator lights are evenly distributed according to the predetermined shape, and the target indicator light is one of the multiple indicator lights; The determining module is used to determine the pose of the handle based on the first position information of the plurality of target indicator lights in the panel image and the identifiers of the plurality of target indicator lights; The predetermined shape includes a circle, and the plurality of indicator lights are evenly distributed along the periphery of the circle; The determining module is further configured to obtain second position information of the multiple target indicator lights relative to the center of the circle based on the identifiers of the multiple target indicator lights; Construct the objective function : ;in, and This refers to the position information of the i-th target indicator light in the panel image. Let be the coordinates of the i-th target indicator light in the world coordinate system. The projection relationship between the camera imaging plane coordinate system and the world coordinate system is as follows: = · + , The equation representing the ellipse formed by the projection of the circle onto the panel image; The objective function Minimize to obtain Target value; Based on the target value, the pose of the handle is obtained, wherein the pose includes: The The parameters used to indicate the rotation of the panel of the handle relative to the camera, The panel used to indicate the displacement of the handle relative to the camera.

6. A control device for a handle indicator light, characterized in that, include: A prediction module is used to predict multiple target indicator lights to be lit next based on the current position of the panel of the handle, wherein the current position is determined by the positioning method according to any one of claims 1-2, the panel of the handle is provided with multiple indicator lights, the multiple indicator lights are arranged in a predetermined shape on the panel, and the multiple indicator lights are evenly distributed according to the predetermined shape, and the target indicator light is one of the multiple indicator lights; The control module is used to control the illumination of the plurality of target indicator lights on the panel of the handle, and to control the extinguishing of the indicator lights other than the plurality of target indicator lights; The prediction module is also used to predict the target position of the panel's next movement based on the current position of the panel on the handle; Based on the target location, determine multiple candidate indicator lights that can be captured by the camera of the terminal device from among the multiple indicator lights set on the panel; Select the target indicator light to be lit next from the plurality of candidate indicator lights.

7. An electronic device, characterized in that, It includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the handle positioning method as claimed in any one of claims 1 to 2, or the steps of the handle indicator control method as claimed in any one of claims 3 to 4.

8. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the handle positioning method as described in any one of claims 1 to 2, or the steps of the handle indicator light control method as described in any one of claims 3 to 4.

Images