Three-Dimensional Object Relative Pose Control Method and Device Based on Fingerprint Images

Abstract

The present invention provides a method and device for controlling the relative pose of a three-dimensional object based on fingerprint images. The method includes: collecting fingerprint sequence images in real time; preprocessing the fingerprint sequence images to remove background noise and enhance fingerprint ridges; inferring the pose information of the current finger in three-dimensional space based on the preprocessed fingerprint sequence images; mapping the pose information into three-dimensional control signals, and controlling the relative pose of the target object in three-dimensional space according to the three-dimensional control signals. The three-dimensional relative pose control method of the present invention can effectively expand the existing human-computer interaction methods and facilitate the pose control of objects in three-dimensional space.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction, and particularly to the problem of three-dimensional pose control. Background Art

[0002] With the continuous development of the field of human-computer interaction, the interaction methods between humans and machines have undergone continuous changes and sufficient development. In the fields of games, virtual reality (VR), security monitoring, 3D design, vehicle control, robot control, spacecraft control, etc., it is often necessary to operate objects such as cameras, 3D models, robotic arms, cars, airplanes, etc. Most of these target objects are located in three-dimensional space, so it is necessary to input three-dimensional control signals to precisely control their poses. In the prior art, there are few pose control systems that directly input three-dimensional signals. The commonly used control input device is a traditional mouse, and the position and pose changes of the target object are controlled by controlling the moving speed and displacement of the mouse on a two-dimensional plane. However, the target object to be operated is often located in three-dimensional space, and complex mapping between two-dimensional input signals and three-dimensional poses is required. Directly using three-dimensional signals to control the pose of the target object is more in line with human intuitive feelings, and at the same time can further improve the efficiency and accuracy of pose control.

[0003] There have been some solutions in the pose control of three-dimensional target objects, but these control methods still have the following limitations and deficiencies:

[0004] The three-dimensional pose control based on 2D input signals does not conform to human intuitive feelings. Users need to practice for a certain period of time before use, and monitor the pose changes of the target object at all times during use to obtain control feedback.

[0005] In addition to the traditional mouse, 3D mice specially designed for three-dimensional pose control have also emerged to cope with more complex three-dimensional pose control scenarios. Most of these devices require complex mechanical structure designs, and their costs are relatively high, which is not conducive to large-scale popularization. Summary of the Invention

[0006] The present invention aims to solve at least one of the technical problems in the related art to some extent.

[0007] To this end, the first object of the present invention is to propose a method for controlling the relative pose of a three-dimensional object based on a fingerprint image, which is used to facilitate the pose control of an object in three-dimensional space.

[0008] The second object of the present invention is to propose a device for controlling the relative pose of a three-dimensional object based on a fingerprint image.

[0009] The third object of the present invention is to propose a computer device.

[0010] The fourth object of the present invention is to propose a computer-readable storage medium.

[0011] To achieve the above object, an embodiment of the first aspect of the present invention proposes a method for controlling the relative pose of a three-dimensional object based on fingerprint images, including: collecting fingerprint sequence images in real time; preprocessing the fingerprint sequence images to remove background noise and enhance fingerprint ridges; inferring the pose information of the current finger in three-dimensional space according to the preprocessed fingerprint sequence images; mapping the pose information into three-dimensional control signals, and controlling the relative pose of the target object in three-dimensional space according to the three-dimensional control signals.

[0012] The method for controlling the relative pose of a three-dimensional object based on fingerprint images proposed by the embodiment of the present invention can predict the three-dimensional pose of a finger given the fingerprint image of a finger collected on a certain interaction device, and as a new type of human-computer interaction system, it is used to control target objects in the real world or the virtual world. In the present invention, first, the three-dimensional pose of the current finger is predicted according to the 2D fingerprint image, and then the three-dimensional pose of the finger is used as an input signal to control the target object in the real world or the virtual world. In the present invention, the input of the three-dimensional relative pose control system is a 2D fingerprint image, and the output is the three-dimensional pose of the current finger, which is used to control the three-dimensional relative pose of the target object. The three-dimensional relative pose control system of the present invention can effectively expand the existing human-computer interaction methods and provide convenience for the pose control of objects in three-dimensional space.

[0013] In addition, the method for controlling the relative pose of a three-dimensional object based on fingerprint images according to the above embodiment of the present invention may further have the following additional technical features:

[0014] Further, in an embodiment of the present invention, the mapping method between the three-dimensional pose of the current finger and the three-dimensional control signal is designed separately according to different applications, including:

[0015] Mapping the three-dimensional pose of the finger into a three-dimensional pose control signal of the target object;

[0016] Mapping the finger pose into a three-dimensional displacement control signal of the target object;

[0017] Mapping the finger pose into a pose control signal of the target object, where the operations of multiple fingers are mixed, and the poses of some fingers are mapped into displacement control signals, and the poses of the other fingers are mapped into pose control signals.

[0018] Further, in an embodiment of the present invention, before controlling the relative pose of the target object in three-dimensional space according to the three-dimensional control signal, it further includes:

[0019] Perform fingerprint recognition verification on the fingerprint images, including frame-by-frame verification and first-frame verification; among them, frame-by-frame verification means that each frame of fingerprint image needs to pass fingerprint recognition and verification before subsequent object pose control is allowed; first-frame verification includes performing fingerprint recognition on the first frame image of the fingerprint image, and after passing the verification, keeping the finger on the control device without leaving to continuously perform subsequent object pose control.

[0020] Furthermore, in an embodiment of the present invention, it further includes:

[0021] Set respective angle mapping functions for different fingerprint images, and call the corresponding angle mapping function based on the result of fingerprint recognition.

[0022] To achieve the above object, an embodiment of the second aspect of the present invention proposes a three-dimensional object relative pose control device based on fingerprint images, including: an acquisition module for real-time acquisition of fingerprint sequence images; a preprocessing module for preprocessing the fingerprint sequence images to remove background noise and enhance fingerprint ridges; a prediction module for inferring the pose information of the current finger in three-dimensional space according to the preprocessed fingerprint sequence images; a control module for mapping the pose information into three-dimensional control signals and controlling the relative pose of the target object in three-dimensional space according to the three-dimensional control signals.

[0023] Furthermore, in an embodiment of the present invention, the control module is further configured to separately design the mapping method between the current finger's three-dimensional pose and the input signal according to different applications, including:

[0024] Map the three-dimensional pose of the finger into a three-dimensional pose control signal of the target object;

[0025] Map the finger pose into a three-dimensional displacement control signal of the target object;

[0026] Map the finger pose into a pose control signal of the target object, where, for the operation of mixing multiple fingers, the poses of some fingers are mapped into displacement control signals, and the poses of the other part are mapped into pose control signals.

[0027] Furthermore, in an embodiment of the present invention, it further includes a verification module for:

[0028] Before controlling the relative pose of the target object in three-dimensional space according to the three-dimensional control signals, perform fingerprint recognition verification on the fingerprint images, including frame-by-frame verification and first-frame verification; among them, frame-by-frame verification means that each frame of fingerprint image needs to pass fingerprint recognition and verification before subsequent object pose control is allowed; first-frame verification includes performing fingerprint recognition on the first frame image of the fingerprint image, and after passing the verification, keeping the finger on the control device without leaving to continuously perform subsequent object pose control.

[0029] Further, in an embodiment of the present invention, it further includes a personalized control module for:

[0030] Set respective angle mapping functions for different fingerprint images, and call the corresponding angle mapping function based on the result of fingerprint recognition.

[0031] To achieve the above object, an embodiment of the third aspect of the present invention provides a computer device, which is characterized by including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the three-dimensional object relative pose control method based on fingerprint images as described above is implemented.

[0032] To achieve the above object, an embodiment of the fourth aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. The computer program is characterized in that when executed by a processor, the three-dimensional object relative pose control method based on fingerprint images as described above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and / or additional aspects and advantages of the present invention will become apparent and easy to understand from the following description of embodiments in conjunction with the drawings, where:

[0034] Figure 1 It is a schematic flow chart of a three-dimensional relative pose control method based on fingerprint images provided by an embodiment of the present invention.

[0035] Figure 2 It is a schematic flow chart of a three-dimensional relative pose control device based on fingerprint images provided by an embodiment of the present invention.

[0036] Figure 3 It is a schematic flow chart of a three-dimensional relative pose control system based on fingerprint images provided by an embodiment of the present invention.

[0037] Figure 4 It is a schematic diagram of finger pose definition provided by an embodiment of the present invention.

[0038] Figure 5 It is a schematic diagram of the coverage range of common finger postures provided by an embodiment of the present invention.

[0039] Figure 6 It is a schematic diagram of a control signal curve provided by an embodiment of the present invention.

[0040] Figure 7 It is a schematic diagram of a pose control system for frame-by-frame verification provided by an embodiment of the present invention.

[0041] Figure 8 It is a schematic diagram of a pose control system for first-frame verification provided by an embodiment of the present invention.

[0042] Figure 9 Schematic diagram of an adaptive pose control system based on fingerprint verification provided by an embodiment of the present invention. Detailed implementation manners

[0043] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0044] The method and device for controlling the relative pose of a three-dimensional object based on a fingerprint image according to an embodiment of the present invention will be described below with reference to the accompanying drawings.

[0045] Figure 1 Schematic flowchart of a method for controlling the relative pose of a three-dimensional object based on a fingerprint image provided by an embodiment of the present invention.

[0046] As Figure 1 shown, the method for controlling the relative pose of a three-dimensional object based on a fingerprint image includes the following steps:

[0047] S1: Collect fingerprint sequence images in real time;

[0048] With the continuous development of fingerprint sensing technology, in addition to the traditional optical fingerprint acquisition method, fingerprint image data can also be obtained in various existing touch screen devices. The ridge information in the fingerprint image contains rich finger pose information. Therefore, the three-dimensional pose of the current finger can be inferred from the fingerprint image, and the finger pose is used as a three-dimensional input signal, so that the three-dimensional pose of the target object can be conveniently and intuitively controlled. Here, the target object includes objects that require multi-degree-of-freedom control such as cameras, 3D models, robotic arms, and spacecraft. At the same time, the finger pose signal based on the fingerprint image can also be combined with the gesture signal based on the touch screen to further expand the diversity of gesture operations. The control system combined with fingerprint identity information also has higher security and personalized settings.

[0049] The present invention is generally divided into three stages, namely, collecting fingerprint images, obtaining finger postures, and controlling the pose of a target object. In the first stage, a fingerprint sequence image in a collection device is obtained, that is, a fingerprint image is obtained in real time. In the second stage, the three-dimensional (3D) posture of the current finger is predicted based on the collected fingerprint sequence image and used as an input of a subsequent control signal. In the third stage, the obtained 3D finger posture is used as an input signal to control the relative pose of the target object in 3D space. The first stage is a basic step of the present invention. The collected fingerprint image contains 3D posture information of the current finger. In the second stage, the 3D posture of the current finger is estimated according to the fingerprint sequence image. In the third stage, the finger 3D posture is used as a posture control signal of a control system to control the pose change of the target object in 3D space. The present invention includes the collection of fingerprint images, the estimation of the 3D finger posture based on the fingerprint images, and the control of the 3D relative pose of the target object. The flowchart of the present invention is shown in the appendix Figure 3 The present invention also proposes to combine fingerprint recognition with pose control to achieve the security and personalization of control.

[0050] The present invention needs to collect fingerprint sequence images. Compared with the capacitance images in traditional touch screen devices, fingerprint images contain more finger shape and posture information and thus can be used to accurately estimate the posture of a finger in 3D space. There are many sensing technologies for obtaining fingerprint images, such as optical fingerprint scanners, optical in-screen fingerprint scanners, and ultrasonic in-screen fingerprint scanners. The present invention can be applied to various fingerprint sensing technologies. Since it is necessary to estimate the 3D posture of a finger in real time according to fingerprint images and then control the movement of a target object, it is necessary to collect fingerprint image data in real time, that is, collect fingerprint sequence image data.

[0051] S2: Preprocess the fingerprint sequence image to remove background noise and enhance fingerprint ridges;

[0052] According to information such as the direction of ridges in the fingerprint image and the shape of the fingerprint area, the posture information of the current finger in 3D space can be inferred. The following only describes a typical finger posture estimation algorithm to assist in understanding the algorithm in this stage of the present invention. Specifically, in the present invention, according to the collected fingerprint image, the 3D posture of the current finger is estimated to be used as a subsequent control signal.

[0053] Since there are large differences in the modalities and qualities of fingerprint images obtained by different sensors, it is necessary to first preprocess the collected images to remove background noise and enhance fingerprint ridges. After the fingerprint image is preprocessed, the 3D posture of the current finger is estimated according to the fingerprint image. In the present invention, taking a deep learning-based posture estimation algorithm as an example, the input of the algorithm is the fingerprint image after the image preprocessing step, and the output is the predicted 3D finger posture. The definition of the 3D finger posture is shown in the appendix Figure 4, which is represented by three angles, namely the roll angle α, the pitch angle β, and the yaw angle γ.

[0054] S3: Based on the preprocessed fingerprint sequence image, infer the pose information of the current finger in three-dimensional space;

[0055] S4: Map the pose information to three-dimensional control signals, and control the relative pose of the target object in three-dimensional space according to the three-dimensional control signals.

[0056] Furthermore, in an embodiment of the present invention, the mapping method between the three-dimensional pose of the current finger and the three-dimensional control signal is designed separately according to different applications, including:

[0057] Map the three-dimensional pose of the finger to the three-dimensional pose control signal of the target object;

[0058] Map the finger pose to the three-dimensional displacement control signal of the target object;

[0059] Map the finger pose to the pose control signal of the target object, where the operations of multiple fingers are mixed, and the poses of some fingers are mapped to displacement control signals, and the poses of the other part are mapped to pose control signals.

[0060] After obtaining the three-dimensional pose P=(p α , p β , p γ ) of the finger from the fingerprint image, this pose can be mapped to three-dimensional control signals to control the pose of the target object (such as a camera, a 3D model, etc.) in three-dimensional space. Since the poses that the finger can easily make are limited (see Appendix Figure 5 ), in order to fully and completely control the three-dimensional pose of the target object, it is necessary to first specify the finger pose zero point P0. There are many ways to define the zero point. The finger pose of the first frame image at the start of the interaction can be selected, or it can be set to a fixed value. Here, taking the roll angle of 0°, the pitch angle of -45°, and the yaw angle of 0° as an example, calculate the relative pose of the finger pose obtained from the fingerprint image in the previous stage relative to the pose zero point P0

[0061] P′=(p′ α , p′ β , p′ γ ),

[0062] and map it to the control signal. The mapping method between the three-dimensional pose of the finger and the control signal can be designed separately according to different specific applications. In the present invention, three typical mapping methods are taken as examples to illustrate the relative pose control system.

[0063] Map the three-dimensional pose of the finger to the three-dimensional pose control signal of the target object. Set S α , S βand S γ are respectively the attitude control signals for the target object in roll, pitch, and yaw angles:

[0064] S α = g(α′)

[0065] S β = g(β′)

[0066] S γ = g(γ′)

[0067] The g(·) function defines the control signal curve, representing the mapping method from finger attitude to object attitude control signal. The definition of a control signal curve is given in the appendix Figure 6 as an example, where θ is the three-dimensional finger attitude and S is the control signal. According to the mapped attitude control signals S α 、S β and S γ , the target object changes its pose Q = (q x , q y , q z , q, α , q β , q γ ) to a new pose Q′ = (q′ x , q′ y , q′ z , q′ α , q′ β , q′ γ ):

[0068] q′ α = q α + Δt·S α

[0069] q′ β = q β + Δt·S β q′ γ = q γ + Δt·S γ where Δt is the time interval between adjacent frames.

[0070] Maps the finger attitude to the three-dimensional position control signal of the target object. Set S x 、S y and S z to be the position control signals for the target object in the X, Y, and Z directions respectively:

[0071] S x = g(α′)

[0072] S y = g(β′)

[0073] S z = g(γ′)

[0074] The definition of a control signal curve is given by way of example, but the setting of the control signal curve can be different from the attitude control signal mapping. According to the mapped position control signals S Figure 6 , S x , S y and S z , the target object changes on the basis of the current pose Q = (q x , q y , q z , q α , q β , q γ ) to obtain a new pose Q′ = (q′ x , q′ y , q′ z , q′ α , q′ β , q′ γ ):

[0075] q′ x = q x + Δt·S x

[0076] q′ y = q y + Δt·S y

[0077] q′ z = q z + Δt·S z

[0078] Map the finger postures to the pose control signals of the target object, that is, mix the operations of multiple fingers. Part of the finger postures are mapped to position control signals, and the other part is mapped to attitude control signals. A typical application scenario is that one finger controls the three-dimensional orientation of the object and the other finger controls the three-dimensional displacement of the object.

[0079] The following is an example of applying the three-dimensional relative pose control system of the present invention to several common target object controls. For vehicle control, the roll angle of the finger can be mapped to the left and right steering control of the vehicle, and the pitch angle can be mapped to the acceleration and deceleration control; for camera control, the roll angle of the finger can be mapped to the left and right rotation of the camera, the pitch angle can be mapped to the up and down pitch of the camera, and the yaw angle can be mapped to the focal length control of the camera; for quadcopter control, the roll angle of the finger can be mapped to the left and right roll and displacement of the aircraft, the pitch angle can be mapped to the front and back roll and displacement of the aircraft, and the yaw angle can be mapped to the up and down displacement of the aircraft; for fixed-wing aircraft control, the mapping method is more direct, and the roll angle of the finger can be mapped to the roll control of the aircraft, the pitch angle can be mapped to the pitch control, and the yaw angle can be mapped to the acceleration and deceleration control, etc.

[0080] Further, in an embodiment of the present invention, before controlling the relative pose of the target object in the three-dimensional space according to the three-dimensional control signal, it further includes:

[0081] Performing fingerprint recognition verification on the fingerprint image, including frame-by-frame verification and first-frame verification; wherein, the frame-by-frame verification includes that each frame of fingerprint image needs to pass fingerprint recognition and verification before subsequent object pose control is allowed; the first-frame verification includes performing fingerprint recognition on the first-frame image of the fingerprint image, and after passing the verification, keeping the finger not leaving the control device to continuously perform subsequent object pose control.

[0082] In addition to the above three-dimensional object pose control based on finger gestures, it can also be combined with gesture signals based on touchscreens or fingerprint images to further expand the diversity of three-dimensional object control methods. In addition, the existing fingerprint recognition technology can also be combined with the three-dimensional relative pose control system based on fingerprints to further broaden the security and privacy permission control of the control system. For example, in a highly confidential control system, only legitimate registered users are allowed to control. The following describes two methods of introducing fingerprint recognition. Att Figure 7 is a schematic diagram of the frame-by-frame verification method. At this time, each frame of image needs to pass the verification of the fingerprint recognition system before subsequent object pose control is allowed. Att Figure 8 is a schematic diagram of the first-frame verification method. At this time, only the first-frame image when the finger presses needs to be fingerprint-recognized. After passing the verification, as long as the finger does not leave the control device, subsequent object pose control can be continuously performed, and once the finger leaves, identity verification needs to be performed again.

[0083] Further, in an embodiment of the present invention, it further includes:

[0084] Setting respective angle mapping functions for different fingerprint images, and calling the corresponding angle mapping functions based on the results of fingerprint recognition.

[0085] In addition, by combining fingerprint recognition technology with fingerprint-based gesture control, personalized control parameters can be achieved. For example, for multiple registered fingerprints (fingerprints of different users or different fingerprints of the same user), respective angle mapping functions are set separately. Based on the result of fingerprint recognition, the corresponding angle mapping function is called, while for unregistered fingers, the default angle mapping function is used (as shown in the appendix). Figure 9 shown

[0086] The three-dimensional object relative pose control method based on fingerprint images proposed in the embodiments of the present invention, given the fingerprint image of a certain finger collected on a certain interaction device, can predict the three-dimensional pose of the finger, and as a new type of human-computer interaction system, is used to control target objects in the real world or the virtual world. In the present invention, first, the three-dimensional pose of the current finger is predicted according to the 2D fingerprint image, and then the three-dimensional pose of the finger is used as an input signal to control the target object in the real world or the virtual world. In the present invention, the input of the three-dimensional relative pose control system is a 2D fingerprint image, and the output is the three-dimensional pose of the current finger, which is used to control the three-dimensional relative pose of the target object. The three-dimensional relative pose control system of the present invention can effectively expand the existing human-computer interaction methods and facilitate the pose control of objects in three-dimensional space.

[0087] To implement the above embodiments, the present invention also proposes a three-dimensional object relative pose control device based on fingerprint images.

[0088] Figure 2 It is a schematic structural diagram of a three-dimensional object relative pose control device provided for the embodiments of the present invention.

[0089] As Figure 2 shown, the three-dimensional object relative pose control device based on fingerprint images includes: an acquisition module 10, a preprocessing module 20, a prediction module 30, and a control module 40. Among them, the acquisition module is used to collect fingerprint sequence images in real time; the preprocessing module is used to preprocess the fingerprint sequence images, remove background noise and enhance fingerprint ridges; the prediction module is used to infer the pose information of the current finger in three-dimensional space according to the preprocessed fingerprint sequence images; the control module is used to map the pose information into three-dimensional control signals and control the relative pose of the target object in three-dimensional space according to the three-dimensional control signals.

[0090] Furthermore, in an embodiment of the present invention, the control module is further used to separately design the mapping method between the three-dimensional pose of the current finger and the input signal according to different applications, including:

[0091] mapping the three-dimensional pose of the finger into a three-dimensional pose control signal of the target object;

[0092] Map the finger gesture to a three-dimensional displacement control signal for the target object;

[0093] Map the finger gesture to a pose control signal for the target object, where the operations of multiple fingers are combined, and the gestures of some fingers are mapped to displacement control signals, and the gestures of the other part are mapped to pose control signals.

[0094] Furthermore, in an embodiment of the present invention, a verification module is further included for:

[0095] Before controlling the relative pose of the target object in the three-dimensional space according to the three-dimensional control signal, perform fingerprint recognition verification on the fingerprint image, including frame-by-frame verification and first-frame verification; wherein, the frame-by-frame verification includes that each frame of fingerprint image needs to pass fingerprint recognition and verification before subsequent object pose control is allowed; the first-frame verification includes performing fingerprint recognition on the first frame image of the fingerprint image, and after the verification passes, keep the finger not leaving the control device to continuously perform subsequent object pose control.

[0096] Furthermore, in an embodiment of the present invention, a personalized control module is further included for:

[0097] Set respective angle mapping functions for different fingerprint images, and call the corresponding angle mapping functions based on the results of fingerprint recognition.

[0098] To achieve the above object, an embodiment of the third aspect of the present invention provides a computer device, which is characterized by including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for controlling the relative pose of a three-dimensional object based on a fingerprint image as described above is implemented.

[0099] To achieve the above object, an embodiment of the fourth aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, and is characterized in that when the computer program is executed by a processor, the method for controlling the relative pose of a three-dimensional object based on a fingerprint image as described above is implemented.

[0100] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without conflict, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0101] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0102] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A three-dimensional object relative pose control method based on fingerprint images, characterized in that, Including the following steps: Collecting fingerprint sequence images in real time; Preprocessing the fingerprint sequence images to remove background noise and enhance fingerprint ridges; Based on the preprocessed fingerprint sequence image, the pose information of the current finger in three-dimensional space is inferred , is the roll angle, is the pitch angle, is the yaw angle; Specify the zero point of finger posture , and calculate the current finger posture Relative to the zero posture The relative posture ; Map the relative pose to a three-dimensional control signal, and control the relative position and pose of the target object in three-dimensional space according to the three-dimensional control signal; Among them, the mapping method between the current finger's three-dimensional posture and the three-dimensional control signal is designed separately according to different applications, including: mapping the three-dimensional posture of the finger to the three-dimensional posture control signal of the target object, or mapping the finger posture to the three-dimensional position control signal of the target object; Map the relative posture of the finger to a three-dimensional posture control signal of the target object, including: Settings , , are respectively the attitude control signals of the target object after mapping in the roll, pitch and yaw angles, expressed as: Among them, The function defines a control signal curve; Based on the mapped attitude control signal, change is made on the basis of the current pose of the target object to obtain a new pose , which is expressed as: Among them, is the time interval between adjacent frames; Mapping the finger posture to the three-dimensional position control signal of the target object includes: Settings , and are respectively the position control signals of the target object in the , and directions: Based on the mapped attitude control signal, change it on the basis of the current pose of the target object to obtain a new pose , which is expressed as: 。 2. The method according to claim 1, wherein The mapping method between the current finger's three-dimensional posture and the three-dimensional control signal also includes: Mapping the finger posture to the three-dimensional displacement control signal of the target object; Mapping the finger posture to the pose control signal of the target object, where, by mixing the operations of multiple fingers, some finger postures are mapped to displacement control signals and the other part is mapped to posture control signals.

3. The method according to claim 1, characterized in that, Before controlling the relative pose of the target object in three-dimensional space according to the three-dimensional control signal, it also includes: Performing fingerprint recognition verification on the fingerprint image, including frame-by-frame verification and first-frame verification; among them, the frame-by-frame verification includes that each frame of fingerprint image needs to pass fingerprint recognition and verification before subsequent object pose control is allowed; the first-frame verification includes performing fingerprint recognition on the first-frame image of the fingerprint image, and after verification, keeping the finger from leaving the control device to continuously perform subsequent object pose control.

4. The method according to claim 1 or 3, characterized in that, It also includes: Setting respective angle mapping functions for different fingerprint images and calling the corresponding angle mapping functions based on the results of the fingerprint recognition.

5. A three-dimensional object relative pose control device based on a fingerprint image, characterized in that, Including: A collection module for collecting fingerprint sequence images in real time; A preprocessing module for preprocessing the fingerprint sequence images to remove background noise and enhance fingerprint ridges; A prediction module, configured to infer the pose information of the current finger in three-dimensional space based on the preprocessed fingerprint sequence image , is the roll angle, is the pitch angle, is the yaw angle; The prediction module is further configured to specify the zero point of the finger posture , and calculate the current finger posture relative to the zero point of the posture for the relative posture ; The control module is configured to map the relative attitude into three-dimensional control signals, and control the relative pose of the target object in the three-dimensional space according to the three-dimensional control signals; Among them, the mapping method between the current finger's three-dimensional posture and the three-dimensional control signal is designed separately according to different applications, including: mapping the three-dimensional posture of the finger to the three-dimensional posture control signal of the target object, or mapping the finger posture to the three-dimensional position control signal of the target object; Map the relative posture of the finger to a three-dimensional posture control signal of the target object, including: Settings , , are respectively the attitude control signals of the target object after mapping in roll, pitch, and yaw angles, expressed as: Among them, The function defines a control signal curve; Based on the mapped attitude control signal, change is made on the basis of the current pose of the target object to obtain a new pose , which is expressed as: Among them, is the time interval between adjacent frames; Mapping the finger posture to the three-dimensional position control signal of the target object includes: Settings , and are respectively the position control signals of the target object in , and directions: Based on the mapped attitude control signal, make a change on the basis of the current pose of the target object to obtain a new pose , which is expressed as: 。 6. The device according to claim 5, characterized in that, The control module is also used to design separately the mapping method between the current finger's three-dimensional posture and the input signal according to different applications, including: Mapping the three-dimensional posture of the finger to the three-dimensional posture control signal of the target object; Mapping the finger posture to the three-dimensional displacement control signal of the target object; Mapping the finger posture to the pose control signal of the target object, where, by mixing the operations of multiple fingers, some finger postures are mapped to displacement control signals and the other part is mapped to posture control signals.

7. The device according to claim 5, characterized in that, It also includes a verification module for: Before controlling the relative pose of the target object in the three-dimensional space according to the three-dimensional control signal, fingerprint recognition verification is performed on the fingerprint image, including frame-by-frame verification and first-frame verification; wherein, the frame-by-frame verification includes that each frame of fingerprint image needs to pass fingerprint recognition and verification before subsequent object pose control is allowed; the first-frame verification includes performing fingerprint recognition on the first frame image of the fingerprint image, and after passing the verification, keeping the finger on the control device without leaving to continuously perform subsequent object pose control.

8. The device according to claim 5, characterized in that, It further includes a personalized control module for: Setting respective angle mapping functions for different fingerprint images and calling the corresponding angle mapping functions based on the results of the fingerprint recognition.

9. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the three-dimensional object relative pose control method based on fingerprint images as described in any one of claims 1-4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the three-dimensional object relative pose control method based on fingerprint images as described in any one of claims 1-4.

Images