Motion capture method and device in vision detection based on AR display equipment and medium
By capturing the user's posture and facial movements in real time on the AR display device, the problem of traditional vision detection methods being applicable to users with limited mobility is solved, and efficient and accurate vision detection assistance is achieved.
Patent Information
- Application Number
- CN202311488268.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-09
- Publication Date
- 2025-05-13
AI Technical Summary
Traditional vision detection methods are not suitable for users with reduced mobility or expression, and cannot effectively assist them in completing vision tests.
The motion capture method based on the AR display device is adopted to acquire the user's posture data and facial images in real time, identify the head and eye movements, calculate the action data, and use it to update the interactive content to assist in vision testing.
It realizes vision detection assistance for users with impaired mobility or inconvenience in expression, improving the efficiency and accuracy of vision tests without gestures or voice commands.
Smart Images
Figure CN119987529A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the AR / VR field, and in particular to a motion capture method, device, and medium based on vision detection in an AR display device. Background Art
[0002] With the increasing development of eye tracking technology, there are more and more ways to test vision. Traditional vision tests usually require users to read symbols on an eye chart at a specific test distance.
[0003] In related technologies, smart glasses are used to test vision. Different images are presented on the smart glasses, and the vision test is completed in combination with the user's voice commands or gesture commands.
[0004] However, for some special users, especially those with limited mobility or difficulty expressing themselves, it is inconvenient to use the above-mentioned smart glasses to measure vision. Therefore, a new method is needed to help them complete vision testing. Summary of the Invention
[0005] In order to meet the vision testing needs of special groups of people, the present application provides a motion capture method, device and medium based on vision detection of AR display device.
[0006] On the one hand, the present application provides a motion capture method based on the vision detection process of an AR display device, which adopts the following technical solutions:
[0007] A motion capture method based on the vision detection process of an AR display device includes the following steps:
[0008] In response to the AR display device entering the vision detection mode, displaying interactive content on the AR display device and acquiring posture data and facial images in real time;
[0009] Calculating first motion data based on the posture data;
[0010] identifying a first feature and a second feature according to the facial image, and calculating a displacement difference between the first feature and the second feature;
[0011] calculating second motion data according to the displacement difference;
[0012] calculating first capture data according to the first motion data and the second motion data;
[0013] identifying a first target and a second target from the first feature;
[0014] calculating a first change rate of the first target and a second change rate of the second target, and if a difference between the first change rate and the second change rate is less than a preset reference change value, calculating second capture data based on the position of the first target on the first feature;
[0015] The interactive content is updated according to the first captured data and / or the second captured data.
[0016] By employing this technical solution, the user's head and eye movements can be captured based on real-time user posture data and facial images. This facilitates the completion of vision tests, providing auxiliary guidance without the need for gestures or voice commands. This method can also improve the efficiency of vision tests for people without disabilities.
[0017] Optionally, the step of calculating the second captured data further includes the following sub-steps:
[0018] Calculating a first pixel change rate of the first feature and a second pixel change rate of the second feature;
[0019] If a pixel change rate difference between the first pixel change rate and the second pixel change rate is greater than a preset pixel change value, anti-correlation adjusting the second captured data according to the pixel change rate difference;
[0020] The larger the pixel change rate difference is, the smaller the change range of the second captured data is.
[0021] By adopting the above technical solution, by extracting features and calculating their change rates, and then adjusting the captured data according to the difference in change rates, it is possible to more accurately capture the user's facial expressions and head rotations and other movements, which helps vision tests provide more accurate test results.
[0022] Optionally, the method further comprises the following steps:
[0023] Based on a first electrode sheet fixed to the torso and a second electrode sheet moving with the head and in contact with the torso, a pulse electrical signal is obtained between the first electrode sheet and the second electrode sheet;
[0024] Calculating first capture reference data according to the amplitude of the pulse electrical signal;
[0025] The first captured data is adjusted according to the first captured reference data.
[0026] By adopting the above technical solution, as the head rotates, the first electrode will rotate with the rotation of the head, thereby changing the distance between the first electrode sheet and the second electrode sheet, that is, changing the resistance value of the circuit connected in series between the first electrode sheet and the second electrode sheet, and generating a change in the amplitude of the pulse electric signal by collecting the change in the contact distance between the first electrode sheet and the second electrode sheet: the amplitude of the pulse electric signal is obtained through a sensor or measuring device, and the amplitude of the pulse electric signal is analyzed, thereby further ensuring the accuracy of the calculation of the first capture action, that is, the accuracy of the head turning action feature.
[0027] Optionally, the method further comprises the following steps:
[0028] adjusting the pulse frequency of the pulse electrical signal according to the amplitude of the pulse electrical signal;
[0029] The larger the amplitude is, the higher the pulse frequency is; the smaller the amplitude is, the lower the pulse frequency is.
[0030] By adopting the above technical solution, the pulse frequency of the pulse electrical signal is positively correlated to the actual detected pulse amplitude, so that the pulse signal has higher energy and reduces noise interference, etc., thereby achieving better signal transmission effect or realizing specific control purposes.
[0031] Optionally, the method further comprises the following steps:
[0032] acquiring magnetic field data at an electrical path formed by the first electrode sheet and the second electrode sheet;
[0033] Calculating second capture reference data according to the magnetic field data;
[0034] The first captured data is adjusted according to the second captured reference data.
[0035] By employing this technical solution, the acquisition and processing of motion capture data can be dynamically adjusted based on the actual environment and conditions, achieving more accurate motion capture or meeting specific application requirements. For example, in virtual reality or augmented reality applications, vision tests can be adjusted based on the user's actual head or eye movements.
[0036] Optionally, the second electrode sheet includes a roller electrode or a track electrode.
[0037] By adopting the above technical solution, a roller electrode, typically composed of two or more rollers, can roll in contact with the object being measured; a track electrode, typically composed of a track or belt, can make planar contact with the object being measured. This method of using a roller electrode or track electrode as the second electrode sheet can expand the measurement range or improve measurement accuracy, while also adapting to the needs of various shapes and sizes. This method can be used to obtain more accurate and comprehensive magnetic field information, thereby achieving smarter and more efficient vision testing.
[0038] Optionally, the method further comprises the following steps:
[0039] acquiring vibration data of an electrical path formed by the first electrode sheet and the second electrode sheet;
[0040] calculating third capture reference data according to the vibration data;
[0041] The first captured data is adjusted according to the third captured reference data.
[0042] By adopting the above technical solution, a vibration sensor is placed next to the first or second electrode sheet, detecting the changes in movement caused by vibration, thereby achieving motion capture at the electrode sheet. In addition, vibration also generates magnetic field change data, which can be combined with magnetic field data for calculation to further achieve accurate motion capture.
[0043] Optionally, the method further comprises the following steps:
[0044] obtaining a vibration component at the torso, and correcting the vibration data according to the vibration component;
[0045] Alternatively, the vibration data is corrected according to the magnetic field data.
[0046] By adopting the above technical solution, the vibration data is corrected or modified using the magnetic field data to obtain more accurate and reliable vibration data.
[0047] On the other hand, the present application provides a motion capture device based on vision detection of an AR display device, which adopts the following technical solution:
[0048] A motion capture device based on vision detection of an AR display device includes a processor in which a program of the motion capture method based on vision detection of an AR display device is run.
[0049] On the other hand, the present application provides a storage medium, which adopts the following technical solution:
[0050] A storage medium stores a program for the above-mentioned motion capture method in vision detection based on an AR display device.
[0051] To summarize, the present application includes at least one of the following beneficial technical effects: by combining software with hardware, it can realize the tracking of head rotation and the recognition of eye movements, and convert these movements into automated assistance for the pointing process during vision detection through calculation, so as to meet the usage needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 This is a step diagram of the motion capture method in the vision detection of AR display device based on this application.
[0053] Figure 2 This application is based on the motion capture method in the vision detection of AR display equipment, and the eye features are white black and white + white black and white.
[0054] Figure 3 In the motion capture method of the present application based on vision detection of an AR display device, the eye features are white and black + white and black.
[0055] Figure 4 This application is based on the motion capture method in the vision detection of AR display equipment, and the eye features are black and white + black and white.
[0056] Figure 5 This is a sub-step diagram for calculating the second captured data in the motion capture method based on vision detection of an AR display device in this application.
[0057] Figure 6 This is a step diagram of obtaining and processing pulse signals through the first electrode sheet and the second electrode sheet in the motion capture method in the vision detection of the AR display device in this application.
[0058] Figure 7 This is a structural schematic diagram of the first electrode sheet and the second electrode sheet in the motion capture method in the vision detection of the AR display device in this application.
[0059] Figure 8 This is a step diagram of the positive correlation between pulse amplitude and pulse frequency in the motion capture method based on vision detection of AR display device in this application.
[0060] Figure 9 This is a diagram of the steps for obtaining magnetic field data in the motion capture method for vision detection in an AR display device in this application.
[0061] Figure 10 This is a structural schematic diagram of the second electrode sheet being a roller electrode in the motion capture method in the vision detection of an AR display device in this application.
[0062] Figure 11This is a structural schematic diagram of the second electrode sheet being a track-type electrode in the motion capture method in the vision detection of an AR display device in this application.
[0063] Figure 12 This is a diagram of the steps for obtaining vibration data in the motion capture method for vision detection of an AR display device in this application.
[0064] Figure 13 This is a diagram of the steps for correcting vibration data in the motion capture method for vision detection in an AR display device according to this application.
[0065] Reference numerals: 1, first electrode sheet; 2, second electrode sheet. DETAILED DESCRIPTION
[0066] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.
[0067] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.
[0068] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0069] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".
[0070] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0071] A motion capture method based on vision detection of AR display device, referring to Figure 1 , including the following steps:
[0072] In response to the AR display device entering the vision detection mode, interactive content is displayed on the AR display device, and posture data and facial images are acquired in real time; wherein, the acquisition of posture data can be achieved through a six-axis sensor to obtain user motion information. The six-axis sensor refers to a three-axis gyroscope + a three-axis accelerometer. The three-axis gyroscope measures the angle between the vertical axis of the gyroscope rotor and the device in a three-dimensional coordinate system and calculates the angular velocity, and uses the angle and angular velocity to determine the motion state of an object in three-dimensional space. The accelerometer obtains the result by measuring the force applied to the component in a certain axis, which is expressed as the magnitude and direction of the axial acceleration (XYZ). Real-time acquisition of facial images can be achieved through the camera. The interactive content includes elements of vision detection, such as characters in different orientations.
[0073] The first motion data is calculated based on the posture data; for example, the first motion data is motion information of the head, such as information including the position, angle, speed, etc. of the head.
[0074] A first feature and a second feature are identified based on the facial image, and the displacement difference between the first feature and the second feature is calculated; through image processing technology, it can be identified that, for example, the first feature includes an eye image, and the content of the eye image includes information such as the eye socket, eyeball, eye white and pupil; the second feature includes the facial image and the background environment outside the face; the displacement difference is calculated based on the first feature and the second feature.
[0075] The second motion data is calculated based on the displacement difference. For example, the second motion data is the eye movement and the movement of the face relative to the background environment.
[0076] The first captured data is calculated based on the first action data and the second action data; the first captured data corresponds to the user's interactive action, which is convenient for the judgment of the subsequent vision test.
[0077] Identify the first target and the second target from the first feature; the identification of the first target and the second target in the first feature can be achieved through image recognition technology, such as the identification of the eyeball and the white of the eye.
[0078] Calculate the first change rate of the first target and the second change rate of the second target. If the difference between the first change rate and the second change rate is less than the preset reference change value, calculate the second captured data according to the position of the first target on the first feature. By calculating the first change rate of the first target and the second change rate of the second target, for example, analyze the change of the eye image. The position of the first target on the first feature can be expressed as follows: for example, with reference to Figure 2 , before rotation: white black white + white black white, refer to Figure 3 , after rotation it becomes: white and black + white and black, refer to Figure 4, or: black and white + black and white. If the calculated difference is small, it means that the movement of the eyeball and the white of the eye has not changed much. In this case, the second capture data is calculated based on this situation, that is, the eye movement and other movements are calculated to facilitate the pointing movement of the subsequent vision test.
[0079] The interactive content is updated according to the first captured data and / or the second captured data.
[0080] By acquiring user posture data and facial images in real time, the system can capture the user's head and eye movements, facilitating vision testing and assisting in the completion of the test without the need for gestures or voice commands. This method can also improve the efficiency of vision testing for people without disabilities.
[0081] Reference Figure 5 , the step of calculating the second captured data further includes the following sub-steps:
[0082] Calculate the first pixel change rate of the first feature and the second pixel change rate of the second feature; and analyze the change rate of the local area in the image to detect the first feature and the second feature action or perform dynamic image analysis.
[0083] If the difference between the first and second pixel change rates is greater than a preset pixel change value, the second captured data will be adjusted inversely based on the pixel change rate difference. If the difference between the first and second pixel change rates exceeds a preset value, the second captured data will need to be adjusted. The adjustment may be to increase, decrease, or remain unchanged, depending on the preset pixel change value and the actual pixel change rate difference. For example, if the first pixel change rate corresponds to the rate of change of the eye image and the second pixel change rate corresponds to the pixel change of the facial image, if the difference is large, it indicates that the second captured data needs to be appropriately adjusted to more accurately reflect the user's motion capture.
[0084] The larger the difference in pixel change rate, the smaller the change in the second captured data. If the difference between the pixel change rates of the two features is large, the adjustment to the second captured data will be smaller; to prevent excessive fluctuations or sudden changes in the data, a smaller adjustment will be selected.
[0085] By extracting features and calculating their rate of change, and then adjusting the captured data based on the difference in rate of change, it is possible to more accurately capture the user's facial expressions and head movements, helping vision tests provide more accurate test results.
[0086] Reference Figure 6 and Figure 7 , the method further comprises the steps of:
[0087] Based on the first electrode sheet 1 fixed to the torso and the second electrode sheet 2 that moves with the head and contacts the torso, a pulse electrical signal is obtained between the first electrode sheet 1 and the second electrode sheet 2. As the head rotates, the position of the second electrode sheet 2 moves closer to or away from the first electrode sheet 1. As the second electrode sheet 2 contacts the torso, the distance between the first electrode sheet 1 and the second electrode sheet 2 changes, thereby changing the electrical signal in the circuit and generating a pulse signal. The pulse electrical signal is obtained through hardware equipment to indirectly detect the rotation of the head.
[0088] The first captured data is adjusted according to the first captured reference data. The first captured data is increased, decreased, or maintained according to the value of the first captured reference data to achieve more accurate acquisition of the head rotation movement.
[0089] By collecting the pulse electrical signal generated by the change in the distance between the electrode sheets, the first capture data is adjusted according to the amplitude of the pulse electrical signal, thereby further ensuring the accuracy of the first capture action calculation, that is, the accuracy of the head turning action feature.
[0090] In other embodiments, the electrode sheets may also be capacitor plates. When the head is rotated, the distance between the capacitor plates changes, thereby changing the electrical signal in the circuit, thereby indirectly achieving further accurate calculation of the head rotation movement.
[0091] Reference Figure 8 , the method further comprises the steps of:
[0092] adjusting the pulse frequency of the pulse electrical signal according to the amplitude of the pulse electrical signal;
[0093] The larger the amplitude, the higher the pulse frequency; the smaller the amplitude, the lower the pulse frequency.
[0094] According to the actual detected pulse amplitude, the pulse frequency of the pulse electrical signal is positively controlled, so that the pulse signal has higher energy and reduces noise interference, etc., thereby achieving better signal transmission effect or realizing specific control purposes.
[0095] Reference Figure 9 , the method further comprises the steps of:
[0096] The magnetic field data at the electrical path formed by the first electrode sheet 1 and the second electrode sheet 2 is obtained. The magnetic field data at the path may be obtained by a magnetic field measuring device or sensor, including information such as the magnitude and direction of the magnetic field.
[0097] The second capture reference data is calculated based on the magnetic field data; the second capture reference data can be calculated based on the acquired magnetic field data, and the second capture reference data, that is, the user's motion data, is calculated based on information such as the rate of change of the magnetic field and the direction of the magnetic field, which can be used for further analysis and processing.
[0098] Adjust the first captured data based on the second captured reference data. Compare and analyze the second captured reference data with the first captured data, and adjust the first captured data as needed. For example, if the second captured reference data indicates a rapidly changing magnetic field, the sensitivity or dynamic range of the first captured data can be increased; if the second captured reference data indicates a slowly changing magnetic field, the sensitivity or dynamic range of the first captured data can be reduced, thereby achieving more accurate motion capture.
[0099] Dynamically adjust the acquisition and processing of motion capture data based on the actual environment and conditions to achieve more accurate motion capture or meet specific application requirements. For example, in virtual reality or augmented reality applications, vision testing can be adjusted based on the user's actual head or eye movements.
[0100] Reference Figure 10 and Figure 11 The second electrode sheet 2 includes a roller electrode or a track electrode. A roller electrode is usually composed of two or more rollers and can roll in contact with the object to be measured; a track electrode is usually composed of a track or belt and can make planar contact with the object to be measured. By using a roller electrode or a track electrode as the second electrode sheet 2, the measurement range can be expanded or the measurement accuracy can be improved, and it can also adapt to the application requirements of different shapes and sizes. This method is used to obtain more accurate and comprehensive magnetic field information, thereby achieving smarter and more efficient vision testing.
[0101] Reference Figure 12 , the method further comprises the steps of:
[0102] Obtain vibration data at the electrical path formed by the first electrode sheet 1 and the second electrode sheet 2; obtain the vibration data at this point by a vibration sensor or measuring equipment, the vibration data including information such as the magnitude, frequency, and direction of the vibration.
[0103] The third capture reference data is calculated based on the vibration data; the third capture reference data is further analyzed and processed based on the information such as the rate of change of the magnetic field and the direction of the magnetic field to achieve more accurate motion capture information.
[0104] Adjust the first captured data based on the third captured reference data. The third captured reference data is compared and analyzed with the first captured data, and the first captured data is adjusted as needed. For example, if the third captured reference data indicates a large vibration amplitude, the sensitivity or dynamic range of the first captured data can be appropriately reduced; if the third captured reference data indicates a small vibration amplitude, the sensitivity or dynamic range of the first captured data can be appropriately increased. This allows for more accurate calculation of head or eye movement data, thereby improving the accuracy of vision testing.
[0105] Vibration sensors placed next to the first electrode sheet 1 or the second electrode sheet 2 detect changes in movement caused by vibration, thereby capturing motion at the electrode sheet. Furthermore, vibration also generates magnetic field change data, which can be combined with magnetic field data for calculation to further accurately capture motion.
[0106] Reference Figure 13 , the method further comprises the steps of:
[0107] Obtain vibration components at the torso and correct vibration data based on the vibration components; obtain vibration components at the torso through a vibration sensor or measuring device, including information such as vibration magnitude, frequency, and direction.
[0108] Alternatively, vibration data can be corrected based on magnetic field data. The vibration component at the torso is compared and analyzed with the vibration data, and the data can be corrected or modified as needed. For example, if the vibration component at the torso indicates abnormal vibration in certain parts of the body, the vibration data for that part can be corrected or adjusted to obtain more accurate and reliable vibration data.
[0109] The magnetic field data is used to correct or modify the vibration data to obtain more accurate and reliable vibration data.
[0110] An embodiment of the present application discloses a motion capture device based on vision detection of an AR display device, including a processor running a program of the above-mentioned motion capture method based on vision detection of an AR display device.
[0111] An embodiment of the present application discloses a storage medium storing a program of the above-mentioned motion capture method based on an AR display device.
[0112] It should be noted that the computer-readable medium described in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.
[0113] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0114] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0116] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0117] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to)
[0118] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A motion capture method based on the vision detection process of an AR display device, characterized in that: The steps include: In response to the AR display device entering a vision detection mode, displaying interactive content in the AR display device, and acquiring posture data and facial images in real time; Calculating first motion data according to the posture data; Identify a first feature and a second feature according to the facial image, and calculate a displacement difference between the first feature and the second feature; Calculating second motion data according to the displacement difference; Calculate first capture data according to the first motion data and the second motion data; identifying a first target and a second target from the first feature; Calculating a first change rate of the first target and a second change rate of the second target, and if a difference between the first change rate and the second change rate is less than a preset reference change value, calculating second capture data according to a position of the first target on the first feature; The interactive content is updated according to the first captured data and / or the second captured data.
2. The motion capture method in the process of vision detection based on AR display device according to claim 1 is characterized in that: The step of calculating the second captured data further includes the following sub-steps: Calculating a first pixel change rate of the first feature and a second pixel change rate of the second feature; If a pixel change rate difference between the first pixel change rate and the second pixel change rate is greater than a preset pixel change value, the second captured data is adjusted inversely according to the pixel change rate difference; The larger the pixel change rate difference is, the smaller the change amplitude of the second captured data is.
3. The motion capture method in the process of vision detection based on AR display device according to claim 1 is characterized in that: The method further comprises the steps of: Based on a first electrode sheet (1) fixed to the trunk and a second electrode sheet (2) moving with the head and in contact with the trunk, a pulse electrical signal between the first electrode sheet (1) and the second electrode sheet (2) is obtained; Calculating first capture reference data according to the amplitude of the pulse electrical signal; The first captured data is adjusted according to the first captured reference data.
4. The motion capture method in the process of vision detection based on AR display device according to claim 3 is characterized in that: The method further comprises the steps of: adjusting the pulse frequency of the pulse electrical signal according to the amplitude of the pulse electrical signal; The larger the amplitude is, the higher the pulse frequency is; the smaller the amplitude is, the lower the pulse frequency is.
5. The motion capture method in the process of vision detection based on AR display device according to claim 3 is characterized in that: The method further comprises the steps of: Acquiring magnetic field data at an electrical path formed by the first electrode sheet (1) and the second electrode sheet (2); Calculating second capture reference data according to the magnetic field data; The first captured data is adjusted according to the second captured reference data.
6. The motion capture method in the process of vision detection based on AR display device according to claim 5 is characterized in that: The second electrode sheet (2) comprises a roller electrode or a track electrode.
7. The motion capture method in the process of vision detection based on AR display device according to claim 6 is characterized in that: The method further comprises the steps of: Acquiring vibration data at an electrical path formed by the first electrode sheet (1) and the second electrode sheet (2); Calculating third capture reference data according to the vibration data; The first captured data is adjusted according to the third captured reference data.
8. The motion capture method in the process of vision detection based on AR display device according to claim 7 is characterized in that: The method further comprises the steps of: Acquire a vibration component at the trunk, and correct the vibration data according to the vibration component; Alternatively, the vibration data is corrected according to the magnetic field data.
9. A motion capture device based on vision detection of an AR display device, characterized in that: It includes a processor, in which runs a program of the motion capture method based on the vision detection process of an AR display device as described in any one of claims 1 to 8.
10. A storage medium, characterized in that: A program storing a motion capture method based on the vision detection process of an AR display device as described in any one of claims 1 to 8.