A motion trajectory recognition method and system for handheld game console

By setting a non-collinear marking point and an infrared camera in the handheld controller, combining acceleration data and marking point calibration processing, the position information distortion problem of accelerometer data during long-term movement is solved, and high accuracy is achieved in complex environments Trajectory recognition.

CN118807193BActive Publication Date: 2024-11-22东莞市三奕电子科技股份有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411312403.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-20
Publication Date
2024-11-22
Estimated Expiration
2044-09-20

AI Technical Summary

Technical Problem

The existing handheld machine motion trajectory recognition method relies on accelerometer data, resulting in distortion of position information during long-term or large-scale movement, lack of an effective external correction mechanism, and the deviation caused by cumulative errors cannot be corrected in time.

Method used

Set at least 3 non-collinear marking points within the imaging range of the infrared camera, and detect the numerical rate of change through acceleration data. If the rate of change is less than or equal to the instantaneous rate of change, the position offset will be calculated by secondary integral. If the rate of change is greater than the instantaneous rate of change, the position offset will be calculated by secondary integral. The mark point calibration process is performed at rate, and the actual offset is obtained by comparing the mark point images and the trajectory is corrected.

Benefits of technology

Through marking point calibration processing and infrared camera-assisted positioning, the accumulated error can be corrected in a timely manner in a fast moving state, improving the accuracy of trajectory recognition and positioning accuracy, and ensuring that high positioning accuracy is maintained in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118807193B_ABST
    Figure CN118807193B_ABST
Patent Text Reader

Abstract

The present invention discloses a motion trajectory recognition method and system for a handheld game console, including setting at least three non-collinear marking points within the imaging range of an infrared camera; collecting acceleration data of a handheld game console handle in real time and detecting the value change rate; when the value change rate is less than or equal to a preset instantaneous change rate, estimating the position offset through quadratic integral calculation, and fitting the handheld game console handle trajectory; when the value change rate is greater than the instantaneous change rate, performing a marking point calibration process, that is, collecting a marking point image through an infrared camera, comparing the marking point images at the current moment with the previous moment to obtain the offset, and correcting the handheld game console handle trajectory accordingly. The present invention can timely correct the accumulated error generated by the integration of accelerometer data in a fast moving state through the marking point calibration process, thereby improving the accuracy of trajectory recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of distance measurement, and in particular to a motion trajectory recognition method and system for a handheld game console. Background Art

[0002] Existing handheld game console motion trajectory recognition methods usually rely on built-in accelerometers to obtain the acceleration data of the handheld game console. These data are then used to calculate the position change of the handheld game console through secondary integral calculation.

[0003] Since the accelerometer outputs acceleration signals, it needs to be integrated twice to get the position information. In this process, each integration will introduce errors, especially when the handheld device is moved for a long time or over a large range, these small errors will gradually accumulate and eventually lead to serious distortion of the position information. The method of trajectory recognition based solely on accelerometer data lacks an effective external correction mechanism and cannot correct the deviation caused by the accumulated error of the accelerometer in a timely manner. Summary of the invention

[0004] In order to solve at least one of the above-mentioned technical problems, the present invention provides a motion trajectory recognition method and system for a handheld game console.

[0005] In a first aspect, the present invention provides a method for identifying a motion trajectory of a handheld game console, the method comprising:

[0006] At least three non-collinear marking points are set within the imaging range of an infrared camera, wherein the infrared camera is set in the handheld game console handle;

[0007] Based on the preset collection frequency, the acceleration data of the handheld controller is collected in real time, and the value change rate of the acceleration data is detected;

[0008] If the value change rate is less than or equal to the preset instantaneous change rate, the estimated position offset at the current acquisition time is obtained by performing a second integration based on the acceleration data, and the estimated position offsets at different acquisition times are fitted to obtain the handheld controller trajectory;

[0009] If the value change rate is greater than the instantaneous change rate, the mark point calibration process is performed;

[0010] The marker point calibration process includes:

[0011] An infrared camera is used to collect a marking point image containing at least three marking points, and the marking point offset is obtained by comparing the marking point images at the current moment with the previous moment. The actual offset of the handheld game controller is obtained based on the marking point offset; the trajectory of the handheld game controller is corrected based on the actual offset.

[0012] Preferably, after performing secondary integration according to the acceleration data to obtain the estimated position offset at the current acquisition time, the method further includes updating the marking point, specifically including:

[0013] Collect low-frequency stable angle data and high-frequency dynamic angle data of the handheld game console, and use complementary filters to fuse the low-frequency stable angle data and high-frequency dynamic angle data to obtain the handheld game console posture angle;

[0014] According to the handheld device posture angle and the estimated position offset, the inner margin between the marking point and the imaging range of the infrared camera is calculated to obtain the inner margin of the marking point;

[0015] If the inner margin distance of any marker point is less than the preset first inner margin distance threshold, a new marker point image is collected by the infrared camera, and the central candidate area is obtained based on the largest marker point inner margin distance. The pixel block with the largest contrast in the central candidate area is selected to replace the current marker point.

[0016] Preferably, the step of calculating the inner margin between the marking point and the imaging range of the infrared camera according to the handheld device posture angle and the estimated position offset to obtain the inner margin of the marking point includes:

[0017] The handheld device posture angle and the estimated position offset are input into a preset marker point position prediction algorithm to obtain the marker point estimated position. The marker point position prediction algorithm is expressed as:

[0018] ,

[0019] ,

[0020] ,

[0021] ,

[0022] ,

[0023] ,

[0024] ,

[0025] ,

[0026] In the formula, is the rotation matrix, is the identity matrix, is the coordinate of the marking point at time t1, are the coordinates of the rotation center, is the estimated position offset of the marker from time t1 to time t2, and , For the handheld posture angle, is the unit vector of the rotation axis, Represents the vector Transformed into the corresponding antisymmetric matrix, Marking point Relative to the center of rotation The position offset, is the coordinate after rotation and translation, is the coordinate of the marked point at time t2;

[0027] According to the estimated position of the marking point, the inner margin between the marking point and the imaging range of the infrared camera is calculated to obtain the inner margin of the marking point.

[0028] Preferably, after acquiring a new marker point image by using an infrared camera, the method further includes executing the marker point calibration process.

[0029] In a second aspect, the present invention further provides a motion trajectory recognition system for a handheld game console, the system comprising:

[0030] A marking point setting module, used to set at least three non-collinear marking points within the imaging range of an infrared camera, wherein the infrared camera is set in the handheld game console;

[0031] The acceleration detection module is used to collect the acceleration data of the handheld controller in real time based on a preset collection frequency and detect the value change rate of the acceleration data;

[0032] The trajectory prediction module is used to obtain the estimated position offset at the current acquisition time by performing a second integration based on the acceleration data if the value change rate is less than or equal to the preset instantaneous change rate, and to fit the estimated position offsets at different acquisition times to obtain the handheld controller trajectory;

[0033] A track calibration module, used to perform a mark point calibration process if the value change rate is greater than the instantaneous change rate;

[0034] The trajectory calibration module comprises:

[0035] A marker point offset prediction unit, used to collect a marker point image containing at least three marker points through an infrared camera, and obtain a marker point offset by comparing the marker point image at a current moment with the marker point image at a previous moment;

[0036] The trajectory calibration unit is used to obtain the actual offset of the handheld controller according to the mark point offset; and calibrate the trajectory of the handheld controller according to the actual offset.

[0037] Preferably, the system further comprises: a marking point updating module, specifically comprising:

[0038] The posture angle calculation unit is used to collect the low-frequency stable angle data and high-frequency dynamic angle data of the handheld game console, and use a complementary filter to fuse the low-frequency stable angle data and the high-frequency dynamic angle data to obtain the handheld game console posture angle;

[0039] The marking point inner margin calculation unit is used to calculate the inner margin between the marking point and the imaging range of the infrared camera according to the handheld device posture angle and the estimated position offset to obtain the marking point inner margin;

[0040] The marking point updating unit is used to collect a new marking point image through an infrared camera if the inner margin of any marking point is less than a preset first inner margin threshold, obtain a central candidate area based on the largest marking point inner margin, and select the pixel block with the largest contrast in the central candidate area to replace the current marking point.

[0041] Preferably, after acquiring a new marker point image by using an infrared camera, the method further includes executing the marker point calibration process.

[0042] In a third aspect, the present invention further provides an electronic device comprising a processor and a memory, wherein the memory is used to store computer program code, and the computer program code comprises computer instructions. When the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation thereof.

[0043] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, wherein the computer program includes program instructions, and when the program instructions are executed by a processor of an electronic device, the processor executes the method as described in the first aspect above and any possible implementation thereof.

[0044] Compared with the prior art, the present invention can achieve at least one of the following technical effects:

[0045] 1) The present invention can correct the accumulated error caused by the integration of accelerometer data in a fast-moving state in a timely manner through the marker point calibration process, thereby improving the accuracy of trajectory recognition. Furthermore, at least three non-collinear marker points and an infrared camera are used for auxiliary positioning, which can maintain a high positioning accuracy even in a complex environment.

[0046] 2) The present invention fuses low-frequency stable angle data and high-frequency dynamic angle data and uses complementary filters to obtain accurate handheld gesture angles, thereby improving the stability and reliability of the entire system. Furthermore, through the marker point update mechanism, the marker points that are about to move out of the imaging range are automatically replaced to ensure that the marker points are always valid, reducing tracking failures caused by marker point loss.

[0047] 3) The present invention obtains the estimated position of the marker point through the marker point position prediction algorithm, which can complete the calculation process faster. Especially in application scenarios that require real-time feedback, this algorithm can provide a faster response speed. In games or interactive applications, a faster response speed means a better user experience. The player's actions can be reflected in the game in real time, improving the realism and immersion of the game. Furthermore, the marker point position prediction algorithm has a small amount of calculation and low requirements on hardware resources. In comparison, neural networks not only require a large amount of training data, but also require more computing resources during operation.

[0048] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments of the present invention or the background technology will be described below.

[0050] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0051] Figure 1 A schematic diagram of a flow chart of a motion trajectory recognition method for a handheld game console provided by an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of the structure of a motion trajectory recognition system for a handheld game console provided by an embodiment of the present invention;

[0053] Figure 3 The embodiment of the present invention provides Figure 2 A schematic diagram of the structure of the submodules of the trajectory calibration module;

[0054] Figure 4 The embodiment of the present invention provides Figure 2 Schematic diagram of the structure of the submodules of the marker point update module. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0057] In the prior art, the method of handheld game consoles to perform trajectory recognition solely based on accelerometer data lacks an effective external correction mechanism and is unable to correct the deviation caused by the accumulated error of the accelerometer in a timely manner.

[0058] See also Figure 1 , Figure 1 The present invention provides a flow chart of a motion trajectory recognition method for a handheld game console. Figure 1 As shown, the method includes:

[0059] S100, setting at least three marking points within an imaging range of an infrared camera, wherein the infrared camera is set in a handheld game console handle;

[0060] In this embodiment, at least three non-collinear marking points are selected to ensure that a stable triangle can be constructed as a reference base. The marking points can be highly reflective spheres or other targets that are easily captured by the infrared camera. The infrared camera is installed in the handheld controller to ensure that its field of view can cover the entire operating area.

[0061] S200, collecting acceleration data of the handheld game controller in real time based on a preset collection frequency, and detecting a value change rate of the acceleration data;

[0062] The built-in acceleration sensor is used to obtain acceleration data at a fixed sampling frequency (for example, 100 Hz), and the rate of change of the acceleration data is obtained by performing differential calculation on the acceleration values ​​between two consecutive sampling points.

[0063] S300, if the value change rate is less than or equal to the preset instantaneous change rate, perform a second integration based on the acceleration data to obtain an estimated position offset at the current acquisition time, fit the estimated position offsets at different acquisition times, and obtain a handheld controller trajectory;

[0064] If the rate of change of the acceleration data is lower than the preset instantaneous rate of change threshold (i.e. the controller is considered to be in a relatively stable state), the current position offset is calculated using the quadratic integral method. A smooth trajectory curve is generated by continuously fitting the position offset of each sampling point.

[0065] S400, if the value change rate is greater than the instantaneous change rate, perform a mark point calibration process;

[0066] The marker point calibration process includes:

[0067] An infrared camera is used to collect a marking point image containing at least three marking points, and the marking point offset is obtained by comparing the marking point images at the current moment with the previous moment. The actual offset of the handheld game controller is obtained based on the marking point offset; the trajectory of the handheld game controller is corrected based on the actual offset.

[0068] When the rate of change of the acceleration data exceeds the set threshold (i.e. the controller is considered to be in a fast-moving state), the marker point calibration process is triggered. The marker point calibration process includes:

[0069] An infrared camera is used to collect a marking point image containing at least three marking points, and the marking point offset is obtained by comparing the marking point images at the current moment with the previous moment. The actual offset of the handheld game controller is obtained based on the marking point offset; the trajectory of the handheld game controller is corrected based on the actual offset.

[0070] Use an infrared camera to capture images containing marker points, and use image processing techniques (such as edge detection or feature matching) to determine the position and changes of the marker points. According to the changes of the marker points, the actual displacement is calculated, and the trajectory estimated by the accelerometer data is corrected accordingly.

[0071] In this embodiment, through the marker point calibration processing, the cumulative error caused by the integration of accelerometer data can be corrected in time in a fast-moving state, thereby improving the accuracy of trajectory recognition. Furthermore, at least three non-collinear marker points and an infrared camera are used for auxiliary positioning, which can maintain a high positioning accuracy even in complex environments.

[0072] Preferably, after performing secondary integration according to the acceleration data to obtain the estimated position offset at the current acquisition time, the method further includes updating the marking point, specifically including:

[0073] Collect low-frequency stable angle data and high-frequency dynamic angle data of the handheld game console, and use complementary filters to fuse the low-frequency stable angle data and high-frequency dynamic angle data to obtain the handheld game console posture angle;

[0074] According to the handheld device posture angle and the estimated position offset, the inner margin between the marking point and the imaging range of the infrared camera is calculated to obtain the inner margin of the marking point;

[0075] If the inner margin distance of any marker point is less than the preset first inner margin distance threshold, a new marker point image is collected by the infrared camera, and the central candidate area is obtained based on the largest marker point inner margin distance. The pixel block with the largest contrast in the central candidate area is selected to replace the current marker point.

[0076] In this embodiment, low-frequency stable angle data of the handheld game controller is obtained through the built-in gyroscope, and high-frequency dynamic angle data is obtained through the accelerometer, and a complementary filter is used to fuse the two types of data to obtain a more accurate handheld game console posture angle. It should be noted that the complementary filter is used to fuse data sources of different frequencies, and the weight factor can be adjusted in real time, so that the gyroscope data is mainly used in the low-frequency part, and the accelerometer data is mainly used in the high-frequency part, thereby obtaining smoother and more accurate posture angle information.

[0077] According to the fused handheld console posture angle and the estimated position offset, the distance between each marker point and the four boundaries of the infrared camera imaging range, that is, the inner margin of the marker point, is calculated.

[0078] When the inner margin distance of a certain marking point is less than the preset first inner margin distance threshold, it means that the marking point is about to move out of the imaging range. At this time, a new marking point image is collected by the infrared camera, and the marking point corresponding to the maximum inner margin distance is found based on the inner margin distances of all current marking points.

[0079] The selection rule of the center candidate area is: find the mark point corresponding to the maximum inner margin, and Figure 4 The edges are scaled down proportionally until the corresponding marking point is on the frame of the scaled-down image, and then the scaling is stopped. The image area corresponding to the current scaled-down viewfinder view is the center candidate area.

[0080] In the selected central candidate area, the pixel block with the highest contrast is found as the new marker point. This can be achieved through image processing technology, such as edge detection or feature extraction methods, and the old marker point that is about to move out of the imaging range is replaced by the new marker point found to ensure that at least 3 marker points are always within the detection range of the infrared camera.

[0081] In this embodiment, by fusing low-frequency stable angle data and high-frequency dynamic angle data, a complementary filter is used to obtain accurate handheld device posture angles, thereby improving the stability and reliability of the entire system. Furthermore, through the marker point update mechanism, the marker points that are about to move out of the imaging range are automatically replaced to ensure that the marker points are always valid, thereby reducing tracking failures caused by marker point loss.

[0082] Preferably, the step of calculating the inner margin between the marking point and the imaging range of the infrared camera according to the handheld device posture angle and the estimated position offset to obtain the inner margin of the marking point includes:

[0083] The handheld device posture angle and the estimated position offset are input into a preset marker point position prediction algorithm to obtain the marker point estimated position. The marker point position prediction algorithm is expressed as:

[0084] ,

[0085] ,

[0086] ,

[0087] ,

[0088] ,

[0089] ,

[0090] ,

[0091] ,

[0092] In the formula, is the rotation matrix, is the identity matrix, is the coordinate of the marking point at time t1, are the coordinates of the rotation center, is the estimated position offset of the marker from time t1 to time t2, and , For the handheld posture angle, is the unit vector of the rotation axis, Represents the vector Transformed into the corresponding antisymmetric matrix, Marking point Relative to the center of rotation The position offset, is the coordinate after rotation and translation, is the coordinate of the marked point at time t2;

[0093] According to the estimated position of the marking point, the inner margin between the marking point and the imaging range of the infrared camera is calculated to obtain the inner margin of the marking point.

[0094] In this embodiment, the position of each marking point is estimated according to the marking point position prediction algorithm, and the steps of marking point position estimation are as follows:

[0095] Set Mark Point The world coordinate system coordinates of , the center of rotation , move the marker point to the center of rotation to get , the rotation matrix Obtained by the Rodriguez rotation formula ,in, is the identity matrix, is the rotation axis unit vector, Represents the unit vector of the rotation axis The corresponding antisymmetric matrix of the transformation is, The coordinates after the rotation matrix transformation are , Add the estimated position offset of the marker from time t1 to time t2 to get Finally, move the marker point from the rotation center back to the selected position to obtain the world coordinate system coordinates of the final marker point .

[0096] set up , the normal vector of the plane where the viewfinder is located is , and the plane where the viewfinder is located takes a known point , in order to Projected onto the plane where the viewfinder is located, you need to find The shortest distance to the plane where the viewfinder is located, and along the direction of the normal vector Move this shortest distance and calculate it using the distance formula from point to plane Distance to the plane of the viewfinder :

[0097] ;

[0098] Will Distance moved along the normal vector ,get .

[0099] count , select two orthogonal vectors on the plane where the viewfinder is located as basis vectors and , and the basis vector and Orthogonality and Normal Vectors , you can choose For any one Orthogonal unit vectors, and for and The cross product of .

[0100] Let the center of the viewfinder be the coordinate origin and use the basis vector and Will Convert to coordinates on the plane where the viewfinder is located:

[0101] ;

[0102] ;

[0103] Set the viewfinder width to , Gao Wei ,judge Is it within the imaging range? If the above conditions are met, it means Within the imaging range, further calculation of marker points The closest distance to the edge of the viewfinder (i.e. the inner margin):

[0104] Calculate the distance to the left: ,

[0105] Calculate the distance to the right: ,

[0106] Calculate the distance to the top: ,

[0107] Calculate the distance to the bottom: ,

[0108] The minimum value is the minimum inner margin of the marking point. .

[0109] In this embodiment, the estimated position of the marker point is calculated by the marker point position prediction algorithm, so the calculation process can be completed more quickly. Especially in application scenarios that require real-time feedback, this algorithm can provide a faster response speed. In games or interactive applications, a faster response speed means a better user experience. The player's actions can be reflected in the game in real time, improving the realism and immersion of the game. Furthermore, the marker point position prediction algorithm has a small amount of calculation and low requirements for hardware resources. In comparison, neural networks not only require a large amount of training data, but also require more computing resources during operation.

[0110] Preferably, after acquiring a new marker point image by using an infrared camera, the method further includes executing the marker point calibration process.

[0111] In this embodiment, when it is detected that the right marking point needs to be updated, not only the marking point calibration processing is performed, but also calculation and calibration are performed based on the old marking point data at the previous moment, to ensure a smooth transition between the new and old marking points, to ensure that the initial position of the new marking point is more accurate, and to reduce the trajectory error caused by the sudden change of the marking point. Furthermore, in the case that there is no data at the previous moment of the new marking point, the last data of the old marking point is retained and calculated and calibrated, which can effectively reduce the occurrence of trajectory fault phenomenon.

[0112] In summary, the method provided in this embodiment can achieve at least one of the following technical effects:

[0113] 1) The present invention can correct the accumulated error caused by the integration of accelerometer data in a fast-moving state in a timely manner through the marker point calibration process, thereby improving the accuracy of trajectory recognition. Furthermore, at least three non-collinear marker points and an infrared camera are used for auxiliary positioning, which can maintain a high positioning accuracy even in a complex environment.

[0114] 2) The present invention fuses low-frequency stable angle data and high-frequency dynamic angle data and uses complementary filters to obtain accurate handheld gesture angles, thereby improving the stability and reliability of the entire system. Furthermore, through the marker point update mechanism, the marker points that are about to move out of the imaging range are automatically replaced to ensure that the marker points are always valid, reducing tracking failures caused by marker point loss.

[0115] 3) The present invention obtains the estimated position of the marker point through the marker point position prediction algorithm, which can complete the calculation process faster. Especially in application scenarios that require real-time feedback, this algorithm can provide a faster response speed. In games or interactive applications, a faster response speed means a better user experience. The player's actions can be reflected in the game in real time, improving the realism and immersion of the game. Furthermore, the marker point position prediction algorithm has a small amount of calculation and low requirements on hardware resources. In comparison, neural networks not only require a large amount of training data, but also require more computing resources during operation.

[0116] See also Figure 2 In one embodiment, a motion trajectory recognition system for a handheld game console is also provided, the system comprising:

[0117] The marking point setting module 100 is used to set at least three marking points within the imaging range of the infrared camera, wherein the infrared camera is set in the handheld game console;

[0118] The acceleration detection module 200 is used to collect the acceleration data of the handheld game controller in real time based on a preset collection frequency and detect the value change rate of the acceleration data;

[0119] The trajectory prediction module 300 is used to obtain the estimated position offset at the current acquisition time by performing a second integration based on the acceleration data if the value change rate is less than or equal to the preset instantaneous change rate, and to fit the estimated position offsets at different acquisition times to obtain the handheld controller trajectory;

[0120] The track calibration module 400 is used to perform a mark point calibration process if the value change rate is greater than the instantaneous change rate;

[0121] See also Figure 3 , the trajectory calibration module 400 includes:

[0122] The marker offset prediction unit 410 is used to collect a marker image containing at least three markers through an infrared camera, and obtain a marker offset by comparing the marker image at a current moment with the marker image at a previous moment;

[0123] The trajectory calibration unit 420 is used to obtain the actual offset of the handheld game controller according to the mark point offset; and calibrate the trajectory of the handheld game controller according to the actual offset.

[0124] Preferably, continue to refer to Figure 2 and Figure 4 The system further includes: a marking point updating module 500, specifically including:

[0125] The posture angle calculation unit 510 is used to collect the low-frequency stable angle data and high-frequency dynamic angle data of the handheld game console, and fuse the low-frequency stable angle data and the high-frequency dynamic angle data using a complementary filter to obtain the handheld game console posture angle;

[0126] The marking point inner margin calculation unit 520 is used to calculate the inner margin between the marking point and the imaging range of the infrared camera according to the handheld device posture angle and the estimated position offset to obtain the marking point inner margin;

[0127] The marker point updating unit 530 is used to collect a new marker point image through an infrared camera if the inner margin of any marker point is less than a preset first inner margin threshold, obtain a central candidate area based on the largest marker point inner margin, and select the pixel block with the largest contrast in the central candidate area to replace the current marker point.

[0128] Preferably, after acquiring a new marker point image by using an infrared camera, the method further includes executing the marker point calibration process.

[0129] It can be understood that the functions or modules included in the system provided in this embodiment can be used to execute the method described in the above method embodiment. Its specific implementation can refer to the description of the above method embodiment. For the sake of brevity, it will not be repeated here.

[0130] The present invention also provides an electronic device, including a processor and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes a method as described in any possible implementation manner.

[0131] The present invention also provides a computer-readable storage medium, in which a computer program is stored. The computer program includes program instructions. When the program instructions are executed by a processor of an electronic device, the processor executes a method as described in any possible implementation manner.

[0132] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. Those skilled in the art can also clearly understand that the descriptions of the various embodiments of the present invention have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, refer to the records of other embodiments.

Claims

1. A method for identifying the motion trajectory of a handheld game console. It is characterized in that The method comprises: At least three non-collinear marking points are set within the imaging range of an infrared camera, wherein the infrared camera is set in the handheld game console handle; Based on the preset collection frequency, the acceleration data of the handheld controller is collected in real time, and the value change rate of the acceleration data is detected; If the value change rate is less than or equal to the preset instantaneous change rate, the estimated position offset at the current acquisition time is obtained by performing a second integration based on the acceleration data, and the estimated position offsets at different acquisition times are fitted to obtain the handheld controller trajectory; If the value change rate is greater than the instantaneous change rate, the mark point calibration process is performed; The marker point calibration process includes: The infrared camera is used to collect a mark point image containing at least three mark points, and the mark point offset is obtained by comparing the mark point images at the current moment with the mark point images at the previous moment. The actual offset of the handheld game controller is obtained according to the mark point offset; and the trajectory of the handheld game controller is corrected according to the actual offset. After the estimated position offset at the current acquisition time is obtained by performing secondary integration according to the acceleration data, the marking point is also updated, specifically including: Collect low-frequency stable angle data and high-frequency dynamic angle data of the handheld game console, and use complementary filters to fuse the low-frequency stable angle data and high-frequency dynamic angle data to obtain the handheld game console posture angle; According to the handheld device posture angle and the estimated position offset, the inner margin between the marking point and the imaging range of the infrared camera is calculated to obtain the inner margin of the marking point; If the inner margin distance of any marker point is less than the preset first inner margin distance threshold, a new marker point image is collected by the infrared camera, and the central candidate area is obtained based on the largest marker point inner margin distance. The pixel with the largest contrast in the central candidate area is selected and the block replaces the current marker point.

2. The motion trajectory recognition method of a handheld game console according to claim 1, It is characterized in that The method of calculating the inner margin between the marking point and the imaging range of the infrared camera according to the handheld device posture angle and the estimated position offset to obtain the inner margin of the marking point includes: The handheld device posture angle and the estimated position offset are input into a preset marker point position prediction algorithm to obtain the marker point estimated position. The marker point position prediction algorithm is expressed as: , , , , , , , , In the formula, is the rotation matrix, is the identity matrix, is the coordinate of the marking point at time t1, are the coordinates of the rotation center, is the estimated position offset of the marker from time t1 to time t2, and , For the handheld posture angle, is the unit vector of the rotation axis, Represents the vector Transformed into the corresponding antisymmetric matrix, Marking point Relative to the center of rotation The position offset, is the coordinate after rotation and translation, is the coordinate of the marked point at time t2; According to the estimated position of the marking point, the inner margin between the marking point and the imaging range of the infrared camera is calculated to obtain the inner margin of the marking point.

3. The motion trajectory recognition method of a handheld game console according to claim 1, It is characterized in that After the new marker point image is collected by the infrared camera, the marker point calibration process is also performed.

4. A motion trajectory recognition system for a handheld game console, It is characterized in that The system comprises: A marking point setting module, used to set at least three non-collinear marking points within the imaging range of an infrared camera, wherein the infrared camera is set in the handheld game console; The acceleration detection module is used to collect the acceleration data of the handheld controller in real time based on a preset collection frequency and detect the value change rate of the acceleration data; The trajectory prediction module is used to obtain the estimated position offset at the current acquisition time by performing a second integration based on the acceleration data if the value change rate is less than or equal to the preset instantaneous change rate, and to fit the estimated position offsets at different acquisition times to obtain the handheld controller trajectory; A track calibration module, used to perform a mark point calibration process if the value change rate is greater than the instantaneous change rate; The trajectory calibration module comprises: A marker point offset prediction unit, used to collect a marker point image containing at least three marker points through an infrared camera, and obtain a marker point offset by comparing the marker point image at a current moment with the marker point image at a previous moment; The track calibration unit is used to obtain the actual offset of the handheld gamepad according to the offset of the marking point; and calibrate the track of the handheld gamepad according to the actual offset; The system further includes: a marking point updating module, specifically including: The posture angle calculation unit is used to collect the low-frequency stable angle data and high-frequency dynamic angle data of the handheld game console, and use a complementary filter to fuse the low-frequency stable angle data and the high-frequency dynamic angle data to obtain the handheld game console posture angle; The marking point inner margin calculation unit is used to calculate the inner margin between the marking point and the imaging range of the infrared camera according to the handheld device posture angle and the estimated position offset to obtain the marking point inner margin; The marking point updating unit is used to collect a new marking point image through an infrared camera if the inner margin of any marking point is less than a preset first inner margin threshold, obtain a central candidate area based on the largest marking point inner margin, and select the pixel block with the largest contrast in the central candidate area to replace the current marking point.

5. The motion trajectory recognition system for a handheld game console according to claim 4, It is characterized in that After the new marker point image is collected by the infrared camera, the marker point calibration process is also performed.

6. An electronic device, It is characterized in that include: A processor and a memory, wherein the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and when the processor executes the computer instructions, the electronic device executes the motion trajectory recognition method for a handheld game console as described in any one of claims 1 to 3.

7. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor of an electronic device, the processor executes the motion trajectory recognition method for a handheld game console according to any one of claims 1 to 3.