A human-computer interaction method and system for somatosensory interaction

The acquisition and conversion of human body image data through somatosensors and mapped to the plane coordinate system of the display screen, solving the problems of human-computer interaction delay and insufficient interaction in the prior art, achieving efficient contactless human-computer interaction, and improving the experience and fun.

CN111870931BActive Publication Date: 2024-11-05HEFEI & EXHIBITION TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010590601.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-06-24
Publication Date
2024-11-05
Estimated Expiration
2040-06-24

AI Technical Summary

Technical Problem

The existing somatosensors only have human body recognition and data generation functions in human-computer interaction, which cannot directly meet the needs of human-computer interaction, and there is interaction delay, resulting in poor human-computer interaction experience.

Method used

The somatosensor obtains the human body image data, converts the three-dimensional coordinate system, maps it to the plane coordinate system of the display screen, calculates the motion vector, and realizes human-computer interaction when the motion vector is greater than the preset threshold, combining the world transformation matrix and the perspective transformation matrix and projection transformation matrix, optimize coordinate transformation and mapping, and reduce interaction latency.

Benefits of technology

It improves the experience of human-computer interaction, realizes contactless human-computer interaction, increases the fun and participatory nature of technology, and supports multi-point interaction and rich interaction needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111870931B_ABST
    Figure CN111870931B_ABST
Patent Text Reader

Abstract

The present invention discloses a human-computer interaction method and system for somatosensory interaction, comprising the following steps: acquiring image data of human limbs through a somatosensory sensor to obtain a three-dimensional coordinate system with the somatosensory sensor as the origin; performing homogeneous coordinate transformation on the three-dimensional coordinate system to obtain a somatosensory two-dimensional coordinate system; selecting a first image in the somatosensory two-dimensional coordinate system as a trigger image, mapping the trigger image to a plane coordinate system preset on a display screen, and obtaining the position coordinates of the trigger image in the plane coordinate system; obtaining a motion vector of the trigger image according to the position coordinates of the trigger image obtained at different times in sequence; when the motion vector is greater than a preset motion threshold, displaying a picture on the display screen to achieve human-computer interaction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a human-computer interaction method and system for somatosensory interaction. Background Art

[0002] Somatosensory games are a new type of electronic game that uses sensors to transmit body movements to gaming devices for control. This breaks away from the traditional control method of simply inputting input via controller buttons, providing players with a better gaming experience. Somatosensory games are not only applicable to electronic games, but also to contactless human-computer interaction, increasing the fun and engagement of technology. However, existing somatosensory sensors, as data collection devices, only have the functions of body recognition and data generation, and cannot directly meet the needs of human-computer interaction. Furthermore, such human-computer interaction devices have interaction delays, resulting in a poor human-computer interaction experience. Summary of the Invention

[0003] Based on the technical problems existing in the background technology, the present invention proposes a human-computer interaction method and system with somatosensory interaction, which improves the experience of human-computer interaction, realizes non-contact human-computer interaction, and increases the fun and participation of science and technology.

[0004] The present invention proposes a human-computer interaction method for somatosensory interaction, comprising the following steps:

[0005] Acquire human limb image data through a somatosensory sensor and obtain a three-dimensional coordinate system with the somatosensory sensor as the origin;

[0006] Perform homogeneous coordinate transformation on the three-dimensional coordinate system to obtain a somatosensory two-dimensional coordinate system;

[0007] Selecting the first image in the somatosensory two-dimensional coordinate system as a trigger image, mapping the trigger image to a plane coordinate system preset on the display screen, and obtaining the position coordinates of the trigger image in the plane coordinate system;

[0008] Obtaining the motion vector of the trigger image according to the position coordinates of the trigger image acquired at different times;

[0009] When the motion vector is greater than the preset motion threshold, the screen displays the picture to achieve human-computer interaction.

[0010] Furthermore, performing homogeneous coordinate transformation on the three-dimensional coordinate system to obtain the somatosensory two-dimensional coordinate system specifically includes:

[0011] The three-dimensional coordinate system with the somatosensory sensor as the origin is multiplied by the world transformation matrix, the perspective transformation matrix and the projection transformation matrix in sequence to obtain the somatosensory two-dimensional coordinate system with the somatosensory sensor as the origin.

[0012] Furthermore, after selecting the first image in the somatosensory two-dimensional coordinate system as the trigger image, the trigger image is mapped to a plane coordinate system preset on the display screen to obtain the position coordinates of each frame of the trigger image, wherein the first image in the somatosensory two-dimensional coordinate system is selected as the trigger image, specifically including:

[0013] The position coordinates of the edge of the display screen are calibrated according to a plane coordinate system preset by the display screen, and the position coordinates of the edge of the display screen are normalized;

[0014] Select a point on the first image as the trigger point, and record the positions of the four corners of the edge of the display screen triggered by the trigger point. The positions of the four corners of the edge are used as the trigger limit positions of the trigger point;

[0015] Obtain and save the coordinate values ​​of the trigger limit position in the plane coordinate system preset on the display screen and the coordinate values ​​in the somatosensory two-dimensional coordinate system after normalization processing, and obtain the coordinate correspondence between the plane coordinate system and the somatosensory two-dimensional coordinate system;

[0016] According to the coordinate correspondence between the plane coordinate system and the somatosensory two-dimensional coordinate system, a mapping formula between the plane coordinate system and the somatosensory two-dimensional coordinate system is calculated;

[0017] The coordinate values ​​in the somatosensory two-dimensional coordinate system are mapped to the plane coordinate system through a mapping formula for display to achieve somatosensory interaction.

[0018] Furthermore, in the step of selecting the first image in the human motion posture image as a trigger image, mapping the trigger image to a plane coordinate system preset on the display screen to obtain position coordinates of each frame of the trigger image, the first image is an image of the hand and elbow, wherein, before mapping the trigger image to the plane coordinate system preset on the display screen, the step includes:

[0019] determining whether the hands and elbows are raised overhead based on the first image;

[0020] If so, the trigger image is mapped to the plane coordinate system preset on the display screen;

[0021] If not, the human limb image data is acquired again through the body sensing sensor.

[0022] Furthermore, in the process of calibrating the position coordinates of the edge of the trigger image according to the normalized position coordinates of the edge of the display screen and mapping the motion parameters in the trigger image onto the display screen, the formula used to map the trigger image onto the display screen is as follows:

[0023]

[0024] The formula used to map the display screen to the trigger image is as follows:

[0025]

[0026] Among them, (y0, y1) is the coordinate of a point in the trigger image in the somatosensory two-dimensional coordinate system, (x0, x1) is the coordinate of a point in the display screen in the plane coordinate system, and a0 and a1 are constants.

[0027] Furthermore, the world transformation matrix is ​​a unit matrix, and the parameters of the somatosensory sensor are: width is 1920, height is 1080, angle is 70°, aspect ratio parameter is set to 16 / 9, cone angle is 70°, near cross section is 0.01, and far cross section is 4.51;

[0028] The construction of the perspective transformation matrix includes:

[0029] Set the position of the color camera of the motion sensor to (0,0,0) and the target position to (0,0,4.5);

[0030] Based on the default up vector, construct the visual transformation and obtain the perspective transformation matrix.

[0031] A somatosensory interactive human-computer interaction system, comprising an acquisition module, a coordinate conversion module, a space mapping module, a coordinate splicing module and a human-computer interaction module;

[0032] The acquisition module is used to acquire image data of human limbs through the somatosensory sensor and obtain a three-dimensional coordinate system with the somatosensory sensor as the origin;

[0033] The coordinate transformation module is used to transform the three-dimensional coordinate system into a homogeneous coordinate system to obtain a somatosensory two-dimensional coordinate system.

[0034] The spatial mapping module is used to select the first image in the somatosensory two-dimensional coordinate system as the trigger image, map the trigger image to the plane coordinate system preset on the display screen, and obtain the position coordinates of each frame of the trigger image;

[0035] The coordinate stitching module is used to obtain the motion vector of the trigger image according to the position coordinates of the trigger image obtained at different times;

[0036] The human-computer interaction module is used to display the screen display image when the motion vector is greater than the preset motion threshold to achieve human-computer interaction.

[0037] A computer-readable storage medium stores a plurality of acquisition and classification programs, wherein the plurality of acquisition and classification programs are used to be called by a processor and execute the human-computer interaction method according to any one of claims 1 to 6.

[0038] A human-computer interaction device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the human-computer interaction method described above is implemented.

[0039] The advantages of the somatosensory interactive human-computer interaction method and system provided by the present invention are as follows: the somatosensory interactive human-computer interaction method and system provided in the structure of the present invention realizes the phenomenon that the display screen displays according to the human body posture by displaying the waving action in the human limb image data in the plane coordinate system of the display screen. Clouds, rain, snow and other weather phenomena can be displayed on the display screen to simulate the alternation of natural phenomena according to the waving posture of the human body. At the same time, the water circulation phenomenon can be simulated by displaying the state of flowing water; the human limb image data obtained by the somatosensory sensor is converted into data that meets the direct needs of human-computer interaction. The display screen displays different pictures according to the human limb movements, which can realize non-contact human-computer interaction and increase the fun and participation of science and technology. In actual application scenarios, this system can be flexibly combined with various sizes and imaging devices, such as transparent screens of various sizes, projectors, spliced ​​TVs, etc., to support multi-point needs of up to 6 people interacting at the same time, meeting rich and diverse interaction needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flowchart of the steps of a somatosensory interactive human-computer interaction method of the present invention;

[0041] Figure 2 Schematic diagram of the correspondence between plane coordinate points and somatosensory two-dimensional coordinate points; DETAILED DESCRIPTION

[0042] The technical solutions of the present invention are described in detail below through specific embodiments. Numerous specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0043] like Figure 1 and 2 As shown, the present invention proposes a somatosensory interactive human-computer interaction method and system, a somatosensory interactive human-computer interaction method, characterized by comprising the following steps:

[0044] S1: Acquire human limb image data through the somatosensory sensor and obtain a three-dimensional coordinate system with the somatosensory sensor as the origin;

[0045] The somatosensory sensor collects data about the audience's body parts and stores it on a per-person basis. This data includes 25 data points for the hands, legs, head, and other parts of the body. The somatosensory sensor is used as the origin for a three-dimensional coordinate system based on the coordinates of the color camera within the sensor. This coordinate system uses a left-handed coordinate system, mirroring the real human body.

[0046] S2: Perform homogeneous coordinate transformation on the three-dimensional coordinate system to obtain a somatosensory two-dimensional coordinate system;

[0047] The three-dimensional coordinate system with the somatosensory sensor as the origin is multiplied by the world transformation matrix, the perspective transformation matrix and the projection transformation matrix in sequence to obtain the somatosensory two-dimensional coordinate system with the somatosensory sensor as the origin.

[0048] When capturing human limb image data from the somatosensor, the human skeleton nodes themselves are generated with the color camera in the somatosensor as the origin and do not contain any additional displacement information. Therefore, the world transformation matrix uses the identity matrix.

[0049] The color space of a general motion sensor is measured in meters and ranges from 0 to 4.5 meters. The color camera position of the motion sensor is set to (0, 0, 0), the target position is set to (0, 0, 4.5), and the default up vector is used to construct a perspective transformation. The perspective transformation is performed on the three-dimensional coordinate system after the world transformation.

[0050] The target position is the orientation vector of the motion sensor's color camera. The combination of these two parameters means the camera is at (0, 0, 0) and facing (0, 0, 4.5). The target position (0, 0, 4.5) is chosen because the maximum distance in the raw motion sensor data is 4.5 meters.

[0051] The final color image of the motion sensor's color space has the following parameters: width 1920, height 1080, angle 70°, aspect ratio 16 / 9, cone angle 70°, near section 0.01, and far section 4.51. The near section and far section are two relative section positions. Using these parameters, a projection transformation matrix is ​​established, and the resulting three-dimensional coordinate system is projected. Data along the Z axis can be ignored, retaining only the data along the X and Y axes. This results in a two-dimensional motion coordinate system with the motion sensor as the origin.

[0052] After establishing the three matrices of world transformation, perspective transformation, and projection transformation through the above parameters, each time the human limb image data of the three-dimensional coordinate system with the somatosensory sensor as the origin is received, it can be converted into human limb image data of the somatosensory two-dimensional coordinate system with the somatosensory sensor as the origin.

[0053] S3: Selecting the first image in the somatosensory two-dimensional coordinate system as a trigger image, mapping the trigger image to a plane coordinate system preset on the display screen, and obtaining the position coordinates of the trigger image in the plane coordinate system;

[0054] In this application, the first image is an image of the hand and elbow. Therefore, before mapping the trigger image to the plane coordinate system preset on the display screen, it is necessary to detect whether the first image is used as the trigger image. The detection and judgment method is as follows:

[0055] Determine whether the hand and elbow are raised above the head based on the first image; if so, map the trigger image to the plane coordinate system preset on the display screen; if not, enter step S1 to re-acquire human limb image data through the somatosensory sensor.

[0056] S4: Obtaining the motion vector of the trigger image according to the position coordinates of the trigger image acquired at different times;

[0057] S5: When the motion vector is greater than a preset motion threshold, the screen displays a picture to achieve human-computer interaction.

[0058] When the hand in the first image waves, different position coordinates of the trigger image in the plane coordinate system are obtained according to the direction and time of the hand waving, and the different position coordinates are connected in sequence to obtain the motion vector of the trigger image. The motion vector of the trigger image corresponds to the waving action of the hand. Therefore, it is determined whether the motion vector is greater than the preset motion threshold. If so, the screen displays the picture. If not, it returns to step S1 to re-acquire the human limb image data.

[0059] Through steps S1 to S5, the display screen is displayed according to the human body posture. Weather phenomena such as clouds, rain, and snow can be displayed on the display screen to simulate the alternation of natural phenomena according to the waving gesture of the human body. At the same time, by displaying the state of flowing water, the circulation phenomenon of water can be simulated.

[0060] At the same time, by mapping the human limb image data to the plane coordinate system of the display screen through a mapping formula, the human limb image data contained in each triggered image can be displayed in the plane coordinate system in a timely manner, and the motion vector of the triggered image can be obtained in a timely manner, thereby realizing the response to the triggered image movement, reducing the interaction delay during human-computer interaction, improving the experience of human-computer interaction, realizing non-contact human-computer interaction, and increasing the fun and participation of science and technology.

[0061] For example: set up a scene box, and place a photoelectric sand table model of mountains, rivers, land and sea in the scene box. Behind the sand table is a display screen. The audience stands in the designated area and waves their arms. The water vapor on the display screen begins to evaporate from the ocean and form clouds. Move your feet, and the clouds will float to the top of the mountain and condense into rain, snow and hail. Waving your arms down will make them fall, comprehensively displaying the process of water cycle on the earth.

[0062] In step S3, the first image in the somatosensory two-dimensional coordinate system is selected as a trigger image, and the trigger image is mapped to a plane coordinate system preset on the display screen to obtain the position coordinates of each frame of the trigger image. The first image in the somatosensory two-dimensional coordinate system is selected as the trigger image, specifically including:

[0063] S31: calibrating the position coordinates of the edge of the display screen according to a preset plane coordinate system of the display screen, and performing normalization processing on the position coordinates of the edge of the display screen;

[0064] A plane coordinate system is constructed with the upper left corner of the display screen as the origin. During the calibration process of the position coordinates of the display screen edge, manual calibration can be performed and the calibrated coordinates can be saved. The purpose of normalization is to facilitate data processing and map the data to the range of 0 to 1 for faster and more convenient processing. For example, the actual display screen resolution is generally 1920×1080. Therefore, in the system, the coordinates of the four points on the upper left, upper right, lower left, and lower right of the display screen are (0, 0), (1920, 0), (0, 1080), and (1920, 1080), respectively.

[0065] S32: Select a point on the first image as a trigger point, and record the positions of the four corners of the edge of the display screen triggered by the trigger point, and the positions of the four corners of the edge are used as the trigger limit positions of the trigger point;

[0066] In this application, the first image is an image of a hand and an elbow, so the hand in the first image can be selected as the trigger point.

[0067] S33: Obtaining and saving the coordinate values ​​of the trigger limit position in the plane coordinate system preset on the display screen and the coordinate values ​​in the somatosensory two-dimensional coordinate system after normalization processing, and obtaining the coordinate correspondence between the plane coordinate system and the somatosensory two-dimensional coordinate system;

[0068] When the first image triggers the four edge corners of the plane coordinate system preset on the screen, the first image also has corresponding coordinate values ​​in the somatosensory two-dimensional coordinate system. Since the four edge corners of the plane coordinate system have been calibrated [(0, 0), (1920, 0), (0, 1080), (1920, 1080)], there will be four sets of coordinate correspondences between the plane coordinate system and the somatosensory two-dimensional coordinate system.

[0069] S34: Calculating a mapping formula between the plane coordinate system and the somatosensory two-dimensional coordinate system according to a coordinate correspondence between the plane coordinate system and the somatosensory two-dimensional coordinate system;

[0070] Through the correspondence between the four sets of coordinates and according to vector calculation, the following mapping formula (1) is obtained:

[0071]

[0072] The mapping formula (2) used to map the display screen to the trigger image is as follows:

[0073]

[0074] Among them, (y0, y1) is the coordinate of a point in the trigger image in the somatosensory two-dimensional coordinate system, (x0, x1) is the coordinate of a point in the display screen in the plane coordinate system, and a0 and a1 are constants.

[0075] a0 and a1 are determined by the following calculation:

[0076] like Figure 2 As shown, o, (0,1,0), (1,0,0), and (1,1,0) are four known coordinate points in the plane coordinate system, which can be obtained by normalizing the position coordinates of the pre-calibrated display screen edge; o, q01, q10, and q11 are known coordinate points in the somatosensory two-dimensional coordinate system, which can also be obtained by pre-calibration. In order to make the four points o, q01, q10, and q11 coplanar, the best method is to express the vector oq11 as a linear combination of the vector oq01 and the vector oq10, which is recorded as q11=a0q10+a1q01.

[0077] As long as any quadrilateral formed by o, q01, q 10, and q 11 is given, the two values ​​of a0 and a1 can be directly calculated, so a0 and a1 can be regarded as constants.

[0078] Knowing the coordinates of any point in the trigger image in the somatosensory two-dimensional coordinate system, the corresponding coordinates of that point in the plane coordinate system of the display screen can be calculated using the mapping formula (1). Knowing the coordinates of any point in the plane coordinate system of the display screen, the corresponding coordinates of that point in the somatosensory two-dimensional coordinate system of the trigger image can be inferred using the mapping formula (2). In this way, any known point can be flexibly used in the interaction to calculate the corresponding point, and then achieve interaction under various conditions.

[0079] S35: Mapping the coordinate values ​​in the somatosensory two-dimensional coordinate system to the plane coordinate system through a mapping formula for display, so as to achieve somatosensory interaction.

[0080] Through the obtained mapping formula, in subsequent human-computer interaction, the coordinate mapping of the three-dimensional coordinate system with the somatosensory sensor as the origin to the plane coordinate system preset on the display screen can be directly realized through this mapping formula. That is, when the hand is at any position, the plane coordinate system of the display screen can display the position coordinates of the corresponding hand. Therefore, when there is a waving action, the motion vector of the waving direction will be obtained in the plane coordinate system. When the motion vector is greater than the preset motion threshold, the display screen will display different pictures.

[0081] A somatosensory interactive human-computer interaction system, comprising an acquisition module, a coordinate conversion module, a space mapping module, a coordinate splicing module and a human-computer interaction module;

[0082] The acquisition module is used to acquire image data of human limbs through the somatosensory sensor and obtain a three-dimensional coordinate system with the somatosensory sensor as the origin;

[0083] The coordinate transformation module is used to transform the three-dimensional coordinate system into a homogeneous coordinate system to obtain a somatosensory two-dimensional coordinate system.

[0084] The spatial mapping module is used to select the first image in the somatosensory two-dimensional coordinate system as the trigger image, map the trigger image to the plane coordinate system preset on the display screen, and obtain the position coordinates of each frame of the trigger image;

[0085] The coordinate stitching module is used to obtain the motion vector of the trigger image according to the position coordinates of the trigger image obtained at different times;

[0086] The human-computer interaction module is used to display the screen display image when the motion vector is greater than the preset motion threshold to achieve human-computer interaction.

[0087] A computer-readable storage medium stores a plurality of acquisition and classification programs, wherein the plurality of acquisition and classification programs are used to be called by a processor and execute the human-computer interaction method as described above.

[0088] A human-computer interaction device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the human-computer interaction method described above is implemented.

[0089] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various media that can store program codes.

[0090] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A human-computer interaction method for somatosensory interaction, characterized in that: The steps include: S1, obtaining human limb image data through a somatosensory sensor to obtain a three-dimensional coordinate system with the somatosensory sensor as the origin; S2, transforming the three-dimensional coordinate system into a homogeneous coordinate system to obtain a somatosensory two-dimensional coordinate system; S3, selecting the first image in the somatosensory two-dimensional coordinate system as a trigger image, mapping the trigger image to a plane coordinate system preset on the display screen, and obtaining the position coordinates of the trigger image in the plane coordinate system; S31, calibrating the position coordinates of the edge of the display screen according to the plane coordinate system preset by the display screen, and normalizing the position coordinates of the edge of the display screen; S32, selecting a point on the first image as a trigger point, and recording the positions of the four corners of the edge of the display screen triggered by the trigger point, and the positions of the four corners of the edge are used as the trigger limit positions of the trigger point; S33, obtaining and saving the coordinate value of the trigger limit position in the plane coordinate system preset on the display screen and the coordinate value in the somatosensory two-dimensional coordinate system after normalization processing, and obtaining the coordinate correspondence between the plane coordinate system and the somatosensory two-dimensional coordinate system; The formula for mapping the somatosensory two-dimensional coordinate system to the plane coordinate system is as follows: The formula for mapping the plane coordinate system to the somatosensory two-dimensional coordinate system is as follows: Wherein, (y0, y1) is the coordinate of a point in the trigger image in the somatosensory two-dimensional coordinate system, (x0, x1) is the coordinate of a point in the display screen in the plane coordinate system, and a0 and a1 are constants; S34, calculating a mapping formula between the plane coordinate system and the body-sensing two-dimensional coordinate system according to the coordinate correspondence between the plane coordinate system and the body-sensing two-dimensional coordinate system; S35, mapping the coordinate values ​​in the somatosensory two-dimensional coordinate system to the plane coordinate system for display through a mapping formula, so as to realize somatosensory interaction; S4, obtaining the motion vector of the trigger image according to the position coordinates of the trigger image acquired at different times; S5. When the motion vector is greater than a preset motion threshold, the screen displays a picture to achieve human-computer interaction.

2. The human-computer interaction method of somatosensory interaction according to claim 1, characterized in that: The step of performing homogeneous coordinate transformation on the three-dimensional coordinate system to obtain the somatosensory two-dimensional coordinate system specifically includes: The three-dimensional coordinate system with the somatosensory sensor as the origin is multiplied by the world transformation matrix, the perspective transformation matrix and the projection transformation matrix in sequence to obtain a somatosensory two-dimensional coordinate system with the somatosensory sensor as the origin.

3. The human-computer interaction method of somatosensory interaction according to claim 2, characterized in that: The world transformation matrix is ​​a unit matrix, and the parameters of the somatosensory sensor are: width is 1920, height is 1080, angle is 70°, aspect ratio parameter is set to 16 / 9, cone angle is 70°, near section is 0.01, and far section is 4.51; The construction of the perspective transformation matrix includes: Set the position of the color camera of the motion sensor to (0,0,0) and the target position to (0,0,4.5); Based on the default up vector, construct the visual transformation and get the perspective transformation matrix.

4. A somatosensory interactive human-computer interaction system, characterized in that: It includes acquisition module, coordinate conversion module, space mapping module, coordinate splicing module and human-computer interaction module; The acquisition module is used to acquire human limb image data through the somatosensory sensor to obtain a three-dimensional coordinate system with the somatosensory sensor as the origin; The coordinate conversion module is used to perform homogeneous coordinate conversion on the three-dimensional coordinate system to obtain a somatosensory two-dimensional coordinate system. The spatial mapping module is used to select the first image in the somatosensory two-dimensional coordinate system as a trigger image, map the trigger image to a plane coordinate system preset on the display screen, and obtain the position coordinates of each frame of the trigger image. The coordinate stitching module is used to obtain the motion vector of the trigger image according to the position coordinates of the trigger image acquired at different times in sequence; The human-computer interaction module is used to display the screen display image when the motion vector is greater than the preset motion threshold to achieve human-computer interaction; The spatial mapping module is specifically used for: According to a preset plane coordinate system of the display screen, the position coordinates of the edge of the display screen are calibrated, and the position coordinates of the edge of the display screen are normalized; Select a point on the first image as a trigger point, and record the positions of the four corners of the edge of the display screen triggered by the trigger point, and the positions of the four corners of the edge are used as the trigger limit positions of the trigger point; Obtain and save the coordinate value of the trigger limit position in the plane coordinate system preset on the display screen and the coordinate value in the somatosensory two-dimensional coordinate system after normalization processing, and obtain the coordinate correspondence between the plane coordinate system and the somatosensory two-dimensional coordinate system; According to the coordinate correspondence between the plane coordinate system and the somatosensory two-dimensional coordinate system, a mapping formula between the plane coordinate system and the somatosensory two-dimensional coordinate system is calculated; The formula for mapping the somatosensory two-dimensional coordinate system to the plane coordinate system is as follows: The formula for mapping the plane coordinate system to the somatosensory two-dimensional coordinate system is as follows: Wherein, (y0, y1) is the coordinate of a point in the trigger image in the somatosensory two-dimensional coordinate system, (x0, x1) is the coordinate of a point in the display screen in the plane coordinate system, and a0 and a1 are constants; The coordinate values ​​in the somatosensory two-dimensional coordinate system are mapped to the plane coordinate system for display through a mapping formula to achieve somatosensory interaction.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of acquisition and classification programs, and the plurality of acquisition and classification programs are used to be called by a processor and execute the human-computer interaction method as described in any one of claims 1 to 3.

6. A human-computer interaction device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the human-computer interaction method as claimed in any one of claims 1 to 3 is implemented.