Photoelectric particle interaction system and method using gestures

Abstract

The invention relates to the technical field of gesture interaction, and particularly discloses a photoelectric particle interaction system and method using gestures, and the method comprises the following steps: recognizing the gesture moving direction of a target person based on a somatosensory interaction device, and controlling a picture based on the gesture moving direction; monitoring images are collected, and the sight estimation direction of a single person is obtained; acquiring a reference direction, determining an included angle between the sight line estimation direction and the reference direction, and determining a normal person; and counting a target number, setting a target time point based on the target number, and controlling whether to perform gesture interaction based on the target time point. According to the invention, natural control experience can be ensured, and the demonstration rhythm is prevented from being interrupted by false touch.

Description

Technical Field

[0001] The present invention relates to the technical field of gesture interaction, and particularly relates to an optoelectronic particle interaction system and method using gestures. Background Art

[0002] Gesture interaction is to capture the movements of the human hand and body using a camera, depth sensor or wearable device, and through computer vision and machine learning algorithms, these movements are parsed into digital instructions in real time, so as to achieve contactless control of digital content or physical devices. Compared with traditional buttons or touchscreens, gesture interaction has the advantages of being more natural and intuitive, having a strong sense of immersion, being hygienic and friendly, etc., and has been widely used in scenarios such as public exhibition devices.

[0003] In demonstration or teaching scenarios, gesture interaction provides the lecturer with the freedom to control the content without touching the device. However, after the lecturer completes a page turn, zoom in or switch, he often needs to stay on the current screen and continue to elaborate. At this time, he will naturally make some random hand movements, such as adjusting the microphone, picking up the pointer, or even just subconsciously arranging the sleeves. These actions unrelated to the operation are still within the capture range of the camera and are parsed as valid gestures by the system. As a result, the image on the screen is enlarged, dragged, or the slides are jumped, interrupting the narrative rhythm and distracting the audience's attention with the visual changes. Summary of the Invention

[0004] The purpose of the present invention is to provide an optoelectronic particle interaction system and method using gestures, and solve the following technical problems: Actions unrelated to the operation are still within the capture range of the camera and are parsed as valid gestures by the system. As a result, the image on the screen is enlarged, the perspective is dragged, or the slides are jumped, interrupting the narrative rhythm and distracting the audience's attention with the visual changes.

[0005] The purpose of the present invention can be achieved by the following technical solutions: An optoelectronic particle interaction method using gestures, comprising the following steps: Mark the person closest to the demonstration screen and in front of the demonstration screen as the target person, identify the gesture movement direction of the target person based on the somatosensory interaction device, and control the picture in the demonstration screen based on the gesture movement direction, where the gesture movement direction includes forward, backward, left, and right; After controlling the picture in the demonstration screen based on the gesture movement direction, periodically collect the monitoring images in front of the demonstration screen, input the monitoring images into a preset line-of-sight estimation model, and obtain the line-of-sight estimation direction of a single person therein; Obtain the position A1 of the person and the position A2 of the target person, determine the reference direction ACK, where the reference direction points from the position A1 to the position A2, obtain the angle between the line-of-sight estimation direction and the reference direction, and when the angle is less than a preset value, mark the corresponding person as a normal person; Count the total number of the normal persons corresponding to a single monitoring image, denoted as the target number, set a target time point based on the target number, and before the target time point, the gesture movement direction recognized by the somatosensory interaction device does not control the picture on the demonstration screen.

[0006] Preferably, controlling the picture on the demonstration screen based on the gesture movement direction includes: Controlling the moving direction of the picture on the demonstration screen to be the same as the gesture movement direction.

[0007] Preferably, setting the target time point based on the target number includes: Generate coordinate points (ti, Ci), where Ci represents the target number corresponding to the i-th collected monitoring image, and ti represents the time point of the i-th collection of the monitoring image. Fit the coordinate points to obtain a fitting curve f(t), where t represents time; Take the inflection point, starting point, and ending point of the fitting curve as reference points, and denote the part of the fitting curve between any two reference points as a sub-curve; Take the line passing through (0, C') and parallel to the x-axis as a reference line, where C' represents a preset target number threshold. Count the part of a single sub-curve that is below the reference line, denoted as the first part, and calculate the ratio of the domain of the first part to the domain of the corresponding sub-curve, denoted as the first ratio; Sort the first ratios in the order of the time axis, take the first ratio that is greater than a preset first ratio threshold in the sorting as the target ratio, take the sub-curve corresponding to the target ratio as the target curve, and take the time point corresponding to the minimum value on the target curve as the target time point.

[0008] Preferably, when the length of the target curve is less than a preset duration, do not take it as the target curve, and select a new target line to determine the target time point.

[0009] Preferably, setting the target time point based on the target number further includes: When the fitting curve is monotonically decreasing, substitute the target number threshold into the fitting curve f(t) to solve for the target time point T.

[0010] Preferably, controlling the picture on the demonstration screen based on the gesture movement direction further includes: Obtain the average hand movement speed of the target person within a preset time period, and standardize and dimensionless the average hand movement speed to obtain a mapping value η; Calculate the number of optoelectronic particles S = (1 + η) * Dys and the optoelectronic particle movement speed V = (1 + η) * Vys, where Dys and Vys respectively represent the preset initial number of optoelectronic particles and the initial speed of optoelectronic particles.

[0011] Preferably, controlling the picture in the demonstration screen based on the gesture movement direction further includes: Based on Compute Shader or Niagara or VFX Graph, update the position of the optoelectronic particles in real time.

[0012] An optoelectronic particle interaction system using gestures, including: Control module: Mark the person closest to the demonstration screen and in front of the demonstration screen as the target person, identify the gesture movement direction of the target person based on the somatosensory interaction device, and control the picture in the demonstration screen based on the gesture movement direction. The gesture movement direction includes forward, backward, left, and right; Analysis module: After controlling the picture in the demonstration screen based on the gesture movement direction, periodically collect the monitoring images in front of the demonstration screen, input the monitoring images into a preset line-of-sight estimation model, and obtain the line-of-sight estimation direction of a single person among them; Obtain the position A1 of the person and the position A2 of the target person, determine the reference direction ACK, where the reference direction points from the position A1 to the position A2, obtain the included angle between the line-of-sight estimation direction and the reference direction, and when the included angle is less than a preset value, mark the corresponding person as a normal person; Control optimization module: Count the total number of the normal persons corresponding to a single monitoring image, denoted as the target number, set a target time point based on the target number, and before the target time point, the gesture movement direction recognized by the somatosensory interaction device does not control the picture in the demonstration screen.

[0013] The beneficial effects of the present invention: Compared with the prior art: 1) After detecting that the speaker has completed an effective gesture, the present invention immediately freezes the impact of subsequent gestures on the screen, and starts audience sight tracking and target number counting; gesture control is automatically restored only when the system determines that the gaze direction of most of the audience has shifted away from the speaker himself and the attention is no longer focused on his natural movements. In this way, the speaker can adjust his sleeves, take props and other non-control actions at will during the explanation without triggering page turning, zooming or dragging of the screen, avoiding accidental interruption of information display and preventing the audience from being distracted by the jumping of the screen; when the audience's eyes return to the screen and look forward to new content, the control is just right to re-activate, so as to synchronize the demonstration rhythm with the audience's attention and ensure the coherence of the narrative.

[0014] 2) The present invention maps the average speed of the speaker's hand to the number and speed of photoelectric particles, and uses ComputeShader / Niagara / VFX Graph for real-time rendering, so that the particle flow and gesture strength form intuitive and consistent feedback. Gentle movements bring soft transitions, and fast sliding produces neat switching. The interface responds to different intensities in a clear hierarchy and expresses intentions naturally. The particle flow direction is synchronized with the picture, providing the audience with dynamic visual cues, helping to quickly grasp the rhythm of the explanation, enhance stage performance, and improve the immersion and attractiveness of teaching and demonstration.

[0015] 3) Through curve fitting and target time point calculation, the system can automatically identify attention attenuation nodes based on the audience's line of sight, and restore gesture control at the best time to achieve precise alignment between content switching and changes in audience interest. This intelligent rhythm scheduling reduces the embarrassment of waiting and repeated gestures, avoids premature page switching that interrupts understanding, and prevents information gaps caused by staying too long, so that the demonstration process strikes a balance between stability and flexibility, the audience experience is more coherent, and the speaker focuses on the content itself rather than the details of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention will be further described below in conjunction with the accompanying drawings.

[0017] Figure 1 It is a flow chart of a photoelectric particle interaction method using hand gestures according to the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] See also Figure 1As shown in the figure, the present invention is an optoelectronic particle interaction method using gestures, including the following steps: Mark the person who is in front of the demonstration screen and has the closest distance to the demonstration screen as the target person. Based on the somatosensory interaction device, identify the gesture movement direction of the target person, and control the picture on the demonstration screen based on the gesture movement direction. The gesture movement directions include forward, backward, left, and right.

[0020] In a preferred embodiment of the present invention, controlling the picture on the demonstration screen based on the gesture movement direction includes: It should be noted that based on the depth sensor of Microsoft Kinect-v2, the straight-line distance from the thoracic vertebra of all human bones to the screen is calculated for each frame and the person with the smallest distance is selected as the target person. At the same time, the spatial coordinates of their shoulders and hips are recorded, and then the three-dimensional trajectory of the right wrist of the target person is continuously obtained through bone tracking; when the trajectory moves forward more than 20 cm along the depth direction towards the screen within one second and the rotation angle of the palm facing the camera does not exceed 30°, the system determines it as a forward gesture. When the trajectory moves backward more than 20 cm along the depth direction within the same time window and the palm faces the presenter's chest, it is determined as a backward gesture. When the horizontal displacement of the wrist exceeds one-third of the shoulder width and the horizontal displacement of the elbow joint is less than one-tenth of the shoulder width, it is determined as a left or right gesture; after identifying the gesture direction, the system calls the demonstration software interface to make the picture pan or flip in the same direction. For example, detecting a forward gesture triggers the display of the next slide, a backward gesture returns to the previous slide, and detecting left or right correspondingly drags the picture to the left or right as a whole. At the same time, during the dragging process, the optoelectronic particle effect moves synchronously with the picture to ensure visual consistency.

[0021] In a preferred case of this embodiment, the moving direction of the picture on the demonstration screen is the same as the gesture movement direction.

[0022] Controlling the picture on the demonstration screen based on the gesture movement direction further includes: Obtain the average hand movement speed of the target person within a preset period, and standardize and remove the dimension of the average hand movement speed to obtain the mapping value η.

[0023] Calculate the number of optoelectronic particles S = (1 + η) * Dys and the optoelectronic particle movement speed V = (1 + η) * Vys, where Dys and Vys respectively represent the preset initial number of optoelectronic particles and the initial speed of optoelectronic particles.

[0024] It is worth noting that the positions of the optoelectronic particles are updated in real time based on Compute Shader or Niagara or VFX Graph.

[0025] It is understandable that first, the continuous three-dimensional coordinates of the target person's wrist are recorded within a time window of every thirty frames, and then the average hand movement speed is obtained by dividing the displacement between the last frame and the first frame by the window duration. If the window span is five hundred milliseconds and the displacement is zero point two meters, the average speed is zero point four meters per second. Subsequently, this speed is divided by the preset reference speed of one meter per second to complete normalization and remove the dimension, obtaining a mapping value η between zero and one. For example, in the above case, η is zero point four (the value range of η is from 0 to 1). Then, the number of photoelectric particles S and the photoelectric particle movement speed V are calculated. On the rendering side, the Compute Shader will first write the particle state into the structured buffer, and then calculate the displacement vector of each particle in parallel and update its world coordinates. When using Niagara, η is used as a system-level floating-point parameter to drive the Spawn Rate and Velocity Scale of the emitter. If it is implemented in the VFX Graph, η is written into the global attribute and then connected to the PositionUpdate and Velocity Update ports through multiplication nodes to ensure that the number and speed of particles change in real time with the gesture strength. At the same time, the root node of the demo screen in Unity or Unreal receives the direction enumeration from the gesture recognition module and directly calls Transform Translate to move in the same direction. Every time a mapping update is detected, the screen translation and particle redrawing are completed in the same frame, realizing the synchronous response of the screen and particles to the lecturer's gestures.

[0026] Through the above method, when the hand moves in a certain direction, the screen moves in the same direction, making the control process naturally understandable without additional learning. Further, by statistically calculating the average hand movement speed within a short time window and normalizing it, the force information is mapped to the number and speed of photoelectric particles, resulting in a sparse and slow particle flow for gentle actions and a dense and fast particle flow for fast actions. The visual feedback and the action intensity form a one-to-one correspondence, thus enhancing the perceptibility and immersion of the operation. Finally, by using the graphics processing unit parallel shader or particle effect module to update the particle positions in real time, it is possible to handle the movement of thousands of particles while ensuring a high frame rate, keeping the particle effect and the screen content always synchronized and avoiding the fragmentation caused by delays, overall improving the smoothness and visual coherence of the interaction, helping the lecturer to more effectively convey information and continuously attract the audience's attention.

[0027] After controlling the screen in the demo screen based on the gesture movement direction, the monitoring images in front of the demo screen are periodically collected, and the monitoring images are input into a preset gaze estimation model to obtain the gaze estimation direction of a single person therein.

[0028] Exemplarily, after completing a screen translation or page turning triggered by a gesture, the system starts a timer to call the front surveillance camera sampling function at a fixed period of every 200 milliseconds (it can be other values). The camera first crops the color image according to the aspect ratio to retain only the color image within a depth range of two meters in front of the screen, and then sends the image to the face detection module to extract the rectangular face frame; for each face frame, the program expands the boundary at a ratio of five to one to cover the eyes and the center of the eyebrows, and then scales the cropped sub-image to sixty-four times sixty-four pixels after linear normalization as the input tensor The pre-trained convolutional neural network line of sight estimation model is fed into the system; the model outputs two angle values ​​representing the horizontal deflection angle and the vertical deflection angle of the eyeball respectively. The system converts this binary group into a three-dimensional unit vector and maps it to the camera coordinate system. For example, when the horizontal angle is negative fifteen degrees and the vertical angle is negative five degrees, the vector points to the lower left of the screen. If there are multiple faces in the image, they are sorted by the depth value of the center point of each face. Only those with a depth of less than three meters will save the line of sight vector and write it to the queue in the same frame for subsequent reference direction and angle calculation steps to call, thereby achieving real-time estimation of the audience's gaze direction.

[0029] Get the position A1 of the person and the position A2 of the target person, determine the reference direction ACK, the reference direction points from position A1 to position A2, get the angle between the line of sight estimation direction and the reference direction, when the angle is less than the preset value, mark the corresponding person as a normal person.

[0030] It is worth noting that by calculating the angle between the audience's gaze direction and its reference direction facing the speaker and screening normal people accordingly, it is possible to quickly determine which audience members are focusing their gaze on the speaker without adding additional hardware, and then accurately assess the real attention of the current crowd. This mechanism avoids misjudgment based solely on the direction of the face or head posture, because the same body posture facing the speaker may deviate from the line of sight, and the angle threshold can effectively eliminate these interferences, so that the system only counts individuals who are actually listening. In this way, the subsequent decision to freeze or resume gesture control is based on more reliable data, which not only prevents the false triggering of screen changes when the audience is still focused on the speaker, but also ensures that the control channel is opened in time when most of the eyes have moved away, fundamentally keeping the demonstration rhythm synchronized with the audience's attention changes, and improving the overall coherence and interaction accuracy of the program.

[0031] The total number of normal people corresponding to a single surveillance image is counted and recorded as the target number. The target time point is set based on the target number. Before the target time point, the gesture movement direction recognized by the somatosensory interaction device does not control the picture on the demonstration screen.

[0032] It is understandable that by continuously counting the number of normal people whose sight lines meet the conditions in each frame of the image and determining the target number accordingly, and then calculating the target time point from the trend of the target number change, the real moment when the audience's overall attention shifts from the speaker to the screen can be dynamically reflected; temporarily blocking the effect of gestures on the screen before this time point can prevent the speaker from mistakenly triggering page turning or zooming due to irrelevant actions made during the explanation, thereby ensuring stable and coherent presentation of information; when the system detects that the target time point has arrived and the audience's sight is mainly focused on the screen, gesture control is reopened, which not only matches the audience's viewing rhythm, but also allows the speaker to operate more smoothly, thereby achieving a high degree of synchronization between the timing of interaction and the audience's attention, and improving the focus and experience quality of the entire presentation.

[0033] In another preferred embodiment of the present invention, setting the target time point based on the target number includes: Generate coordinate points (ti, Ci), where Ci represents the number of targets corresponding to the surveillance image collected for the i-th time, and ti represents the time point when the surveillance image is collected for the i-th time. Fit the coordinate points to obtain the fitting curve f(t), where t represents time.

[0034] The inflection point, starting point and end point of the fitting curve are taken as reference points, and the part of the fitting curve between any two reference points is recorded as a sub-curve.

[0035] The straight line passing through (0, C') and parallel to the x-axis is used as the reference line, C' represents the preset target number threshold, and the part of a single sub-curve that is smaller than the part below the reference line is counted as the first part. The ratio of the domain of the first part to the domain of the corresponding sub-curve is calculated, which is recorded as the first ratio.

[0036] The first ratios are sorted in timeline order, the first first ratio in the sorting that is greater than a preset first ratio threshold is taken as the target ratio, the sub-curve corresponding to the target ratio is taken as the target curve, and the time point corresponding to the minimum value on the target curve is taken as the target time point.

[0037] It should be noted that fitting the discrete target number sequence into a continuous function using a curve first is to filter out the random fluctuations caused by single-frame jitter, making the attention change trend smoother and more reliable; then dividing the sub-curve with the inflection point as the boundary can distinguish a complete attention decline process from other local fluctuations, thus avoiding mis-triggering control before a stable decay is formed; the inflection point mentioned here refers to the position where the trend of the fitted curve f(t) changes from rising to falling or from falling to rising; mathematically, it can be understood as the point where the sign of the slope of the first derivative of the curve changes or the second derivative changes from positive to negative or from negative to positive at this point, which marks the directional reversal of the change trend of the audience's target number. Therefore, dividing the whole curve with the inflection point can clearly separate a complete attention decay segment from the fluctuations before and after, facilitating the subsequent accurate selection of the target time point; the reference line represents the lowest attention threshold that the system can accept. Statistically calculating the time ratio of the sub-curve falling below the reference line interval is to measure the duration of the audience's attention being lower than the threshold during this period; selecting the first ratio exceeding the threshold in chronological order is essentially to find the first segment that meets the characteristics of continuous low attention among many fluctuations, enabling the system to judge as early as possible and reliably that the audience has shifted their gaze away from the speaker; finally, taking the minimum point of this segment as the target time point can ensure that it truly falls at the bottom of the attention valley, reducing the interference caused by premature or late restoration of gesture control from the root, achieving the interaction state switch at a more accurate timing, and enhancing the intelligence and stability of the overall rhythm.

[0038] It is worth noting that in the present invention, the fitting curve is updated in real time, that is, after obtaining a monitoring image, it is updated once to avoid the interference caused by late restoration of gesture control.

[0039] In a preferred case of this embodiment, when the length of the target curve is less than the preset duration, it is not regarded as the target curve, and a new target line is selected to determine the target time point.

[0040] It can be understood that when the span of the selected target curve on the abscissa is less than the minimum duration preset by the system, the program first calculates the actual time difference between the first and last sampling points of this curve. If this difference is less than a threshold such as two seconds, it is considered that this downward trend is not sufficient to prove that the audience's attention has been fully transferred. Then, the current curve is removed from the candidate list, and the next sub-curve sorted in chronological order is checked. This cycle continues until a target curve that meets the duration requirement is found; if no eligible curve is found after traversing, the system automatically relaxes the threshold or directly uses the position of the overall minimum value as the backup target time point; Another preferred case of this embodiment is that setting the target time point based on the target number further includes: When the fitting curve is monotonically decreasing, substituting the target number threshold into the fitting curve f(t) to solve for the target time point T.

[0041] It should be noted that when it is detected that the fitting curve remains monotonically decreasing from the beginning to the end, that is, the number of targets of each adjacent two sampling points shows a downward or flat trend and there is no recovery, it indicates that the loss of the audience's attention is continuous and stable. At this time, there is no need to calculate in segments, and the system directly substitutes the preset target number threshold into the function expression corresponding to the fitting curve, and obtains the time coordinate of the intersection of the curve and the threshold through the numerical iteration method. For example, when the fitting curve linearly decreases from 15 people to 0 people between 0 and 60 seconds and the threshold is 5 people, it can be calculated that the intersection point is at 30 seconds, and the system then sets 30 seconds as the target time point for restoring gesture control.

[0042] An optoelectronic particle interaction system using gestures, comprising: Control module: Mark the person closest to the demonstration screen in front of the demonstration screen as the target person, identify the gesture movement direction of the target person based on the somatosensory interaction device, and control the picture in the demonstration screen based on the gesture movement direction. The gesture movement directions include forward, backward, left, and right; Analysis module: After controlling the picture in the demonstration screen based on the gesture movement direction, periodically collect the monitoring images in front of the demonstration screen, input the monitoring images into a preset line-of-sight estimation model, and obtain the line-of-sight estimation direction of a single person therein; Obtain the position A1 of the person and the position A2 of the target person, determine the reference direction ACK, the reference direction points from position A1 to position A2, obtain the included angle between the line-of-sight estimation direction and the reference direction, and when the included angle is less than the preset value, mark the corresponding person as a normal person; Control optimization module: Count the total number of normal people corresponding to a single monitoring image, denoted as the target number, set the target time point based on the target number, and before the target time point, the gesture movement direction recognized by the somatosensory interaction device does not control the picture in the demonstration screen.

[0043] The above has described an embodiment of the present invention in detail, but the content described is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of the application of the present invention should still fall within the scope covered by the patent of the present invention.

Claims

1. A photoelectric particle interaction method using gestures, characterized in that: The following steps are involved: Mark the person who is in front of the demonstration screen and closest to the demonstration screen as a target person, identify the gesture movement direction of the target person based on the somatosensory interaction device, and control the image in the demonstration screen based on the gesture movement direction, where the gesture movement direction includes forward, backward, left and right; After the image in the demonstration screen is controlled based on the gesture movement direction, monitoring images in front of the demonstration screen are periodically collected, and the monitoring images are input into a preset sight line estimation model to obtain the sight line estimation direction of a single person therein; The position A1 of the person and the position A2 of the target person are obtained, a reference direction ACK is determined, the reference direction points from the position A1 to the position A2, and an angle between the line of sight estimation direction and the reference direction is obtained. When the angle is less than a preset value, the corresponding person is marked as a normal person; The total number of normal persons corresponding to a single monitoring image is counted and recorded as a target number. A target time point is set based on the target number. Before the target time point, the movement direction of the gesture recognized by the somatosensory interaction device does not control the picture in the demonstration screen.

2. The photoelectric particle interaction method using gestures according to claim 1, characterized in that: Controlling the picture in the demonstration screen based on the gesture movement direction includes: The moving direction of the picture in the demonstration screen is controlled to be the same as the moving direction of the gesture.

3. The photoelectric particle interaction method using gestures according to claim 1, characterized in that: Setting a target time point based on the target number includes: Generate a coordinate point (ti, Ci), where Ci represents the number of targets corresponding to the surveillance image collected for the i-th time, and ti represents the time point when the surveillance image is collected for the i-th time. Fit the coordinate point to obtain a fitting curve f(t), where t represents time. The inflection point, the starting point and the end point of the fitting curve are taken as reference points, and the part of the fitting curve between any two reference points is recorded as a sub-curve; A straight line passing through (0, C') and parallel to the x-axis is used as a reference line, C' represents a preset target number threshold, a portion of a single sub-curve that is smaller than the reference line is counted as a first portion, and a ratio of the domain of the first portion to the domain of the corresponding sub-curve is calculated, which is recorded as a first ratio; The first ratios are sorted in timeline order, the first first ratio in the sort that is greater than a preset first ratio threshold is taken as the target ratio, the sub-curve corresponding to the target ratio is taken as the target curve, and the time point corresponding to the minimum value on the target curve is taken as the target time point.

4. The photoelectric particle interaction method using gestures according to claim 3, characterized in that: When the length of the target curve is less than the preset time length, it is not used as the target curve, and a new target line is selected to determine the target time point.

5. The photoelectric particle interaction method using gestures according to claim 3, characterized in that: Setting the target time point based on the target number also includes: When the fitting curve is monotonically decreasing, the target number threshold is substituted into the fitting curve f(t) to obtain the target time point T.

6. The photoelectric particle interaction method using gestures according to claim 2, characterized in that: Controlling the picture in the demonstration screen based on the gesture movement direction also includes: Obtaining an average hand movement speed of the target person within a preset period, and normalizing the average hand movement speed to remove the dimension, to obtain a mapping value η; Calculate the number of photoelectric particles S = (1 + η) * Dys and the moving speed of photoelectric particles V = (1 + η) * Vys, where Dys and Vys represent the preset initial number of photoelectric particles and the initial speed of photoelectric particles, respectively.

7. The photoelectric particle interaction method using gestures according to claim 6, characterized in that: Controlling the picture in the demonstration screen based on the gesture movement direction also includes: The positions of the photoelectric particles are updated in real time based on Compute Shader, Niagara or VFX Graph.

8. An optoelectronic particle interaction system using gestures, characterized in that: include: Control module: Mark the person who is in front of the demonstration screen and closest to the demonstration screen as the target person, identify the gesture movement direction of the target person based on the somatosensory interaction device, and control the picture in the demonstration screen based on the gesture movement direction, where the gesture movement direction includes forward, backward, left and right; Analysis module: after controlling the image in the demonstration screen based on the direction of the gesture movement, periodically collects monitoring images in front of the demonstration screen, inputs the monitoring images into a preset sight line estimation model, and obtains the estimated sight line direction of a single person therein; The position A1 of the person and the position A2 of the target person are obtained, a reference direction ACK is determined, the reference direction points from the position A1 to the position A2, and an angle between the line of sight estimation direction and the reference direction is obtained. When the angle is less than a preset value, the corresponding person is marked as a normal person; Control optimization module: Count the total number of normal people corresponding to a single monitoring image, record it as the target number, set the target time point based on the target number, and before the target time point, the gesture movement direction recognized by the somatosensory interaction device does not control the picture in the demonstration screen.

Images