An optoelectronic particle interaction system and method using gestures

Abstract

The present invention relates to the technical field of gesture interaction, and specifically discloses an optoelectronic particle interaction system and method using gestures. The method includes the following steps: identifying the gesture movement direction of a target person based on a somatosensory interaction device, and controlling a screen based on the gesture movement direction; collecting a monitoring image and obtaining the estimated line-of-sight direction of a single person therein; obtaining a reference direction, determining the included angle between the estimated line-of-sight direction and the reference direction, and determining a normal person; counting the number of targets, setting a target time point based on the number of targets, and controlling whether to perform gesture interaction based on the target time point. The present invention can ensure a natural control experience while avoiding accidental touch actions from interrupting the demonstration rhythm.

Description

Technical Field

[0001] The present invention relates to the technical field of gesture interaction, and particularly relates to an optoelectronic particle interaction system and method using gestures. Background Art

[0002] Gesture interaction is to capture the hand and limb movements of the human body using a camera, depth sensor or wearable device, and through computer vision and machine learning algorithms, these movements are parsed into digital instructions in real time, so as to achieve contactless control of digital content or physical devices. Compared with traditional buttons or touchscreens, gesture interaction has the advantages of being more natural and intuitive, having a strong sense of immersion, being hygienic and friendly, etc., and has been widely applied in scenarios such as public exhibition devices.

[0003] In a demonstration or teaching scenario, gesture interaction provides the explainer with the freedom to manipulate the content without touching the device. However, when the explainer completes a page turn, zoom in or switch, he often needs to stay on the current screen and continue to elaborate. At this time, he will naturally make some random hand movements, such as adjusting the microphone, picking up the pointer, or even just subconsciously arranging the sleeves. These actions unrelated to manipulation are still within the capture range of the camera and are parsed as valid gestures by the system. As a result, the image on the screen is enlarged, dragged, or the slides jump, interrupting the narrative rhythm and distracting the audience's attention with the visual changes. Summary of the Invention

[0004] The purpose of the present invention is to provide an optoelectronic particle interaction system and method using gestures, and solve the following technical problems:

[0005] Actions unrelated to manipulation are still within the capture range of the camera and are parsed as valid gestures by the system. As a result, the image on the screen is enlarged, the viewing angle is dragged, or the slides jump, interrupting the narrative rhythm and distracting the audience's attention with the visual changes.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] An optoelectronic particle interaction method using gestures, comprising the following steps:

[0008] Mark the person closest to the demonstration screen and in front of the demonstration screen as the target person, identify the gesture movement direction of the target person based on the somatosensory interaction device, and control the picture in the demonstration screen based on the gesture movement direction, where the gesture movement direction includes forward, backward, left, and right;

[0009] After controlling the picture in the demonstration screen based on the gesture movement direction, periodically collect the monitoring image in front of the demonstration screen, input the monitoring image into a preset line-of-sight estimation model, and obtain the line-of-sight estimation direction of a single person therein;

[0010] Obtain the position A1 of the person and the position A2 of the target person, determine the reference direction ACK, where the reference direction points from the position A1 to the position A2, obtain the included angle between the line-of-sight estimation direction and the reference direction, and when the included angle is less than a preset value, mark the corresponding person as a normal person;

[0011] Count the total number of the normal persons corresponding to a single monitoring image, denoted as the target number, set a target time point based on the target number, and before the target time point, the gesture movement direction recognized by the somatosensory interaction device does not control the picture in the presentation screen.

[0012] Preferably, controlling the picture in the presentation screen based on the gesture movement direction includes:

[0013] Control the moving direction of the picture in the presentation screen to be the same as the gesture movement direction.

[0014] Preferably, setting the target time point based on the target number includes:

[0015] Generate coordinate points (ti, Ci), where Ci represents the target number corresponding to the i-th collected monitoring image, ti represents the time point of the i-th collection of the monitoring image, perform fitting on the coordinate points to obtain a fitting curve f(t), and t represents time;

[0016] Take the inflection point, starting point, and ending point of the fitting curve as reference points, and denote the part of the fitting curve between any two reference points as a sub-curve;

[0017] Take the straight line passing through (0, C') and parallel to the x-axis as a reference line, where C' represents a preset target number threshold, count the part of a single sub-curve that is below the reference line, denoted as the first part, and calculate the ratio of the domain of the first part to the domain of the corresponding sub-curve, denoted as the first ratio;

[0018] Sort the first ratios in the order of the time axis, take the first ratio that is greater than a preset first ratio threshold in the sorting as the target ratio, take the sub-curve corresponding to the target ratio as the target curve, and take the time point corresponding to the minimum value on the target curve as the target time point.

[0019] Preferably, when the length of the target curve is less than a preset duration, do not take it as the target curve, and select a new target line to determine the target time point.

[0020] Preferably, setting the target time point based on the target number further includes:

[0021] When the fitting curve is monotonically decreasing, substitute the target number threshold into the fitting curve f(t) to solve for the target time point T.

[0022] Preferably, controlling the picture on the demonstration screen based on the gesture moving direction further includes:

[0023] Obtain the average hand moving speed of the target person within a preset period, standardize and remove the dimension of the average hand moving speed to obtain a mapping value η;

[0024] Calculate the number of optoelectronic particles S = (1 + η) * Dys and the optoelectronic particle moving speed V = (1 + η) * Vys, where Dys and Vys respectively represent the preset initial number of optoelectronic particles and the initial speed of optoelectronic particles.

[0025] Preferably, controlling the picture on the demonstration screen based on the gesture moving direction further includes:

[0026] Based on Compute Shader or Niagara or VFX Graph, update the positions of the optoelectronic particles in real time.

[0027] An optoelectronic particle interaction system using gestures includes:

[0028] Control module: Mark the person closest to the demonstration screen and in front of it as the target person, identify the gesture moving direction of the target person based on the somatosensory interaction device, and control the picture on the demonstration screen based on the gesture moving direction, where the gesture moving direction includes forward, backward, left, and right;

[0029] Analysis module: After controlling the picture on the demonstration screen based on the gesture moving direction, periodically collect the monitoring images in front of the demonstration screen, input the monitoring images into a preset line-of-sight estimation model to obtain the line-of-sight estimation direction of a single person among them;

[0030] Obtain the position A1 of the person and the position A2 of the target person, determine the reference direction ACK, where the reference direction points from the position A1 to the position A2, obtain the included angle between the line-of-sight estimation direction and the reference direction, and when the included angle is less than the preset value, mark the corresponding person as a normal person;

[0031] Control optimization module: Count the total number of the normal persons corresponding to a single monitoring image, denoted as the target number, set a target time point based on the target number, and before the target time point, the gesture moving direction recognized by the somatosensory interaction device does not control the picture on the demonstration screen.

[0032] The beneficial effects of the present invention: Compared with the prior art:

[0033] 1) After detecting that the speaker has completed an effective gesture, the present invention immediately freezes the impact of subsequent gestures on the screen, and starts audience sight tracking and target number counting; gesture control is automatically restored only when the system determines that the gaze direction of most of the audience has shifted away from the speaker himself and the attention is no longer focused on his natural movements. In this way, the speaker can adjust his sleeves, take props and other non-control actions at will during the explanation without triggering page turning, zooming or dragging of the screen, avoiding accidental interruption of information display and preventing the audience from being distracted by the jumping of the screen; when the audience's eyes return to the screen and look forward to new content, the control is just right to re-activate, so as to synchronize the demonstration rhythm with the audience's attention and ensure the coherence of the narrative.

[0034] 2) The present invention maps the average speed of the speaker's hand to the number and speed of photoelectric particles, and uses ComputeShader / Niagara / VFX Graph for real-time rendering, so that the particle flow and gesture strength form intuitive and consistent feedback. Gentle movements bring soft transitions, and fast sliding produces neat switching. The interface responds to different intensities in a clear hierarchy and expresses intentions naturally. The particle flow direction is synchronized with the picture, providing the audience with dynamic visual cues, helping to quickly grasp the rhythm of the explanation, enhance stage performance, and improve the immersion and attractiveness of teaching and demonstration.

[0035] 3) Through curve fitting and target time point calculation, the system can automatically identify attention attenuation nodes based on the audience's line of sight, and restore gesture control at the best time to achieve precise alignment between content switching and changes in audience interest. This intelligent rhythm scheduling reduces the embarrassment of waiting and repeated gestures, avoids premature page switching that interrupts understanding, and prevents information gaps caused by staying too long, so that the demonstration process strikes a balance between stability and flexibility, the audience experience is more coherent, and the speaker focuses on the content itself rather than the details of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention will be further described below in conjunction with the accompanying drawings.

[0037] Figure 1 It is a flow chart of a photoelectric particle interaction method using hand gestures according to the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] See also Figure 1As shown in the figure, the present invention is an optoelectronic particle interaction method using gestures, including the following steps:

[0040] Mark the person closest to the demonstration screen in front of the demonstration screen as the target person, identify the gesture movement direction of the target person based on the somatosensory interaction device, and control the picture on the demonstration screen based on the gesture movement direction. The gesture movement directions include forward, backward, left, and right.

[0041] In a preferred embodiment of the present invention, controlling the picture on the demonstration screen based on the gesture movement direction includes:

[0042] It should be noted that based on the depth sensor of Microsoft Kinect-v2, the straight-line distance from the thoracic vertebra of all human bones to the screen is calculated for each frame and the person with the smallest distance is selected as the target person. At the same time, the spatial coordinates of their shoulders and hips are recorded, and then the three-dimensional trajectory of the right wrist of the target person is continuously obtained through bone tracking; when the trajectory moves forward more than 20 cm in the depth direction towards the screen within one second and the rotation angle of the palm facing the camera does not exceed 30°, the system determines it as a forward gesture. When the trajectory moves backward more than 20 cm in the depth direction within the same time window and the palm faces the presenter's chest, it is determined as a backward gesture. When the horizontal displacement of the wrist exceeds one-third of the shoulder width and the horizontal displacement of the elbow joint is less than one-tenth of the shoulder width, it is determined as a left or right gesture; after identifying the gesture direction, the system calls the demonstration software interface to make the picture pan or flip in the same direction. For example, detecting a forward gesture triggers the display of the next slide, a backward gesture returns to the previous slide, and detecting left or right correspondingly drags the picture to the left or right as a whole, while keeping the optoelectronic particle effect moving synchronously with the picture during the dragging process to ensure visual consistency.

[0043] In a preferred case of this embodiment, the moving direction of the picture on the demonstration screen is the same as the gesture movement direction.

[0044] Controlling the picture on the demonstration screen based on the gesture movement direction further includes:

[0045] Obtain the average moving speed of the target person's hand within a preset period, standardize and remove the dimension of the average moving speed of the hand to obtain the mapping value η.

[0046] Calculate the number of optoelectronic particles S = (1 + η) * Dys and the optoelectronic particle moving speed V = (1 + η) * Vys, where Dys and Vys respectively represent the preset initial number of optoelectronic particles and the initial speed of optoelectronic particles.

[0047] It should be noted that the positions of the optoelectronic particles are updated in real time based on Compute Shader or Niagara or VFX Graph.

[0048] It can be understood that first, the continuous three-dimensional coordinates of the target person's wrist are recorded within a time window of every thirty frames. Then, the average hand movement speed is obtained by dividing the displacement between the last frame and the first frame by the window duration. If the window span is five hundred milliseconds and the displacement is zero point two meters, the average speed is zero point four meters per second. Subsequently, this speed is divided by the preset reference speed of one meter per second to complete normalization and remove the dimension, obtaining a mapping value η between zero and one. For example, in the above case, η is zero point four (the value range of η is from 0 to 1). Next, the number of optoelectronic particles S and the movement speed V of the optoelectronic particles are calculated. On the rendering side, the Compute Shader will first write the particle state into the structured buffer, and then calculate the displacement vector of each particle in parallel and update its world coordinates. When using Niagara, η is used as a system-level floating-point parameter to drive the Spawn Rate and Velocity Scale of the emitter. If it is implemented in the VFX Graph, η is written into the global attribute and then connected to the PositionUpdate and Velocity Update ports through multiplication nodes to ensure that the number and speed of particles change in real time with the gesture intensity. At the same time, the root node of the demonstration screen in Unity or Unreal receives the direction enumeration from the gesture recognition module and directly calls Transform Translate to move in the same direction. Every time a mapping update is detected, the screen translation and particle redrawing are completed in the same frame, realizing the synchronous response of the screen and particles to the lecturer's gestures.

[0049] Through the above method, when the hand moves in a certain direction, the screen moves in the same direction, making the control process naturally understandable without additional learning. Further, by statistically calculating the average hand movement speed within a short time window and normalizing it, the intensity information is mapped to the number and speed of optoelectronic particles, resulting in a sparse and slow particle flow for gentle actions and a dense and fast particle flow for rapid actions. The visual feedback and the action intensity form a one-to-one correspondence, thereby enhancing the perceptibility and immersion of the operation. Finally, the particle positions are updated in real time using the graphics processing unit parallel shader or the particle effect module, which can handle the movement of thousands of particles while ensuring a high frame rate, keeping the particle effect and the screen content always synchronized and avoiding the sense of fragmentation caused by delays, overall improving the smoothness and visual coherence of the interaction, helping the lecturer to more effectively convey information and continuously attract the audience's attention.

[0050] After controlling the screen in the demonstration screen based on the gesture movement direction, the monitoring image in front of the demonstration screen is periodically collected, and the monitoring image is input into a preset line-of-sight estimation model to obtain the line-of-sight estimation direction of a single person therein.

[0051] Exemplarily, after completing a screen translation or page turning triggered by a gesture, the system starts a timer to call the front surveillance camera sampling function at a fixed period of every 200 milliseconds (it can be other values). The camera first crops the color image according to the aspect ratio to retain only the color image within a depth range of two meters in front of the screen, and then sends the image to the face detection module to extract the rectangular face frame; for each face frame, the program expands the boundary at a ratio of five to one to cover the eyes and the center of the eyebrows, and then scales the cropped sub-image to sixty-four times sixty-four pixels after linear normalization as the input tensor The pre-trained convolutional neural network line of sight estimation model is fed into the system; the model outputs two angle values representing the horizontal deflection angle and the vertical deflection angle of the eyeball respectively. The system converts this binary group into a three-dimensional unit vector and maps it to the camera coordinate system. For example, when the horizontal angle is negative fifteen degrees and the vertical angle is negative five degrees, the vector points to the lower left of the screen. If there are multiple faces in the image, they are sorted by the depth value of the center point of each face. Only those with a depth of less than three meters will save the line of sight vector and write it to the queue in the same frame for subsequent reference direction and angle calculation steps to call, thereby achieving real-time estimation of the audience's gaze direction.

[0052] Get the position A1 of the person and the position A2 of the target person, determine the reference direction ACK, the reference direction points from position A1 to position A2, get the angle between the estimated line of sight direction and the reference direction, and when the angle is less than the preset value, mark the corresponding person as a normal person.

[0053] It is worth noting that by calculating the angle between the audience's gaze direction and its reference direction facing the speaker and screening normal people accordingly, it is possible to quickly determine which audience members are focusing their gaze on the speaker without adding additional hardware, and then accurately assess the real attention of the current crowd. This mechanism avoids misjudgment based solely on the direction of the face or head posture, because the same body posture facing the speaker may deviate from the line of sight, and the angle threshold can effectively eliminate these interferences, so that the system only counts individuals who are actually listening. In this way, the subsequent decision to freeze or resume gesture control is based on more reliable data, which not only prevents the false triggering of screen changes when the audience is still focused on the speaker, but also ensures that the control channel is opened in time when most of the eyes have moved away, fundamentally keeping the demonstration rhythm synchronized with the audience's attention changes, and improving the overall coherence and interaction accuracy of the program.

[0054] The total number of normal people corresponding to a single surveillance image is counted and recorded as the target number. The target time point is set based on the target number. Before the target time point, the gesture movement direction recognized by the somatosensory interaction device does not control the picture on the demonstration screen.

[0055] It is understandable that by continuously counting the number of normal people whose sight lines meet the conditions in each frame of the image and determining the target number accordingly, and then calculating the target time point from the trend of the target number change, the real moment when the audience's overall attention shifts from the speaker to the screen can be dynamically reflected; temporarily blocking the effect of gestures on the screen before this time point can prevent the speaker from mistakenly triggering page turning or zooming due to irrelevant actions made during the explanation, thereby ensuring stable and coherent presentation of information; when the system detects that the target time point has arrived and the audience's sight is mainly focused on the screen, gesture control is reopened, which not only matches the audience's viewing rhythm, but also allows the speaker to operate more smoothly, thereby achieving a high degree of synchronization between the timing of interaction and the audience's attention, and improving the focus and experience quality of the entire presentation.

[0056] In another preferred embodiment of the present invention, setting the target time point based on the target number includes:

[0057] Generate coordinate points (ti, Ci), where Ci represents the number of targets corresponding to the surveillance image collected for the i-th time, and ti represents the time point when the surveillance image is collected for the i-th time. Fit the coordinate points to obtain the fitting curve f(t), where t represents time.

[0058] The inflection point, starting point and end point of the fitting curve are taken as reference points, and the part of the fitting curve between any two reference points is recorded as a sub-curve.

[0059] The straight line passing through (0, C') and parallel to the x-axis is used as the reference line, C' represents the preset target number threshold, and the part of a single sub-curve that is smaller than the part below the reference line is counted as the first part. The ratio of the domain of the first part to the domain of the corresponding sub-curve is calculated, which is recorded as the first ratio.

[0060] The first ratios are sorted in timeline order, the first first ratio in the sorting that is greater than a preset first ratio threshold is taken as the target ratio, the sub-curve corresponding to the target ratio is taken as the target curve, and the time point corresponding to the minimum value on the target curve is taken as the target time point.

[0061] It should be noted that fitting the discrete target number sequence into a continuous function using a curve first is to filter out the random fluctuations caused by single-frame jitter, making the attention change trend smoother and more reliable; then, the sub-curves are segmented with inflection points as boundaries, which can distinguish a complete attention decline process from other local fluctuations, thus avoiding mis-triggering control before a stable decay is formed; the inflection point mentioned here refers to the position where the trend of the fitted curve f(t) changes from rising to falling or from falling to rising; mathematically, it can be understood as the point where the sign of the slope of the first derivative of the curve changes or the second derivative changes from positive to negative or from negative to positive at this point, which marks the directional reversal of the change trend of the audience's target number. Therefore, by segmenting the entire curve with inflection points, a complete attention decay segment can be clearly separated from the fluctuations before and after, facilitating the subsequent accurate selection of the target time point; the reference line represents the lowest attention threshold that the system can accept. Calculate the ratio of the time when the sub-curve falls below the reference line interval, which is equal to measuring the duration of the audience's attention being lower than the threshold during this period; select the ratio that exceeds the threshold for the first time in chronological order. Substantially, it is to find the first section that meets the characteristics of continuous low attention among many fluctuations, enabling the system to judge as early as possible and reliably that the audience has shifted their gaze from the presenter; finally, take the minimum point of this section as the target time point, which can ensure that it truly falls at the bottom of the attention valley, reducing the interference caused by premature or late restoration of gesture control from the root, achieving the interaction state switch at a more accurate time, and enhancing the intelligence and stability of the overall rhythm.

[0062] It is worth noting that in the present invention, the fitted curve is updated in real time, that is, after obtaining a monitoring image, it is updated once to avoid the interference caused by late restoration of gesture control.

[0063] In a preferred case of this embodiment, when the length of the target curve is less than the preset duration, it is not used as the target curve, and a new target line is selected to determine the target time point.

[0064] It can be understood that when the span of the selected target curve on the abscissa is less than the minimum duration preset by the system, the program first calculates the actual time difference between the first and last sampling points of the curve. If the difference is less than a threshold such as two seconds, it is considered that the downward trend of this section is not sufficient to prove that the audience's attention has been fully transferred. Therefore, the current curve is removed from the candidate list, and the next sub-curve sorted in chronological order is checked. This cycle continues until a target curve that meets the duration requirement is found; if no eligible curve is found after traversing, the system automatically relaxes the threshold or directly uses the position of the overall minimum value as the backup target time point;

[0065] Another preferred case of this embodiment is that setting the target time point based on the target number further includes:

[0066] When the fitting curve is monotonically decreasing, substitute the target number threshold into the fitting curve f(t) to solve for the target time point T.

[0067] It should be noted that when it is detected that the fitting curve remains monotonically decreasing from start to end, that is, the target numbers of every two adjacent sampling points show a downward or flat trend without any recovery, it indicates that the loss of the audience's attention is continuous and stable. At this time, there is no need to calculate in segments. The system directly substitutes the preset target number threshold into the function expression corresponding to this fitting curve, and uses the numerical iteration method to obtain the time coordinate where the curve intersects the threshold. For example, when the fitting curve linearly decreases from 15 people to 0 people between 0 and 60 seconds and the threshold is 5 people, it can be calculated that the intersection point is at 30 seconds. The system then sets 30 seconds as the target time point to restore gesture control.

[0068] An optoelectronic particle interaction system using gestures, comprising:

[0069] A control module: marks the person closest to the demonstration screen in front of the demonstration screen as the target person, identifies the gesture movement direction of the target person based on the somatosensory interaction device, and controls the picture on the demonstration screen based on the gesture movement direction. The gesture movement directions include forward, backward, left, and right;

[0070] An analysis module: after controlling the picture on the demonstration screen based on the gesture movement direction, periodically collects the monitoring images in front of the demonstration screen, inputs the monitoring images into a preset line-of-sight estimation model to obtain the line-of-sight estimation direction of a single person therein;

[0071] Obtain the position A1 of the person and the position A2 of the target person, determine the reference direction ACK. The reference direction points from position A1 to position A2. Obtain the angle between the line-of-sight estimation direction and the reference direction. When the angle is less than the preset value, mark the corresponding person as a normal person;

[0072] A control optimization module: counts the total number of normal people corresponding to a single monitoring image, denoted as the target number, sets the target time point based on the target number. Before the target time point, the gesture movement directions recognized by the somatosensory interaction device do not control the picture on the demonstration screen.

[0073] The above has described an embodiment of the present invention in detail, but the content described is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of the application of the present invention should still fall within the patent coverage scope of the present invention.

Claims

1. An optoelectronic particle interaction method using gestures, characterized in that Including the following steps: Mark the person closest to the demonstration screen and in front of it as the target person, identify the gesture movement direction of the target person based on the somatosensory interaction device, and control the picture on the demonstration screen based on the gesture movement direction, where the gesture movement direction includes forward, backward, left, and right; After controlling the picture on the demonstration screen based on the gesture movement direction, periodically collect the monitoring images in front of the demonstration screen, input the monitoring images into a preset line-of-sight estimation model, and obtain the line-of-sight estimation direction of a single person among them; Obtain the position A1 of the person and the position A2 of the target person, determine the reference direction ACK, where the reference direction points from the position A1 to the position A2, obtain the included angle between the line-of-sight estimation direction and the reference direction, and when the included angle is less than the preset value, mark the corresponding person as a normal person; Count the total number of the normal persons corresponding to a single monitoring image, denoted as the target number, set a target time point based on the target number, and before the target time point, the gesture movement direction recognized by the somatosensory interaction device does not control the picture on the demonstration screen; Setting the target time point based on the target number includes: Generate coordinate points (ti, Ci), where Ci represents the target number corresponding to the i-th collected monitoring image, ti represents the time point of collecting the i-th monitoring image, fit the coordinate points to obtain a fitting curve f(t), and t represents time; Take the inflection point, starting point, and ending point of the fitting curve as reference points, and denote the part of the fitting curve between any two reference points as a sub-curve; Take the straight line passing through (0, C') and parallel to the x-axis as the reference line, where C' represents a preset target number threshold, count the part of a single sub-curve that is less than and below the reference line, denoted as the first part, and calculate the ratio of the domain of the first part to the domain of the corresponding sub-curve, denoted as the first ratio; Sort the first ratios in the order of the time axis, take the first ratio greater than a preset first ratio threshold in the sorting as the target ratio, take the sub-curve corresponding to the target ratio as the target curve, and take the time point corresponding to the minimum value on the target curve as the target time point; Setting the target time point based on the target number further includes: When the fitting curve is monotonically decreasing, substitute the target number threshold into the fitting curve f(t) and solve to obtain the target time point T; Controlling the picture on the demonstration screen based on the gesture movement direction further includes: Obtain the average moving speed of the target person's hand within a preset time period, standardize and remove the dimension of the average moving speed of the hand to obtain a mapping value η; Calculate the number of optoelectronic particles S = (1 + η) * Dys and the optoelectronic particle moving speed V = (1 + η) * Vys, where Dys and Vys respectively represent the preset initial number of optoelectronic particles and the initial speed of optoelectronic particles; 2. The optoelectronic particle interaction method using gestures according to claim 1, wherein Controlling the picture on the demonstration screen based on the gesture movement direction includes: Control the moving direction of the picture on the demonstration screen to be the same as the gesture movement direction.

3. The optoelectronic particle interaction method using gestures according to claim 1, characterized in that, When the length of the target curve is less than the preset duration, it is not regarded as the target curve, and a new target line is selected to determine the target time point.

4. A method for optoelectronic particle interaction using gestures according to claim 1, characterized in that Controlling the picture in the demonstration screen based on the gesture moving direction further includes: Updating the positions of the optoelectronic particles in real time based on Compute Shader or Niagara or VFX Graph.

5. An optoelectronic particle interaction system using gestures, characterized in that, Including: Control module: Mark the person closest to the demonstration screen and in front of it as the target person, identify the gesture moving direction of the target person based on the somatosensory interaction device, and control the picture in the demonstration screen based on the gesture moving direction. The gesture moving direction includes forward, backward, left, and right. Analysis module: After controlling the picture in the demonstration screen based on the gesture moving direction, periodically collect the monitoring images in front of the demonstration screen, input the monitoring images into a preset line-of-sight estimation model, and obtain the line-of-sight estimation direction of a single person therein. Obtain the position A1 of the person and the position A2 of the target person, determine the reference direction ACK. The reference direction points from the position A1 to the position A2. Obtain the included angle between the line-of-sight estimation direction and the reference direction. When the included angle is less than the preset value, mark the corresponding person as a normal person. Control optimization module: Count the total number of normal persons corresponding to a single monitoring image, denoted as the target number. Set the target time point based on the target number. Before the target time point, the gesture moving direction recognized by the somatosensory interaction device does not control the picture in the demonstration screen. Setting the target time point based on the target number includes: Generate coordinate points (ti, Ci), where Ci represents the target number corresponding to the i-th collected monitoring image, and ti represents the time point when the i-th monitoring image is collected. Fit the coordinate points to obtain a fitting curve f(t), where t represents time. Take the inflection point, starting point, and ending point of the fitting curve as reference points, and denote the part of the fitting curve between any two reference points as a sub-curve. Take the straight line passing through (0, C') and parallel to the x-axis as the reference line, where C' represents a preset target number threshold. Count the part of a single sub-curve that is less than and below the reference line, denoted as the first part, and calculate the ratio of the domain of the first part to the domain of the corresponding sub-curve, denoted as the first ratio. Sort the first ratios in the order of the time axis, take the first ratio greater than a preset first ratio threshold in the sorting as the target ratio, take the sub-curve corresponding to the target ratio as the target curve, and take the time point corresponding to the minimum value on the target curve as the target time point. Setting the target time point based on the target number further includes: When the fitting curve is monotonically decreasing, substitute the target number threshold into the fitting curve f(t) to solve for the target time point T. Controlling the picture in the demonstration screen based on the gesture moving direction further includes: Obtain the average moving speed of the target person's hand within a preset period, standardize the average moving speed of the hand to remove the dimension, and obtain a mapping value η. Calculate the number of optoelectronic particles \(S=(1 + \eta)\times Dys\) and the optoelectronic particle movement speed \(V=(1 + \eta)\times Vys\), where \(Dys\) and \(Vys\) respectively represent the preset initial number of optoelectronic particles and the initial speed of optoelectronic particles.

Images