Simulator visual angle control method and device based on gesture control

Abstract

The invention provides a simulator visual angle control method and device based on gesture control, and the method comprises the steps: recognizing a gesture state from an operator gesture image in front of a visual display, and recognizing a gesture type in the gesture image under the condition that the gesture state meets a preset activation condition; extracting corresponding gesture motion data from the continuous gesture images according to the gesture types; generating a corresponding visual angle control instruction according to the recognized gesture type and the extracted gesture motion data; converting the visual angle control instruction into eye point data for visual rendering; and driving a visual rendering engine based on the eye point data, and updating a display visual angle to complete visual angle control. According to the method and the device, the problems of limited operation distance and sight shielding caused by over-high man-machine interaction space coupling degree of touch operation in a visit display scene of the aviation simulator in the prior art are solved, and an operator can seamlessly control a visual angle without shielding under an explanation pose.

Description

Technical Field

[0001] This invention relates to the field of simulation and modeling technology, and in particular to a method and apparatus for controlling the viewpoint of a simulator based on gesture control. Background Technology

[0002] When using flight simulators for pilot training, an instructor control console system is typically provided. This system generally integrates two touch-screen displays. One display runs the instructor console software, allowing flight instructors to perform operations such as simulation operation control, aircraft position setting, training subjects, and fault settings. The other display shows the third-person view of the flight simulator, enabling instructors to clearly and intuitively observe the aircraft's position, attitude response, and changes in actuators, thereby achieving comprehensive monitoring of the flight status.

[0003] Existing third-person perspective viewpoints typically have one or more pre-set fixed observation points, and their viewpoints cannot be changed in real time with external control, making it difficult to meet users' needs for comprehensive observation from different distances, orientations, and positions. To improve observation flexibility, the industry has developed viewpoint control methods based on external devices such as keyboards, mice, and touchscreens. These methods achieve real-time control of the viewpoint by acquiring input signals from specific hardware devices, thus improving the freedom of observation.

[0004] However, in educational applications such as visits and demonstrations of flight simulators, the limitations of the aforementioned control methods become particularly prominent. In such scenarios, system configuration tends to be simplified, and the third-person view is often output as an independent signal to an external display device. At this point, existing control methods generally face the following common problems: First, limited operating distance leads to low interaction efficiency. Operators must enter the effective sensing area (usually a close-range area) closely associated with the display device to make adjustments, severely restricting their freedom of movement during the explanation process and creating spatial conflicts with the continuous presentation. Second, operation easily causes visual obstruction. Since interactive actions need to be completed near the display device, the operator's body will partially obstruct the display area, hindering the visitor's view and thus reducing the viewing experience. Summary of the Invention

[0005] This invention provides a gesture-based method and device for controlling the visual perspective of a simulator, addressing the limitations on operating distance and obstructed vision caused by the high spatial coupling of human-computer interaction during touch operations in existing aviation simulator demonstration scenarios. This allows the operator to seamlessly and unobstructedly control the visual perspective while in a lecturing position. The technical solution proposed by this invention is as follows: In a first aspect, the present invention provides a simulator viewpoint control method based on gesture control, comprising: Capture the operator's hand gestures in front of the visual display using an image acquisition device; The gesture state is identified from the gesture image, and the gesture type in the gesture image is identified when the gesture state meets a preset activation condition. Based on the gesture type, extract the corresponding gesture motion data from consecutive gesture images; Based on the identified gesture type and the extracted gesture motion data, corresponding view control commands are generated; The view control commands are converted into eye point data for scene rendering; The visual rendering engine is driven by the eye point data to update the display viewpoint and complete the viewpoint control.

[0006] Optionally, the gesture type is one of the following: single-finger movement gesture, hand distance change gesture, single-hand lateral movement gesture; the view control command is a view position and posture adjustment command, view zoom command, or view mode switching command corresponding to the gesture type; Specifically, when a single index finger is detected to be extended and moving, it is determined to be a single-finger movement gesture; when both hands are detected to be spread apart and the distance between the hands changes, it is determined to be a hand distance change gesture; when a single hand is detected to be spread apart and its lateral movement distance exceeds a preset threshold, it is determined to be a single-hand lateral movement gesture. The gesture motion data is as follows: For the single-finger movement gesture, it refers to the change in position of the finger during the movement process; The gesture referring to the change in hand distance refers to the amount of change in the distance between the hands; For the single-hand lateral movement gesture, it refers to the lateral movement distance of the single hand.

[0007] Optionally, the preset activation condition is that the hand position height is higher than the shoulder position height.

[0008] Optionally, for both the viewpoint position and pose adjustment command and the viewpoint scaling command, the step of converting the viewpoint control command into eye-point data for scene rendering includes: The gesture motion data is converted into position offset and attitude offset in the carrier coordinate system; Through coordinate system transformation, the position offset and attitude offset in the carrier coordinate system are converted into the eye point position and attitude in the geodetic coordinate system; the eye point position and attitude in the geodetic coordinate system include: longitude, latitude, altitude, heading angle and pitch angle.

[0009] Optionally, when the single-handed lateral movement gesture is detected, the following operations are performed: Switching between multiple predefined viewpoint modes, which include at least a slanted 3D viewpoint, a top-view 2D viewpoint, and a first-person viewpoint; When switching to first-person view, hide the third-person view model and set the eye point data to be consistent with the simulator's main view.

[0010] Optionally, the method further includes: After completing a gesture-based view control operation, the gesture state is reset and the system enters a cooling state.

[0011] Secondly, the present invention also provides a simulator viewpoint control device based on gesture control, comprising the following modules: The image acquisition module is used to capture images of the operator's gestures in front of the visual display through an image acquisition device; The state management module is used to identify the gesture state from the gesture image, and to identify the gesture type in the gesture image when the gesture state meets the preset activation conditions. The gesture acquisition and processing module is used to extract corresponding gesture motion data from continuous gesture images according to the gesture type. The gesture data processing module is used to generate corresponding view control commands based on the recognized gesture type and the extracted gesture motion data. A coordinate system transformation module is used to convert the view control commands into eye point data for scene rendering; The viewpoint switching control module is used to drive the visual rendering engine based on the eyepoint data, update the display viewpoint, and complete the viewpoint control.

[0012] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the gesture-based simulator view control method as described in the first aspect above.

[0013] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the gesture-based simulator viewpoint control method as described in the first aspect above.

[0014] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the gesture-based simulator viewpoint control method described in the first aspect above.

[0015] Based on the above technical solution, the beneficial effects of the present invention compared with the prior art are as follows: This invention provides a gesture-based simulator visual perspective control method and device. By capturing the operator's gesture images in front of the visual display using an image acquisition device, the operator can perform gesture operations without being close to the display, even while in a lecturing position. This solves the problem of limited operating distance, improves freedom of movement, avoids spatial conflicts with the lecturing activity, and ensures the continuity and professionalism of the presentation. In existing touch operations, the operator's body partially obstructs the display area, hindering the visitor's view and reducing the viewing quality. The method of this invention eliminates the need for the operator to touch the display, preventing obstruction of the display area. Visitors can view the aircraft's position, attitude, movements, and surrounding flight environment on the display without obstruction, thus improving the viewing quality.

[0016] Existing human-computer interaction methods suffer from excessive spatial coupling, meaning there's an irreconcilable conflict between the close proximity required for operation and the long distance needed for viewing. Touch-based real-time view control methods often result in physical overlap between the operation and display interfaces during visits, causing operational interference with the display and vice versa. This invention separates operation and display spatially, allowing operators to control the viewpoint from a greater distance using gestures without affecting the display. This enables seamless, unobstructed viewpoint control from the operator's position while explaining, reducing the spatial coupling of human-computer interaction.

[0017] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram illustrating popular science application scenarios such as visiting and displaying the aviation simulator provided in this embodiment of the invention.

[0021] Figure 2 This is one of the flowcharts illustrating the simulator viewpoint control method based on gesture control provided in this embodiment of the invention.

[0022] Figure 3 This is the second flowchart of the simulator viewpoint control method based on gesture control provided in the embodiments of the present invention.

[0023] Figure 4 This is a schematic diagram of the structure of the simulator view control device based on gesture control provided in an embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] Existing third-person view displays typically have one or more preset relative positions, and the viewpoint cannot change in real-time with external control of the eye point, failing to meet users' needs for omnidirectional observation of the aircraft from different viewing distances, orientations, and positions. Therefore, the industry has proposed a real-time third-person view control method for simulators based on keyboard and mouse controls. This method calculates the eye point in both the carrier coordinate system and the geodetic coordinate system by acquiring the mouse's horizontal and vertical movement and scroll wheel values ​​in real time, enabling control of the eye point's pitch angle, yaw angle, and distance from the aircraft, thus allowing for observation of the aircraft model from multiple angles. However, this keyboard and mouse-based simulator third-person view control method requires both the third-person view display screen and the instructor console software display screen to be located on the instructor console seat, sharing a single keyboard and mouse device connected to the instructor console computer. Therefore, keyboard and mouse operations can only originate from the instructor console, resulting in the viewpoint control function being integrated into the instructor console software. This design leads to problems in practical applications such as inconvenient operation, display device limitations, a strong sense of disconnect in operation, and an inability to flexibly switch between first-person and second-person views.

[0027] To overcome the limitations of keyboard and mouse control, a real-time visual control method for aviation simulators based on touchscreen control has been further developed. This method connects the video output of the third-view rendering computer and the USB touch output to the same touchscreen display, using the touchscreen as the viewpoint input device through a touch data acquisition program. Operators can control the position and posture of their eye points in real time through single-point horizontal or vertical movement, multi-point combinations, or double-clicking, achieving movement, zooming, and switching of the viewpoint. Although touch technology improves the intuitiveness of interaction, it still has limitations in educational applications such as visits and demonstrations of aviation simulators (see...). Figure 1As shown), the instructor console software is usually disabled, and the third-person view is output as a separate video signal to an external touchscreen display (i.e., ...). Figure 1 The visual display in the aircraft (such as the one used in the flight simulator) is used to show the aircraft's position, attitude, movements, and surrounding flight environment. However, touchscreen displays face two major problems: First, the limited operating distance leads to low interaction efficiency. Operators must enter the effective sensing area of ​​the touchscreen display (generally within 1 meter) to make adjustments, severely restricting freedom of movement and creating spatial conflicts with the explanation, affecting the continuity and professionalism of the presentation. Second, during touch operation, the operator's body partially obstructs the display area, hindering the visitor's view and reducing the viewing experience.

[0028] The root cause of the above problems lies in the high spatial coupling of existing human-computer interaction models, namely, the irreconcilable contradiction between the close distance required for operation and the long distance (2-3 meters or more) required for viewing. Because the real-time visual perspective control method based on touch controls results in the physical overlap between the operation interface and the display interface during visits and exhibitions, it leads to problems where operation affects the display, and the display is inconvenient for operation.

[0029] To address the issues of excessively high spatial coupling in human-computer interaction, conflicting interaction and display distances, resulting in operational inconvenience and obstructed vision, this invention proposes a gesture-based simulator visual perspective control method and device. By installing an image acquisition device on the visual display and connecting it to a third-view visual computer, video images in front of the display are acquired. The video image information captures the operator's lateral or vertical finger movements, changes in hand-hand distance, and lateral hand swiping gestures. This gesture data is then calculated in real-time as the position and posture data of the visual eye point, enabling comprehensive control and observation functions such as movement, zooming, and switching between different perspectives of the simulator's visual view. Visitors can also adjust the visual perspective according to their observation needs using pre-defined gesture control methods.

[0030] Reference Figure 2 As shown, the gesture-based simulator view control method includes the following: S110. Capture the operator's gesture image in front of the visual display using an image acquisition device.

[0031] Use an image acquisition device to capture images of the operator's gestures, including basic information such as the shape and outline of the gestures. The image acquisition device can be a high-definition camera, etc. Install the image acquisition device in front of the visual display, ensuring its field of view covers the area where the operator might perform gestures. The installation height and angle need to be adjusted according to the actual usage scenario to ensure clear capture of various gestures. The image acquisition device continuously captures images in front of the visual display at a certain frame rate.

[0032] S120. Identify the gesture state from the gesture image, and if the gesture state meets the preset activation conditions, identify the gesture type in the gesture image.

[0033] The acquired gesture images undergo preprocessing, including noise reduction and contrast enhancement, to improve image quality and facilitate subsequent gesture state recognition. Image processing algorithms and pattern recognition techniques are used to identify the gesture state. For example, the activity level of a gesture can be determined by analyzing its contours, color distribution, and other features. If a gesture remains relatively still for a period of time, it is considered inactive; conversely, if the gesture shows significant movement or change, it is considered active. Preset activation conditions are set based on the actual application scenario and requirements. For example, gestures may be required to operate within a specific area to be considered active, or the movement speed of the gesture may need to reach a certain threshold. Only when the recognized gesture state meets these preset activation conditions is the user's intention to control the viewpoint determined, and the process proceeds to the next step of gesture type recognition. If the preset activation conditions are not met, the current frame image is ignored, and subsequent images are captured and analyzed, awaiting the triggering of a valid command.

[0034] Once the gesture state meets the activation condition, the specific type of the gesture is further identified. Gesture types are predefined static or dynamic hand postures corresponding to different viewpoint control functions. These can include clenching a fist, opening the palm, pointing with a finger, and making specific shapes with both hands. Machine learning-based gesture recognition algorithms, such as convolutional neural networks, can be used. The model is trained with a large number of gesture image samples to accurately recognize different types of gestures. During the recognition process, the current gesture image is input into the trained model, and the model outputs the corresponding gesture type.

[0035] S130. Based on the gesture type, extract the corresponding gesture motion data from the continuous gesture images.

[0036] Depending on the type of gesture, a motion tracking algorithm is selected to extract gesture motion data. For example, for simple single-point gestures (such as pointing with a finger), an optical flow-based motion tracking algorithm can be used to track the movement trajectory of the finger by analyzing the motion of pixels in the gesture image. For complex multi-point gestures, such as making specific shapes with both hands, a feature-point-based motion tracking algorithm can be used. This involves first extracting key feature points from the gesture image, such as finger joints, and then tracking the positional changes of these feature points in consecutive image frames to obtain the gesture motion data.

[0037] Gesture motion data extracted from continuous gesture images includes information such as gesture displacement and velocity. For example, the displacement of a gesture can be obtained by calculating the positional changes of gesture feature points in adjacent image frames; the velocity can be calculated based on the displacement and time interval. This motion data can accurately reflect the dynamic changes of the operator's gestures, providing a basis for subsequent generation of viewpoint control commands.

[0038] S140. Generate corresponding view control commands based on the recognized gesture type and the extracted gesture motion data.

[0039] Establish mapping rules in advance between gesture types and motion data and view control commands. For example, a gesture of clenching a fist and moving it upward corresponds to a command to rotate the view upward; a gesture of opening the palm and moving it to the right corresponds to a command to translate the view to the right, and so on. For the motion data of the gesture, such as the magnitude of displacement and speed, corresponding mapping relationships can also be set. For example, the greater the displacement of the gesture, the greater the amplitude of the view rotation or translation; the faster the gesture moves, the faster the rate of view change.

[0040] Based on the identified gesture type and extracted gesture motion data, corresponding view control commands are generated according to preset mapping rules. These commands can be specific numerical values, such as the angle of view rotation or the distance of translation. The gesture type determines the category of the control command, for example, whether it is translation, rotation, or scaling, while the gesture motion data determines the specific parameters of the command, such as the direction and speed of translation and the scaling ratio. Finally, a view control command containing a clear operational intent and quantified parameters is generated.

[0041] S150, The view control command is converted into eye point data for scene rendering.

[0042] The generated view control commands need to be understood by the visual rendering engine. Therefore, these commands are converted into eye-point data required for visual rendering. Eye-point data defines the position and orientation of the observer's eyes in virtual 3D space, including a 3D coordinate (position) and an orientation angle (pose). This conversion process is calculated using a coordinate transformation matrix based on the type and parameters of the control commands, ultimately outputting a new set of eye-point coordinates and pose angles that can directly drive the rendering engine.

[0043] S160. Based on the eye point data, drive the visual rendering engine to update the display viewpoint and complete the viewpoint control.

[0044] The calculated eye-point data is passed to the visual rendering engine. The visual rendering engine is the module in the simulator responsible for generating and displaying the 3D visual scene. It renders the corresponding visual scene based on the eye-point position and viewing direction. Eye-point data is sent to the visual rendering engine via an interface or protocol. After receiving the eye-point data, the engine recalculates the rendering parameters of the visual scene, such as the projection matrix and view matrix. Then, the updated parameters are used to render the 3D model, generating a new visual scene, which is displayed on the visual display. The viewing angle (such as observation position, observation angle, and field of view) on the display screen changes smoothly and continuously, thus completing a closed loop from physical gestures to virtual viewpoint control. This achieves the goal of controlling the visual viewpoint based on the operator's gestures. The operator can adjust the display viewpoint in real time through different gestures for a better viewing experience.

[0045] Existing touchscreen technologies require operators to enter the effective sensing area of ​​the touchscreen display (typically within 1 meter) to make adjustments, severely limiting freedom of movement. This invention provides a gesture-based simulator viewpoint control method. By capturing the operator's gesture images in front of the display using an image acquisition device, the operator can perform gesture operations without being close to the display, even while in a lecturing position (generally 2-3 meters or more from the display). This solves the problem of limited operating distance, improves freedom of movement, avoids spatial conflicts with lecturing activities, and ensures the continuity and professionalism of the presentation. In existing touchscreen operations, the operator's body partially obstructs the display area, hindering the visitor's view and reducing the viewing experience. The method of this invention eliminates the need for the operator to touch the display, preventing obstruction of the display area. Visitors can view the aircraft's position, attitude, movements, and surrounding flight environment on the display without obstruction, further improving the viewing experience.

[0046] Existing human-computer interaction methods suffer from excessive spatial coupling, meaning there's an irreconcilable conflict between the close proximity required for operation and the long distance needed for viewing. Touch-based real-time view control methods often result in physical overlap between the operation and display interfaces during visits, causing operational interference with the display and vice versa. This invention separates operation and display spatially, allowing operators to control the viewpoint from a greater distance using gestures without affecting the display. This enables seamless, unobstructed viewpoint control from the operator's position while explaining, reducing the spatial coupling of human-computer interaction.

[0047] In an optional embodiment, the gesture type is one of the following: single-finger movement gesture, hand-to-hand distance change gesture, single-hand lateral movement gesture; the view control command is a view position and posture adjustment command, view zoom command, or view mode switching command corresponding to the gesture type; Specifically, when a single index finger is detected to be extended and moving, it is determined to be a single-finger movement gesture; when both hands are detected to be spread apart and the distance between the hands changes, it is determined to be a hand distance change gesture; when a single hand is detected to be spread apart and its lateral movement distance exceeds a preset threshold, it is determined to be a single-hand lateral movement gesture. The gesture motion data is as follows: For the single-finger movement gesture, it refers to the change in position of the finger during the movement process; The gesture referring to the change in hand distance refers to the amount of change in the distance between the hands; For the single-hand lateral movement gesture, it refers to the lateral movement distance of the single hand.

[0048] The system captures the operator's hand gestures in front of the visual display using an image acquisition device. Image processing algorithms (such as OpenCV) and deep learning models (such as MediaPipe) are used to recognize the gesture state. When a single index finger is detected extending and moving, it is identified as a single-finger movement gesture. The single finger can be any finger; the index finger is used as an example to illustrate the motion data extraction process. The initial position of the index finger (x0, y0) is recorded. During the movement of the index finger, its current position (x1, y1) is acquired in real time and sent to the gesture data processing module. When the index finger is retracted, data acquisition and transmission stop. The change in position during the index finger's movement is calculated, i.e., the lateral change deltaX = (x1, y0) / y0. x0) and longitudinal change deltaY=(y1) x0, y0 are the initial x and y coordinates of the index finger in the gesture image. x1, y1 are the real-time x and y coordinates of the index finger in the gesture image.

[0049] The hand-distance change gesture recognition process is as follows: Gesture images are captured using an image acquisition device. The gesture state with fingers spread is recognized, and changes in the distance between the hands are detected. The initial positions of the hands (left hand (x...)) are recorded. 0l ,y 0l ), right hand (x) 0r ,y 0r ), calculate the initial Euclidean distance between the two hands disDoublehand0=sqrt((x 0r x 0l ) 2 (y 0r y 0l ) 2 When the distance between the hands changes, the current position of both hands is obtained in real time (left hand (x)). 1l ,y 1l ), right hand (x)1r ,y 1r ), calculate the Euclidean distance when both hands move: disDoublehand=sqrt((x 1r x 1l ) 2 (y 1r y 1l ) 2 The change in distance between the hands is calculated, i.e., the zoom factor zoomFactor = disDoublehand / disDoublehand0. This data is then sent to the gesture data processing module. Data acquisition and transmission cease when both hands are retracted or the fingers are no longer extended.

[0050] The process of recognizing a single-handed lateral movement gesture is as follows: A gesture image is captured using an image acquisition device. The gesture state with the five fingers spread is recognized, and the lateral movement of the hand is detected. The initial position of the hand (left hand (x...)) is recorded. 0l ,y 0l ) or right hand (x 0r ,y 0r When moving one hand laterally, the current position of that hand (left hand (x)) is obtained in real time. 1l ,y 1l ) or right hand (x 1r ,y 1r When one hand moves laterally a distance X l =x 1l x 0l or X r =x 1r x 0r Exceeding the preset threshold (deltaX) l or deltaX r When a gesture is detected as a single-handed lateral movement, the lateral movement distance is recorded. (deltaX) l deltaX r These are the preset thresholds for the left and right hands, respectively. X l X r These represent the horizontal movement distances to the left and right hands, respectively.

[0051] The generation process of the viewpoint position and posture adjustment command corresponding to a single-finger movement gesture is as follows: Based on the positional change of the index finger (deltaX, deltaY), calculate the lateral angle H and vertical angle V between the eye point and the third-view target. angleH = angleH0 deltaX*PI / 180*RATIO,angleV=angleV0 deltaY*PI / 180*RATIO, where angleH0 and angleV0 are the angles between the previous calculations in the horizontal and vertical directions, respectively, and PI is pi. RATIO is the rate of change adjustment coefficient, which controls the response speed during gesture control.

[0052] Based on the calculated lateral angle H and vertical angle V, and the current eye-point observation distance radiusDis, calculate the positional offsets (diffX, diffY, diffZ) in the vehicle coordinate system relative to the third-view model along the X, Y, and Z axes. The eye-point observation distance is the distance between the eye point and the observed target.

[0053] diffX= radiusDis*cos(angleV)*cos(angleH); diffY=radiusDis*cos(angleV)*sin(angleH); diffZ= radiusDis*sin(angleV).

[0054] The generation process of the view zoom command corresponding to the hand distance change gesture is as follows: Based on the change in distance between the hands (i.e., the aforementioned view zoom factor), the distance between the eye point and the observed target is adjusted as radiusDis = radiusDis0 / zoomFactor. When the fingers of both hands are spread and close together, the distance between the hands decreases, the zoomFactor decreases, the distance between the eye point and the observed target increases from radiusDis, and the view zooms out. When the hands are spread apart, the distance increases, the zoomFactor increases, the distance between the eye point and the observed target decreases from radiusDis, and the view zooms out. radiusDis0 is the initial distance. At the end of each distance change, the current radiusDis is assigned to radiusDis0 as the initial distance value for the next distance change. After the distance between the eye point and the third-view model changes, the positional offset in the X, Y, and Z axes relative to the third-view model in the carrier coordinate system is recalculated.

[0055] diffX= radiusDis*cos(angleV)*cos(angleH); diffY=radiusDis*cos(angleV)*sin(angleH); diffZ= radiusDis*sin(angleV).

[0056] The generation process for the view mode switching command corresponding to a single-hand lateral movement gesture is as follows: When the lateral movement distance of a single hand exceeds a preset threshold, a view mode switching command (switchL or switchR) is generated and sent to the view mode switching control module. switchL indicates switching the view to the left, and switchR indicates switching the view to the right. The view mode is switched sequentially or in reverse order according to the command direction, including oblique 3D view, top-down view, and first-person view of the hidden model.

[0057] It should be noted that, in this invention, "carrier" refers to a moving object such as an aircraft or vehicle being tracked and calculated in the simulator. A third-person perspective model refers to a three-dimensional model used in the visual system to display the full view of the carrier or its surrounding environment.

[0058] This invention allows operators to directly control the viewing angle in real-time via gestures from their designated viewing position, eliminating the need for frequent movement close to the screen. This significantly improves operational efficiency and makes the explanation process smoother and more natural. Since operation doesn't require close proximity to the screen, the operator doesn't obstruct the visitor's view, ensuring a high-quality experience. Visitors can clearly observe the aircraft's position, attitude response, changes in its actuators, and the surrounding flight environment outside the simulator cockpit. This method provides comprehensive observation capabilities for the simulator's visual environment, including various viewing modes such as a 3D oblique view, a top-down view, and a first-person view with a hidden model. Visitors can choose the appropriate viewing mode based on their observation needs for a more comprehensive and in-depth viewing experience.

[0059] In an optional embodiment, the preset activation condition is that the hand position is higher than the shoulder position. When the hand is hanging down, even if the corresponding gesture state is recognized, no position data acquisition or gesture data calculation will be performed.

[0060] Specifically, an image acquisition device installed above the display captures hand gestures in real time. Image processing libraries (such as OpenCV) and deep learning models (such as MediaPipe) are used to analyze the gesture images, acquiring real-time positional data of the person's shoulders and hands. Specifically, MediaPipe can identify key points on the human body, including the shoulders and hands, and provide their positions in the image coordinate system. After acquiring the shoulder and hand position data, the height (y-coordinate value) of the hand position is compared with the height of the shoulder position. If the height of the detection unit is higher than the height of the shoulder position, a preset activation condition is met, and the gesture data acquisition module is activated to record and extract gesture motion data. If the height of the hand position is not higher than the height of the shoulder position, the gesture data acquisition module remains inactive, neither recording nor processing gesture data, thus avoiding false triggering.

[0061] Once the gesture data acquisition module is activated, it will extract corresponding gesture motion data based on the recognized gesture type (such as single-finger movement, changes in the distance between both hands, and lateral movement of one hand). This gesture motion data will be converted into view control commands to adjust the view position and posture of the simulator's visual environment. After coordinate system transformation, the view control commands are sent to the visual rendering engine to achieve real-time rendering of the visual environment and view control.

[0062] In everyday environments, people's hand movements are very frequent. If activation conditions are not set for gesture recognition, unintentional body movements can easily trigger viewpoint control, leading to unstable visuals or misoperations. This invention, by setting an activation condition that the hand position is higher than the shoulder position, can effectively filter out most unintentional body movements, reduce false triggers, and improve the stability and reliability of the system. Setting an activation condition that the hand position is higher than the shoulder position ensures that the system only responds and extracts gesture motion data when the operator genuinely intends to perform a gesture operation. This helps improve the accuracy of operations, allowing operators to more precisely control the viewpoint position and posture of the scene, thereby enhancing the quality of the visitor experience.

[0063] In exhibition and demonstration scenarios, operators need to interact with and explain to visitors. Frequent false triggers or inaccurate operations by the gesture recognition system can distract operators and negatively impact the presentation. By setting activation conditions, the gesture recognition system can be ensured to respond accurately when needed and remain silent when not, thereby enhancing operator focus and confidence, and improving the overall user experience.

[0064] In an optional embodiment, for both the viewpoint position and pose adjustment command and the viewpoint scaling command, the conversion of the viewpoint control command into eye-point data for scene rendering described in S150 above includes: S1501. Convert the gesture motion data into position offset and attitude offset in the carrier coordinate system.

[0065] Reference Figure 3 As shown, gesture motion data is converted into position and orientation data in the carrier coordinate system, specifically offset and orientation offset. For gesture motion data related to viewpoint position and orientation adjustment commands, such as lateral movement of a single finger, the initial position (x0, y0) and real-time position (x1, y1) of the single finger are recorded, and the lateral movement deltaX = x1 is calculated. x0; For vertical movement, calculate the vertical movement amount deltaY=y1 y0. These movements reflect the motion of the gesture in a two-dimensional plane.

[0066] For gesture motion data related to view zoom commands, when the system detects that the fingers of both hands are spread and close together or spread apart, it records the initial Euclidean distance disDoublehand0 of the hands and the real-time Euclidean distance disDoublehand during the movement, and calculates the change in the distance between the hands, deltaDis = disDoublehand. disDoublehand0.

[0067] Establish a carrier coordinate system, with a fixed reference point on the simulator (such as the simulator center) as the origin, and define the x, y, and z axes. Map the two-dimensional motion data of the gesture in the video frame to the carrier coordinate system. For viewpoint position and attitude adjustment commands, the lateral movement deltaX corresponds to the position offset along the x-axis in the carrier coordinate system, and the vertical movement deltaY corresponds to the position offset along the y-axis. Based on the preset correspondence between gesture movement and viewpoint adjustment, determine the corresponding attitude offset. For example, a single finger lateral movement may correspond to a rotation of the viewpoint around the vertical axis (i.e., a change in yaw angle), and vertical movement may correspond to a rotation of the viewpoint around the horizontal axis (i.e., a change in pitch angle).

[0068] For the view zoom command, the change in the distance between the hands, deltaDis, is converted into a scaling offset of the distance between the eye point and the observed target in the carrier coordinate system after proportional conversion, which affects the change in the distance between the eye point and the observed target, radiusDis.

[0069] S1502. Through coordinate system transformation, the position offset and attitude offset in the carrier coordinate system are converted into the eye point position and attitude in the geodetic coordinate system; the eye point position and attitude in the geodetic coordinate system include: longitude, latitude, altitude, heading angle and pitch angle.

[0070] After processing by the gesture data module, the positional offset of the observation point in the carrier coordinate system (i.e., diffX, diffY, diffZ mentioned above), as well as the attitude offsets in the horizontal and vertical directions (i.e., angleH, angleV mentioned above), are obtained. This information is then used as input parameters, and the carrier-to-geocentric coordinate system transformation function converts the positional data in the carrier coordinate system into the longitude, latitude, altitude, pitch angle, roll angle, and heading angle of the observation point in the geocentric coordinate system. The carrier refers to the aircraft model in the observed flight simulator, and the carrier coordinate system is a local coordinate system with the aircraft itself as the origin. The geocentric coordinate system (such as WGS-84) is a global coordinate system that uses longitude, latitude, and altitude to describe any point on Earth.

[0071] vPosLLASet=CalBodyPointPos2LLA(OriginLLA,OriginPosture,vCgPos,DisplaceRelative) In the formula, vPosLLASet is the function output, representing the eyepoint position and attitude in the geodetic coordinate system, and CalBodyPointPos2LLA is the carrier-to-geodetic coordinate system transformation function. The input and output quantities of this transformation function are both data containers containing three floating-point numbers: x, y, and z.

[0072] This vehicle-to-geodetic coordinate system transformation function requires the vehicle's current position in the geodetic coordinate system (OriginLLA, including longitude, latitude, and altitude) and its current attitude (OriginPosture, including pitch angle OriginPosture.x, roll angle OriginPosture.y, and heading angle OriginPosture.z) as input. This attitude information describes the rotation relationship between the vehicle's coordinate system and the geodetic coordinate system. OriginPosture.x and OriginPosture.y are typically set to 0, while OriginPosture.z represents the vehicle's current heading. Mathematical tools such as rotation matrices are used to transform the position and attitude offsets in the vehicle's coordinate system to the geodetic coordinate system. The function input vCgPos is the coordinates of the vehicle's center point in the vehicle's coordinate system; typically, the x, y, and z coordinates vCgPos.x, vCgPos.y, and vCgPos.z are all 0. The function input DisplaceRelative is a container containing three floating-point numbers, representing the relative position of the eye point in the carrier coordinate system, that is, the position offset of the observation eye point in the carrier coordinate system (diffX, diffY, diffZ).

[0073] For location calculation, based on the positional offset in the carrier coordinate system and the carrier's current position in the geodetic coordinate system, a coordinate transformation algorithm is used to calculate the new position (longitude, latitude, and altitude) of the eye point in the geodetic coordinate system. For example, if there is a positional offset along the x-axis in the carrier coordinate system, the new longitude value of the eye point is calculated using the geographic coordinate transformation formula, combined with the carrier's longitude information; similarly, latitude and altitude are calculated. Specifically, the carrier-to-geodetic coordinate system transformation function CalBodyPointPos2LLA calculates the direction cosine matrix using OriginPosture, transforming the position point from the carrier coordinate system to the Earth coordinate system. Then, combined with the target's current position in the geodetic coordinate system, OriginLLA uses the geodetic datum model (WGS84) to convert the position point from the Earth coordinate system to the geodetic coordinate system, thus determining the eye point's position in the geodetic coordinate system.

[0074] For attitude calculation, the attitude offset in the vehicle coordinate system is synthesized with the current attitude of the vehicle. The heading angle of the eye point in the geodetic coordinate system is the difference between OriginPosture.z and the calculated lateral angle H, converted to degrees. The pitch angle of the eye point in the geodetic coordinate system is the negative value of the calculated vertical angle V, converted to degrees. The roll angle of the eye point in the geodetic coordinate system is 0.

[0075] This invention accurately converts gesture motion data into offsets in the carrier coordinate system, precisely reflecting the operator's gesture intentions. For example, a small lateral movement of a single finger corresponds to a slight rotation of the viewing angle, and subtle changes in the distance between the hands can precisely control the zoom ratio of the view, making the viewing angle adjustment more delicate and accurate. By transforming the coordinate system, the offsets in the carrier coordinate system are converted into the eye position and posture in the geodetic coordinate system, taking into account the actual position and posture of the carrier, ensuring that the viewing angle control matches the geographical information of the real scene. In scenarios such as simulated flight, the observation angle can be accurately adjusted according to the carrier's flight status and position, improving the realism of the simulation. This invention supports multiple gesture operations corresponding to different viewing angle adjustment methods, such as single-finger movement to control viewing angle rotation, changes in the distance between the hands to control viewing angle zoom, and single-hand lateral movement to switch viewing angle modes. This diverse operation method provides the operator with flexible viewing angle control means, allowing for quick and convenient adjustment of the viewing angle according to different display needs and scene characteristics. This invention uses gestures as control input, conforming to people's natural interaction habits. Operators do not need additional equipment; they can control the viewing angle simply through simple gestures, making the operation more natural and intuitive. This natural interaction method reduces the difficulty of operation, improves the operator's efficiency and comfort, and thus enhances the overall user experience.

[0076] In an optional embodiment, upon recognizing the single-handed lateral movement gesture, the following operations are performed: Switching between multiple predefined view modes, including at least a slanted 3D view, a top-view 2D view, and a first-person view; when switching to the first-person view, the third-person view model is hidden, and the eye point data is set to be consistent with the simulator's main view.

[0077] Specifically, specific gesture features can be set to recognize single-handed lateral movement gestures, such as rapid, large-amplitude lateral movement of one hand exceeding a preset threshold, or specific crossing or waving motions of both hands. When a gesture matching these features is detected, it is determined that a single-handed lateral movement gesture has been recognized. In the oblique 3D perspective mode, the observer looks down or up at a certain angle at the simulated scene, simultaneously seeing multiple surfaces and spatial relationships of objects, presenting a three-dimensional effect, suitable for displaying the overall structure and spatial layout of objects. In the top-view 2D perspective mode, the observer looks vertically downward from directly above the object, and the scene is presented in a two-dimensional plane, highlighting the planar layout and positional relationships of objects, used to display scenes such as area planning and site layout. In the first-person perspective mode, the perspective of direct human observation is simulated, giving users an immersive feeling, as if they are observing and operating in the simulated scene themselves.

[0078] The viewpoint mode switching logic is as follows: Maintain a viewpoint mode state variable to record the current viewpoint mode. When a single-handed horizontal movement gesture is detected, the state variable is updated according to a preset switching order (e.g., oblique 3D view → top-view 2D view → first-person view → oblique 3D view…) or a preset switching rule. The preset switching rule can be that different gestures correspond to different switching directions. For example, if the current viewpoint is oblique 3D, after a switching gesture is detected, the state variable is updated to top-view 2D; if another switching gesture is detected, it is updated to first-person view, and so on.

[0079] Within the simulator, a third-person perspective model is used to externally display the relative positional relationship between the simulated scene and the simulator itself. When switching to first-person perspective, the display properties of the third-person perspective model are controlled via the graphics rendering engine's interface (functions provided by OpenGL, DirectX, etc.), setting it to an invisible state, i.e., hiding the model from the view. Eye point data from the simulator's main view is acquired, including the eye point's position coordinates (x, y, z) in 3D space and the viewing direction vector. The viewing direction vector determines the orientation of the view. This data is assigned to the eye point parameters used for first-person rendering. During graphics rendering, scene projection and view transformations are performed based on the new eye point data, ensuring complete synchronization between the first-person view and the simulator's main view, giving the user the feeling of being physically present in the simulated scene.

[0080] This invention allows for quick switching between multiple predefined perspective modes via simple gestures, eliminating the need for complex menu operations or button combinations. Operators can adjust the perspective at any time according to presentation needs; for example, using a 3D perspective to introduce the overall structure of an object, switching to a top-down 2D perspective to explain the floor plan, and switching to a first-person perspective to provide visitors with an immersive experience. This significantly improves operational efficiency and flexibility. Different perspective modes are suitable for different presentation scenarios and content. The 3D perspective is suitable for displaying objects or scenes with complex spatial structures, the top-down 2D perspective is convenient for displaying floor plan planning and layout, and the first-person perspective provides an immersive experience. This multi-perspective mode switching function allows the simulator to better adapt to various presentation needs and meet the observation habits and interests of different users.

[0081] When switching to first-person perspective, the third-person model is hidden and the eye-point data is set to match the simulator's main view, allowing the user to be fully immersed in the simulated scene. The user feels as if they are part of the scene, observing the surrounding environment from a first-person perspective, enhancing the realism and immersion of the experience. For example, in simulated flight or driving scenarios, the first-person perspective allows users to more intuitively experience the flight or driving process, improving training and demonstration effectiveness. The entire perspective mode switching process is achieved through gesture recognition and automatic data setting, resulting in a smooth and natural transition without noticeable lag or jumps. This ensures a consistent observation experience when switching perspectives, further enhancing immersion.

[0082] In an optional embodiment, the present invention prevents accidental and continuous triggering through a state management mechanism, ensuring stable control of the viewing angle. The state management module is used to determine the activation of the gesture state, reset the gesture state, and cool down the execution of the viewing angle switching gesture, ensuring that each gesture operation accurately triggers the corresponding viewing angle control command, avoiding viewing angle instability caused by accidental or continuous operation. The method further includes: S170. After completing a gesture-based view control, reset the gesture state and enter the cooling state.

[0083] After a round of gesture motion data is converted into model control data and eye-point data, i.e., after completing one round of gesture-based viewpoint control through S130-S160, before proceeding to the next round of gesture state judgment, refer to... Figure 3As shown, the state management module performs state resets and view switching cooldowns. Gesture state information needs to be reset to inactive, with the gesture position empty. While the current gesture state is active, other gesture states cannot be entered to prevent calculation errors caused by multiple external gesture inputs when multiple users are present. A series of variables describing gesture states can be maintained, such as gesture recognition flags, gesture action type identifiers, and gesture duration counters. After completing a gesture-based view control operation, these variables are reset. For example, the gesture recognition flag is set to an inactive state (e.g., changing from 1 to 0), the gesture action type identifier is set to its initial default value (if no value corresponds to the action state), and the gesture duration counter is cleared.

[0084] The aforementioned model control data refers to the switching signals used to control whether the third-view aircraft model is displayed in the view. Specifically, this includes the visualization attributes of the third-view model. When the attribute is set to visible, it is in third-view mode, such as an oblique 3D view or a top-view 2D view. A complete aircraft model that can be observed from the outside is rendered and displayed in the view. The observer is essentially operating a virtual camera, observing from outside the aircraft. When the attribute is set to invisible, it switches to first-view mode, hiding the previously displayed third-view aircraft model. At this time, the view is completely consistent with the main view in the simulator cockpit, displaying the scene seen from the pilot's seat or inside the aircraft, i.e., the scene outside the windshield, the instrument panel, etc.

[0085] View switching is achieved by moving one hand horizontally beyond a certain horizontal distance threshold. When the horizontal movement distance is large, multiple view switchings may occur if the threshold is exceeded consecutively. However, only one view switching is needed for each actual movement. The state management module's cooling mechanism sets a cooldown time threshold. After a view switching is completed, a cooling state is entered. If the cooldown time is less than the threshold, no further view switching is triggered. A cooldown timer is started, with a fixed cooldown interval (the aforementioned cooldown time threshold). This interval can be adjusted according to the actual application scenario and requirements to prevent users from performing too many gestures, which could lead to view control confusion.

[0086] Once the timer starts, a countdown begins. During the cooling-off period, gesture recognition-related functional modules, such as the gesture recognition module and gesture data processing module, are temporarily disabled. Even if the sensor detects the user's gesture, it will not be recognized or processed, and no view control commands will be triggered. For example, if the user makes a gesture to zoom in during the cooling-off period, the gesture will not be responded to, and the view will not change. When the cooling-off timer finishes counting down, the cooling-off period is considered over. At this point, the gesture recognition module will be reactivated, allowing gesture-based view control operations to resume.

[0087] This invention enters a cooling state after completing a gesture-based view control operation, effectively preventing continuous false triggering of view control commands due to unconscious hand tremors or repetitive movements. By resetting the gesture state, it ensures that each new gesture operation starts from the initial, unrecognized state, avoiding interference from residual state information from previous gesture operations. This helps improve the accuracy of gesture recognition, thereby enabling more precise execution of view control commands. The cooling state provides users with a control mechanism for the rhythm of operation. After a gesture operation, users have a buffer period to adjust their hand posture and prepare for the next operation, making the operation smoother and more natural.

[0088] The following describes the simulator view control device based on gesture control provided by the present invention. The simulator view control device based on gesture control described below and the simulator view control method based on gesture control described above can be referred to in correspondence.

[0089] The present invention provides a gesture-based simulator viewpoint control device, referring to... Figure 4 As shown, it includes: Image acquisition module 210 is used to capture the gesture image of the operator in front of the visual display through an image acquisition device; The state management module 220 is used to identify the gesture state from the gesture image, and to identify the gesture type in the gesture image when the gesture state meets the preset activation conditions. The gesture acquisition and processing module 230 is used to extract corresponding gesture motion data from continuous gesture images according to the gesture type. The gesture data processing module 240 is used to generate corresponding view control commands based on the recognized gesture type and the extracted gesture motion data. The coordinate system transformation module 250 is used to convert the view control command into eye point data for scene rendering; The viewpoint switching control module 260 is used to drive the visual rendering engine based on the eye point data, update the display viewpoint, and complete the viewpoint control.

[0090] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions from the memory 330 to execute a gesture-based simulator viewpoint control method.

[0091] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0092] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to execute the gesture-based simulator view control method provided by the above methods.

[0093] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the gesture-based simulator viewpoint control method provided by the above methods.

[0094] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0095] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for controlling the viewpoint of a simulator based on gesture control, characterized in that, include: Capture the operator's hand gestures in front of the visual display using an image acquisition device; The gesture state is identified from the gesture image, and the gesture type in the gesture image is identified when the gesture state meets a preset activation condition. Based on the gesture type, extract the corresponding gesture motion data from consecutive gesture images; Based on the identified gesture type and the extracted gesture motion data, corresponding view control commands are generated; The view control commands are converted into eye point data for scene rendering; The visual rendering engine is driven by the eye point data to update the display viewpoint and complete the viewpoint control.

2. The simulator viewpoint control method based on gesture control according to claim 1, characterized in that, The gesture type is one of the following: single-finger movement gesture, hand-to-hand distance change gesture, single-hand lateral movement gesture; the view control command is a view position and posture adjustment command, view zoom command, or view mode switching command corresponding to the gesture type. Specifically, when a single index finger is detected to be extended and moving, it is determined to be a single-finger movement gesture; when both hands are detected to be spread apart and the distance between the hands changes, it is determined to be a hand distance change gesture; when a single hand is detected to be spread apart and its lateral movement distance exceeds a preset threshold, it is determined to be a single-hand lateral movement gesture. The gesture motion data is as follows: For the single-finger movement gesture, it refers to the change in position of the finger during the movement process; The gesture referring to the change in hand distance refers to the amount of change in the distance between the hands; For the single-hand lateral movement gesture, it refers to the lateral movement distance of the single hand.

3. The simulator viewpoint control method based on gesture control according to claim 1, characterized in that, The preset activation condition is that the height of the hand position is higher than the height of the shoulder position.

4. The simulator viewpoint control method based on gesture control according to claim 2, characterized in that, For both the viewpoint position and pose adjustment command and the viewpoint scaling command, the step of converting the viewpoint control command into eye-point data for scene rendering includes: The gesture motion data is converted into position offset and attitude offset in the carrier coordinate system; Through coordinate system transformation, the position offset and attitude offset in the carrier coordinate system are converted into the eye point position and attitude in the geodetic coordinate system; the eye point position and attitude in the geodetic coordinate system include: longitude, latitude, altitude, heading angle and pitch angle.

5. The simulator viewpoint control method based on gesture control according to claim 2, characterized in that, Upon recognizing the single-handed lateral movement gesture, the following operations are performed: Switching between multiple predefined viewpoint modes, which include at least a slanted 3D viewpoint, a top-view 2D viewpoint, and a first-person viewpoint; When switching to first-person view, hide the third-person view model and set the eye point data to be consistent with the simulator's main view.

6. The simulator viewpoint control method based on gesture control according to claim 1, characterized in that, The method further includes: After completing a gesture-based view control operation, the gesture state is reset and the system enters a cooling state.

7. A simulator viewpoint control device based on gesture control, characterized in that, include: The image acquisition module is used to capture images of the operator's gestures in front of the visual display through an image acquisition device; The state management module is used to identify the gesture state from the gesture image, and to identify the gesture type in the gesture image when the gesture state meets the preset activation conditions. The gesture acquisition and processing module is used to extract corresponding gesture motion data from continuous gesture images according to the gesture type. The gesture data processing module is used to generate corresponding view control commands based on the recognized gesture type and the extracted gesture motion data. A coordinate system transformation module is used to convert the view control commands into eye point data for scene rendering; The viewpoint switching control module is used to drive the visual rendering engine based on the eyepoint data, update the display viewpoint, and complete the viewpoint control.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the simulator viewpoint control method based on gesture control as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the simulator viewpoint control method based on gesture control as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the simulator viewpoint control method based on gesture control as described in any one of claims 1 to 6.

Images