ROV simulator visual rendering display method and platform

Abstract

The invention relates to an ROV simulator visual rendering display method and platform, and the method is characterized in that the method comprises the steps: obtaining an initialization resource, carrying out the modeling of a three-dimensional virtual model, and randomly placing a virtual camera in a virtual scene; receiving user interaction operation data; according to data acquired by the virtual camera and user operation data, drawing a three-dimensional virtual model through a four-stage mode of event response, data updating, model cutting and primitive drawing; and forming a display window by using three or more physical display screens, and receiving and displaying the drawn three-dimensional virtual model. Compared with the prior art, the mode that one graphic rendering computer is externally connected with three display screens to output 12 paths of virtual cameras is adopted, the input of hardware purchase, energy consumption and space arrangement is reduced, the visible cost is greatly reduced, a rendering program is compiled in a four-stage mode, and the rendering efficiency is improved. And the resource performance consumption between the CPU and the GPU of the graphic computer is balanced.

Description

Technical Field

[0001] The invention relates to the field of underwater robot construction operation simulation, and in particular to an ROV simulator visual rendering display method and platform. Background Art

[0002] ROV plays an important role in the research and development of underwater resources. The ROV maneuvering training simulator is a semi-physical simulation training platform built with semi-physical simulation technology and virtual reality technology. It can simulate the real ROV control system, so that operators can complete the real ROV training process through the platform, recognize and perceive the requirements and operation procedures of ROV surface and surface operations, learn operating procedures, and ultimately improve operation efficiency and safety. At the same time, it reduces the possibility of danger in surface and surface operations and greatly reduces training costs. Many maritime service providers in the world use ROV simulators to train pilot teams. Chinese patent application CN105319987A discloses a motion control simulation system for an operational ROV training simulator, which mainly describes the motion control mathematical model of ROV construction operations, can simulate the six-degree-of-freedom motion control of ROV, and considers multiple interference factors, but does not consider the rendering and effect display of the output simulation scene. Chinese patent application CN110379247A discloses a multi-task, multi-role ROV collaborative training simulation system and method, and describes a multi-task, multi-role ROV collaborative simulation training method that records the training process and evaluates the training results, with a focus on the storage, organization, call and analysis of operation process data. Chinese patent application CN114694440A discloses an ROV simulation trainer, which mainly describes an ROV simulation trainer including an operation panel, a display bracket, an LCD screen, a control system and an operation cabinet, with a focus on the construction of a hardware platform. Chinese patent application CN113741213A discloses a deep-water semi-submersible platform mooring recovery installation operation simulation platform, which describes an installation operation simulation platform for recovering ROVs in a semi-submerged state, with a focus on the control of the relative position between the mother ship and the ROV, and is mainly about the comprehensive management and evaluation of the use of motion state parameters.

[0003] In the prior art, when outputting the visual rendering image of the ROV simulator, the saving of real-time performance is not well considered. When dealing with complex underwater environments and ROV interactions, problems such as slow rendering speed and low frame rate may be faced, thereby affecting the real-time performance of the simulator. This may cause users to experience obvious delays or freezes when operating the ROV, reducing the user experience of the simulator. With the continuous development of underwater environments and ROV technology, simulators need to be able to simulate more complex and diverse scenes. However, the prior art may find it difficult to adapt to these changes, resulting in the simulator requiring a lot of modification and debugging when facing new requirements. This not only increases development costs, but may also affect the stability and reliability of the simulator. Summary of the invention

[0004] The purpose of the present invention is to provide a ROV simulator visual rendering display method and platform, which reduces the investment in manpower and material resources by establishing a three-dimensional virtual model and visual rendering display. At the same time, a mode in which one graphics rendering computer is connected to three display screens to output multi-channel virtual camera data is adopted, which significantly reduces the investment in hardware purchase, energy consumption, and space layout, and greatly reduces the tangible cost.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A ROV simulator visual rendering and display method, the method comprising:

[0007] Obtain initialization resources, build a 3D virtual model, and place virtual cameras arbitrarily in the virtual scene;

[0008] Receive user interaction operation data;

[0009] According to the data obtained by the virtual camera and the user operation data, the visual rendering image of the three-dimensional virtual model is drawn through a four-stage mode of event response, data update, model clipping and primitive drawing;

[0010] A display window is formed by using three or more physical display screens to receive and display the rendered images of the drawn three-dimensional virtual model scene.

[0011] Furthermore, the initialization resources include a widget configuration file, a viewport configuration file, a network configuration file, and a serial port configuration file.

[0012] Furthermore, the three-dimensional virtual model includes a three-dimensional model of a mother ship, a three-dimensional model of an ROV and a three-dimensional model of a construction site, and the three-dimensional model of the mother ship and the three-dimensional model of the ROV are connected by dynamically drawn data ropes.

[0013] Furthermore, in the ROV three-dimensional model, the virtual robotic arm and the virtual robotic gripper are set as DOF ​​node manipulation, and the change range of the DOF node position state is controlled by setting the maximum and minimum values ​​of each DOF node position state.

[0014] Furthermore, the user interactive operation data is received via serial port communication and network communication, and the user interactive operation data includes network events, serial port events and GUI operation events.

[0015] Furthermore, the specific steps of drawing the visual rendering image of the three-dimensional virtual model include:

[0016] In the event response phase, user interaction operation data is received and parsed, and the 3D virtual model is controlled to respond to the user interaction operation data;

[0017] In the data update phase, according to the result of the event response, the status data of the 3D virtual model is updated, including the position status matrix of the mother ship 3D model, the position status matrix of the ROV 3D model, the DOF node position status matrix, and the observation matrix, projection matrix, and window matrix of each virtual camera;

[0018] In the model clipping stage, based on the status data of the 3D virtual model, the CPU is used to determine the model within the perspective projection frustum of the virtual camera, and the GPU is used to clip the invisible model.

[0019] In the primitive drawing stage, the model within the perspective projection frustum of the virtual camera is drawn;

[0020] Among them, the CPU computing power is used to run the model within the range of the virtual camera perspective projection frustum in the event response stage, data update stage and model clipping stage, and the GPU computing power is used to run the clipping and primitive drawing stages of the invisible model in the model clipping stage.

[0021] Furthermore, the position state matrix of the three-dimensional model of the mother ship is obtained by converting the latitude and longitude information of the absolute geographic coordinates of the three-dimensional model of the mother ship;

[0022] The position state matrix of the ROV three-dimensional model is obtained by taking the mother ship three-dimensional model as a reference point and using relative coordinates (xyzhpr) to calculate the relative state position change between the ROV three-dimensional model and the mother ship three-dimensional model;

[0023] The DOF node position state matrix includes the DOF node position state matrix of the virtual mechanical arm and the DOF node position state matrix of the virtual mechanical gripper, the DOF nodes are deployed on the ROV three-dimensional model, and the DOF node position state matrix is ​​obtained by calculating the relative state position change of the DOF nodes on the ROV three-dimensional model using relative coordinates (xyzhpr);

[0024] Among them, in the relative coordinates (xyzhpr), the three parameters x, y, and z represent the position of the object in the three-dimensional space, h is the rotation angle of the object around the x-axis, p is the rotation angle of the object around the y-axis, and r is the rotation angle of the object around the z-axis.

[0025] Furthermore, the process of drawing the model within the range of the virtual camera perspective projection frustum includes:

[0026] Calculate the model matrix according to the state data of the cropped three-dimensional virtual model;

[0027] According to the model matrix, a visual rendering image of the three-dimensional virtual model is drawn on the display screen.

[0028] Furthermore, the process of calculating the model matrix includes:

[0029] Convert the longitude, latitude and height information of the mother ship's three-dimensional model into geographic coordinates (xyz) as the first-layer conversion matrix of the model matrix;

[0030] Calculate the position state of the ROV three-dimensional model relative to the mother ship three-dimensional model as the second-layer transformation matrix;

[0031] Calculate the relative position state of the DOF nodes in the ROV three-dimensional model as the third-layer transformation matrix;

[0032] The relative position states of the geometric vertices of the model within the perspective projection frustum of the virtual camera are calculated as the fourth-layer transformation matrix.

[0033] A ROV simulator visual rendering display platform comprises a visual graphics rendering computer and three physical display screens. The physical display screens are sequentially arranged in an extended mode from bottom to top according to a physical space layout to form a display window. Each physical display screen receives and displays a three-dimensional virtual model scene shot by at most four virtual cameras.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. The present invention simulates the working environment above and below the water surface by building a three-dimensional virtual model of the working scene above and below the water surface that needs to be observed, and adopts a method of arranging virtual cameras at any position in the virtual scene for observation, thereby simulating the working environment above and below the water surface, reducing the input of manpower and material resources, and improving the safety of the simulated water environment operation. When outputting the simulation image, a single graphics computer multi-display screen rendering mode is adopted, which reduces the hardware construction cost. When drawing the output image of the three-dimensional virtual model, a four-stage mode is adopted, which ensures the orderliness of the three-dimensional model data scheduling and balances the resource performance consumption between the graphics computer CPU and GPU;

[0036] 2. The technology of the present invention adopts the method of associating DOF node model matrix to calculate and update the observation matrix of the virtual camera, which reduces the repeatability of the calculation without significantly increasing the calculation complexity and reduces the memory consumption of the computer;

[0037] 3. The technology of the present invention can be applied to the digital twin of water operation scenarios. By accessing the network digital twin platform, the simulation of the operation environment and scenarios above and below the water surface can be realized. A variety of ROV hardware operating devices can be connected through the network and serial port communication. It is not limited to the hardware model of a certain equipment manufacturer. It has strong expansion flexibility and wide versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the method of the present invention;

[0039] Figure 2 This is a structural diagram of the platform equipment framework of the present invention;

[0040] Figure 3 It is a schematic diagram of nodes of the three-dimensional virtual model of the present invention;

[0041] Figure 4 It is the coordinate conversion relationship diagram of the present invention;

[0042] Figure 5 It is the matrix transformation relationship diagram of the present invention;

[0043] Figure 6 This is a scene scheduling flow chart of the present invention. DETAILED DESCRIPTION

[0044] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0045] This embodiment discloses a method for rendering and displaying a scene of an ROV simulator, wherein the ROV (Remote Operated Vehicle) is a remotely operated unmanned submersible. The specific steps of the method are as follows: Figure 1 As shown, including:

[0046] S1, obtain initialization resources, build a three-dimensional virtual model, and place a virtual camera arbitrarily in the virtual scene;

[0047] S2, receiving user interaction operation data;

[0048] S3, drawing a visual rendering image of the three-dimensional virtual model through a four-stage mode of event response, data update, model clipping and primitive drawing according to the data obtained by the virtual camera and the user operation data;

[0049] S4, using three physical display screens to form a display window, receiving and displaying the drawn three-dimensional virtual model scene rendering image.

[0050] Figure 2 The visual display platform framework structure diagram designed for this embodiment. In this embodiment, ROV operation panels produced by different hardware manufacturers are used to send communication data to the visual graphics rendering computer through the network and serial port. The graphics card configured by one visual graphics rendering computer can display and output at least three physical display screens. The three physical display screens are arranged vertically from bottom to top in the order of display screen 0, display screen 1, and display screen 2 in space. On the operating system of the visual graphics rendering computer, the three display screens are set to the extended mode from bottom to top according to the physical space layout, so that the three physical display screens form a complete display window.

[0051] In this embodiment, each display screen is designed to receive signals rendered by 4 virtual cameras. The 4 signals contained in widget0 corresponding to display screen 0 are viewport0, viewport1, viewport2, and viewport3 respectively. The 4 signals contained in widget1 corresponding to display screen 1 are viewport4, viewport5, viewport6, and viewport7 respectively. The 4 signals contained in widget2 corresponding to display screen 2 are viewport8, viewport9, viewport10, and viewport11 respectively. The viewport contained in each widget has two display modes: full screen and quarter screen. The 12 viewports contained in the three display screens can receive signals rendered and output by 12 virtual cameras at most. There is not a one-to-one correspondence between viewport and virtual camera in a fixed order. Multiple different viewports can receive signals rendered and output by 1 virtual camera, and signals rendered and output by multiple virtual cameras can be displayed on different viewports in different orders.

[0052] The graphics card configured in the computer for rendering visual graphics in this embodiment must have at least three video signal output ports, and can ensure that the three video signal output ports can display simultaneously. Considering the display effect and procurement cost, it is best to purchase a television, monitor or projector of the same model, size and resolution as the physical display screen. The three display screens can be fixed on an immovable physical wall or a movable metal bracket. In three-dimensional virtual reality display technology, there are many ways to display multiple virtual camera viewpoints with multiple display screens. In this embodiment, the three display screens are identified as a whole graphics context (Graphics Context, GC) or a display window after using the extended display mode. Three widgets are arranged in sequence on the entire display window, and four viewports are distributed on each widget. By modifying the observation projection matrix of each virtual camera, different observation angles of each viewport are achieved.

[0053] In step S1, the initialization resources include: a widget configuration file, a viewport configuration file, a network configuration file, and a serial port configuration file.

[0054] After obtaining the initialization resources, you can start modeling the 3D virtual model, such as Figure 3 As shown, Figure 3 The node schematic diagram of the three-dimensional virtual model designed for this embodiment can use conventional modeling software such as 3dmax, blender, sketchup, and presagis creator to produce the three-dimensional virtual scene model required for this embodiment. The three-dimensional virtual model includes a mother ship three-dimensional model, an ROV three-dimensional model, and a construction site three-dimensional model, wherein the mother ship three-dimensional model and the ROV three-dimensional model are connected together by dynamically drawn data ropes, and the length and shape of the ropes change with the relative position between the mother ship and the ROV. A hierarchical tree management method is used to organize all models including the mother ship three-dimensional model, the ROV three-dimensional model, and the construction site three-dimensional model into a three-dimensional model database.

[0055] In the ROV three-dimensional model, the virtual robotic arm and virtual robotic gripper need to move in six degrees of freedom (DOF) directions to perform engineering operations. The robotic arm and robotic gripper are set as DOF ​​nodes, and the change range of the DOF node position state is controlled by setting the maximum and minimum values ​​of each DOF node position state. Unlike the ROV operating equipment in the real world, the virtual monitoring rendering camera can be arranged at various positions in the three-dimensional virtual scene model. These positions can be any corner of the mother ship's three-dimensional model, any corner of the ROV three-dimensional model, or even any position above or below the water. If necessary, during the navigator's operation of the ROV simulator, the number and position of the monitoring rendering cameras arranged in the three-dimensional virtual scene can also be manually changed. To illustrate the principle, Figure 3 The schematic diagram only shows 9 virtual monitoring rendering cameras, 2 cameras are configured at the stern of the ROV, 1 camera is configured at the bottom of the ROV, 4 cameras are configured at the bow of the ROV, and 2 cameras are configured on the 2 mechanical grippers. This embodiment uses perspective projection to render the virtual operation scene, and sets a separate perspective projection mode for each monitoring rendering camera. The perspective angle and horizontal and vertical ratio of each monitoring rendering camera can be changed separately. The horizontal and vertical ratio values ​​are Figure 3 The resolution of the physical display is related to the aspect ratio of the widget. Figure 3 The perspective projection shown uses a right-handed coordinate system, but the same principle applies to a left-handed coordinate system.

[0056] In step S2, the user interaction operation data is received through serial port communication and network communication, and the user interaction operation data includes network events, serial port events and GUI operation events.

[0057] In step S3, the specific steps of drawing the visual rendering image of the three-dimensional virtual model include:

[0058] In the event response phase, user interaction operation data is received and parsed, and the 3D virtual model is controlled to respond to the user interaction operation data;

[0059] In the data update phase, according to the result of the event response, the status data of the 3D virtual model is updated, including the position status matrix of the mother ship 3D model, the position status matrix of the ROV 3D model, the DOF node position status matrix, and the observation matrix, projection matrix, and window matrix of each virtual camera;

[0060] In the model clipping stage, according to the status data of the 3D virtual model, the CPU is used to determine the model within the perspective projection frustum of the virtual camera, and the GPU is used to clip the invisible model;

[0061] In the primitive drawing stage, the model within the perspective projection frustum of the virtual camera is drawn.

[0062] like Figure 4 As shown, Figure 4 It is a coordinate conversion relationship diagram in the three-dimensional virtual scene. According to the structural hierarchy of the three-dimensional model relative to the geographic coordinate system, the position relationship of the virtual scene rendering model points is divided into absolute position coordinates and relative position coordinates. For the three-dimensional model of the mother ship floating on the water surface, the absolute geographic coordinates converted from the longitude and latitude information are used. The ROV three-dimensional model of underwater construction is connected to the mother ship model through a data cable. The relative state position of the ROV and the mother ship will change. The ROV three-dimensional model uses relative coordinates (xyzhpr). Each virtual robot arm DOF node and virtual mechanical gripper DOF node set will change state position relative to the reference point of the ROV three-dimensional model, and relative coordinates (xyzhpr) are used for state rendering. The observation position of the virtual monitoring rendering camera will change. The relative observation position eye, relative observation focus center, and relative head-up direction up of the camera are set, thereby setting the observation matrix viewMatrix.

[0063] by Figure 4 For example, the position state matrix of the mother ship three-dimensional model is obtained by converting the latitude and longitude information of the absolute geographic coordinates of the mother ship three-dimensional model;

[0064] The position state matrix of the ROV three-dimensional model is obtained by taking the mother ship three-dimensional model as a reference point and using relative coordinates (xyzhpr) to calculate the relative state position change between the ROV three-dimensional model and the mother ship three-dimensional model;

[0065] The DOF node position state matrix includes the DOF node position state matrix of the virtual robotic arm and the DOF node position state matrix of the virtual robotic gripper. The DOF nodes are deployed on the ROV three-dimensional model. The DOF node position state matrix is ​​obtained by calculating the relative state position change of the DOF nodes on the ROV three-dimensional model using relative coordinates (xyzhpr).

[0066] In the model clipping stage, the process of drawing the model within the range of the virtual camera perspective projection frustum includes:

[0067] Calculate the model matrix according to the state data of the cropped three-dimensional virtual model;

[0068] According to the model matrix, a visual rendering image of the three-dimensional virtual model is drawn on the display screen.

[0069] The process of calculating the model matrix is ​​a matrix transformation, including:

[0070] Convert the longitude, latitude and height information of the mother ship's three-dimensional model into geographic coordinates (xyz) as the first-layer conversion matrix of the model matrix;

[0071] Calculate the position state of the ROV three-dimensional model relative to the mother ship three-dimensional model as the second-layer transformation matrix;

[0072] Calculate the relative position state of the DOF nodes in the ROV three-dimensional model as the third-layer transformation matrix;

[0073] The relative position states of the geometric vertices of the model within the perspective projection frustum of the virtual camera are calculated as the fourth-layer transformation matrix.

[0074] Figure 5 is a matrix transformation relationship diagram. In a three-dimensional virtual scene, the conversion between relative coordinates and absolute coordinates is only for the convenience of calculation. The final rendered scene needs to be converted into screen coordinates for pixel texture drawing. It is necessary to set the model matrix modelMatrix for the mother ship three-dimensional model and the ROV three-dimensional model, set the observation matrix viewMatrix and the projection matrix projectMatrix separately for each virtual monitoring rendering camera, and set the window range (x ywidth height) for each viewport to calculate the window matrix windowMatrix. In the transformation hierarchy of the model matrix modelMatrix, the conversion matrix gisMatrix of the longitude, latitude and height information of the mother ship three-dimensional model to the geographic coordinates (xyz) is the first-level conversion matrix, the conversion matrix rovMatrix of the position state of the ROV three-dimensional model relative to the mother ship three-dimensional model is the second-level conversion matrix, the relative position state conversion matrix dofMatrix of the DOF node in the ROV three-dimensional model is the third-level conversion matrix, and the fourth-level conversion matrix objMatrix is ​​the relative position state conversion of each point of the drawing primitive. In the process of converting the model matrix modelMatrix, the hierarchical relationship calculation WorldToLocalMatrix from the first-layer transformation matrix to the fourth-layer transformation matrix and the hierarchical calculation LocalToWorldMatrix from the fourth-layer transformation matrix to the first-layer transformation matrix are particularly important. Considering the universality of the Gis coordinate system, in the process of converting the earth's spherical coordinates to plane coordinates, the model matrix modelMatrix uses a right-handed coordinate system, where the north direction is the positive direction of the Y axis, the east direction is the positive direction of the X axis, and the vertical upward direction is the positive direction of the Z axis. In the 12 virtual cameras, the windowMatrix, projectMatrix, and viewMatrix of each virtual camera need to be calculated. Figure 5The hierarchical calculation of the transformation matrix of the i-th virtual camera [i-1] is shown, where the virtual camera's observation matrix viewMatrix contains the transformation of the positive direction of the Y axis from due north to vertically upward.

[0075] Figure 6Flowchart of the program software designed for this embodiment to dynamically load and schedule the three-dimensional scene model. This embodiment adopts the pipeline concept of openGL and completes the drawing of all primitives on the device graphics context display window. After the program starts running, it is necessary to complete the loading of initialization resources. These initialization resources include: widget configuration file, viewport configuration file, network configuration file and serial port configuration file. This embodiment is designed to load up to 12 virtual cameras on a grapchicsContext [0:11]. At present, the mainstream three-dimensional graphics rendering engine divides the drawing of each frame of graphics into at least three stages: user interaction App, clipping model Cull, and drawing primitive Draw. The frame drawing process of this embodiment adopts a four-stage mode of event response, data update, model clipping, and primitive drawing. These four correctly running stages constitute a complete process Process. Under the premise of ensuring reasonable resource scheduling, each virtual camera needs to correctly run a complete Process. Among them, the event response stage, data update stage and partial model clipping stage mainly consume the resources of the graphics rendering computer CPU, and the partial model clipping stage and primitive drawing stage mainly consume the resources of the graphics rendering computer GPU. The event response Event stage describes the user's interactive operations, and can receive network events, serial port events, and GUI operation events. All the operation data of the navigator in the ROV simulator are interactively processed in the event response Event stage. The Update stage handles the data calculation and update of the scene rendering program, mainly the update of each matrix Matrix, including the position state matrix of the mother ship's three-dimensional model, the position state matrix of the ROV three-dimensional model, and the position state matrix of each DOF node, and the view matrix, projection matrix, project matrix, and window matrix of each virtual camera. The Cull stage determines whether each primitive in the three-dimensional model scene tree needs to be effectively drawn. In order to balance the resource consumption and performance of the CPU and GPU, the conventional clipping algorithm uses the CPU to determine the primitives within the perspective projection frustum of the virtual camera, and sends these valid primitives to the GPU for further clipping. The Draw stage completes the drawing of each valid primitive on the display window. All operations completed in the previous three stages of event response, data update, and model clipping are preparations for the drawing of primitives. The primitive drawing stage draws the rendering effect of the three-dimensional model on the display screen. After the 12 virtual cameras [0:11] complete the process operation, they determine whether the application needs to be completed. If the application has completed all operations and needs to exit, it releases all loaded resources and then exits. Otherwise, it continues to loop to complete the rendering and drawing of the next frame.

Claims

1. A ROV simulator visual rendering and display method, characterized in that: The method comprises: Obtain initialization resources, build a 3D virtual model, and place virtual cameras arbitrarily in the virtual scene; Receive user interaction operation data; According to the data obtained by the virtual camera and the user operation data, the visual rendering image of the three-dimensional virtual model is drawn through a four-stage mode of event response, data update, model clipping and primitive drawing; A display window is formed by using three or more physical display screens to receive and display the rendered images of the drawn three-dimensional virtual model scene.

2. A ROV simulator visual rendering display method according to claim 1, characterized in that: The initialization resources include a widget configuration file, a viewport configuration file, a network configuration file, and a serial port configuration file.

3. A ROV simulator visual rendering display method according to claim 1, characterized in that: The three-dimensional virtual model includes a three-dimensional model of a mother ship, a three-dimensional model of an ROV and a three-dimensional model of a construction site. The three-dimensional model of the mother ship and the three-dimensional model of the ROV are connected by a dynamically drawn data rope.

4. A ROV simulator visual rendering display method according to claim 3, characterized in that: In the ROV three-dimensional model, the virtual mechanical arm and the virtual mechanical gripper are set as DOF ​​node manipulation, and the change range of the DOF node position state is controlled by setting the maximum value and the minimum value of each DOF node position state.

5. A ROV simulator visual rendering display method according to claim 1, characterized in that: The user interaction operation data is received through serial port communication and network communication, and the user interaction operation data includes network events, serial port events and GUI operation events.

6. A ROV simulator visual rendering display method according to claim 1, characterized in that: The specific steps of drawing the visual rendering image of the three-dimensional virtual model include: In the event response phase, user interaction operation data is received and parsed, and the 3D virtual model is controlled to respond to the user interaction operation data; In the data update phase, according to the result of the event response, the status data of the 3D virtual model is updated, including the position status matrix of the mother ship 3D model, the position status matrix of the ROV 3D model, the DOF node position status matrix, and the observation matrix, projection matrix, and window matrix of each virtual camera; In the model clipping stage, based on the status data of the 3D virtual model, the CPU is used to determine the model within the perspective projection frustum of the virtual camera, and the GPU is used to clip the invisible model. In the primitive drawing stage, the model within the perspective projection frustum of the virtual camera is drawn; Among them, the CPU computing power is used to run the model within the range of the virtual camera perspective projection frustum in the event response stage, data update stage and model clipping stage, and the GPU computing power is used to run the clipping and primitive drawing stages of the invisible model in the model clipping stage.

7. A ROV simulator visual rendering and display method according to claim 6, characterized in that: The position state matrix of the mother ship three-dimensional model is obtained by converting the latitude and longitude information of the absolute geographic coordinates of the mother ship three-dimensional model; The position state matrix of the ROV three-dimensional model is obtained by taking the mother ship three-dimensional model as a reference point and using relative coordinates (xyzhpr) to calculate the relative state position change between the ROV three-dimensional model and the mother ship three-dimensional model; The DOF node position state matrix includes the DOF node position state matrix of the virtual mechanical arm and the DOF node position state matrix of the virtual mechanical gripper, the DOF nodes are deployed on the ROV three-dimensional model, and the DOF node position state matrix is ​​obtained by calculating the relative state position change of the DOF nodes on the ROV three-dimensional model using relative coordinates (xyzhpr); Among them, in the relative coordinates (xyzhpr), the three parameters x, y, and z represent the position of the object in the three-dimensional space, h is the rotation angle of the object around the x-axis, p is the rotation angle of the object around the y-axis, and r is the rotation angle of the object around the z-axis.

8. A ROV simulator visual rendering and display method according to claim 6, characterized in that: The process of drawing the model within the range of the virtual camera perspective projection frustum includes: Calculate the model matrix according to the state data of the cropped three-dimensional virtual model; According to the model matrix, a visual rendering image of the three-dimensional virtual model is drawn on the display screen.

9. A ROV simulator visual rendering display method according to claim 8, characterized in that: The process of calculating the model matrix includes: Convert the longitude, latitude and height information of the mother ship's three-dimensional model into geographic coordinates (xyz) as the first-layer conversion matrix of the model matrix; Calculate the position state of the ROV three-dimensional model relative to the mother ship three-dimensional model as the second-layer transformation matrix; Calculate the relative position state of the DOF nodes in the ROV three-dimensional model as the third-layer transformation matrix; The relative position states of the geometric vertices of the model within the perspective projection frustum of the virtual camera are calculated as the fourth-layer transformation matrix.

10. An ROV simulator visual rendering display platform, characterized in that: The platform includes a visual graphics rendering computer and three physical display screens. The physical display screens are arranged in an extended mode from bottom to top according to the physical space layout to form a display window. Each physical display screen receives and displays three-dimensional virtual model scenes captured by at most four virtual cameras.

Images