Method for building digital twin smart fishing port based on oblique photography
By generating 3D reality models through oblique photography and drone aerial photography, and combining them with open-source WebGIS frameworks and IoT technology, the problem of high cost and low efficiency in scene construction and rendering in digital twin fishing port systems has been solved, achieving high-precision and low-cost fishing port management and supervision.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- OCEAN UNIV OF CHINA
- Filing Date
- 2023-10-13
- Publication Date
- 2026-07-07
Smart Images
Figure CN117671130B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for constructing and using a digital twin smart fishing port based on oblique photography, belonging to the field of geographic information. Background Technology
[0002] Digital twins are virtual scenes constructed by digitizing real-world scenarios using physical models, IoT technology, big data analytics, virtual simulation, and other new technologies. Data and operations are interconnected between the virtual and real worlds. Digital twin technology allows users to remotely understand the state of physical entities and perform responsive actions within the virtual world. Compared to the flat, two-dimensional world, digital twin scenes offer a more intuitive visual experience, effectively improving user efficiency in target management and simplifying maintenance. With increasing demand, smart fishing ports have emerged. While initial port area visualization has been achieved, issues such as insufficient realism in display effects and limited interactive functions between the virtual and real worlds are prevalent. Oblique photogrammetry involves mounting multiple sensors on a flight platform to simultaneously collect ground image data from multiple angles. After data processing, accurate and complete location and texture data of ground features are obtained, producing realistic 3D reality models with high spatial accuracy.
[0003] In the area of digital twin fishing ports, the Xiangshan County Big Data Development Center primarily utilizes technologies such as dynamic sensing, digital twins, algorithm recognition, data analysis, mechanism reshaping, and data-driven empowerment to create digital twin fishing ports, addressing a series of challenges in fisheries safety governance, including the invisibility, indistinction, and poor management. However, this system operates at the virtual simulation level. Wang Shuo, Xuan Yingying, and others applied 3D real-scene modeling technology to a power grid digital twin system, demonstrating that 3D real-scene models can enrich the display layer of the digital twin power grid, providing a base layer for various sensor devices and power infrastructure that can be placed, displayed, and measured, effectively improving the construction and display effects of digital twin scenarios. In the area of oblique photogrammetry, with the development of UAV technology, the implementation threshold has gradually decreased, and work efficiency has significantly improved. The Phantom 4 RTK, as a small multi-rotor high-precision aerial surveying UAV, is designed for low-altitude photogrammetry applications. It features a centimeter-level navigation and positioning system and a high-performance imaging system, is portable and easy to use, and offers high aerial surveying efficiency. It also provides two oblique photogrammetry flight route planning schemes: tic-tac-toe and five-way flight. It can not only produce 3D reality models, but also provide centimeter-level high-resolution DOM (orthophotos) and DSM (digital surface models), enabling high-precision and low-cost modeling of fishing ports.
[0004] Currently, the main challenges of browser-based digital twin technology lie in two aspects: First, scene construction, especially the construction of twin scenes. Real-world scenes are typically large, and how to quickly and cost-effectively construct scenes that closely resemble real-world environments is the focus of digital twin project development. Second, scene rendering. Currently, there are two main rendering solutions: one is rendering through WebGL (3D graphics protocol) frameworks, such as Three.js and Cesium. The advantage of this solution is its low development and usage cost, but it suffers from lag issues when loading complex scenes. The other solution is cloud rendering through game engine development, using pixel streaming. While this solution can achieve high-performance front-end rendering, its usage cost is much higher than WebGL. Balancing cost and user experience is a co-existing problem for digital twin smart fishing ports. Therefore, this paper proposes a digital twin smart fishing port system based on oblique photogrammetry. Summary of the Invention
[0005] To overcome the shortcomings of existing technologies, this invention provides a method for constructing and using a digital twin smart fishing port based on oblique photogrammetry. The technical solution of this invention is as follows:
[0006] A method for constructing and using a digital twin smart fishing port based on oblique photogrammetry includes:
[0007] (1) Digital twin scenario construction; (2) Fishing port video fusion;
[0008] (3) Management of twin berths in fishing ports; (4) Radar tracking of targets in port areas.
[0009] Step (1) specifically includes:
[0010] (1-1) Drone aerial photography: After conducting oblique photogrammetry flight surveys of the fishing port area, the aerial photography area covering the fishing port is drawn using a marking language; to ensure the accuracy of the oblique photogrammetry results, the image control points should be evenly distributed in the port area; after marking the image control points by ground spraying or selecting landmark features, the image control points are measured using RTK equipment.
[0011] That is: before the flight, the environment of the fishing port is inspected to determine whether the current environmental conditions are suitable for flight. If the conditions are met, the drone is installed. When the drone's self-check is correct and the RTK module is working properly, the aerial photography work can begin.
[0012] (1-2) Fishing Port Modeling: The raw data captured by the drone is exported, and ContextCapture software is used to sequentially read the raw data, perform the first aerial triangulation, puncture point measurement, the second aerial triangulation, and modeling, generating raw 3D real scene model data in OSGB and OBJ formats. The first aerial triangulation is used to calculate the exterior orientation elements of each photo; the second aerial triangulation is used to improve the calculation accuracy of the exterior orientation elements of the photos and correct the model.
[0013] (1-3) Fishing port model restoration: The original 3D real scene model was optimized using ModelFun software, and the OSGB and OBJ format dataset models were imported. After the model restoration was completed, the OSGB data was regenerated.
[0014] (1-4) Detailed modeling of fishing port: Based on the modeling of the original 3D real scene model, determine the areas in the port area that need to be modeled in detail, use modeling software to perform detailed modeling, and generate OBJ model after modeling is completed;
[0015] (1-5) Rendering of fishing port model: Convert the repaired results into 3D Tiles format; use Cesium or MapTalks open source GIS framework to load 3D Tiles data on the browser side, adjust parameters according to project requirements, and realize the front-end rendering of three-dimensional real scene.
[0016] Step (2) specifically refers to:
[0017] (2-1) After performing detailed modeling of the camera in the twin scene, load it to the corresponding position in the scene and add response events to control the camera to turn on or off. When the event response is turned on, use the Cesium camera to create a view cone to simulate the field of view of the surveillance camera. Based on the camera position, azimuth angle and focal length parameters provided by the backend interface, automatically switch the view of the twin scene to the corresponding position of the surveillance screen.
[0018] (2-2) Adjust the camera frustum so that it accurately reflects the actual focal length, angle and position of the camera; use Cesium's ShadowMap function to perform field-of-view analysis on the camera frustum range;
[0019] (2-3) Create a video tag and generate video texture by playing the video stream; use GLSL language to write texture processing logic. The visible area within the view frustum is calculated with respect to the camera and the shadow coordinates through the video texture and the corresponding vertex to obtain the corresponding gl_FragColor. The shadow area retains the original texture of the model.
[0020] (2-3) To avoid the video distortion caused by projection affecting the visual experience, an additional set of video components is created to synchronously display the playback screen in normal state. The video player and the camera position in the twin scene are bound by the callback function provided by Cesium.
[0021] The specific steps (3) are as follows:
[0022] (3-1) Implement intelligent twin management of berths in the port, provide berth reservation function, fishermen can view the map-based area division and usage status of berths in real time through WeChat mini program, and select vacant berths for online reservation and intelligent route planning;
[0023] (3-2) The current berth usage status in the port is displayed in the form of a two-dimensional map combined with vector elements in the mini-program; select an available berth to make a reservation. After receiving the reservation command, the backend pushes the reservation information to the twin scene via WebSocket. The twin scene uses gradient walls of different colors to display the real-time status of the berth; click on the berth to display the recent usage statistics of the berth; the backend obtains the coordinates and orientation information of the fishing boat in real time through the positioning equipment installed on the fishing boat, in conjunction with the near-port radar or monitoring screen. The twin scene plans the berthing route of the fishing boat based on the transmitted information.
[0024] (3-3) The A* algorithm is used to provide necessary berth route planning for fishing vessels. First, data preprocessing is performed. The berths are vectorized using remote sensing satellite images of the port area. The vector elements are converted into TIFF. The TIFF is binarized using a raster calculator. The TIFF is resampled according to the complexity of the port berths. The post-processing results are read using GDAL and exported as a two-dimensional x*y array. The array, range coordinates, and pixel parameters are saved to the database. During route planning, the latitude and longitude coordinates of the vessel position are passed in. The latitude and longitude coordinates are converted to rectangular coordinates according to the coordinate range and pixel parameters corresponding to the array. After reading the array, the code performs a breadth-first search to obtain the set of route nodes. The set of nodes is converted from the rectangular coordinate system to the geographic coordinate system. A GeoJson object is created and returned to the system as the result. The twin scene loads the GeoJson object to render the planned route.
[0025] Step (4) specifically refers to:
[0026] A two-way communication protocol is established between the virtual reality scene and the radar data server. The radar data server analyzes the maritime targets of interest currently being tracked by the radar and pushes the basic information and geospatial parameters of the current targets to the virtual reality scene in real time. After receiving the data, the virtual reality scene determines the rendering status of the target ships in the scene. If the system is tracking the target ship for the first time, it renders the ship model and adds response events to display the ship's basic information. If the target is being continuously tracked by the system, it dynamically adjusts the ship's geospatial parameters to display the ship's position and attitude in real time. The screen coordinates of the current radar model are directly used as video controls, calculated using the browser window coordinates. The screen coordinates are dynamically adjusted to a suitable position using CSS, ensuring they do not obstruct other models. The video control is dynamically bound to the location of the radar model in the twin scene and automatically plays real-time monitoring footage. By converting the geographic coordinates at sea to the spatial coordinates of the twin scene, specifically using the Cesium spatial coordinate transformation function toolkit, the geographic coordinate system is converted to the Cartesian coordinate system. The spatial coordinates of the locked ship and radar in the twin scene are calculated, and a pyramid is drawn as the tracking light cone. The base of the pyramid faces the locked ship, and the apex is located at the radar position. The azimuth angle parameters of the light cone are adjusted in real time using keyframe animation functions to ensure that the light cone continuously locks onto the tracked ship.
[0027] The advantages of this invention are as follows: This invention is a digital twin smart fishing port system based on oblique photogrammetry. It utilizes a drone equipped with an RTK (Real-Time Kinematic) positioning module to perform oblique photogrammetry modeling of the port area. Combined with a detailed model created using professional modeling software, a 3D scene of the fishing port is constructed. This 3D scene is then rendered and displayed on a web-based platform, achieving low-cost, high-precision, and rapid 3D modeling of the fishing port. By leveraging IoT (Internet of Things) and data fusion technologies, information exchange between the virtual and real scenes is achieved. Combined with mature artificial intelligence technologies, operations such as video fusion, radar tracking, and berth management within the port area are realized. This results in a truly digital twin-capable smart fishing port system, solving a series of problems in traditional smart fishing port systems, such as chaotic management, difficult supervision, and high costs. This provides a new technical solution for realizing a digital twin system for smart fishing ports.
[0028] This invention uses a combination of a 3D real-world model generated by oblique photography as the main component and a fine model generated by refined modeling as a supplement to construct an efficient and realistic digital twin fishing port scene. The front end uses open-source WebGIS (Web Geographic Information System) frameworks such as Cesium or Maptalks to perform high-performance rendering and display of the twin scene. With the help of hardware and software devices and mature Internet technologies such as IoT, it can complete the twin functions such as video fusion, berth management, and radar tracking.
[0029] It also has the following advantages:
[0030] (1) High-performance rendering on the browser side, data is transmitted over the network, and it can be seamlessly switched with the traditional two-dimensional fishing port. Users do not need to install additional software or set up an environment.
[0031] (2) It has the ability to display large-area fishing port 3D models. The macroscopic 3D model is built using 3D real scene construction, with fast loading efficiency, high spatial accuracy and realistic texture. The microscopic model shows the details of the area of interest through fine modeling, providing users with an immersive experience.
[0032] (3) Realize data and operation interoperability between the virtual and real worlds. The digital twin smart fishing port scenario has functions such as video fusion, radar linkage, and berth management, which effectively enhances the management and supervision capabilities of the fishing port. Attached Figure Description
[0033] Figure 1 This is a flowchart illustrating the present invention.
[0034] Figure 2 This is a schematic diagram of the route planning process of the present invention.
[0035] Figure 3 This is a schematic diagram of the fishing port modeling process of the present invention.
[0036] Figure 4 This is a schematic diagram of the basic data processing flow for berth route planning in this invention.
[0037] Figure 5 This is a schematic diagram of the berth route planning process of the present invention. Detailed Implementation
[0038] The present invention will be further described below with reference to specific embodiments, and the advantages and features of the present invention will become clearer as a result. However, these embodiments are merely exemplary and do not constitute any limitation on the scope of the present invention. Those skilled in the art should understand that modifications or substitutions can be made to the details and form of the technical solutions of the present invention without departing from the spirit and scope of the present invention, but all such modifications and substitutions fall within the protection scope of the present invention.
[0039] See Figures 1 to 5 The present invention relates to a method for constructing and using a digital twin smart fishing port based on oblique photography, including: (1) digital twin scene construction; (2) fishing port video fusion; (3) fishing port twin berth management; and (4) port area target radar tracking.
[0040] Step (1) specifically includes:
[0041] (1-1) Drone Flight: Conduct flight surveys of the fishing port area, use markup language to map the aerial photography area to ensure coverage of the fishing port. After loading the markup language KML, the drone can automatically plan flight routes within the area. To ensure the accuracy of the oblique photogrammetry results, the image control point measurement is carried out using the Qianxun Xingyao X RTK equipment. The selected image control points are evenly distributed in the port area. The image control points are painted or selected by the user based on ground landmarks such as zebra crossings and road directional arrows. Before the flight, the fishing port environment is checked to determine whether the environment meets the requirements for flight. If the conditions are met, the drone is installed. After the drone self-checks and finds no errors, the network RTK module is checked to see if it is connected normally. Since CORS (Continuously Operating Reference Station) has not yet achieved nationwide coverage, it is necessary to build a base station through RTK when necessary. When the RTK module is connected normally, the oblique photogrammetry work of the port area is completed by flying according to the DJI Phantom 4 RTK oblique photogrammetry plan.
[0042] (1-2) Fishing Port Modeling: The raw data was exported from the UAV, and ContextCapture software was used to sequentially perform image reading, first aerial triangulation, point pricking, second aerial triangulation, and modeling, generating raw 3D reality model data in both OSGB and OBJ formats. ； To ensure model accuracy, the number of triangulation points is flexibly adjusted based on the survey area. The final output is data in two formats: OSGB (a commonly used oblique photogrammetry data storage format) and OBJ (3D model file format). Since this process is time-consuming, a cluster can be set up to improve production efficiency when necessary. Specifically, the first aerial triangulation is used to calculate the exterior orientation elements of each photograph; the second aerial triangulation is used to improve the accuracy of the exterior orientation element calculation and correct the model.
[0043] (1-3) Fishing Port Model Restoration: Due to the technical characteristics of oblique photogrammetry, some details of the model are lost. Although the model accuracy can be improved by flying at lower altitudes and using height layering, problems such as holes and deformations cannot be solved. Therefore, model restoration is necessary. The original 3D reality model is optimized using ModelFun software, and OSGB and OBJ format datasets are imported. After model restoration, OSGB data is regenerated. Based on existing data, structural restoration, dock clearing, road surface screen placement, and texture restoration are performed on the model. After model restoration, it is ensured that there are no moving objects interfering with the fishing port dock, and the data is re-exported.
[0044] (1-4) Detailed modeling of fishing port: Based on the modeling of the original 3D real scene model, determine the areas in the port area that need to be modeled in detail. Use 3ds Max or Blender modeling software to perform detailed modeling. After the modeling is completed, export the OBJ format for use.
[0045] (1-5) Rendering of fishing port model: Convert the repaired results into 3D Tiles format; use Cesium or MapTalks open source GIS framework to load 3D Tiles data on the browser side, adjust parameters according to project requirements, and realize the front-end rendering of three-dimensional real scene.
[0046] Browsers cannot directly load OSGB and OBJ format tile data. Software such as Cesium Lab or DasViewer is used to convert the repaired results into 3D Tiles format. 3D Tiles is an open 3D spatial data standard. Each 3D Tiles consists of a JSON (Lightweight Data Interchange) file storing node information and several tile package folders. By performing layered and hierarchical tile processing on the model, the client can load and render the model on demand. The converted results are treated as static resources and accessed via reverse proxy using servers such as Nginx (a high-performance HTTP and reverse proxy web server). The browser uses the Cesium or MapTalks open-source GIS framework to load the 3D Tiles data, and parameter tuning is performed according to project requirements to achieve high-performance rendering of the 3D real-world scene on the front end.
[0047] The 3D real-scene scene has spatial information. By assigning spatial coordinates to the finely produced model and adjusting the model's scaling, it can be correctly displayed in the 3D real-scene scene, realizing the construction of a digital twin smart fishing port scene.
[0048] Step (2) specifically involves: The purpose of fishing port video fusion is to project and fuse one or more surveillance videos within the fishing port area with a spatially related three-dimensional virtual scene, enabling the virtual scene to reflect the surveillance footage in real time and realistically. Specifically:
[0049] (2-1) After performing detailed modeling of the camera in the twin scene, load it to the corresponding position in the scene and add response events to control the camera to turn on or off. When the event response is turned on, use the Cesium camera to create a view cone to simulate the field of view of the surveillance camera. Based on the camera position, azimuth angle and focal length parameters provided by the backend interface, automatically switch the view of the twin scene to the corresponding position of the surveillance screen.
[0050] (2-2) Adjust the camera frustum so that it accurately reflects the actual focal length, angle and position of the camera; use Cesium's ShadowMap function to perform field-of-view analysis on the camera frustum range;
[0051] (2-3) Create a video tag and generate video texture by playing HLS (HTTP streaming media network transmission protocol) or FLV (streaming media format) video stream; use GLSL language to write texture processing logic, calculate the relative coordinates and shadow coordinates of the visible area within the view frustum with respect to the camera through the video texture and corresponding vertices to obtain the corresponding gl_FragColor, and retain the original texture of the model in the shadow area to achieve the combination of virtual and real, so that users can watch real monitoring screens in virtual scenes;
[0052] (2-4) To avoid the video distortion caused by projection affecting the visual experience, an additional set of video components is created to synchronously display the playback screen in normal state. The video player and the camera position in the twin scene are bound by the callback function provided by Cesium. When the user drags the virtual scene, the video player will automatically move with the movement of the camera position in the scene.
[0053] The effectiveness of video fusion is related to the viewing angle of the surveillance camera. The closer the viewing angle is to 90° with the monitored surface, the smaller the image projection distortion and the more realistic the experience. Video fusion upgrades the monitoring image from two-dimensional to three-dimensional, achieving a combination of virtual and real. At the same time, the scene supports the function of rotating surveillance videos, allowing managers to conduct comprehensive inspections of port area monitoring without manual intervention, effectively improving management and supervision efficiency.
[0054] The specific steps (3) are as follows:
[0055] (3-1) Implement intelligent twin management of berths in the port and provide berth reservation function. Fishermen can view the map-based area division and usage status of berths in real time through a WeChat mini program with reservation function, and select vacant berths for online reservation and intelligent route planning, so as to realize the visualization of berth management and the efficiency of berth use.
[0056] Open the WeChat mini program, log in, and enter the berth reservation interface. Use the 2D map and vector elements in the WeChat mini program to determine the current berth availability in the port, and select a suitable vacant berth to make a reservation.
[0057] (3-2) The current usage status of berths in the port is displayed in the form of a two-dimensional map combined with vector elements; select an available berth to make a reservation. After receiving the reservation command, the backend pushes the reservation information to the twin scene via WebSocket. The twin scene uses gradient walls of different colors to display the real-time status of the berth; click on the berth to display the recent usage statistics of the berth; the backend obtains the coordinates and orientation information of the fishing boat in real time through the positioning equipment installed on the fishing boat, in conjunction with the near-port radar or monitoring screen. The twin scene plans the berthing route for the fishing boat based on the transmitted information.
[0058] (3-3) The A* algorithm is used to provide necessary berth route planning for fishing vessels. First, data preprocessing is performed. The berths are vectorized using remote sensing satellite images of the port area. The vector elements are converted into TIFF. The TIFF is binarized using a raster calculator. The TIFF is resampled according to the complexity of the port berths. The post-processing results are read using GDAL and exported as a two-dimensional x*y array. The array, range coordinates, and pixel parameters are saved to the database. During route planning, the latitude and longitude coordinates of the vessel position are passed in. The latitude and longitude coordinates are converted into rectangular coordinates according to the coordinate range and pixel parameters corresponding to the array. After reading the array, the code performs a breadth-first search to obtain the set of route nodes. The set of nodes is converted from the rectangular coordinate system to the geographic coordinate system. A GeoJson object is created and returned to the system as the result. The twin scene loads the GeoJson object to render the planned route.
[0059] Step (4) specifically refers to:
[0060] A two-way communication protocol is established between the twin scene and the radar data server. The radar data server analyzes the maritime vessels of interest currently being tracked by the radar and pushes the basic information and geospatial parameters of the current targets to the twin scene in real time. After receiving the data, the twin scene determines the rendering status of the target vessels in the scene. If the system is tracking the target vessel for the first time, it renders the vessel model and adds response events to display the vessel's basic information. If the target is being continuously tracked by the system, it dynamically adjusts the vessel's geospatial parameters to display the vessel's position and attitude in real time. The screen coordinates of the current radar model are directly used as the screen coordinates of the video controls through browser window coordinate calculation and dynamically adjusted using CSS. In a suitable location, ensuring no obstruction of other models, the video control is dynamically bound to the location of the radar model in the twin scene and automatically plays real-time monitoring footage. By converting the geographic coordinates at sea to the spatial coordinates of the twin scene, specifically using the Cesium spatial coordinate transformation function toolkit, the geographic coordinate system is converted to the Cartesian coordinate system. The spatial coordinates of the locked vessel and the radar in the twin scene are calculated, and a pyramid is drawn as the tracking light cone. The base of the pyramid faces the locked vessel, and the apex is located at the radar position. The azimuth angle parameters of the light cone are adjusted in real time using keyframe animation functions, so that the light cone continuously locks onto the tracked vessel. Even if the target area has a dense distribution of ships, the manager can clearly understand the vessel currently locked by the radar.
[0061] Considering development costs and hardware performance limitations, data transmission will not be continuous; it will be pushed out every few seconds. Interpolation processing is performed on the positions of consecutive data points to ensure that the movement path of the ships in the scene is continuous and uniform, avoiding ship position jumps.
[0062] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for constructing and using a digital twin smart fishing port based on oblique photogrammetry, characterized in that, include: (1) Digital twin scene construction; (2) Fishing port video fusion; (3) Management of twin berths in fishing ports; (4) Radar tracking of targets in port areas; The specific steps (3) are as follows: (3-1) Implement intelligent twin management of berths in the port, provide berth reservation function, fishermen can view the map-based area division and usage status of berths in real time, and select vacant berths for online reservation and intelligent route planning; (3-2) The current usage status of berths in the port is displayed in the form of a two-dimensional map combined with vector elements in the WeChat mini program; select an available berth to make a reservation. After receiving the reservation command, the backend pushes the reservation information to the twin scene via WebSocket. The twin scene uses gradient walls of different colors to display the real-time status of the berth; click on the berth to display the recent usage statistics of the berth; the backend obtains the coordinates and orientation information of the fishing boat in real time through the positioning equipment installed on the fishing boat, in conjunction with the near-port radar or monitoring screen. The twin scene plans the berthing route of the fishing boat based on the transmitted information. (3-3) The A* algorithm is used to provide necessary berth route planning for fishing boats. First, data preprocessing is performed. The berths are vectorized using remote sensing satellite images of the port area. The vector elements are converted into TIFF. The TIFF is binarized using a raster calculator. The TIFF is resampled according to the complexity of the port berths. The post-processing results are read using GDAL and a two-dimensional x*y array is exported. The array, range coordinates, and pixel parameters are saved to the database. When planning the route, the latitude and longitude coordinates of the boat position are passed in. After the code reads the array, it converts the latitude and longitude coordinates into rectangular coordinates according to the coordinate range and pixel parameters of the array. After performing a breadth-first search in the array, the set of route nodes is obtained. The set of nodes is converted from rectangular coordinates to geographic coordinates. A GeoJson object is created and returned to the system as the result. The twin scene can render the planned route by loading GeoJson. The specific steps (4) are as follows: The twin scene establishes a two-way communication protocol with the radar data server. The radar data server analyzes the maritime vessels of interest currently being tracked by the radar and pushes the basic information and geospatial parameters of the current targets to the twin scene in real time. After receiving data, the twin scene determines the rendering status of the target vessel. If the system is tracking the target vessel for the first time, it renders the vessel model and adds response events to display the vessel's basic information. If the target is being continuously tracked by the system, it dynamically adjusts the vessel's geospatial parameters to display the vessel's position and attitude in real time. It calculates the screen coordinates of the current radar model using browser window coordinates, and dynamically adjusts the video control to a suitable position using CSS, ensuring it doesn't obstruct other models. The video control is then dynamically bound to the radar model's location in the twin scene and automatically plays real-time monitoring footage. Finally, it converts the geographic coordinates at sea to the twin scene's spatial coordinates using the Cesium spatial coordinate transformation toolkit, converting the geographic coordinate system to a Cartesian coordinate system. It calculates the spatial coordinates of the locked vessel and radar in the twin scene, and draws a pyramid as the tracking light cone. The pyramid's base faces the locked vessel, and its apex is located at the radar position. Keyframe animation functions are used to adjust the light cone's azimuth parameters in real time, ensuring the light cone continuously locks onto the tracked vessel.
2. The method for constructing and using a digital twin smart fishing port based on oblique photogrammetry according to claim 1, characterized in that, Step (1) specifically includes: (1-1) Aerial photography by drones: After conducting oblique photogrammetry flight surveys of the fishing port area, the aerial photography area covering the fishing port was drawn using a marking language; to ensure the accuracy of the oblique photogrammetry results, the selected image control points were evenly distributed throughout the port area; after marking the image control points by ground spraying or selecting landmark features, the image control points were measured using RTK equipment; Before the flight, the environment of the fishing port is inspected to determine whether the current environmental conditions are suitable for flight. If the conditions are met, the drone is installed. When the drone's self-check is correct and the RTK module is working properly, the aerial photography work can begin. (1-2) Fishing port modeling: The raw data captured by the drone is exported, and the raw data is read, the first aerial triangulation is performed, the puncture point is determined, the second aerial triangulation is performed, and the model is generated in two formats: OSGB and OBJ. The first aerial triangulation is used to calculate the exterior orientation elements of each photo; the second aerial triangulation is used to improve the calculation accuracy of the exterior orientation elements of the photo and correct the model. (1-3) Fishing port model restoration: The original 3D real scene model was optimized using ModelFun software, and the OSGB and OBJ format dataset models were imported. After the model restoration was completed, the OSGB data was regenerated. (1-4) Detailed modeling of fishing port: Based on the modeling of the original 3D real scene model, determine the areas in the port area that need detailed modeling, use modeling software to perform detailed modeling, and generate OBJ model after modeling is completed; (1-5) Rendering of fishing port model: Convert the repaired results into 3D Tiles format; use Cesium or MapTalks open source GIS framework to load 3D Tiles data on the browser side, adjust parameters according to project requirements, and realize front-end rendering of three-dimensional real scene.
3. The method for constructing and using a digital twin smart fishing port based on oblique photogrammetry according to claim 1 or 2, characterized in that, The specific steps (2) are as follows: (2-1) After performing detailed modeling of the camera in the twin scene, load it to the corresponding position in the scene and add response events to control the camera to turn on or off. When the event response is turned on, use the Cesium camera to create a view cone to simulate the field of view of the surveillance camera. Based on the camera position, azimuth angle and focal length parameters provided by the backend interface, automatically switch the view of the twin scene to the corresponding position of the surveillance screen. (2-2) Adjust the camera frustum so that it accurately reflects the actual focal length, angle and position of the camera; use Cesium's ShadowMap function to perform field-of-view analysis on the camera frustum range; (2-3) Create a video tag and generate video texture by playing the monitoring video stream; use GLSL language to write texture processing logic. The visible area within the view frustum is calculated with respect to the camera and the shadow coordinates through the video texture and the corresponding vertex to obtain the corresponding gl_FragColor. The shadow area retains the original texture of the model. (2-4) To avoid the video distortion caused by projection affecting the visual experience, an additional set of video components is created to synchronously display the playback screen in normal state. The video player and the camera position in the twin scene are bound by the callback function provided by Cesium.