Port loading and unloading process simulation display data organization method and system based on VR technology

Abstract

The present invention discloses a method and system for organizing data of port loading and unloading process simulation display based on VR technology, including: performing regional environment analysis and port facility analysis according to target port information, and constructing port environment sub-model and port facility sub-model; constructing a general assembly model according to the port environment sub-model and the port facility sub-model, and performing port virtual scene rendering and dynamic association to obtain a target port three-dimensional model; performing scene motion modeling based on the target port three-dimensional model, formulating scene motion rules, and obtaining a target port operation model; constructing a target port virtual environment model, and setting user interaction rules and scene triggering rules; obtaining port loading and unloading information, extracting the loading and unloading process characteristics and loading and unloading process characteristics of the target port, and establishing loading and unloading simulation rules in combination with the target port virtual environment model to adapt the loading and unloading scene. The design efficiency and construction quality are improved, and the recognition and experience of the automated container terminal project by the general audience are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of port loading and unloading process simulation, and in particular to a method and system for organizing data for port loading and unloading process simulation display based on VR technology. Background Art

[0002] Port loading and unloading technology is a highly professional technical work, and it is also an important work in the design, construction and operation of port engineering terminals. Intuitively showing the design intention of terminal loading and unloading technology to owners, management departments and other stakeholders can effectively help all parties reach a consensus, promote the design progress, and serve the future management of the owner. At present, the main methods of displaying port loading and unloading technology include on-site field display, multimedia display, 3D model and sand table display, but there are great safety hazards in on-site display, multimedia display is an indirect display, and the audience cannot independently and intuitively understand the working details of the loading and unloading process. 3D model and sand table display are static displays, which cannot help the audience understand the details of loading and unloading technology in dynamic operations. Therefore, a simulation display method that can intuitively display port loading and unloading technology is needed to meet professional technical display. Summary of the invention

[0003] The present invention overcomes the defects of the prior art and provides a method and system for organizing data for simulation display of port loading and unloading processes based on VR technology. Its important purpose is to improve design efficiency and construction quality and enhance the general audience's understanding and experience of automated container terminal projects.

[0004] To achieve the above-mentioned purpose, the first aspect of the present invention provides a method for organizing data for simulation display of port loading and unloading processes based on VR technology, comprising:

[0005] Acquire target port information, perform regional environment analysis and port facility analysis based on the target port information, and construct a port environment sub-model and a port facility sub-model;

[0006] Constructing a general assembly model according to the port environment sub-model and the port facility sub-model, and performing port virtual scene rendering and dynamic association to obtain a three-dimensional model of the target port;

[0007] Based on the three-dimensional model of the target port, scene motion modeling is performed, scene motion classification is performed, scene motion rules are formulated according to the classification results, and a target port operation model is obtained;

[0008] Building a virtual environment model of the target port based on the target port three-dimensional model and the target port motion model, and setting user interaction rules and scene triggering rules;

[0009] The port loading and unloading information is obtained, the loading and unloading process characteristics and loading and unloading process characteristics of the target port are extracted, and the loading and unloading simulation rules are established in combination with the virtual environment model of the target port to adapt the loading and unloading scenarios.

[0010] In this solution, the target port information is obtained, regional environment analysis and port facility analysis are performed according to the target port information, and a port environment sub-model and a port facility sub-model are constructed, specifically including:

[0011] Acquiring target port information, wherein the target port information includes target port area geographic information and target port facility information;

[0012] Perform regional environmental analysis based on the target port regional geographic information, extract regional geographic features of the target port, and obtain regional geographic feature information;

[0013] Acquire regional terrain parameters of the target port according to the regional geographical feature information, set environmental categories, and calculate the Euclidean distance between the regional geographical feature information and each environmental category;

[0014] The calculated Euclidean distance is compared with a preset threshold, and the corresponding regional geographical features are divided into various environmental categories according to the judgment results to obtain environmental parameters of each environmental category;

[0015] Acquire the target port facility information according to the target port information, perform facility structure analysis, analyze the wharf structure, loading and unloading facilities, storage facilities and auxiliary facilities structure of the target port, and obtain the port facility parameters of the target port;

[0016] Obtain a component library, wherein the component library contains all components used to construct a port and components used to construct a terrain, namely, an environment component, a building component, and a terrain component;

[0017] The regional terrain parameters, environmental parameters and port facility parameters are imported into the component library for component pairing analysis, and the port environment sub-model and the port facility sub-model are constructed based on BIM technology.

[0018] In this solution, the assembly model is constructed according to the port environment sub-model and the port facility sub-model, and the port virtual scene rendering and dynamic association are performed to obtain the target port three-dimensional model, which specifically includes:

[0019] Export the port environment sub-model and the port facility sub-model and perform data standardization, and import the model files into UE4 using the Datasmith plug-in;

[0020] The port facility sub-model is used as a sub-component of the port environment sub-model, static mesh components of each sub-model are added, initial state settings are performed, dynamic association logic is set, and corresponding environment components are associated with facility components;

[0021] Set up Datasmith DirectLink based on the Direct Link workflow, build model update rules, bind the sub-model's update events to UE4, and perform real-time updates through the set Datasmith DirectLink to obtain the final assembly model.

[0022] Preset updated test data, perform performance test on the assembly model, and perform parameter adjustment and model optimization according to the updated effect of the assembly model;

[0023] The assembly model is imported into a VR renderer to render a virtual port scene, and lighting rendering, material replacement, and environmental parameter adjustment are performed on the entire port scene. A number of viewing distance levels are set, and each viewing distance level corresponds to a different rendering quality for LOD optimization;

[0024] The static meshes in the rendering scene are extracted for mesh merging and a rendering performance test is performed. The model operation efficiency is optimized based on the test results to obtain a three-dimensional model of the target port.

[0025] In this solution, the scene motion modeling is performed based on the three-dimensional model of the target port, the scene motion category is divided, and the scene motion rules are formulated according to the division results to obtain the target port operation model, which specifically includes:

[0026] Extracting motion elements based on the three-dimensional model of the target port, including static elements and dynamic elements, and classifying scene motion according to the extracted motion elements into basic motion and random motion, thereby obtaining scene motion classification information;

[0027] Extracting motion elements corresponding to basic motion classes according to the scene motion classification information, defining basic motion rules, performing basic kinematic modeling, and obtaining a basic motion model;

[0028] Obtain historical motion instances of random motion through big data retrieval, extract features of each historical motion instance, including motion speed, motion path and motion sequence, and obtain feature information of historical motion instances;

[0029] Set a motion sequence benchmark, construct a motion change node in combination with the feature information of the historical motion instance, analyze the nodes where each historical motion instance differs from the motion sequence benchmark as motion change nodes, use the corresponding historical motion instance as a node-attached feature, and construct a state transition space for each motion change node;

[0030] The historical motion instances corresponding to each motion change node are used as the transfer state, the similarity between each transfer state is calculated, and the transfer state is aggregated based on the calculated similarity;

[0031] Feature extraction is performed based on the results of transition state aggregation, the mutual information value between each feature and the corresponding transition state is calculated, and the value is compared with the preset threshold to obtain the root feature analysis information;

[0032] The state transition probability of each motion change node corresponding to the transition state is calculated through the constructed state transition space, and a state transition matrix is ​​constructed. The root feature analysis information is used as a priori features of the state transition, and a random motion rule is constructed in combination with the state transition matrix to perform random motion modeling to obtain a random motion model.

[0033] A target port motion model is constructed according to the basic motion model and the random motion model.

[0034] In this solution, the target port virtual environment model is constructed based on the target port three-dimensional model and the target port motion model, and user interaction rules and scene triggering rules are set, specifically including:

[0035] Obtain the target port three-dimensional model and the target port motion model for model integration, build the target port virtual environment model through the virtual engine UE4, and set user interaction rules, wherein the user interaction rules include interface interaction, physical interaction and event interaction;

[0036] Different scene roles are set based on the virtual environment model of the target port, and corresponding scene role task paths are set. By analyzing the user's real-time experience position and time, it is judged whether the scene loss phenomenon occurs. If so, the distance to the user's real-time experience position is calculated based on the extracted user's real-time task node, and a guide mark is generated for prompting;

[0037] Set scene trigger rules based on the set scene roles and scene role task paths, divide scene events into preset events and random events, set preset event trigger rules through scene roles and scene role task paths, set random event trigger logic and trigger features based on UE4 blueprint visual scripts and form random event trigger rules;

[0038] The scene triggering rules are formed according to the preset event triggering rules and the random event triggering rules, and the scene triggering rules include time triggering rules, location triggering rules and state triggering rules.

[0039] In this solution, the port loading and unloading information is obtained, the loading and unloading process characteristics and loading and unloading process characteristics of the target port are extracted, and the loading and unloading simulation rules are established in combination with the virtual environment model of the target port to adapt the loading and unloading scene, which specifically includes:

[0040] Acquire port loading and unloading information, perform feature extraction on the port loading and unloading information, extract loading and unloading process features and loading and unloading process features of each loading and unloading type, and obtain loading and unloading feature information;

[0041] According to the characteristics of the loading and unloading process, the loading and unloading process nodes of the target port are set, and the loading and unloading process of each loading and unloading process node is matched according to the characteristics of the loading and unloading process to form a loading and unloading process diagram of the target port;

[0042] Obtain historical loading and unloading instances of the target port, match and analyze each loading and unloading instance with the corresponding loading and unloading process according to the loading and unloading flow chart, and obtain loading and unloading instance matching information according to the similarity between each historical loading and unloading instance and each loading and unloading process and loading and unloading technology in the loading and unloading flow chart;

[0043] Extract features from each historical loading and unloading instance to obtain the loading and unloading event features of each instance, including loading and unloading routes, loading and unloading processes, the number of vehicles on the loading and unloading routes, and port throughput, and obtain the loading and unloading event feature information;

[0044] According to the loading and unloading process nodes, loading and unloading processes and loading and unloading events, entity triples are formed and a loading and unloading knowledge graph is constructed. The characteristic information of the loading and unloading events is used as the auxiliary features of each loading and unloading event, and the occurrence probability of each loading and unloading event is set;

[0045] Based on the loading and unloading knowledge graph, loading and unloading simulation rules are set, and the loading and unloading routes and loading and unloading processes of users' real-time loading and unloading experience are used as prior conditions to match loading and unloading events. The number of vehicles on the loading and unloading routes and the port throughput are simulated according to the matching results of loading and unloading events, loading and unloading simulation events are generated, and real-time loading and unloading routes are generated through path planning algorithms;

[0046] The loading and unloading simulation rules are adapted to the target port virtual environment model for loading and unloading scenarios.

[0047] A second aspect of the present invention provides a port loading and unloading process simulation display data organization system based on VR technology, the system comprising: a memory, a processor, the memory containing a port loading and unloading process simulation display data organization method program based on VR technology, the port loading and unloading process simulation display data organization method program based on VR technology when executed by the processor implements the following steps:

[0048] Acquire target port information, perform regional environment analysis and port facility analysis based on the target port information, and construct a port environment sub-model and a port facility sub-model;

[0049] Constructing a general assembly model according to the port environment sub-model and the port facility sub-model, and performing port virtual scene rendering and dynamic association to obtain a three-dimensional model of the target port;

[0050] Based on the three-dimensional model of the target port, scene motion modeling is performed, scene motion classification is performed, scene motion rules are formulated according to the classification results, and a target port operation model is obtained;

[0051] Building a virtual environment model of the target port based on the target port three-dimensional model and the target port motion model, and setting user interaction rules and scene triggering rules;

[0052] The port loading and unloading information is obtained, the loading and unloading process characteristics and loading and unloading process characteristics of the target port are extracted, and the loading and unloading simulation rules are established in combination with the virtual environment model of the target port to adapt the loading and unloading scenarios.

[0053] The present invention discloses a method and system for organizing data of port loading and unloading process simulation display based on VR technology, including: performing regional environment analysis and port facility analysis according to target port information, and constructing port environment sub-model and port facility sub-model; constructing a general assembly model according to the port environment sub-model and the port facility sub-model, and performing port virtual scene rendering and dynamic association to obtain a target port three-dimensional model; performing scene motion modeling based on the target port three-dimensional model, formulating scene motion rules, and obtaining a target port operation model; constructing a target port virtual environment model, and setting user interaction rules and scene triggering rules; obtaining port loading and unloading information, extracting the loading and unloading process characteristics and loading and unloading process characteristics of the target port, and establishing loading and unloading simulation rules in combination with the target port virtual environment model to adapt the loading and unloading scene. The design efficiency and construction quality are improved, and the recognition and experience of the automated container terminal project by the general audience are enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the drawings required for use in the embodiments or exemplary descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to the drawings without paying creative work.

[0055] Figure 1 A flow chart of a method for organizing data for simulation display of port loading and unloading processes based on VR technology provided by one embodiment of the present invention;

[0056] Figure 2 A loading and unloading simulation flow chart provided for an embodiment of the present invention;

[0057] Figure 3 A block diagram of a data organization system for port loading and unloading process simulation display based on VR technology provided by an embodiment of the present invention;

[0058] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0059] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0061] Figure 1 A flow chart of a method for organizing data for simulation display of port loading and unloading processes based on VR technology provided by one embodiment of the present invention;

[0062] like Figure 1 As shown, the present invention provides a flow chart of a method for organizing data for simulation display of port loading and unloading processes based on VR technology, including:

[0063] S102, obtaining target port information, performing regional environment analysis and port facility analysis according to the target port information, and constructing a port environment sub-model and a port facility sub-model;

[0064] S104, constructing a general assembly model according to the port environment sub-model and the port facility sub-model, and performing port virtual scene rendering and dynamic association to obtain a three-dimensional model of the target port;

[0065] S106, performing scene motion modeling based on the three-dimensional model of the target port, performing scene motion classification, formulating scene motion rules according to the classification results, and obtaining a target port operation model;

[0066] S108, constructing a target port virtual environment model based on the target port three-dimensional model and the target port motion model, and setting user interaction rules and scene triggering rules;

[0067] S110, obtaining port loading and unloading information, extracting the loading and unloading process characteristics and loading and unloading process characteristics of the target port, establishing loading and unloading simulation rules in combination with the virtual environment model of the target port, and performing loading and unloading scene adaptation.

[0068] It should be noted that the present invention provides a method and system for organizing data for simulation display of port loading and unloading processes based on VR technology. According to the target port information, the regional environment and port facilities are analyzed, and two sub-models are constructed to represent the port environment and facilities respectively. The general assembly model is constructed based on the constructed sub-models, and the three-dimensional model of the target port is generated through virtual scene rendering and dynamic association. Then, based on the three-dimensional model of the target port, scene motion modeling is performed, and motion rules are formulated according to the motion characteristics to obtain the operation model of the port. Finally, the three-dimensional model and the operation model are combined to construct a virtual environment model, and the rules for user interaction and scene triggering are set. At the same time, the loading and unloading operations of the port are simulated, and the loading and unloading information is obtained, the relevant features are extracted, and the loading and unloading simulation rules are formulated in combination with the virtual environment model to achieve the adaptation of the loading and unloading scenes. Improve the design efficiency and construction quality, and enhance the general audience's understanding and experience of the automated container terminal project.

[0069] Further, in a preferred embodiment of the present invention, the acquisition of target port information, performing regional environment analysis and port facility analysis according to the target port information, and constructing a port environment sub-model and a port facility sub-model specifically include:

[0070] Acquiring target port information, wherein the target port information includes target port area geographic information and target port facility information;

[0071] Perform regional environmental analysis based on the target port regional geographic information, extract regional geographic features of the target port, and obtain regional geographic feature information;

[0072] Acquire regional terrain parameters of the target port according to the regional geographical feature information, set environmental categories, and calculate the Euclidean distance between the regional geographical feature information and each environmental category;

[0073] The calculated Euclidean distance is compared with a preset threshold, and the corresponding regional geographical features are divided into various environmental categories according to the judgment results to obtain environmental parameters of each environmental category;

[0074] Acquire the target port facility information according to the target port information, perform facility structure analysis, analyze the wharf structure, loading and unloading facilities, storage facilities and auxiliary facilities structure of the target port, and obtain the port facility parameters of the target port;

[0075] Obtain a component library, wherein the component library contains all components used to construct a port and components used to construct a terrain, namely, an environment component, a building component, and a terrain component;

[0076] The regional terrain parameters, environmental parameters and port facility parameters are imported into the component library for component pairing analysis, and the port environment sub-model and the port facility sub-model are constructed based on BIM technology.

[0077] It should be noted that the information of the target port is first obtained, including regional geographic information and facility information. Regional geographic information involves the geographical location of the target port, the surrounding terrain, climatic conditions, etc., while facility information covers the specific details of the dock, loading and unloading equipment, storage facilities and auxiliary facilities. Next, a regional environmental analysis is performed based on the regional geographic information of the target port to extract the regional geographical features of the port, such as the shape of the coastline, altitude, etc. Then, the regional terrain parameters of the port are obtained based on these geographical feature information, different environmental categories are set, and the Euclidean distance between each environmental category is calculated. By comparing the calculated Euclidean distance, it is determined which environmental category each geographical feature belongs to, so that the regional geographical features are divided into the corresponding environmental categories, and the environmental parameters of each category are obtained. Facility structure analysis is performed for the facility information of the target port. This includes analyzing the layout and function of the port's dock structure, loading and unloading facilities, storage facilities and auxiliary facilities, and obtaining the specific parameters of each facility, such as the length and width of the dock, the type and number of loading and unloading equipment, the capacity and distribution of storage facilities, and the functional zoning of auxiliary facilities. Subsequently, various components used to build ports and terrain are obtained from the component library, which are divided into environmental components, building components and terrain components. The environment component is used to simulate the natural environment around the port, the building component is used to construct various buildings and facilities in the port, and the terrain component is used to simulate the topography of the port and its surroundings. Finally, the obtained regional terrain parameters, environmental parameters and port facility parameters are imported into the component library for pairing analysis, and the port environment sub-model and port facility sub-model are constructed based on the building information model (BIM) technology.

[0078] Furthermore, in a preferred embodiment of the present invention, the assembly model is constructed according to the port environment sub-model and the port facility sub-model, and the port virtual scene rendering and dynamic association are performed to obtain the target port three-dimensional model, which specifically includes:

[0079] Export the port environment sub-model and the port facility sub-model and perform data standardization, and import the model files into UE4 using the Datasmith plug-in;

[0080] The port facility sub-model is used as a sub-component of the port environment sub-model, static mesh components of each sub-model are added, initial state settings are performed, dynamic association logic is set, and corresponding environment components are associated with facility components;

[0081] Set up Datasmith DirectLink based on the Direct Link workflow, build model update rules, bind the sub-model's update events to UE4, and perform real-time updates through the set Datasmith DirectLink to obtain the final assembly model.

[0082] Preset updated test data, perform performance test on the assembly model, and perform parameter adjustment and model optimization according to the updated effect of the assembly model;

[0083] The assembly model is imported into a VR renderer to render a virtual port scene, and lighting rendering, material replacement, and environmental parameter adjustment are performed on the entire port scene. A number of viewing distance levels are set, and each viewing distance level corresponds to a different rendering quality for LOD optimization;

[0084] The static meshes in the rendering scene are extracted for mesh merging and a rendering performance test is performed. The model operation efficiency is optimized based on the test results to obtain a three-dimensional model of the target port.

[0085] It should be noted that using the Datasmith plugin with UE4, the Direct Link workflow allows you to set up Datasmith DirectLink between one or more source applications and multiple destinations (such as Unreal Engine-based applications or Twinmotion). This link updates the Unreal Engine level model you set up, eliminating the need to re-export the *.udatasmith file from the source BIM model every time you make a model change. This makes it easier to update and visualize incremental changes to 3D scenes in near real time, and can convert the entire port project scene (including animations and metadata) from 3dsMax, Revit, SketchUp, Civil 3D, SolidWorks, and various other CAD, BIM, and GIS formats with high fidelity, greatly reducing the workload of format conversion. By setting detailed lighting and shadow details within specific parts, combined with setting the model LOD or level of detail, you can switch between showing the overall effect from a far perspective and showing more model details from a close perspective.

[0086] Further, in a preferred embodiment of the present invention, the scene motion modeling is performed based on the three-dimensional model of the target port, the scene motion category is divided, and the scene motion rules are formulated according to the division results to obtain the target port operation model, which specifically includes:

[0087] Extracting motion elements based on the three-dimensional model of the target port, including static elements and dynamic elements, and classifying scene motion according to the extracted motion elements into basic motion and random motion, thereby obtaining scene motion classification information;

[0088] Extracting motion elements corresponding to basic motion classes according to the scene motion classification information, defining basic motion rules, performing basic kinematic modeling, and obtaining a basic motion model;

[0089] Obtain historical motion instances of random motion through big data retrieval, extract features of each historical motion instance, including motion speed, motion path and motion sequence, and obtain feature information of historical motion instances;

[0090] Set a motion sequence benchmark, construct a motion change node in combination with the feature information of the historical motion instance, analyze the nodes where each historical motion instance differs from the motion sequence benchmark as motion change nodes, use the corresponding historical motion instance as a node-attached feature, and construct a state transition space for each motion change node;

[0091] The historical motion instances corresponding to each motion change node are used as the transfer state, the similarity between each transfer state is calculated, and the transfer state is aggregated based on the calculated similarity;

[0092] Feature extraction is performed based on the results of the transfer state aggregation, the mutual information value between each feature and the corresponding transfer state is calculated, and the value is compared with the preset threshold to obtain the root feature analysis information;

[0093] The state transition probability of each motion change node corresponding to the transition state is calculated through the constructed state transition space, and a state transition matrix is ​​constructed. The root feature analysis information is used as a priori features of the state transition, and a random motion rule is constructed in combination with the state transition matrix to perform random motion modeling to obtain a random motion model.

[0094] A target port motion model is constructed according to the basic motion model and the random motion model.

[0095] It should be noted that in the process of terminal loading and unloading simulation, the dynamic models in the three-dimensional scene, including berthing ships, quay cranes, container yard rail cranes and container horizontal transport trolleys and other major port loading and unloading equipment, need to be modeled based on basic kinematics. The reason is that mastering the characteristics of the movement of port machinery and equipment can make the simulation effect more consistent with the actual situation. It is mainly summarized into two types of motion models: 1) Basic motion model, mainly including ships, quay cranes at the front of the terminal, and container tire cranes in the yard. It mainly involves the swing of the spreader during the berthing of the ship, the loading and unloading operation of the quay crane, and the stacking of containers by the tire crane in the yard. The characteristics of this type of motion model are that the action is relatively repetitive and regular, which can be decomposed into single-point or unidirectional actions, and the burden on the overall calculation and graphic display of the system is relatively small; 2) Random motion model, mainly including loading and unloading transport trolleys, external transport container trucks, etc. It mainly involves the selection of loading and unloading operation points, driving paths, speeds, automatic avoidance principles and braking, and the selection of routes for external container trucks to enter and leave the port. The characteristics of this type of motion model are that the action follows random rules and needs to be repeatedly adjusted to form a closed running route. It has a heavy burden on the overall calculation and graphic display of the system, so the number of such models should be strictly controlled. The kinematic model and geometric modeling are independent of each other. Based on the imported geometric model, the set motion is restricted and programmed accordingly according to the law of actual motion, which improves the kinematic modeling efficiency of the model and realizes the simulation of the virtual scene of automated port operation.

[0096] Furthermore, in a preferred embodiment of the present invention, the target port virtual environment model is constructed based on the target port three-dimensional model and the target port motion model, and user interaction rules and scene triggering rules are set, specifically including:

[0097] Obtain the target port three-dimensional model and the target port motion model for model integration, build the target port virtual environment model through the virtual engine UE4, and set user interaction rules, wherein the user interaction rules include interface interaction, physical interaction and event interaction;

[0098] Different scene roles are set based on the virtual environment model of the target port, and corresponding scene role task paths are set. By analyzing the user's real-time experience position and time, it is judged whether the scene loss phenomenon occurs. If so, the distance to the user's real-time experience position is calculated based on the extracted user's real-time task node, and a guide mark is generated for prompting;

[0099] Set scene trigger rules based on the set scene roles and scene role task paths, divide scene events into preset events and random events, set preset event trigger rules through scene roles and scene role task paths, set random event trigger logic and trigger features based on UE4 blueprint visual scripts and form random event trigger rules;

[0100] The scene triggering rules are formed according to the preset event triggering rules and the random event triggering rules, and the scene triggering rules include time triggering rules, location triggering rules and state triggering rules.

[0101] It should be noted that the user interface (UI) and the switching function of each module scene are realized through programming with Blue Print visual scripts. Based on the UI guidance function, the experiencer can move freely in the virtual scene and perform corresponding human-computer interaction operations. The experiencer loses his way in an unfamiliar virtual scene, and it takes a long time to find the location to complete the task, which affects the experience effect. The UI guidance function can solve this problem. When the experiencer is experiencing the project, according to the guidance signs and operation panel prompts that appear on the ground, he quickly moves to the designated location to complete the corresponding operation, and smoothly returns to the main menu after completing the corresponding module experience. At the same time, in the simulation environment, the experiencer's redundant UI will distort the scene. Considering the experiencer's unintentional blindness, when there is too much information on the interface, the experiencer is difficult to respond to each sensory input, thus generating anxiety. Therefore, a simple blue cursor is used as a guidance sign to remind the experiencer how to move and operate the corresponding interactive information. It is also used for information prompts that appear when entering the key areas or important links of the virtual scene, the purpose is to remind the experiencer of the name of the selected experience project and enhance the experiencer's immersion in the VR system. For example, when the experiencer chooses the loading and unloading process in the yard area, the perspective follows the equipment perspective, experiencing the loading and unloading process from the front of the dock to the yard. At the same time, it is equipped with handle vibration and automatic avoidance experience settings, allowing the experiencer to smoothly experience the fully automated and efficient loading and unloading process. When the external container truck enters and exits the gate module, the experiencer's driving experience in the truck cockpit is simulated. When passing through the three gates, the experiencer needs to perform corresponding operations according to the prompts. In addition, the truck driving sound effects are set, and the real-life voice is placed in the virtual environment. The experiencer enters the project under the dual senses of hearing and vision, achieving an effect that is difficult to distinguish between true and false.

[0102] It should be noted that in the scene trigger rule setting, the time trigger rule: trigger events according to time settings, such as routine inspections or maintenance at a fixed time every day. Location trigger rule: set a location trigger point to trigger the corresponding event when the user or device reaches a specific location. State trigger: trigger events according to the state change of equipment or personnel, such as abnormal equipment operation status, personnel entering a dangerous area, etc. By building logical nodes, the trigger logic is built using the blueprint in UE4 to ensure the accurate execution of trigger conditions and trigger events. At the same time, a response mechanism is set for each trigger event, such as notifying the user, performing specific operations, updating the scene status, etc. In addition, in setting user interaction rules, interface interactions include menus, buttons, information panels, etc., which facilitate users to interact with the virtual environment. Physical interaction enables users to physically interact with objects in the environment, such as picking up, moving, and operating devices. Event interactions, such as clicking, dragging, and triggering, enable users to interact with elements in the scene through simple operations.

[0103] Furthermore, in a preferred embodiment of the present invention, the obtaining of port loading and unloading information, extracting the loading and unloading process characteristics and loading and unloading process characteristics of the target port, establishing loading and unloading simulation rules in combination with the virtual environment model of the target port, and performing loading and unloading scene adaptation specifically include:

[0104] Acquire port loading and unloading information, perform feature extraction on the port loading and unloading information, extract loading and unloading process features and loading and unloading process features of each loading and unloading type, and obtain loading and unloading feature information;

[0105] According to the characteristics of the loading and unloading process, the loading and unloading process nodes of the target port are set, and the loading and unloading process of each loading and unloading process node is matched according to the characteristics of the loading and unloading process to form a loading and unloading process diagram of the target port;

[0106] Obtain historical loading and unloading instances of the target port, match and analyze each loading and unloading instance with the corresponding loading and unloading process according to the loading and unloading flow chart, and obtain loading and unloading instance matching information according to the similarity between each historical loading and unloading instance and each loading and unloading process and loading and unloading technology in the loading and unloading flow chart;

[0107] Extract features from each historical loading and unloading instance to obtain the loading and unloading event features of each instance, including loading and unloading routes, loading and unloading processes, the number of vehicles on the loading and unloading routes, and port throughput, and obtain the loading and unloading event feature information;

[0108] According to the loading and unloading process nodes, loading and unloading processes and loading and unloading events, entity triples are formed and a loading and unloading knowledge graph is constructed. The characteristic information of the loading and unloading events is used as the auxiliary features of each loading and unloading event, and the occurrence probability of each loading and unloading event is set;

[0109] Based on the loading and unloading knowledge graph, loading and unloading simulation rules are set, and the loading and unloading routes and loading and unloading processes of users' real-time loading and unloading experience are used as prior conditions to match loading and unloading events. The number of vehicles on the loading and unloading routes and the port throughput are simulated according to the matching results of loading and unloading events, loading and unloading simulation events are generated, and real-time loading and unloading routes are generated through path planning algorithms;

[0110] The loading and unloading simulation rules are adapted to the target port virtual environment model for loading and unloading scenarios.

[0111] It should be noted that loading and unloading simulation is an important part of port loading and unloading simulation. First, the port loading and unloading information is obtained, including the loading and unloading process characteristics and loading and unloading process characteristics of different loading and unloading types. By extracting the characteristics of the port loading and unloading information, the loading and unloading feature information is obtained, which reflects the specific operation steps and processes of each loading and unloading type. Based on the extracted loading and unloading process characteristics, the loading and unloading process nodes of the target port are set. These nodes are key links in the loading and unloading process, such as loading, transportation, unloading, etc. Combined with the loading and unloading process characteristics, the loading and unloading process is matched for each loading and unloading process node to form a complete loading and unloading process diagram, which describes in detail all the steps and operations from the arrival of the goods at the port to their final departure. Next, the historical loading and unloading instances of the target port are obtained to provide data and experience in actual operations. According to the loading and unloading process diagram, each historical loading and unloading instance is matched and analyzed with the corresponding loading and unloading process. By comparing the similarity between the historical loading and unloading instances and the various processes and processes in the loading and unloading process diagram, the matching information of the loading and unloading instances is obtained. The loading and unloading event features of each historical loading and unloading instance are extracted, including loading and unloading routes, loading and unloading processes, the number of vehicles used in the loading and unloading process, and the throughput of the port, so as to describe the specific situation and parameters of each loading and unloading event. According to the loading and unloading process nodes, loading and unloading processes, and loading and unloading events, entity triples are constructed and a loading and unloading knowledge graph is established. The loading and unloading knowledge graph is a structured information representation method. It takes the loading and unloading event feature information as an auxiliary feature and sets the occurrence probability of each loading and unloading event, which can fully reflect the various factors in the loading and unloading process. Based on the loading and unloading knowledge graph, the loading and unloading simulation rules are set. The loading and unloading routes and loading and unloading processes of the user's real-time loading and unloading experience are used as prior conditions to match the loading and unloading events. According to the matching results, the number of vehicles on the loading and unloading routes and the throughput of the port are simulated to generate specific loading and unloading simulation events. At the same time, the real-time loading and unloading path is generated through the path planning algorithm, so that the user can experience different loading and unloading scenarios in the loading and unloading simulation experience, and feel the loading and unloading experience of path change and automatic avoidance. Finally, the set loading and unloading simulation rules are adapted to the virtual environment model of the target port to ensure that the loading and unloading simulation rules can be accurately executed in the virtual environment, thereby realizing the true simulation and optimization of the port loading and unloading scenes.

[0112] Figure 2 A loading and unloading simulation flow chart provided for an embodiment of the present invention;

[0113] like Figure 2 As shown, the present invention provides a loading and unloading simulation flow chart, including:

[0114] S202, obtaining loading and unloading routes and loading and unloading processes based on the user's real-time loading and unloading experience scenario, and matching loading and unloading events;

[0115] S204, randomly generating the number of vehicles on the loading and unloading routes and the throughput of the port according to the matching results of the loading and unloading events;

[0116] S206, based on the user's real-time loading and unloading experience position, the vehicle motion state transition probability in the real-time loading and unloading route is calculated according to the random motion rule, and the motion state transition is performed according to the state transition probability;

[0117] S208, performing event simulation based on the vehicle and user's real-time loading and unloading experience position after state transfer, generating a real-time loading and unloading path through a path planning algorithm, and providing loading and unloading guidance to the user.

[0118] It should be noted that when the user is experiencing the loading and unloading process, the user's loading and unloading route and loading and unloading process are obtained through the user's loading and unloading experience scene, and loading and unloading events are matched, such as matching corresponding road blockages and increased traffic, etc. Then, the corresponding loading and unloading route vehicles and port throughput are generated according to the matching results, and event presets are performed. Then, according to the user's real-time loading and unloading experience position, the state transition probability of the vehicle in the target user's loading and unloading experience route is calculated through random motion rules, such as lane change, collision, etc., and then the state transition is performed according to the state transition probability to generate random events. Subsequently, event simulation is performed based on the vehicle after state transfer and the user's real-time loading and unloading experience position, and it is determined whether it can block the user's loading and unloading experience, and event prompts are performed. At the same time, path planning is performed based on the judgment results, thereby enhancing the user's experience of loading and unloading process experience.

[0119] Figure 3 A port loading and unloading process simulation display data organization system 3 based on VR technology is provided in one embodiment of the present invention. The system comprises: a memory 31 and a processor 32. The memory 31 contains a port loading and unloading process simulation display data organization method program based on VR technology. When the port loading and unloading process simulation display data organization method program based on VR technology is executed by the processor 32, the following steps are implemented:

[0120] Acquire target port information, perform regional environment analysis and port facility analysis based on the target port information, and construct a port environment sub-model and a port facility sub-model;

[0121] Constructing a general assembly model according to the port environment sub-model and the port facility sub-model, and performing port virtual scene rendering and dynamic association to obtain a three-dimensional model of the target port;

[0122] Based on the three-dimensional model of the target port, scene motion modeling is performed, scene motion classification is performed, scene motion rules are formulated according to the classification results, and a target port operation model is obtained;

[0123] Building a virtual environment model of the target port based on the target port three-dimensional model and the target port motion model, and setting user interaction rules and scene triggering rules;

[0124] The port loading and unloading information is obtained, the loading and unloading process characteristics and loading and unloading process characteristics of the target port are extracted, and the loading and unloading simulation rules are established in combination with the virtual environment model of the target port to adapt the loading and unloading scenarios.

[0125] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0126] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0127] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0128] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0129] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0130] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for organizing data for port loading and unloading process simulation display based on VR technology, characterized in that: include: Acquire target port information, perform regional environment analysis and port facility analysis based on the target port information, and construct a port environment sub-model and a port facility sub-model; Constructing a general assembly model according to the port environment sub-model and the port facility sub-model, and performing port virtual scene rendering and dynamic association to obtain a three-dimensional model of the target port; Based on the three-dimensional model of the target port, scene motion modeling is performed, scene motion classification is performed, scene motion rules are formulated according to the classification results, and a target port operation model is obtained; Building a virtual environment model of the target port based on the target port three-dimensional model and the target port motion model, and setting user interaction rules and scene triggering rules; Obtain port loading and unloading information, extract the loading and unloading process characteristics and loading and unloading process characteristics of the target port, establish loading and unloading simulation rules in combination with the virtual environment model of the target port, and adapt the loading and unloading scene; The step of acquiring port loading and unloading information, extracting the loading and unloading process characteristics and loading and unloading process characteristics of the target port, establishing loading and unloading simulation rules in combination with the virtual environment model of the target port, and adapting the loading and unloading scene specifically includes: Acquire port loading and unloading information, perform feature extraction on the port loading and unloading information, extract loading and unloading process features and loading and unloading process features of each loading and unloading type, and obtain loading and unloading feature information; According to the characteristics of the loading and unloading process, the loading and unloading process nodes of the target port are set, and the loading and unloading process of each loading and unloading process node is matched according to the characteristics of the loading and unloading process to form a loading and unloading process diagram of the target port; Obtain historical loading and unloading instances of the target port, match and analyze each loading and unloading instance with the corresponding loading and unloading process according to the loading and unloading flow chart, and obtain loading and unloading instance matching information according to the similarity between each historical loading and unloading instance and each loading and unloading process and loading and unloading technology in the loading and unloading flow chart; Extract features from each historical loading and unloading instance to obtain the loading and unloading event features of each instance, including loading and unloading routes, loading and unloading processes, the number of vehicles on the loading and unloading routes, and port throughput, and obtain the loading and unloading event feature information; According to the loading and unloading process nodes, loading and unloading processes and loading and unloading events, entity triples are formed and a loading and unloading knowledge graph is constructed. The characteristic information of the loading and unloading events is used as the auxiliary features of each loading and unloading event, and the occurrence probability of each loading and unloading event is set; Based on the loading and unloading knowledge graph, loading and unloading simulation rules are set, and the loading and unloading routes and loading and unloading processes of users' real-time loading and unloading experience are used as prior conditions to match loading and unloading events. The number of vehicles on the loading and unloading routes and the port throughput are simulated according to the matching results of loading and unloading events, loading and unloading simulation events are generated, and real-time loading and unloading routes are generated through path planning algorithms; The loading and unloading simulation rules are adapted to the target port virtual environment model for loading and unloading scenarios.

2. According to the method for organizing data of port loading and unloading process simulation display based on VR technology in claim 1, it is characterized in that: The step of acquiring target port information, performing regional environment analysis and port facility analysis according to the target port information, and constructing a port environment sub-model and a port facility sub-model specifically includes: Acquiring target port information, wherein the target port information includes target port area geographic information and target port facility information; Perform regional environmental analysis based on the target port regional geographic information, extract regional geographic features of the target port, and obtain regional geographic feature information; Acquire regional terrain parameters of the target port according to the regional geographical feature information, set environmental categories, and calculate the Euclidean distance between the regional geographical feature information and each environmental category; The calculated Euclidean distance is compared with a preset threshold, and the corresponding regional geographical features are divided into various environmental categories according to the judgment results to obtain environmental parameters of each environmental category; Acquire the target port facility information according to the target port information, perform facility structure analysis, analyze the wharf structure, loading and unloading facilities, storage facilities and auxiliary facilities structure of the target port, and obtain the port facility parameters of the target port; Obtain a component library, wherein the component library contains all components used to construct a port and components used to construct a terrain, namely, an environment component, a building component, and a terrain component; The regional terrain parameters, environmental parameters and port facility parameters are imported into the component library for component pairing analysis, and the port environment sub-model and the port facility sub-model are constructed based on BIM technology.

3. The method for organizing data for port loading and unloading process simulation display based on VR technology according to claim 1 is characterized in that: The method of constructing the assembly model according to the port environment sub-model and the port facility sub-model, and performing port virtual scene rendering and dynamic association to obtain a target port three-dimensional model specifically includes: Export the port environment sub-model and the port facility sub-model and perform data standardization, and import the model files into UE4 using the Datasmith plug-in; The port facility sub-model is used as a sub-component of the port environment sub-model, static mesh components of each sub-model are added, initial state settings are performed, dynamic association logic is set, and corresponding environment components are associated with facility components; Set up Datasmith DirectLink based on the Direct Link workflow, build model update rules, bind the sub-model's update events to UE4, and perform real-time updates through the set Datasmith DirectLink to obtain the final assembly model. Preset updated test data, perform performance test on the assembly model, and perform parameter adjustment and model optimization according to the updated effect of the assembly model; The assembly model is imported into a VR renderer to render a virtual port scene, and lighting rendering, material replacement, and environmental parameter adjustment are performed on the entire port scene. A number of viewing distance levels are set, and each viewing distance level corresponds to a different rendering quality for LOD optimization; The static meshes in the rendering scene are extracted for mesh merging and a rendering performance test is performed. The model operation efficiency is optimized based on the test results to obtain a three-dimensional model of the target port.

4. The method for organizing data for port loading and unloading process simulation display based on VR technology according to claim 1 is characterized in that: The scene motion modeling is performed based on the three-dimensional model of the target port, the scene motion category is divided, and the scene motion rules are formulated according to the division results to obtain the target port operation model, which specifically includes: Extracting motion elements based on the three-dimensional model of the target port, including static elements and dynamic elements, and classifying scene motion according to the extracted motion elements into basic motion and random motion, thereby obtaining scene motion classification information; Extracting motion elements corresponding to basic motion classes according to the scene motion classification information, defining basic motion rules, performing basic kinematic modeling, and obtaining a basic motion model; Obtain historical motion instances of random motion through big data retrieval, extract features of each historical motion instance, including motion speed, motion path and motion sequence, and obtain feature information of historical motion instances; Set a motion sequence benchmark, construct a motion change node in combination with the feature information of the historical motion instance, analyze the nodes where each historical motion instance differs from the motion sequence benchmark as motion change nodes, use the corresponding historical motion instance as a node-attached feature, and construct a state transition space of each motion change node; The historical motion instances corresponding to each motion change node are used as the transfer state, the similarity between each transfer state is calculated, and the transfer state is aggregated based on the calculated similarity; Feature extraction is performed based on the results of the transfer state aggregation, the mutual information value between each feature and the corresponding transfer state is calculated, and the value is compared with the preset threshold to obtain the root feature analysis information; The state transition probability of each motion change node corresponding to the transition state is calculated through the constructed state transition space, and a state transition matrix is ​​constructed. The root feature analysis information is used as a priori feature of the state transition, and a random motion rule is constructed in combination with the state transition matrix to perform random motion modeling to obtain a random motion model. A target port motion model is constructed according to the basic motion model and the random motion model.

5. The method for organizing data for port loading and unloading process simulation display based on VR technology according to claim 1 is characterized in that: The target port virtual environment model is constructed based on the target port three-dimensional model and the target port motion model, and user interaction rules and scene triggering rules are set, specifically including: Obtain the target port three-dimensional model and the target port motion model for model integration, build the target port virtual environment model through the virtual engine UE4, and set user interaction rules, wherein the user interaction rules include interface interaction, physical interaction and event interaction; Different scene roles are set based on the target port virtual environment model, and corresponding scene role task paths are set. By analyzing the user's real-time experience position and time, it is judged whether the scene loss phenomenon occurs. If so, the distance to the user's real-time experience position is calculated based on the extracted user's real-time task node, and a guide mark is generated for prompting; Set scene trigger rules based on the set scene roles and scene role task paths, divide scene events into preset events and random events, set preset event trigger rules through scene roles and scene role task paths, set random event trigger logic and trigger features based on UE4 blueprint visual scripts and form random event trigger rules; The scene triggering rules are formed according to the preset event triggering rules and the random event triggering rules, and the scene triggering rules include time triggering rules, location triggering rules and state triggering rules.

6. A data organization system for port loading and unloading process simulation display based on VR technology, characterized in that: The system includes: a memory and a processor, wherein the memory contains a method program for organizing data of a port loading and unloading process simulation display based on VR technology, and when the method program for organizing data of a port loading and unloading process simulation display based on VR technology is executed by the processor, the following steps are implemented: Acquire target port information, perform regional environment analysis and port facility analysis based on the target port information, and construct a port environment sub-model and a port facility sub-model; Constructing a general assembly model according to the port environment sub-model and the port facility sub-model, and performing port virtual scene rendering and dynamic association to obtain a three-dimensional model of the target port; Based on the three-dimensional model of the target port, scene motion modeling is performed, scene motion classification is performed, scene motion rules are formulated according to the classification results, and a target port operation model is obtained; Building a virtual environment model of the target port based on the target port three-dimensional model and the target port motion model, and setting user interaction rules and scene triggering rules; Obtain port loading and unloading information, extract the loading and unloading process characteristics and loading and unloading process characteristics of the target port, establish loading and unloading simulation rules in combination with the virtual environment model of the target port, and adapt the loading and unloading scene; The step of acquiring port loading and unloading information, extracting the loading and unloading process characteristics and loading and unloading process characteristics of the target port, establishing loading and unloading simulation rules in combination with the virtual environment model of the target port, and adapting the loading and unloading scene specifically includes: Acquire port loading and unloading information, perform feature extraction on the port loading and unloading information, extract loading and unloading process features and loading and unloading process features of each loading and unloading type, and obtain loading and unloading feature information; According to the characteristics of the loading and unloading process, the loading and unloading process nodes of the target port are set, and the loading and unloading process of each loading and unloading process node is matched according to the characteristics of the loading and unloading process to form a loading and unloading process diagram of the target port; Obtain historical loading and unloading instances of the target port, match and analyze each loading and unloading instance with the corresponding loading and unloading process according to the loading and unloading flow chart, and obtain loading and unloading instance matching information according to the similarity between each historical loading and unloading instance and each loading and unloading process and loading and unloading technology in the loading and unloading flow chart; Extract features from each historical loading and unloading instance to obtain the loading and unloading event features of each instance, including loading and unloading routes, loading and unloading processes, the number of vehicles on the loading and unloading routes, and port throughput, and obtain the loading and unloading event feature information; According to the loading and unloading process nodes, loading and unloading processes and loading and unloading events, entity triples are formed and a loading and unloading knowledge graph is constructed. The characteristic information of the loading and unloading events is used as the auxiliary features of each loading and unloading event, and the occurrence probability of each loading and unloading event is set; Based on the loading and unloading knowledge graph, loading and unloading simulation rules are set, and the loading and unloading routes and loading and unloading processes of users' real-time loading and unloading experience are used as prior conditions to match loading and unloading events. The number of vehicles on the loading and unloading routes and the port throughput are simulated according to the matching results of loading and unloading events, loading and unloading simulation events are generated, and real-time loading and unloading routes are generated through path planning algorithms; The loading and unloading simulation rules are adapted to the target port virtual environment model for loading and unloading scenarios.

Images